Pivotal Knowledge Base

Follow

Deploying PHD 1.1.1 with Isilon: Yarn ResourceManager will not start if using the default cluster template

Environment

  • PCC 2.2.1
  • PHD 1.1.1.0
  • ISILON 

Scenario

You have just deployed PHD successfully onto an Isilon appliance. When you start the cluster the ResourceManager fails to start.

Symptom

2014-10-13 16:31:37,412 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1612)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:329)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:225)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:710)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1580)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1604)
        ... 4 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1486)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1578)
        ... 5 more

Cause

The procedure for deploying PHD 1.1.1.0 on an Isilon appliance requires Pivotal Command Center version 2.2.1. PCC 2.2.1 is normally delivered with PHD 2.0.1.0.

When setting up for deployment, one of the steps is to perform a "fetch-template". If you don't specify the PHD-1.1.1.0 template, it defaults to the PHD-2.0.1.0 version. Deploying with the default, in this case, causes the ResourceManager to fail because it expects one Java class, but another is installed instead.

In PHD 2.0.1.0 the DefaultResourceCalculator class is:

<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
</property>

While in PHD 1.1.10 the DefaultResourceCalculator class is:

<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator</value>
</property>

Fix
In the scenario above, you attempted to deploy using cluster-id cluster1 with cluster config directory /home/gpadmin/ClusterConfig1.

Stop all currently running PHD services:

icm_client stop -l phdid

Fetch the correct template:

icm_client fetch-template -o /home/gpadmin/ClusterConfigIsilonPHD111 -v PHD-1.1.1.0

Now copy over the previous clusterConfig.xml over:

cp /home/gpadmin/ClusterConfig1/clusterConfig.xml /home/gpadmin/ClusterConfigIsilonPHD111/

Edit /home/gpadmin/ClusterConfigIsilonPHD111/clusterConfig.xml
Make sure you have the gphdStackVer attribute set as follows:

<gphdStackVer>PHD-1.1.1.0</gphdStackVer>

Reconfigure the cluster:

icm_client reconfigure -l cluster1 -c /home/gpadmin/ClusterConfigIsilonPHD111/

See also article regarding Yarn resourcemanager does not start after icm deployment with isilon

Comments

Powered by Zendesk