Pivotal Knowledge Base

Follow

Lost queues in Capacity scheduler after PHD upgrade.

Environment

PHD 1.1.X to PHD2.0.X upgrade using ICM

Problem

After the PHD upgrade, only default queue is available with capacity scheduler.

To list the available queues, run the following command:

hadoop queue -list

Root Cause

During upgrade, ICM pushes the configurations available under the cluster configuration directory, if capacity-scheduler.xml file configured with queues was not used to reconfigure, then default capacity-scheduler.xml file is pushed to the nodes and user loses the configured queues.

Solution

1: You should still have a backup of capacity-scheduler.xml if you did fetch-configuration before upgrading. We assume that icm_client reconfigure was used to push capacity-scheduler.xml to all the nodes earlier. In case, backup of capacity-scheduler.xml is not available, please follow this article to manually add the queues again.

2: Modify the original capacity-scheduler.xml by updating

org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator

to

org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator

3: After updating capacity-scheduler.xml, use icm_client reconfigure to push the updated capacity-scheduler.xml file to the cluster nodes.

 

 

 

 

 

 

 
 
 
 
 
 
 
 
 
 

Comments

Powered by Zendesk