Pivotal Knowledge Base

Follow

How to relocate yarn-site.xml (yarn.nodemanager.local.dirs) in an existing HD/PHD cluster

Environment

 Product  Version
 Pivotal HD  1.x, 2.x
 OS  RHEL 6.x

Problem

The mount points for yarn.nodemanager.local.dirs are running out of space. Is there a way to relocate to different mount points?

Resolution

1. Create new directories
2. Grant the same permissions to those new directories to match with the old directories
3. Change the parameter in yarn-site.xml (via icm_client or by manually editing it) and make sure this is done across all Yarn node managers. Before changing any parameters please make a backup of the current configuration of the HD cluster. If icm_client is being used, backup the configuration using the following command

icm_client fetch-configuration -l CLUSTERNAME -o LOCALDIR

4. If you've only used icm_client to change the configurations in the past then make sure you use icm_client to make the same changes here (with command listed below) or else you can manually edit the yarn-site.xml file directly

icm_client reconfigure -l CLUSTERNAME -c LOCALDIR

5. Restart Yarn

icm_client restart -l CLUSTERNAME -s yarn

6. Check to make sure the new mount points are in use

Changes can be made to yarn-site.xml for the yarn.nodemanager.local.dirs parameter on any node managers and you can do this one at a time.

For rollback purposes, please make sure that you do not unmount the old directories until confirmation that things are working properly for the new mount points.

IMPORTANT: Please make sure that the new parent and child directories do have the same exact permissions as the old mount point directories.

 

 

Comments

Powered by Zendesk