Pivotal Knowledge Base

Follow

MapReduce jobs fails with "Not able to initialize user directories in any of the configured local directories"

Environment

  • PHD 1.x

Symptom:

Executing MapReduce job fails with an error message 

14/02/27 01:59:06 INFO mapreduce.Job: Job job_1393464189937_0030 running in uber mode : false
14/02/27 01:59:06 INFO mapreduce.Job: map 0% reduce 0%
14/02/27 01:59:06 INFO mapreduce.Job: Job job_1393464189937_0030 failed with state FAILED due to: Application application_1393464189937_0030 failed 1 times due to AM Container for appattempt_1393464189937_0030_000001 exited with exitCode: -1000 due to: java.io.IOException: Not able to initialize user directories in any of the configured local directories for user hdfs
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.createUserLocalDirs(DefaultContainerExecutor.java:357)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:88)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:860)

Solution

- Identify the yarn directories specified using the parameter yarn.nodemanager.local-dirs from /etc/gphd/hadoop/conf/yarn-site.xml on yarn node manager.

- Delete the files / folders under usercache from all the directories listed in yarn-site.xml and all the node managers.

Ex: rm -rf /data/dn/yarn/nm-local-dir/usercache/*

Miscellaneous

Comments

  • Avatar
    Karthikeyan Nagalingam

    I am new to Hadoop. I don't have the usercache folder in yarn.nodemanager.local-dirs folder. can you guide me, to fix the issue.

    Regards,
    Karthikeyan,.N

Powered by Zendesk