|Pivotal HD / HDP||3.x / 2.3,2.4|
How to collect logs associated with MapReduce, Tez, Hive jobs or any other jobs running in a YARN cluster. These logs will usually be requested by Pivotal Support for troubleshooting purposes.
Identify the job or application ID: This can be achieved by either,
CLI, by Running:
yarn application -list -appStates FINISHED,FAILED,KILLED
Reviewing the YARN ResourceManager UI available via Ambari GUI / YARN / Quick Links / ResourceManager UI:
Run the following command to fetch and compress the logs:
yarn logs -applicationId <app_id> -appOwner <user> |& gzip -c > <app_id>.log.gz
The option -appOwner can be omitted if the current user is the same user who ran the job.