Pivotal Knowledge Base

Follow

How to Configure YARN Fair Scheduler on a PHD Cluster

Purpose

This is a quick reference for YARN Fair Scheduler configuration basics

  1. Add the following parameters to /etc/gphd/hadoop/conf/yarn-site.xml on the resource manager node or via icm_client reconfigure
    <property>
      <name>yarn.resourcemanager.scheduler.class</name>
      <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
    </property>

    Note: By default, Pivotal Hadoop Distrbution uses Capacity Scheduler class, you can change the value of yarn.resourcemanager.scheduler.class to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

  2. Include Fair Scheduler allocation file param in yarn-site.xml
    <property>
      <name>yarn.scheduler.fair.allocation.file</name>
      <value>/etc/gphd/hadoop/conf/fair-scheduler.xml</value>
    </property>

    You can add more control by adding properties in yarn-site.xml. For example, adding control of container assignment. Click here for a reference.

  3. There are five types of elements which can be set up in an allocation file
    • Queue element -- Representing queues. It has the following properties:
      • minResources -- Setting the minimum resources of a queue
      • maxResources -- Setting the maximum resources of a queue
      • maxRunningApps -- Setting the maximum number of apps from a queue to run at once
      • weight -- Sharing the cluster non-proportional with other queues. Default to 1
      • schedulingPolicy -- Values are "fifo"/"fair"/"drf" or any class that extends org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.SchedulingPolicy
      • aclSubmitApps -- Listing the users who can submit apps to the queue
      • minSharePreemptionTimeout -- Specifying the number of seconds the queue is under its minimum share before it tries to preempt containers to take resources from other queues.
    • User elements -- Representing user behaviors. It can contain a single properties to set maximum number apps for a particular user.
    • userMaxAppsDefault element -- Setting the default running app limit for users if the limit is not otherwise specified.
    • fairSharePreemptionTimeout element -- Setting the number of seconds a queue is under its fair share before it tries to preempt containers to take resources from other queues.
    • defaultQueueSchedulingPolicy element -- Specifying the default scheduling policy for queues; overriden by the schedulingPolicy element in each queue if specified.

    Below is a sample allocation configuration file (as mentioned under yarn.scheduler.fair.allocation.file) where a queue called test is configured. Just like YARN Capacity Scheduler the fair schedule supports sub-queues.

    <?xml version="1.0"?>
    <allocations>
      <queue name="test">
        <minResources>1000 mb, 1 vcores</minResources>
        <maxResources>5000 mb, 1 vcores</maxResources>
        <maxRunningApps>10</maxRunningApps>
        <aclSubmitApps>hdfs,gpadmin</aclSubmitApps>
        <weight>2.0</weight>
        <schedulingPolicy>fair</schedulingPolicy>
        <queue name="test-sub">
            <aclSubmitApps>root</aclSubmitApps>
            <minResources>500 mb, 1 vcores</minResources>
        </queue>
      </queue>
      <user name="root">
        <maxRunningApps>10</maxRunningApps>
      </user>
     <user name="hdfs">
        <maxRunningApps>5</maxRunningApps>
      </user>
      <user name="gpadmin">
        <maxRunningApps>5</maxRunningApps>
      </user>
      <userMaxAppsDefault>5</userMaxAppsDefault>
     <fairSharePreemptionTimeout>30</fairSharePreemptionTimeout>
    </allocations>

    Note: All queues are children of a predefined queue called root. A queue path starts with root and uses dot as delimiters. For example, a queue under root named queue1 has a sub queue called queue1-sub. The queue path of queue1-sub should be specified as root.queue1.queue1-sub. Child queues inherit the permission from the parent queues.

  4. Restarting Resourcemanager
    service hadoop-yarn-resourcemanager restart
  5. Submitting a job to the test queue
    hadoop jar /usr/lib/gphd/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount -D mapreduce.job.queuename=test /input /output/

You can check the queue and application status through resource manager web interface:
http://<Resource_manager>:8088/cluster/scheduler

 

Comments

  • Avatar
    sandeep chaturvdi

    tried to configure cluster using the recommendations mentioned here.. seen a strange behavior:

    a) configured cluster to use fair scheduler
    b) allocated 36 GB to yarn out of 64 GB available on data nodes
    c) ran MR job but the job always uses 50% of the resources i.e. memory.
    d) I see only 18 GB getting utilized by mappers and rest 18 is still free
    e) what I can do to make my MR job use entire configured memory?

    d) how do I tell MR job to use entire 36 GB configured for yarn app?

  • Avatar
    Christian

    I got the same problem but with capacity scheduler, cluseter server has 96 GB ram and only use a max capacity of 72 GB
    the queues that I have configured are:

    ======================
    Queue Name : default
    Queue State : running

    Scheduling Info : Capacity: 20.0, MaximumCapacity: 50.0, CurrentCapacity: 0.0

    Queue Name : hdpappLOW
    Queue State : running

    Scheduling Info : Capacity: 20.0, MaximumCapacity: 100.0, CurrentCapacity: 0.0

    Queue Name : hdplake
    Queue State : running

    Scheduling Info : Capacity: 10.0, MaximumCapacity: 50.0, CurrentCapacity: 0.0

    Queue Name : hdpappHIGH
    Queue State : running
    Scheduling Info : Capacity: 50.0, MaximumCapacity: 100.0, CurrentCapacity: 104.1

    so any idea about it?

Powered by Zendesk