Examples of DominantResourceCalculator

org.apache.hadoop.yarn.server.resourcemanager.resource.DominantResourceCalculator
A {@link ResourceCalculator} which uses the concept of dominant resource to compare multi-dimensional resources. Essentially the idea is that the in a multi-resource environment, the resource allocation should be determined by the dominant share of an entity (user or queue), which is the maximum share that the entity has been allocated of any resource. In a nutshell, it seeks to maximize the minimum dominant share across all entities. For example, if user A runs CPU-heavy tasks and user B runs memory-heavy tasks, it attempts to equalize CPU share of user A with Memory-share of user B. In the single resource case, it reduces to max-min fairness for that resource. See the Dominant Resource Fairness paper for more details: www.cs.berkeley.edu/~matei/papers/2011/nsdi_drf.pdf
org.apache.hadoop.yarn.util.resource.DominantResourceCalculator
A {@link ResourceCalculator} which uses the concept of dominant resource to compare multi-dimensional resources. Essentially the idea is that the in a multi-resource environment, the resource allocation should be determined by the dominant share of an entity (user or queue), which is the maximum share that the entity has been allocated of any resource. In a nutshell, it seeks to maximize the minimum dominant share across all entities. For example, if user A runs CPU-heavy tasks and user B runs memory-heavy tasks, it attempts to equalize CPU share of user A with Memory-share of user B. In the single resource case, it reduces to max-min fairness for that resource. See the Dominant Resource Fairness paper for more details: www.cs.berkeley.edu/~matei/papers/2011/nsdi_drf.pdf

Examples of org.apache.hadoop.yarn.server.resourcemanager.resource.DominantResourceCalculator


  }
  
  @Test
  public void testNormalizeRequestWithDominantResourceCalculator() {
    ResourceCalculator resourceCalculator = new DominantResourceCalculator();
    
    Resource minResource = Resources.createResource(1024, 1);
    Resource clusterResource = Resources.createResource(10 * 1024, 10);
    
    ResourceRequest ask = new ResourceRequestPBImpl();

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterCapacity, minimumAllocation, maximumAllocation, incrAllocation);


    // Release containers
    for (ContainerId releasedContainerId : release) {
      RMContainer rmContainer = getRMContainer(releasedContainerId);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterResource, minimumAllocation, maximumAllocation, incrAllocation);


    // Set amResource for this app
    if (!application.getUnmanagedAM() && ask.size() == 1
        && application.getLiveContainers().isEmpty()) {

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

    assertEquals(maxResource.getMemory(), ask.getCapability().getMemory());
  }
  
  @Test (timeout = 30000)
  public void testNormalizeRequestWithDominantResourceCalculator() {
    ResourceCalculator resourceCalculator = new DominantResourceCalculator();
    
    Resource minResource = Resources.createResource(1024, 1);
    Resource maxResource = Resources.createResource(10240, 10);
    Resource clusterResource = Resources.createResource(10 * 1024, 10);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterCapacity, minimumAllocation, maximumAllocation, incrAllocation);


    // Release containers
    for (ContainerId releasedContainerId : release) {
      RMContainer rmContainer = getRMContainer(releasedContainerId);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

    assertEquals(maxResource.getMemory(), ask.getCapability().getMemory());
  }
  
  @Test (timeout = 30000)
  public void testNormalizeRequestWithDominantResourceCalculator() {
    ResourceCalculator resourceCalculator = new DominantResourceCalculator();
    
    Resource minResource = Resources.createResource(1024, 1);
    Resource maxResource = Resources.createResource(10240, 10);
    Resource clusterResource = Resources.createResource(10 * 1024, 10);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterCapacity, minimumAllocation, maximumAllocation, incrAllocation);


    // Release containers
    for (ContainerId releasedContainerId : release) {
      RMContainer rmContainer = getRMContainer(releasedContainerId);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

    assertEquals(maxResource.getMemory(), ask.getCapability().getMemory());
  }
  
  @Test (timeout = 30000)
  public void testNormalizeRequestWithDominantResourceCalculator() {
    ResourceCalculator resourceCalculator = new DominantResourceCalculator();
    
    Resource minResource = Resources.createResource(1024, 1);
    Resource maxResource = Resources.createResource(10240, 10);
    Resource clusterResource = Resources.createResource(10 * 1024, 10);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterCapacity, minimumAllocation, maximumAllocation, incrAllocation);


    // Release containers
    for (ContainerId releasedContainerId : release) {
      RMContainer rmContainer = getRMContainer(releasedContainerId);

View Full Code Here

Examples of org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

          "or non existant application " + appAttemptId);
      return EMPTY_ALLOCATION;
    }


    // Sanity check
    SchedulerUtils.normalizeRequests(ask, new DominantResourceCalculator(),
        clusterCapacity, minimumAllocation, maximumAllocation, incrAllocation);


    // Release containers
    for (ContainerId releasedContainerId : release) {
      RMContainer rmContainer = getRMContainer(releasedContainerId);

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.