Examples of org.apache.hadoop.yarn.api.ApplicationMasterProtocol.allocate()

Class org.apache.hadoop.yarn.api.ApplicationMasterProtocol

Examples of org.apache.hadoop.yarn.api.ApplicationMasterProtocol.allocate()

org.apache.hadoop.yarn.api.ApplicationMasterProtocol.allocate()

The main interface between an ApplicationMaster and the ResourceManager.

The ApplicationMaster uses this interface to provide a list of {@link ResourceRequest} and returns unused {@link Container} allocated toit via {@link AllocateRequest}. Optionally, the ApplicationMaster can also blacklist resources which it doesn't want to use.

This also doubles up as a heartbeat to let the ResourceManager know that the ApplicationMaster is alive. Thus, applications should periodically make this call to be kept alive. The frequency depends on {@link YarnConfiguration#RM_AM_EXPIRY_INTERVAL_MS} which defaults to{@link YarnConfiguration#DEFAULT_RM_AM_EXPIRY_INTERVAL_MS}.

The ResourceManager responds with list of allocated {@link Container}, status of completed containers and headroom information for the application.

The ApplicationMaster can use the available headroom (resources) to decide how to utilized allocated resources and make informed decisions about future resource requests.
@param request allocation request @return allocation response @throws YarnException @throws IOException @throws InvalidApplicationMasterRequestException This exception is thrown when an ApplicationMaster calls allocate without registering first. @throws InvalidResourceBlacklistRequestException This exception is thrown when an application provides an invalid specification for blacklist of resources. @throws InvalidResourceRequestException This exception is thrown when a {@link ResourceRequest} is out ofthe range of the configured lower and upper limits on the resources. @see AllocateRequest @see AllocateResponse


    @Override
    protected ApplicationMasterProtocol createSchedulerProxy() {
      ApplicationMasterProtocol scheduler = mock(ApplicationMasterProtocol.class);
      try {
        when(scheduler.allocate(isA(AllocateRequest.class)))
          .thenThrow(RPCUtil.getRemoteException(new IOException("forcefail")));
      } catch (YarnException e) {
      } catch (IOException e) {
      }
      return scheduler;

View Full Code Here


    AllocateRequest allocateRequest =
        AllocateRequest.newInstance(0, 0.0f, null, null, blacklistRequest);
    boolean error = false;
    try {
      client.allocate(allocateRequest);
    } catch (InvalidResourceBlacklistRequestException e) {
      error = true;
    }


    rm.stop();

View Full Code Here

    assertEquals(2, snoopRequest.getNumContainers());
    
    ApplicationMasterProtocol realRM = amClient.rmClient;
    try {
      ApplicationMasterProtocol mockRM = mock(ApplicationMasterProtocol.class);
      when(mockRM.allocate(any(AllocateRequest.class))).thenAnswer(
          new Answer<AllocateResponse>() {
            public AllocateResponse answer(InvocationOnMock invocation)
                throws Exception {
              amClient.removeContainerRequest(
                             new ContainerRequest(capability, nodes,

View Full Code Here

      rpc.stopProxy(rmClient, conf); // To avoid using cached client
      rmClient = createRMClient(rm, conf, rpc, currentUser);
      AllocateRequest allocateRequest =
          Records.newRecord(AllocateRequest.class);
      try {
        rmClient.allocate(allocateRequest);
        Assert.fail("You got to be kidding me! "
            + "Using App tokens after app-finish should fail!");
      } catch (Throwable t) {
        LOG.info("Exception found is ", t);
        // The exception will still have the earlier appAttemptId as it picks it

View Full Code Here


      // One allocate call.
      AllocateRequest allocateRequest =
          Records.newRecord(AllocateRequest.class);
      Assert.assertTrue(
          rmClient.allocate(allocateRequest).getAMCommand() == null);


      // Simulate a master-key-roll-over
      AMRMTokenSecretManager appTokenSecretManager =
          rm.getRMContext().getAMRMTokenSecretManager();
      SecretKey oldKey = appTokenSecretManager.getMasterKey();

View Full Code Here

      // Another allocate call. Should continue to work.
      rpc.stopProxy(rmClient, conf); // To avoid using cached client
      rmClient = createRMClient(rm, conf, rpc, currentUser);
      allocateRequest = Records.newRecord(AllocateRequest.class);
      Assert.assertTrue(
          rmClient.allocate(allocateRequest).getAMCommand() == null);
    } finally {
      rm.stop();
      if (rmClient != null) {
        rpc.stopProxy(rmClient, conf); // To avoid using cached client
      }

View Full Code Here


    AllocateRequest allocateRequest =
        AllocateRequest.newInstance(0, 0.0f, null, null, blacklistRequest);
    boolean error = false;
    try {
      client.allocate(allocateRequest);
    } catch (InvalidResourceBlacklistRequestException e) {
      error = true;
    }


    rm.stop();

View Full Code Here

      rpc.stopProxy(rmClient, conf); // To avoid using cached client
      rmClient = createRMClient(rm, conf, rpc, currentUser);
      AllocateRequest allocateRequest =
          Records.newRecord(AllocateRequest.class);
      try {
        rmClient.allocate(allocateRequest);
        Assert.fail("You got to be kidding me! "
            + "Using App tokens after app-finish should fail!");
      } catch (Throwable t) {
        LOG.info("Exception found is ", t);
        // The exception will still have the earlier appAttemptId as it picks it

View Full Code Here


      // One allocate call.
      AllocateRequest allocateRequest =
          Records.newRecord(AllocateRequest.class);
      Assert.assertTrue(
          rmClient.allocate(allocateRequest).getAMCommand() == null);


      // Simulate a master-key-roll-over
      AMRMTokenSecretManager appTokenSecretManager =
          rm.getRMContext().getAMRMTokenSecretManager();
      SecretKey oldKey = appTokenSecretManager.getMasterKey();

View Full Code Here

      // Another allocate call. Should continue to work.
      rpc.stopProxy(rmClient, conf); // To avoid using cached client
      rmClient = createRMClient(rm, conf, rpc, currentUser);
      allocateRequest = Records.newRecord(AllocateRequest.class);
      Assert.assertTrue(
          rmClient.allocate(allocateRequest).getAMCommand() == null);
    } finally {
      rm.stop();
      if (rmClient != null) {
        rpc.stopProxy(rmClient, conf); // To avoid using cached client
      }

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.