Examples of com.amazonaws.services.elasticmapreduce.model.RunJobFlowRequest

Package com.amazonaws.services.elasticmapreduce.model

Examples of com.amazonaws.services.elasticmapreduce.model.RunJobFlowRequest

com.amazonaws.services.elasticmapreduce.model.RunJobFlowRequest
Container for the parameters to the {@link com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduce#runJobFlow(RunJobFlowRequest) RunJobFlow operation}.
RunJobFlow creates and starts running a new job flow. The job flow will run the steps specified. Once the job flow completes, the cluster is stopped and the HDFS partition is lost. To prevent loss of data, configure the last step of the job flow to store results in Amazon S3. If the JobFlowInstancesDetail : KeepJobFlowAliveWhenNoSteps parameter is set to TRUE , the job flow will transition to the WAITING state rather than shutting down once the steps have completed.

A maximum of 256 steps are allowed in each job flow.

For long running job flows, we recommended that you periodically store your results.
@see com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduce#runJobFlow(RunJobFlowRequest)

      MojoFailureException {
    return getService().runJobFlow(getRequest());
  }


  private RunJobFlowRequest getRequest() {
    RunJobFlowRequest req = new RunJobFlowRequest();


    req.setLogUri(logUri);
    req.setName(jobName);


    JobFlowInstancesConfig instances = getInstances();
    req.setInstances(instances);


    HadoopJarStepConfig hadoopJarStepConfig = new HadoopJarStepConfig()
        .withArgs(args).withMainClass(mainClass).withJar(path);


    req.getSteps().add(new StepConfig("custom-jar-exec", hadoopJarStepConfig));


    return req;
  }

View Full Code Here

      MojoFailureException {
    return getService().runJobFlow(getRequest());
  }


  private RunJobFlowRequest getRequest() {
    RunJobFlowRequest req = new RunJobFlowRequest();


    req.setLogUri(logUri);
    req.setName(jobName);


    JobFlowInstancesConfig instances = getInstances();
    req.setInstances(instances);


    HadoopJarStepConfig hadoopJarStepConfig = new HadoopJarStepConfig()
        .withArgs(args).withMainClass(mainClass).withJar(path);


    req.getSteps().add(new StepConfig("custom-jar-exec", hadoopJarStepConfig));


    return req;
  }

View Full Code Here

      MojoFailureException {
    return service.runJobFlow(getRequest());
  }


  private RunJobFlowRequest getRequest() {
    RunJobFlowRequest req = new RunJobFlowRequest();


    req.setLogUri(logUri);
    req.setName(jobName);


    JobFlowInstancesConfig instances = getInstances();
    req.setInstances(instances);


    HadoopJarStepConfig hadoopJarStepConfig = new HadoopJarStepConfig()
        .withArgs(args).withMainClass(mainClass).withJar(path);


    req.getSteps().add(new StepConfig("custom-jar-exec", hadoopJarStepConfig));


    return req;
  }

View Full Code Here

      MojoFailureException {
    return service.runJobFlow(getRequest());
  }


  private RunJobFlowRequest getRequest() {
    RunJobFlowRequest req = new RunJobFlowRequest();


    req.setLogUri(logUri);
    req.setName(jobName);


    JobFlowInstancesConfig instances = getInstances();
    req.setInstances(instances);


    HadoopJarStepConfig hadoopJarStepConfig = new HadoopJarStepConfig()
        .withArgs(args).withMainClass(mainClass).withJar(path);


    req.getSteps().add(new StepConfig("custom-jar-exec", hadoopJarStepConfig));


    return req;
  }

View Full Code Here

TOP

Related Classes of com.amazonaws.services.elasticmapreduce.model.RunJobFlowRequest

br.com.ingenieux.mojo.mapreduce.RunJobFlowsMojo

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.