Examples of SleepJob

org.apache.hadoop.SleepJob
Dummy class for testing MR framefork. Sleeps for a defined period of time in mapper and reducer. Generates fake input for map / reduce jobs. Note that generated number of input pairs is in the order of numMappers * mapSleepTime / 100, so the job uses some disk space.
org.apache.hadoop.examples.SleepJob
Dummy class for testing MR framefork. Sleeps for a defined period of time in mapper and reducer. Generates fake input for map / reduce jobs. Note that generated number of input pairs is in the order of numMappers * mapSleepTime / 100, so the job uses some disk space.
org.apache.hadoop.mapreduce.SleepJob
Dummy class for testing MR framefork. Sleeps for a defined period of time in mapper and reducer. Generates fake input for map / reduce jobs. Note that generated number of input pairs is in the order of numMappers * mapSleepTime / 100, so the job uses some disk space.

Examples of org.apache.hadoop.examples.SleepJob

  //got a KillJobAction).
  private static final Log LOG =
        LogFactory.getLog(TestEmptyJobWithDFS.class.getName());
  private void runSleepJob(JobConf conf) throws Exception {
    String[] args = { "-m", "1", "-r", "10", "-mt", "1000", "-rt", "10000" };
    ToolRunner.run(conf, new SleepJob(), args);
  }

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

                                    String queueName) 
                                      throws IOException {
    JobConf clientConf = new JobConf();
    clientConf.set("mapred.job.tracker", "localhost:"
        + miniMRCluster.getJobTrackerPort());
    SleepJob job = new SleepJob();
    job.setConf(clientConf);
    clientConf = job.setupJobConf(numMappers, numReducers, 
        mapSleepTime, (int)mapSleepTime/100,
        reduceSleepTime, (int)reduceSleepTime/100);
    if (queueName != null) {
      clientConf.setQueueName(queueName);
    }

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

                                    String queueName) 
                                      throws IOException {
    JobConf clientConf = new JobConf();
    clientConf.set("mapred.job.tracker", "localhost:"
        + miniMRCluster.getJobTrackerPort());
    SleepJob job = new SleepJob();
    job.setConf(clientConf);
    clientConf = job.setupJobConf(numMappers, numReducers, 
        mapSleepTime, (int)mapSleepTime/100,
        reduceSleepTime, (int)reduceSleepTime/100);
    if (queueName != null) {
      clientConf.setQueueName(queueName);
    }

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

  private void launchSleepJob(String mapSleepTime, String reduceSleepTime,
                              String mapTaskCount, String reduceTaskCount,
                              Configuration conf) throws Exception {
    String[] args = { "-m", mapTaskCount, "-r", reduceTaskCount,
                      "-mt", mapSleepTime, "-rt", reduceSleepTime };
    ToolRunner.run(conf, new SleepJob(), args);
  }

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

                         "-m", "1", 
                         "-r", "1", 
                         "-mt", "1000", 
                         "-rt", "1000",
                         "-recordt","100"};
      SleepJob job = new SleepJob();
      JobConf jobConf = new JobConf(conf); 
      int exitStatus = ToolRunner.run(jobConf, job, jobArgs);
      Assert.assertEquals("Exit Code:", 0, exitStatus);
      UtilsForTests.waitFor(100);
      JobClient jobClient = jtClient.getClient();

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

   * @return the job id of the high ram job
   * @throws Exception is thrown when the method fails to run the high ram job
   */
  public JobID runHighRamJob (Configuration conf, JobClient jobClient, 
      JTProtocol remoteJTClient,String assertMessage) throws Exception {
    SleepJob job = new SleepJob();
    String jobArgs []= {"-D","mapred.cluster.max.map.memory.mb=2048", 
                        "-D","mapred.cluster.max.reduce.memory.mb=2048", 
                        "-D","mapred.cluster.map.memory.mb=1024", 
                        "-D","mapreduce.job.complete.cancel.delegation.tokens=false",
                        "-D","mapred.cluster.reduce.memory.mb=1024",

View Full Code Here

Examples of org.apache.hadoop.examples.SleepJob

     }
  }


  
  public JobID runSleepJob(boolean signalJob) throws Exception{
    SleepJob job = new SleepJob();
    job.setConf(conf);
    conf = job.setupJobConf(5, 1, 100, 5, 100, 5);
    JobConf jconf = new JobConf(conf);
    //Controls the job till all verification is done 
    FinishTaskControlAction.configureControlActionForJob(conf);
    //Submitting the job
    RunningJob rJob = cluster.getJTClient().getClient().submitJob(jconf);

View Full Code Here

Examples of org.apache.hadoop.mapreduce.SleepJob

      IOException {
    String[] args = { "-m", "0", "-r", "0", "-mt", "0", "-rt", "0" };
    boolean throwsException = false;
    String msg = null;
    try {
      ToolRunner.run(jobConf, new SleepJob(), args);
    } catch (RemoteException re) {
      throwsException = true;
      msg = re.unwrapRemoteException().getMessage();
    }
    assertTrue(throwsException);

View Full Code Here

Examples of org.apache.hadoop.mapreduce.SleepJob

      IOException {
    String[] args = { "-m", "0", "-r", "0", "-mt", "0", "-rt", "0" };
    boolean throwsException = false;
    String msg = null;
    try {
      ToolRunner.run(jobConf, new SleepJob(), args);
    } catch (RemoteException re) {
      throwsException = true;
      msg = re.unwrapRemoteException().getMessage();
    }
    assertTrue(throwsException);

View Full Code Here

Examples of org.apache.hadoop.mapreduce.SleepJob

  @Test
  public void testFailedTaskJobStatus()
      throws IOException, InterruptedException, ClassNotFoundException {
    Configuration conf = new Configuration(cluster.getConf());
    TaskInfo taskInfo = null;
    SleepJob job = new SleepJob();
    job.setConf(conf);
    Job slpJob = job.createJob(3, 1, 4000, 4000, 100, 100);
    JobConf jobConf = new JobConf(conf);
    jobConf.setMaxMapAttempts(20);
    jobConf.setMaxReduceAttempts(20);
    slpJob.submit();
    RunningJob runJob =

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.