Examples of org.apache.hadoop.mapreduce.TaskAttemptID

org.apache.hadoop.mapreduce.TaskAttemptID
TaskAttemptID represents the immutable and unique identifier for a task attempt. Each task attempt is one particular instance of a Map or Reduce Task identified by its TaskID. TaskAttemptID consists of 2 parts. First part is the {@link TaskID}, that this TaskAttemptID belongs to. Second part is the task attempt number.
An example TaskAttemptID is : attempt_200707121733_0003_m_000005_0 , which represents the zeroth task attempt for the fifth map task in the third job running at the jobtracker started at 200707121733.
Applications should never construct or parse TaskAttemptID strings , but rather use appropriate constructors or {@link #forName(String)} method. @see JobID @see TaskID

    };
  }


  @Override
  public TaskAttemptID newTaskAttemptID(JobID jobId, boolean isMap, int taskId, int id) {
    return new TaskAttemptID(jobId.getJtIdentifier(), jobId.getId(), isMap, taskId, id);
  }

View Full Code Here

    }
  }


  @Override
  public org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext(Configuration conf, final Progressable progressable) {
    return new org.apache.hadoop.mapreduce.TaskAttemptContext(conf, new TaskAttemptID()) {
      @Override
      public void progress() {
        progressable.progress();
      }
    };

View Full Code Here

    };
  }


  @Override
  public TaskAttemptID newTaskAttemptID(JobID jobId, boolean isMap, int taskId, int id) {
    return new TaskAttemptID(jobId.getJtIdentifier(), jobId.getId(), isMap, taskId, id);
  }

View Full Code Here

      return new TaskID();
    }


    @Override
    public TaskAttemptID createTaskAttemptID() {
      return new TaskAttemptID();
    }

View Full Code Here

      if (job.getConfiguration().get("mapred.job.tracker", "")
              .equalsIgnoreCase("local")) {
        try {
          // This call needs to be removed after MAPREDUCE-1447 is fixed.
          outputFormat.getOutputCommitter(createTaskAttemptContext(
                  job.getConfiguration(), new TaskAttemptID())).abortJob(job, JobStatus.State.FAILED);
        } catch (IOException e) {
          throw new IOException("Failed to abort job", e);
        } catch (InterruptedException e) {
          throw new IOException("Failed to abort job", e);
        }

View Full Code Here

  }


  public void initialize(DoFn<?, ?> fn) {
    if (context == null) {
      configureLocalFiles();
      context = TaskInputOutputContextFactory.create(getConfiguration(), new TaskAttemptID(),
          new SparkReporter(counters));
    }
    fn.setContext(context);
    fn.initialize();
  }

View Full Code Here

    splits.toArray(sorted);
    Builder.sortSplits(sorted);


    int numTrees = Builder.getNbTrees(conf); // total number of trees


    TaskAttemptContext task = new TaskAttemptContext(conf, new TaskAttemptID());


    firstOutput = new MockContext(new Step1Mapper(), conf, task.getTaskAttemptID(), numTrees);


    /* first instance id in hadoop's order */
    //int[] firstIds = new int[nbSplits];

View Full Code Here


      // expected number of trees that this mapper will build
      int mapNbTrees = Step1Mapper.nbTrees(NUM_MAPPERS, NUM_TREES, partition);


      MockContext context = new MockContext(new Step1Mapper(),
          new Configuration(), new TaskAttemptID(), mapNbTrees);


      MockStep1Mapper mapper = new MockStep1Mapper(treeBuilder, dataset, seed,
          partition, NUM_MAPPERS, NUM_TREES);


      // make sure the mapper computed firstTreeId correctly

View Full Code Here


  public static <K1, V1, K2, V2> Mapper<K1, V1, K2, V2>.Context build(Mapper<K1, V1, K2, V2> mapper,
                                                                      Configuration configuration,
                                                                      RecordWriter<K2, V2> output)
    throws IOException, InterruptedException {
    return mapper.new Context(configuration, new TaskAttemptID(), null, output, null, new DummyStatusReporter(), null);
  }

View Full Code Here

                                                                       RecordWriter<K2, V2> output,
                                                                       Class<K1> keyClass,
                                                                       Class<V1> valueClass)
    throws IOException, InterruptedException {
    return reducer.new Context(configuration,
                               new TaskAttemptID(),
                               new MockIterator(),
                               null,
                               null,
                               output,
                               null,

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of org.apache.hadoop.mapreduce.TaskAttemptID

com.digitalpebble.behemoth.io.sequencefile.SequenceFileConverterMapperTest

com.facebook.giraph.hive.input.benchmark.InputBenchmark

com.facebook.giraph.hive.tailer.Tailer

com.facebook.hiveio.benchmark.InputBenchmark

com.facebook.hiveio.input.HiveInput$RecordIterator

com.hadoop.mapreduce.TestLzoTextInputFormat

com.thinkaurelius.faunus.formats.graphson.GraphSONRecordReaderTest

com.tinkerpop.gremlin.giraph.hdfs.GiraphElementIterator

com.twitter.elephantbird.mapred.input.DeprecatedInputFormatWrapper$RecordReaderWrapper

com.twitter.elephantbird.pig.load.TestRCFileProtobufStorage

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.