List of findCounter() Examples

Examples of findCounter()

org.apache.hadoop.mapred.Counters.findCounter()
Find the counter for the given enum. The same enum will always return the same counter. @param key the counter key @return the matching counter object
org.apache.hadoop.mapreduce.CounterGroup.findCounter()
Internal to find a counter in a group. @param counterName the name of the counter @param displayName the display name of the counter @return the counter that was found or added
org.apache.hadoop.mapreduce.Counters.findCounter()
org.apache.tez.common.counters.CounterGroup.findCounter()
org.apache.tez.common.counters.TezCounters.findCounter()

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

    }


    count = counters.findCounter(Task.Counter.MAP_SKIPPED_RECORDS);
    contribution.incrementReadSkipCount(safeLongToInt(count.getValue()));


    count = counters.findCounter(Task.Counter.REDUCE_OUTPUT_RECORDS);
    contribution.incrementWriteCount(safeLongToInt(count.getValue()));


    count = counters.findCounter(Task.Counter.REDUCE_SKIPPED_RECORDS);


    for (int i = 0; i < safeLongToInt(count.getValue()); i++) {

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

    contribution.incrementReadSkipCount(safeLongToInt(count.getValue()));


    count = counters.findCounter(Task.Counter.REDUCE_OUTPUT_RECORDS);
    contribution.incrementWriteCount(safeLongToInt(count.getValue()));


    count = counters.findCounter(Task.Counter.REDUCE_SKIPPED_RECORDS);


    for (int i = 0; i < safeLongToInt(count.getValue()); i++) {
      contribution.incrementWriteSkipCount();
    }
  }

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

                  job.getConfiguration()).getClass()
              .getName());
      job.waitForCompletion(false);
      if (job.isComplete()) {
        Counters counters = job.getCounters();
        totalRecordsRead = counters.findCounter(
            DedupRecordCounter.TOTAL_RECORDS_READ).getValue();
        badRecords = counters.findCounter(
            DedupRecordCounter.BAD_RECORD).getValue();
        output = counters.findCounter(DedupRecordCounter.OUTPUT)
            .getValue();

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

      job.waitForCompletion(false);
      if (job.isComplete()) {
        Counters counters = job.getCounters();
        totalRecordsRead = counters.findCounter(
            DedupRecordCounter.TOTAL_RECORDS_READ).getValue();
        badRecords = counters.findCounter(
            DedupRecordCounter.BAD_RECORD).getValue();
        output = counters.findCounter(DedupRecordCounter.OUTPUT)
            .getValue();
        duplicateRecords = totalRecordsRead - output;
        logger.info("Total records read are: " + totalRecordsRead);

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

        Counters counters = job.getCounters();
        totalRecordsRead = counters.findCounter(
            DedupRecordCounter.TOTAL_RECORDS_READ).getValue();
        badRecords = counters.findCounter(
            DedupRecordCounter.BAD_RECORD).getValue();
        output = counters.findCounter(DedupRecordCounter.OUTPUT)
            .getValue();
        duplicateRecords = totalRecordsRead - output;
        logger.info("Total records read are: " + totalRecordsRead);
        logger.info("Bad Records are: " + badRecords);
        logger.info("Output records are: " + output);

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

          @Override
          public boolean dump(Job job) {
              Counters counters;
            try {
              counters = job.getCounters();
                    long wsize=counters.findCounter("higo", "dumpcount").getValue();
                    if(wsize>0)
                    {
                      try {
                  TableJoin.LOG.info("update "+tableName+ ",dump");
                  TableJoin.updatePercent(tableName, "Stage-2 map = 100%,  reduce = 100%", "DUMP")  ;

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

      if (start != 0 && now - start <= Integer.MAX_VALUE) {
        splitsBlock.getProgressWallclockTime().extend(newProgress,
            (int) (now - start));
      }


      Counter cpuCounter = counters.findCounter(TaskCounter.CPU_MILLISECONDS);
      if (cpuCounter != null && cpuCounter.getValue() <= Integer.MAX_VALUE) {
        splitsBlock.getProgressCPUTime().extend(newProgress,
            (int) cpuCounter.getValue()); // long to int? TODO: FIX. Same below
      }

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

      if (cpuCounter != null && cpuCounter.getValue() <= Integer.MAX_VALUE) {
        splitsBlock.getProgressCPUTime().extend(newProgress,
            (int) cpuCounter.getValue()); // long to int? TODO: FIX. Same below
      }


      Counter virtualBytes = counters
        .findCounter(TaskCounter.VIRTUAL_MEMORY_BYTES);
      if (virtualBytes != null) {
        splitsBlock.getProgressVirtualMemoryKbytes().extend(newProgress,
            (int) (virtualBytes.getValue() / (MEMORY_SPLITS_RESOLUTION)));
      }

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

      if (virtualBytes != null) {
        splitsBlock.getProgressVirtualMemoryKbytes().extend(newProgress,
            (int) (virtualBytes.getValue() / (MEMORY_SPLITS_RESOLUTION)));
      }


      Counter physicalBytes = counters
        .findCounter(TaskCounter.PHYSICAL_MEMORY_BYTES);
      if (physicalBytes != null) {
        splitsBlock.getProgressPhysicalMemoryKbytes().extend(newProgress,
            (int) (physicalBytes.getValue() / (MEMORY_SPLITS_RESOLUTION)));
      }

View Full Code Here

Examples of org.apache.hadoop.mapreduce.Counters.findCounter()

        throw new IllegalStateException("You should call run() first");
      }


      Counters counters = job.getCounters();


      Counter referenced = counters.findCounter(Counts.REFERENCED);
      Counter unreferenced = counters.findCounter(Counts.UNREFERENCED);
      Counter undefined = counters.findCounter(Counts.UNDEFINED);
      Counter multiref = counters.findCounter(Counts.EXTRAREFERENCES);


      boolean success = true;

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.