Examples of ReduceDriver

org.apache.hadoop.mrunit.ReduceDriver
Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results.
org.apache.hadoop.mrunit.mapreduce.ReduceDriver
Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results. This is designed to handle a single (k, v*) -> (k, v)* case from the Reducer, representing a single unit test. Multiple input (k, v*) sets should go in separate unit tests.

Examples of org.apache.hadoop.mrunit.ReduceDriver

  }


  @SuppressWarnings({ "unchecked", "rawtypes" })
  @Test
  public void testOutputFormatWithMismatchInOutputClasses() throws IOException {
    final ReduceDriver driver = ReduceDriver.newReduceDriver(reducer);
    driver.withOutputFormat(TextOutputFormat.class, TextInputFormat.class);
    driver.withInputKey(new Text("a"));
    driver.withInputValue(new LongWritable(1)).withInputValue(
        new LongWritable(2));
    driver.withOutput(new LongWritable(), new Text("a\t3"));
    driver.runTest();
  }

View Full Code Here

Examples of org.apache.hadoop.mrunit.mapreduce.ReduceDriver

    List list = new ArrayList();
    for (Object value : values)
    {
      list.add(createWritable(value, reducer.getValueInType()));
    }
    ReduceDriver reduceDriver = new ReduceDriver<Text, LongWritable, Text, LongWritable>();
    reduceDriver.withInput(createWritable(key, reducer.getKeyInType()), list);
    reduceDriver.setReducer(reducer);
    List results = reduceDriver.run();
    Collections.sort(results, PairComparer.INSTANCE);
    String header = String.format("(%s, %s)\r\n\r\n -> reduces via %s to -> \r\n", key, list, reducer.getClass()
        .getSimpleName());
    Approvals.verifyAll(header, results, Echo.INSTANCE);
  }

View Full Code Here

Examples of org.apache.hadoop.mrunit.mapreduce.ReduceDriver

   * Tests that method context.getNumReduceTasks() correctly returns value from configuration
   * */
  @Test
  public void testContext_getNumReduceTasks() throws Exception {
    Reducer reducer = new TestReducer();
    ReduceDriver driver = ReduceDriver.newReduceDriver(reducer);
    driver.getConfiguration().setInt("mapred.reduce.tasks", 10);


    List<Text> values = new ArrayList();
    values.add(new Text("bb"));
    driver.withInput(new Text("aa"), values);
    assertEquals("Reducer not executed", 1, driver.run().size());


  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.