Examples of cascading.flow.hadoop2.Hadoop2MR1Planner

cascading.flow.hadoop2.Hadoop2MR1FlowConnector

Class Hadoop2MR1Planner is the core Hadoop MapReduce planner used by default through the {@link cascading.flow.hadoop2.Hadoop2MR1FlowConnector}.

Notes:

Custom JobConf properties
A custom JobConf instance can be passed to this planner by calling {@link #copyJobConf(java.util.Map,org.apache.hadoop.mapred.JobConf)}on a map properties object before constructing a new {@link cascading.flow.hadoop2.Hadoop2MR1FlowConnector}.

A better practice would be to set Hadoop properties directly on the map properties object handed to the FlowConnector. All values in the map will be passed to a new default JobConf instance to be used as defaults for all resulting Flow instances.

For example, {@code properties.set("mapred.child.java.opts","-Xmx512m");} would convince Hadoopto spawn all child jvms with a heap of 512MB.

    }


  @Override
  public FlowConnector getFlowConnector( Map<Object, Object> properties )
    {
    return new Hadoop2MR1FlowConnector( properties );
    }

    }


  @Override
  public FlowConnector getFlowConnector()
    {
    return new Hadoop2MR1FlowConnector( HadoopUtil.createProperties( getPlannerConfig() ) );
    }

Examples of cascading.flow.hadoop2.Hadoop2MR1Planner

Related Classes of cascading.flow.hadoop2.Hadoop2MR1Planner