Package org.apache.crunch.impl.spark.fn

Examples of org.apache.crunch.impl.spark.fn.FlatMapDoFn


  private JavaRDDLike<?, ?> getJavaRDDLikeInternal(SparkRuntime runtime) {
    JavaRDDLike<?, ?> parentRDD = ((SparkCollection) getOnlyParent()).getJavaRDDLike(runtime);
    fn.configure(runtime.getConfiguration());
    if (parentRDD instanceof JavaRDD) {
      return ((JavaRDD) parentRDD).mapPartitions(new FlatMapDoFn(fn, runtime.getRuntimeContext()));
    } else {
      return ((JavaPairRDD) parentRDD).mapPartitions(new FlatMapPairDoFn(fn, runtime.getRuntimeContext()));
    }
  }
View Full Code Here

TOP

Related Classes of org.apache.crunch.impl.spark.fn.FlatMapDoFn

Copyright © 2018 www.massapicom. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.