Package org.apache.crunch.impl.spark.fn

Examples of org.apache.crunch.impl.spark.fn.FlatMapIndexFn

53
54
55
56
57
58
59
60
  private JavaRDDLike<?, ?> getJavaRDDLikeInternal(SparkRuntime runtime) {
    JavaRDDLike<?, ?> parentRDD = ((SparkCollection) getOnlyParent()).getJavaRDDLike(runtime);
    fn.configure(runtime.getConfiguration());
    return parentRDD.mapPartitionsWithIndex(
        new FlatMapIndexFn(fn, parentRDD instanceof JavaPairRDD, runtime.getRuntimeContext()),
        false);
  }
View Full Code Here
68
69
70
71
72
73
74
75
76
    }
    JavaRDDLike<?, ?> parentRDD = ((SparkCollection) getOnlyParent()).getJavaRDDLike(runtime);
    fn.configure(runtime.getConfiguration());
    return parentRDD
        .mapPartitionsWithIndex(
            new FlatMapIndexFn(fn, parentRDD instanceof JavaPairRDD, runtime.getRuntimeContext()),
            false)
        .mapPartitionsToPair(new CrunchPairTuple2());
  }
View Full Code Here
TOP

Related Classes of org.apache.crunch.impl.spark.fn.FlatMapIndexFn

  • org.apache.crunch.impl.spark.collect.DoCollection
  • org.apache.crunch.impl.spark.collect.DoTable
  • java.util.Iterator
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.