Package com.tdunning.plume.local.lazy.op

Examples of com.tdunning.plume.local.lazy.op.MultipleParallelDo

76
77
78
79
80
81
82
83
84
85
86
87
88
          pDo.getFunction().process(obj, emitter);
        }
      // MultipleParallelDo -> parallel operations that read the same collection
      // In this version of executor, we will only compute the current collection, not its neighbors
      } else if(op instanceof MultipleParallelDo) {
        MultipleParallelDo mPDo = (MultipleParallelDo) op;
        parent = execute((LazyCollection)mPDo.getOrigin());
        DoFn function = (DoFn)mPDo.getDests().get(output); // get the function that corresponds to this collection
        for (Object obj : parent) {
          function.process(obj, emitter);
        }
      // GroupByKey
      } else if(op instanceof GroupByKey) {
View Full Code Here
87
88
89
90
91
92
93
94
95
96
97
98
      toProcess = Pair.create(key, value);
    }

    for(DeferredOp op: l.getDownOps()) {
      if(op instanceof MultipleParallelDo) {
        MultipleParallelDo mPDo = ((MultipleParallelDo)op);
        for(Object entry: mPDo.getDests().entrySet()) {
          Map.Entry<PCollection, DoFn> en = (Map.Entry<PCollection, DoFn>)entry;
          LazyCollection<?> lCol = (LazyCollection<?>)en.getKey();
          DeferredOp childOp = null;
          if(lCol.getDownOps() != null && lCol.getDownOps().size() > 0) {
            childOp = lCol.getDownOps().get(0);
View Full Code Here
TOP

Related Classes of com.tdunning.plume.local.lazy.op.MultipleParallelDo

  • com.tdunning.plume.local.lazy.LocalExecutor
  • com.tdunning.plume.local.lazy.MSCRMapper
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.