Examples of FileDataSink

eu.stratosphere.api.java.record.operators.FileDataSink
Operator for nodes which act as data sinks, storing the data they receive in a file instead of sending it to another contract. The encoding of the data in the file is handled by the {@link FileOutputFormat}. @see FileOutputFormat
org.apache.flink.api.java.record.operators.FileDataSink
s.apache.org/jira/browse/FLINK-1106">FLINK-1106 for more details. Operator for nodes which act as data sinks, storing the data they receive in a file instead of sending it to another contract. The encoding of the data in the file is handled by the {@link FileOutputFormat}. @see FileOutputFormat

Examples of eu.stratosphere.api.java.record.operators.FileDataSink

    MapOperator mp1 = MapOperator.builder(Mp1.class).input(jn1).build();


    // Mp2 filters out all p values which can be divided by z
    MapOperator mp2 = MapOperator.builder(Mp2.class).setBroadcastVariable("z", mp1).input(jn2).build();


    FileDataSink output = new FileDataSink(new ContractITCaseOutputFormat(), resultPath);
    output.setDegreeOfParallelism(1);
    output.setInput(mp2);


    return new Plan(output);
  }

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

    
    ReduceOperator globalRed = ReduceOperator.builder(new IdentityReduce(), IntValue.class, 0).build();
    globalRed.addInput(redA);
    globalRed.addInput(redB);
    
    FileDataSink sink = new FileDataSink(new DummyOutputFormat(), OUT_FILE, globalRed);
    
    // return the plan
    Plan plan = new Plan(sink, "Union Property Propagation");
    
    OptimizedPlan oPlan = compileNoStats(plan);

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

        .input1(buildTriads).input2(projectOutCounts)
        .name("Close Triads").build();
    closeTriads.setParameter("INPUT_SHIP_STRATEGY", "SHIP_REPARTITION_HASH");
    closeTriads.setParameter("LOCAL_STRATEGY", "LOCAL_STRATEGY_HASH_BUILD_SECOND");


    FileDataSink triangles = new FileDataSink(new TriangleOutputFormat(), output, closeTriads, "Triangles");


    Plan p = new Plan(triangles, "Enumerate Triangles");
    p.setDefaultParallelism(numSubTasks);
    return p;
  }

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink


    /* Connect aggregate: */
    sumAmountAggregate.setInput(partListJoin);


    /* Connect sink: */
    FileDataSink result = new FileDataSink(new StringIntPairStringDataOutFormat(), this.outputPath, "Results sink");
    result.setInput(sumAmountAggregate);


    Plan p = new Plan(result, "TPC-H query 9");
    p.setDefaultParallelism(this.degreeOfParallelism);
    return p;
  }

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

      .build();
    closeTriads.setParameter("INPUT_LEFT_SHIP_STRATEGY", "SHIP_REPARTITION_HASH");
    closeTriads.setParameter("INPUT_RIGHT_SHIP_STRATEGY", "SHIP_REPARTITION_HASH");
    closeTriads.setParameter("LOCAL_STRATEGY", "LOCAL_STRATEGY_HASH_BUILD_SECOND");


    FileDataSink triangles = new FileDataSink(new CsvOutputFormat(), output, "Output");
    CsvOutputFormat.configureRecordFormat(triangles)
      .recordDelimiter('\n')
      .fieldDelimiter(' ')
      .field(StringValue.class, 0)
      .field(StringValue.class, 1)
      .field(StringValue.class, 2);


    triangles.setInput(closeTriads);
    closeTriads.setSecondInput(edges);
    closeTriads.setFirstInput(buildTriads);
    buildTriads.setInput(edges);


    Plan plan = new Plan(triangles, "Enumerate Triangles");

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

      .keyField(StringValue.class, 1, 1)
      .name("Find Shortest Paths")
      .build();
    findShortestPaths.setDegreeOfParallelism(numSubTasks);


    FileDataSink result = new FileDataSink(new PathOutFormat(),output, "New Paths");
    result.setDegreeOfParallelism(numSubTasks);


    result.setInput(findShortestPaths);
    findShortestPaths.setFirstInput(pathsInput);
    findShortestPaths.setSecondInput(concatPaths);
    concatPaths.setFirstInput(pathsInput);
    concatPaths.setSecondInput(pathsInput);

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

    MapOperator mapper = MapOperator.builder(new TokenizeLine())
      .input(source)
      .name("Tokenize Lines")
      .build();


    FileDataSink out = new FileDataSink(new CsvOutputFormat(), output, mapper, "Selection");
    CsvOutputFormat.configureRecordFormat(out)
      .recordDelimiter('\n')
      .fieldDelimiter(' ')
      .field(StringValue.class, 0)
      .field(IntValue.class, 1);

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

      .input2(aggInput2)
      .name("JoinLiO")
      .build();


    // create DataSinkContract for writing the result
    FileDataSink result = new FileDataSink(new CsvOutputFormat(), output, joinLiO, "Output");
    CsvOutputFormat.configureRecordFormat(result)
      .recordDelimiter('\n')
      .fieldDelimiter('|')
      .lenient(true)
      .field(IntValue.class, 0)

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink


    iteration.setNextWorkset(updateRanks);
    iteration.setSolutionSetDelta(oldRankComparison);
    
    // create DataSinkContract for writing the final ranks
    FileDataSink result = new FileDataSink(CsvOutputFormat.class, output, iteration, "Final Ranks");
    CsvOutputFormat.configureRecordFormat(result)
      .recordDelimiter('\n')
      .fieldDelimiter(' ')
      .field(LongValue.class, 0)
      .field(DoubleValue.class, 1);

View Full Code Here

Examples of org.apache.flink.api.java.record.operators.FileDataSink

      .input(joinLiO, partJoin2, partJoin1)
      .name("AggLio")
      .build();


    // create DataSinkContract for writing the result
    FileDataSink result = new FileDataSink(new CsvOutputFormat(), output, aggLiO, "Output");
    CsvOutputFormat.configureRecordFormat(result)
      .recordDelimiter('\n')
      .fieldDelimiter('|')
      .lenient(true)
      .field(LongValue.class, 0)

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.