Examples of org.apache.flink.api.java.record.operators.DeltaIteration

org.apache.flink.api.java.record.operators.DeltaIteration

s.apache.org/jira/browse/FLINK-1106">FLINK-1106 for more details. A DeltaIteration is similar to a {@link BulkIteration}, but maintains state across the individual iteration steps. The state is called the solution set, can be obtained via {@link #getSolutionSet()}, and be accessed by joining (or CoGrouping) with it. The solution set is updated by producing a delta for it, which is merged into the solution set at the end of each iteration step.

The delta iteration must be closed by setting a delta for the solution set ( {@link #setSolutionSetDelta(org.apache.flink.api.common.operators.Operator)}) and the new workset (the data set that will be fed back, {@link #setNextWorkset(org.apache.flink.api.common.operators.Operator)}). The DeltaIteration itself represents the result after the iteration has terminated. Delta iterations terminate when the feed back data set (the workset) is empty. In addition, a maximum number of steps is given as a fall back termination guard.

Elements in the solution set are uniquely identified by a key. When merging the solution set delta, contained elements with the same key are replaced.

This class is a subclass of {@code DualInputOperator}. The solution set is considered the first input, the workset is considered the second input.

    ReduceOperator reducedSource = ReduceOperator.builder(IdentityReduce.class).
        input(source).
        name("Identity reduce source").
        build();


    DeltaIteration iteration = new DeltaIteration(0,"Loop");
    iteration.setMaximumNumberOfIterations(10);
    iteration.setInitialSolutionSet(source);
    iteration.setInitialWorkset(mappedSource);


    JoinOperator nextWorkset = JoinOperator.builder(DummyNonPreservingMatchStub.class, IntValue.class, 0,0).
        input1(iteration.getWorkset()).
        input2(reducedSource).
        name("Next work set").
        build();


    JoinOperator solutionSetDelta = JoinOperator.builder(DummyNonPreservingMatchStub.class, IntValue.class, 0,
        0).
        input1(iteration.getSolutionSet()).
        input2(nextWorkset).
        name("Solution set delta").
        build();


    iteration.setNextWorkset(nextWorkset);
    iteration.setSolutionSetDelta(solutionSetDelta);


    FileDataSink sink = new FileDataSink(DummyOutputFormat.class, OUT_FILE, iteration, "Iteration sink");
    List<FileDataSink> sinks = new ArrayList<FileDataSink>();
    sinks.add(sink);

Examples of org.apache.flink.api.java.record.operators.DeltaIteration

Related Classes of org.apache.flink.api.java.record.operators.DeltaIteration