List of fromCollection() Examples

Examples of fromCollection()

eu.stratosphere.api.java.ExecutionEnvironment.fromCollection()
Creates a DataSet from the given non-empty collection. The type of the data set is that of the elements in the collection. The elements need to be serializable (as defined by {@link java.io.Serializable}), because the framework may move the elements into the cluster if needed.
The framework will try and determine the exact type from the collection elements. In case of generic elements, it may be necessary to manually supply the type information via {@link #fromCollection(Collection,TypeInformation)}.
Note that this operation will result in a non-parallel data source, i.e. a data source with a degree of parallelism of one. @param data The collection of elements to create the data set from. @return A DataSet representing the given collection. @see #fromCollection(Collection,TypeInformation)
org.apache.flink.api.java.ExecutionEnvironment.fromCollection()
Creates a DataSet from the given non-empty collection. The type of the data set is that of the elements in the collection. The elements need to be serializable (as defined by {@link java.io.Serializable}), because the framework may move the elements into the cluster if needed.
The framework will try and determine the exact type from the collection elements. In case of generic elements, it may be necessary to manually supply the type information via {@link #fromCollection(Collection,TypeInformation)}.
Note that this operation will result in a non-parallel data source, i.e. a data source with a degree of parallelism of one. @param data The collection of elements to create the data set from. @return A DataSet representing the given collection. @see #fromCollection(Collection,TypeInformation)
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.fromCollection()
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.fromCollection()
Creates a DataStream from the given non-empty collection. The type of the DataStream is that of the elements in the collection. The elements need to be serializable (as defined by java.io.Serializable), because the framework may move the elements into the cluster if needed. @param data The collection of elements to create the DataStream from. @param < OUT> type of the returned stream @return The DataStream representing the elements.

Examples of eu.stratosphere.api.java.ExecutionEnvironment.fromCollection()

  
  @Test
  public void testFieldsAggregate() {
    
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple5<Integer, Long, String, Long, Integer>> tupleDs = env.fromCollection(emptyTupleData, tupleTypeInfo);


    // should work
    try {
      tupleDs.aggregate(Aggregations.SUM, 1);
    } catch(Exception e) {

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

    public static String runProgram(String resultPath) throws Exception {
      
      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);
      
      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);
      int keyPosition = 0;
      
      DeltaIteration<Tuple2<Long, Long>, Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterateDelta(initialSolutionSet, MAX_ITERATIONS, keyPosition);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

      
      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);
      
      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);
      int keyPosition = 0;
      
      DeltaIteration<Tuple2<Long, Long>, Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterateDelta(initialSolutionSet, MAX_ITERATIONS, keyPosition);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

    public static String runProgram(String resultPath) throws Exception {


      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);


      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);


      IterativeDataSet<Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterate(MAX_ITERATIONS);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()


      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);


      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);


      IterativeDataSet<Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterate(MAX_ITERATIONS);


      // register the aggregator

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

    public static String runProgram(String resultPath) throws Exception {


      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);


      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);


      IterativeDataSet<Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterate(MAX_ITERATIONS);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()


      final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(DOP);


      DataSet<Tuple2<Long, Long>> initialSolutionSet = env.fromCollection(verticesInput);
      DataSet<Tuple2<Long, Long>> edges = env.fromCollection(edgesInput);


      IterativeDataSet<Tuple2<Long, Long>> iteration =
          initialSolutionSet.iterate(MAX_ITERATIONS);


      // register the convergence criterion

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()


  @Override
  protected void testProgram() throws Exception {
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    
    DataSet<Tuple2<String, String>> data = env.fromCollection(input);
    
    data.mapPartition(new TestMapPartition()).output(new LocalCollectionOutputFormat<Tuple2<String,Integer>>(result));
    
    env.execute();
  }

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

  
  @Test
  public void testFieldsAggregate() {
    
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple5<Integer, Long, String, Long, Integer>> tupleDs = env.fromCollection(emptyTupleData, tupleTypeInfo);


    // should work
    try {
      tupleDs.aggregate(Aggregations.SUM, 1);
    } catch(Exception e) {

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromCollection()

    } catch(Exception e) {
      Assert.fail();
    }
    
    // should not work: not applied to tuple dataset
    DataSet<Long> longDs = env.fromCollection(emptyLongData, BasicTypeInfo.LONG_TYPE_INFO);
    try {
      longDs.aggregate(Aggregations.MIN, 1);
      Assert.fail();
    } catch(InvalidProgramException uoe) {
      // we're good here

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.