List of fromElements() Examples

Examples of fromElements()

eu.stratosphere.api.java.ExecutionEnvironment.fromElements()
Creates a new data set that contains the given elements. The elements must all be of the same type, for example, all of the {@link String} or {@link Integer}. The sequence of elements must not be empty. Furthermore, the elements must be serializable (as defined in {@link java.io.Serializable}, because the execution environment may ship the elements into the cluster.
The framework will try and determine the exact type from the collection elements. In case of generic elements, it may be necessary to manually supply the type information via {@link #fromCollection(Collection,TypeInformation)}.
Note that this operation will result in a non-parallel data source, i.e. a data source with a degree of parallelism of one. @param data The elements to make up the data set. @return A DataSet representing the given list of elements.
org.apache.flink.api.java.CollectionEnvironment.fromElements()
org.apache.flink.api.java.ExecutionEnvironment.fromElements()
Creates a new data set that contains the given elements. The elements must all be of the same type, for example, all of the {@link String} or {@link Integer}. The sequence of elements must not be empty. Furthermore, the elements must be serializable (as defined in {@link java.io.Serializable}, because the execution environment may ship the elements into the cluster.
The framework will try and determine the exact type from the collection elements. In case of generic elements, it may be necessary to manually supply the type information via {@link #fromCollection(Collection,TypeInformation)}.
Note that this operation will result in a non-parallel data source, i.e. a data source with a degree of parallelism of one. @param data The elements to make up the data set. @return A DataSet representing the given list of elements.
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.fromElements()
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.fromElements()
Creates a new DataStream that contains the given elements. The elements must all be of the same type, for example, all of the String or Integer. The sequence of elements must not be empty. Furthermore, the elements must be serializable (as defined in java.io.Serializable), because the execution environment may ship the elements into the cluster. @param data The collection of elements to create the DataStream from. @param < OUT> type of the returned stream @return The DataStream representing the elements.

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

      
      // ------------ construct the test program ------------------
      {
        
        @SuppressWarnings("unchecked")
        DataSet<Tuple2<String, Double>> initialVertices = env.fromElements(new Tuple2<String, Double>("abc", 3.44));
  
        @SuppressWarnings("unchecked")
        DataSet<Tuple2<String, String>> edges = env.fromElements(new Tuple2<String, String>("a", "c"));

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

        
        @SuppressWarnings("unchecked")
        DataSet<Tuple2<String, Double>> initialVertices = env.fromElements(new Tuple2<String, Double>("abc", 3.44));
  
        @SuppressWarnings("unchecked")
        DataSet<Tuple2<String, String>> edges = env.fromElements(new Tuple2<String, String>("a", "c"));
        
        
        VertexCentricIteration<String, Double, Long, ?> vertexIteration = 
            VertexCentricIteration.withPlainEdges(edges, new UpdateFunction(), new MessageFunctionNoEdgeValue(), NUM_ITERATIONS);
        vertexIteration.addBroadcastSetForMessagingFunction(BC_SET_MESSAGES_NAME, bcVar);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()


  @Override
  protected void testProgram() throws Exception {
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();


    DataSet<String> text = env.fromElements(WordCountData.TEXT);
    DataSet<Tuple2<String, Integer>> words = text.flatMap(new WordCount.Tokenizer());
    DataSet<Tuple2<String, Integer>> result = words.groupBy(0).aggregate(Aggregations.SUM, 1);


    result.output(new LocalCollectionOutputFormat<Tuple2<String, Integer>>(resultsCollected));
    env.execute("Word Count Collection");

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

      
    // set up execution environment
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    
    // read vertex and edge data
    DataSet<Long> vertices = env.fromElements(ConnectedComponentsData.getEnumeratingVertices(NUM_VERTICES).split("\n"))
        .map(new VertexParser());
    
    DataSet<Tuple2<Long, Long>> edges = env.fromElements(ConnectedComponentsData.getRandomOddEvenEdges(NUM_EDGES, NUM_VERTICES, SEED).split("\n"))
        .flatMap(new EdgeParser());

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

    
    // read vertex and edge data
    DataSet<Long> vertices = env.fromElements(ConnectedComponentsData.getEnumeratingVertices(NUM_VERTICES).split("\n"))
        .map(new VertexParser());
    
    DataSet<Tuple2<Long, Long>> edges = env.fromElements(ConnectedComponentsData.getRandomOddEvenEdges(NUM_EDGES, NUM_VERTICES, SEED).split("\n"))
        .flatMap(new EdgeParser());
    
    // assign the initial components (equal to the vertex id)
    DataSet<Tuple2<Long, Long>> verticesWithInitialId = vertices.map(new DuplicateValue<Long>());

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

  @Test
  public void testNoBreakerForIndependentVariable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      
      DataSet<String> source1 = env.fromElements("test");
      DataSet<String> source2 = env.fromElements("test");
      
      source1.map(new IdentityMapper<String>()).withBroadcastSet(source2, "some name").print();
      
      Plan p = env.createProgramPlan();

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

  public void testNoBreakerForIndependentVariable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      
      DataSet<String> source1 = env.fromElements("test");
      DataSet<String> source2 = env.fromElements("test");
      
      source1.map(new IdentityMapper<String>()).withBroadcastSet(source2, "some name").print();
      
      Plan p = env.createProgramPlan();
      OptimizedPlan op = compileNoStats(p);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

   @Test
  public void testBreakerForDependentVariable() {
      try {
        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        
        DataSet<String> source1 = env.fromElements("test");
        
        source1.map(new IdentityMapper<String>()).map(new IdentityMapper<String>()).withBroadcastSet(source1, "some name").print();
        
        Plan p = env.createProgramPlan();
        OptimizedPlan op = compileNoStats(p);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

  public void testPartitioningNotPushedDown() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      
      @SuppressWarnings("unchecked")
      DataSet<Tuple3<Long, Long, Long>> input = env.fromElements(new Tuple3<Long, Long, Long>(0L, 0L, 0L));
      
      input
        .groupBy(0, 1).sum(2)
        .groupBy(0).sum(1)
        .print();

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.fromElements()

  public void testPartitioningReused() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      
      @SuppressWarnings("unchecked")
      DataSet<Tuple3<Long, Long, Long>> input = env.fromElements(new Tuple3<Long, Long, Long>(0L, 0L, 0L));
      
      input
        .groupBy(0).sum(1)
        .groupBy(0, 1).sum(2)
        .print();

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.