Examples of org.apache.flink.test.recordJobs.kmeans.udfs.PointOutFormat

Package org.apache.flink.test.recordJobs.kmeans.udfs

Examples of org.apache.flink.test.recordJobs.kmeans.udfs.PointOutFormat

org.apache.flink.test.recordJobs.kmeans.udfs.PointOutFormat
Writes records that contain an id and a CoordVector. The output format is line-based, i.e. one record is written to a line and terminated with '\n'. Within a line the first '|' character separates the id from the CoordVector. The vector consists of a vector of decimals. The decimals are separated by '|'. The is is the id of a data point or cluster center and the vector the corresponding position (coordinate vector) of the data point or cluster center. Example line: "42|23.23|52.57|74.43| Id: 42 Coordinate vector: (23.23, 52.57, 74.43)

    
    IterativeDataSet<Record> iteration = initialInput.iterate(2);
    
    DataSet<Record> result = iteration.union(iteration).map(new IdentityMapper());
    
    iteration.closeWith(result).write(new PointOutFormat(), this.resultPath);
    
    env.execute();
  }

View Full Code Here

        .name("Reduce something").build();


    MapOperator dummyMap = MapOperator.builder(new IdentityMapper()).input(dummyReduce).build();
    iteration.setNextPartialSolution(dummyMap);


    FileDataSink finalResult = new FileDataSink(new PointOutFormat(), output, iteration, "Output");


    Plan plan = new Plan(finalResult, "Iteration with chained map test");
    plan.setDefaultParallelism(numSubTasks);
    return plan;
  }

View Full Code Here

        .input(computeFinalDistance)
        .name("Find Nearest Final Centers")
        .build();


    // create DataSinkContract for writing the new cluster positions
    FileDataSink finalClusters = new FileDataSink(new PointOutFormat(), output+"/centers", iteration, "Cluster Positions");


    // write assigned clusters
    FileDataSink clusterAssignments = new FileDataSink(new PointOutFormat(), output+"/points", findNearestFinalCluster, "Cluster Assignments");
    
    List<FileDataSink> sinks = new ArrayList<FileDataSink>();
    sinks.add(finalClusters);
    sinks.add(clusterAssignments);

View Full Code Here

TOP

Related Classes of org.apache.flink.test.recordJobs.kmeans.udfs.PointOutFormat

org.apache.flink.test.iterative.IterationWithChainingITCase

org.apache.flink.test.iterative.IterationWithUnionITCase

org.apache.flink.test.recordJobs.kmeans.KMeansCross

org.apache.flink.types.IntValue

java.text.DecimalFormatSymbols

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.