Examples of com.intel.hadoop.graphbuilder.idnormalize.mapreduce.HashIdMR

Package com.intel.hadoop.graphbuilder.idnormalize.mapreduce

Examples of com.intel.hadoop.graphbuilder.idnormalize.mapreduce.HashIdMR

com.intel.hadoop.graphbuilder.idnormalize.mapreduce.HashIdMR
This MapReduce class maps a list of unique vertex into 2 parts of output: A dictionary from rawId to newId, and a new vertex data file using newId. The domain of newId is consecutive integers from 0 to |V|-1.
Input directory: list of unique vertex data. Output directory:
- $outputdir/vidmap for rawid to newid dictionary.
- $outputdir/vdata for normalized vertex data.

 * 
 * @author Haijie Gu
 */
public class HashIdMapReduceTest {
  public static void main(String[] args) throws IOException {
    HashIdMR driver = new HashIdMR(new BasicGraphParser(), new StringParser(),
        new EmptyParser());
    driver.run(args[0], args[1]);
  }

View Full Code Here

        .getName());
    FieldParser edataparser = ParserFactory.getParserByClassName(edataClass()
        .getName());


    try {
      HashIdMR job1 = new HashIdMR(graphparser, vidparser, vdataparser);
      job1.run(input + "/vdata", output);


      SortDictMR job2 = new SortDictMR(numParts, true, vidparser);
      job2.run(output + "/vidmap", output + "/temp/partitionedvidmap");


      SortEdgeMR job3 = new SortEdgeMR(numParts, graphparser, vidparser,

View Full Code Here

TOP

Related Classes of com.intel.hadoop.graphbuilder.idnormalize.mapreduce.HashIdMR

com.intel.hadoop.graphbuilder.job.AbstractIdNormalizationJob

com.intel.hadoop.graphbuilder.test.idnormalize.HashIdMapReduceTest

org.apache.hadoop.fs.Path

org.apache.hadoop.mapred.JobConf

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.