Examples of org.apache.mahout.math.hadoop.similarity.RowSimilarityJob

Package org.apache.mahout.math.hadoop.similarity

Examples of org.apache.mahout.math.hadoop.similarity.RowSimilarityJob

org.apache.mahout.math.hadoop.similarity.RowSimilarityJob
miacs.umd.edu/~jimmylin/publications/Elsayed_etal_ACL2008_short.pdf)

Command line arguments specific to this class are:
1. -Dmapred.input.dir=(path): Directory containing a {@link DistributedRowMatrix} as aSequenceFile
2. -Dmapred.output.dir=(path): output path where the computations output should go (a {@link DistributedRowMatrix}stored as a SequenceFile)
3. --numberOfColumns: the number of columns in the input matrix
4. --similarityClassname (classname): an implementation of {@link DistributedVectorSimilarity} used to compute thesimilarity
5. --maxSimilaritiesPerRow (integer): cap the number of similar rows per row to this number (100)
General command line options are documented in {@link AbstractJob}.

Please consider supplying a --tempDir parameter for this job, as is needs to write some intermediate files

Note that because of how Hadoop parses arguments, all "-D" arguments must appear before all other arguments.


    int numberOfUsers = TasteHadoopUtils.readIntFromFile(getConf(), countUsersPath);


    /* Once DistributedRowMatrix uses the hadoop 0.20 API, we should refactor this call to something like
     * new DistributedRowMatrix(...).rowSimilarity(...) */
    ToolRunner.run(getConf(), new RowSimilarityJob(), new String[] {
      "-Dmapred.input.dir=" + itemUserMatrixPath.toString(),
      "-Dmapred.output.dir=" + similarityMatrixPath.toString(),
      "--numberOfColumns", String.valueOf(numberOfUsers),
      "--similarityClassname", similarityClassName,
      "--maxSimilaritiesPerRow", String.valueOf(maxSimilarItemsPerItem + 1),

View Full Code Here


    if (shouldRunNextPhase(parsedArgs, currentPhase)) {
      /* Once DistributedRowMatrix uses the hadoop 0.20 API, we should refactor this call to something like
       * new DistributedRowMatrix(...).rowSimilarity(...) */
      try {
        ToolRunner.run(getConf(), new RowSimilarityJob(), new String[] {
          "-Dmapred.input.dir=" + itemUserMatrixPath.toString(),
          "-Dmapred.output.dir=" + similarityMatrixPath.toString(),
          "--numberOfColumns", String.valueOf(numberOfUsers),
          "--similarityClassname", similarityClassname,
          "--maxSimilaritiesPerRow", String.valueOf(maxSimilaritiesPerItem + 1),

View Full Code Here


    int numberOfUsers = TasteHadoopUtils.readIntFromFile(getConf(), countUsersPath);


    /* Once DistributedRowMatrix uses the hadoop 0.20 API, we should refactor this call to something like
     * new DistributedRowMatrix(...).rowSimilarity(...) */
    ToolRunner.run(getConf(), new RowSimilarityJob(), new String[] {
      "-Dmapred.input.dir=" + itemUserMatrixPath,
      "-Dmapred.output.dir=" + similarityMatrixPath,
      "--numberOfColumns", String.valueOf(numberOfUsers),
      "--similarityClassname", similarityClassName,
      "--maxSimilaritiesPerRow", String.valueOf(maxSimilarItemsPerItem + 1),

View Full Code Here


    if (shouldRunNextPhase(parsedArgs, currentPhase)) {
      /* Once DistributedRowMatrix uses the hadoop 0.20 API, we should refactor this call to something like
       * new DistributedRowMatrix(...).rowSimilarity(...) */
      try {
        ToolRunner.run(getConf(), new RowSimilarityJob(), new String[] {
          "-Dmapred.input.dir=" + itemUserMatrixPath,
          "-Dmapred.output.dir=" + similarityMatrixPath,
          "--numberOfColumns", String.valueOf(numberOfUsers),
          "--similarityClassname", similarityClassname,
          "--maxSimilaritiesPerRow", String.valueOf(maxSimilaritiesPerItem + 1),

View Full Code Here

TOP

Related Classes of org.apache.mahout.math.hadoop.similarity.RowSimilarityJob

org.apache.hadoop.conf.Configuration

org.apache.hadoop.fs.Path

org.apache.hadoop.io.IntWritable

org.apache.hadoop.mapreduce.Job

org.apache.mahout.cf.taste.hadoop.item.RecommenderJob

org.apache.mahout.cf.taste.hadoop.similarity.item.ItemSimilarityJob

org.apache.mahout.math.hadoop.DistributedRowMatrix.MatrixEntryWritable

org.apache.mahout.math.RandomAccessSparseVector

org.apache.mahout.math.SequentialAccessSparseVector

org.apache.mahout.math.VarIntWritable

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.