Examples of edu.ucla.sspace.ri.HadoopRandomIndexing

• edu.ucla.sspace.ri.HadoopRandomIndexing
  An implementation of Random Indexing that uses Hadoop to perform a distributed co-occurrence counting. This class is designed to only be used on systems where a Hadoop system is currently installed and running. Furthermore, this class depends on all external resources and corpora being present in the Hadoop distributed file system.

  This class supports the following properties.

  Property: {@value #WINDOW_SIZE_PROPERTY}
  Default: {@value #DEFAULT_WINDOW_SIZE}

  This property sets the number of words before and after that are counted as co-occurring. With the default value, {@value #DEFAULT_WINDOW_SIZE} words are counted before and {@value #DEFAULT_WINDOW_SIZE} words are counter after. This class always uses asymmetric window.

  Property: {@value #VECTOR_LENGTH_PROPERTY}
  Default: {@value #DEFAULT_VECTOR_LENGTH}

  This property sets the number of dimensions to be used for the index and semantic vectors.

  Property: {@value #USE_PERMUTATIONS_PROPERTY}
  Default: {@code false}

  This property specifies whether to enable permuting the index vectors of co-occurring words. Enabling this option will cause the word semantics to include word-ordering information. However this option is best used with a larger corpus.

  Property: {@value #PERMUTATION_FUNCTION_PROPERTY}
  Default: {@link edu.ucla.sspace.index.DefaultPermutationFunction DefaultPermutationFunction}

  This property specifies the fully qualified class name of a {@link PermutationFunction} instance that will be usedto permute index vectors. If the {@value #USE_PERMUTATIONS_PROPERTY} isset to {@code false}, the value of this property has no effect.

  Property: {@value #USE_SPARSE_SEMANTICS_PROPERTY}
  Default: {@code true}

  This property specifies whether to use a sparse encoding for each word's semantics. Using a sparse encoding can result in a large saving in memory, while requiring more time to process each document.

  @see RandomIndexing @see WordCooccurrenceCountingJob

Related Classes of edu.ucla.sspace.ri.HadoopRandomIndexing