Examples of edu.ucla.sspace.matrix.LogEntropyTransform

edu.ucla.sspace.matrix.LogEntropyTransform
Transforms a matrix using log-entropy weighting. The input matrix is assumed to be formatted as rows representing terms and columns representing documents. Each matrix cell indicates the number of times the row's word occurs within the column's document. See the following papers for details and analysis:
- Landauer, T. K., Foltz, P. W., & Laham, D. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259-284.
- S. Dumais, “Enhancing performance in latent semantic indexing (LSI) retrieval,” Bellcore, Morristown (now Telcordia Technologies), Tech. Rep. TM-ARH-017527, 1990.
- P. Nakov, A. Popova, and P. Mateev, “Weight functions impact on LSA performance,” in Proceedings of the EuroConference Recent Advances in Natural Language Processing, (RANLP’01), 2001, pp. 187–193.
@author David Jurgens

    }
    
    protected SemanticSpace getSpace() {
        try {
            int dimensions = argOptions.getIntOption("dimensions", 300);
            Transform transform = new LogEntropyTransform();
            if (argOptions.hasOption("preprocess"))
                transform = ReflectionUtil.getObjectInstance(
                        argOptions.getStringOption("preprocess"));
            String algName = argOptions.getStringOption("svdAlgorithm", "ANY");
            SingularValueDecomposition factorization = SVD.getFactorization(

View Full Code Here

    }
    
    protected SemanticSpace getSpace() {
        try {
            int dimensions = argOptions.getIntOption("dimensions", 300);
            Transform transform = new LogEntropyTransform();
            if (argOptions.hasOption("preprocess"))
                transform = ReflectionUtil.getObjectInstance(
                        argOptions.getStringOption("preprocess"));
            String algName = argOptions.getStringOption("svdAlgorithm", "ANY");
            MatrixFactorization factorization =

View Full Code Here

     * {@Link LatentSemanticAnalysis} with the default parameters set in the
     * original paper.  This construct initializes this instance such that the
     * document space is <i>not</i> retained.
     */
    public LatentSemanticAnalysis() throws IOException {
        this(false, 300, new LogEntropyTransform(),
             SVD.getFastestAvailableFactorization(), 
             false, new StringBasisMapping());
    }

View Full Code Here

     * document space is <i>not</i> retained.
     * 
     * @param dimensions The number of dimensions to retain in the reduced space
     */
    public LatentSemanticAnalysis(int numDimensions) throws IOException {
        this(false, numDimensions, new LogEntropyTransform(),
             SVD.getFastestAvailableFactorization(), 
             false, new StringBasisMapping());
    }

View Full Code Here

     * @param dimensions The number of dimensions to retain in the reduced space
     */
    public LatentSemanticAnalysis(int numDimensions,
                                  SingularValueDecomposition svdMethod)
            throws IOException {
        this(false, numDimensions, new LogEntropyTransform(),
             svdMethod, 
             false, new StringBasisMapping());
    }

View Full Code Here

     *        accessible
     */
    public LatentSemanticAnalysis(int numDimensions, 
                                  boolean retainDocumentSpace) 
            throws IOException {
        this(retainDocumentSpace, numDimensions, new LogEntropyTransform(),
             SVD.getFastestAvailableFactorization(), 
             false, new StringBasisMapping());
    }

View Full Code Here

    public void processSpace(Properties properties) {
        try {
            // first ensure that we are no longer writing to the matrix
            termDocumentMatrixBuilder.finish();


            Transform transform = new LogEntropyTransform();


            String transformClass =
            properties.getProperty(MATRIX_TRANSFORM_PROPERTY);
            if (transformClass != null) {
                try {
                    Class clazz = Class.forName(transformClass);
                    transform = (Transform)(clazz.newInstance());
                }
                // perform a general catch here due to the number of possible
                // things that could go wrong.  Rethrow all exceptions as an
                // error.
                catch (Exception e) {
                    throw new Error(e);
                }
            }


            LSA_LOGGER.info("performing " + transform + " transform");


                // Get the finished matrix file from the builder
            File termDocumentMatrix = termDocumentMatrixBuilder.getFile();
            if (LSA_LOGGER.isLoggable(Level.FINE)) {
                LSA_LOGGER.fine("stored term-document matrix in format " +
                        termDocumentMatrixBuilder.getMatrixFormat()
                        + " at " + termDocumentMatrix.getAbsolutePath());
            }


            // Convert the raw term counts using the specified transform
            File transformedMatrix = transform.transform(termDocumentMatrix,
                    termDocumentMatrixBuilder.getMatrixFormat());


            if (LSA_LOGGER.isLoggable(Level.FINE)) {
                LSA_LOGGER.fine("transformed matrix to " +
                        transformedMatrix.getAbsolutePath());

View Full Code Here

TOP

Related Classes of edu.ucla.sspace.matrix.LogEntropyTransform

edu.ucla.sspace.lsa.LatentSemanticAnalysis

edu.ucla.sspace.mains.LSAMain

edu.ucla.sspace.mains.PLSAMain

edu.ucla.sspace.vector.SparseDoubleVector

java.io.IOError

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.