Examples of cc.mallet.topics.ParallelTopicModel

cc.mallet.topics.ParallelTopicModel
Simple parallel threaded implementation of LDA, following Newman, Asuncion, Smyth and Welling, Distributed Algorithms for Topic Models JMLR (2009), with SparseLDA sampling scheme and data structure from Yao, Mimno and McCallum, Efficient Methods for Topic Model Inference on Streaming Document Collections, KDD (2009). @author David Mimno, Andrew McCallum

        features = FeatureConstraintUtil.selectFeaturesByInfoGain(list,numConstraints.value);
      }
      else if (featureSelection.value.equals("lda")) {
        try {
          ObjectInputStream ois = new ObjectInputStream(new FileInputStream(ldaFile.value));
          ParallelTopicModel lda = (ParallelTopicModel)ois.readObject();
          features = FeatureConstraintUtil.selectTopLDAFeatures(numConstraints.value, lda, list.getDataAlphabet());
        }
        catch (Exception e) {
          e.printStackTrace();
        }

View Full Code Here

        features = FeatureConstraintUtil.selectFeaturesByInfoGain(list,numConstraints.value);
      }
      else if (featureSelection.value.equals("lda")) {
        try {
          ObjectInputStream ois = new ObjectInputStream(new FileInputStream(ldaFile.value));
          ParallelTopicModel lda = (ParallelTopicModel)ois.readObject();
          features = FeatureConstraintUtil.selectTopLDAFeatures(numConstraints.value, lda, list.getDataAlphabet());
        }
        catch (Exception e) {
          e.printStackTrace();
        }

View Full Code Here

        instances.addThruPipe(new DataBagSourceIterator(documents));


        // Create a model with numTopics, alpha_t = 0.01, beta_w = 0.01
        // Note that the first parameter is passed as the sum over topics, while
        // the second is the parameter for a single dimension of the Dirichlet prior.
        ParallelTopicModel model = new ParallelTopicModel(numTopics, 1.0, 0.01);
        model.addInstances(instances);
        model.setNumThreads(1); // Important, since this is being run in the reduce, just use one thread
        model.setTopicDisplay(0,0);
        model.setNumIterations(2000);
        model.estimate();


        // Get the results
        Alphabet dataAlphabet = instances.getDataAlphabet();
        ArrayList<TopicAssignment> assignments = model.getData();


        // Convert the results into comprehensible topics
        for (int topicNum = 0; topicNum < model.getNumTopics(); topicNum++) {
            TreeSet<IDSorter> sortedWords = model.getSortedWords().get(topicNum);
            Iterator<IDSorter> iterator = sortedWords.iterator();


            DataBag topic = BagFactory.getInstance().newDefaultBag();
            
            // Get the set of keywords with weights for this topic and add them as tuples

View Full Code Here

        int numTopics;
        
        public MalletLDA(Corpus corpus, int numTopics, int numIterations, double alpha, double beta) {
                this.corpus = corpus;
                this.numTopics = numTopics;
                LDA = new ParallelTopicModel(numTopics,alpha,beta);
                LDA.setNumIterations(numIterations);
        }

View Full Code Here

        }
        
        public MalletLDA(Corpus corpus, int numTopics, int numIterations) {
            this.corpus = corpus;
            this.numTopics = numTopics;
            LDA = new ParallelTopicModel(numTopics);
            LDA.setNumIterations(numIterations);
    }

View Full Code Here

TOP

Related Classes of cc.mallet.topics.ParallelTopicModel

cc.mallet.classify.tui.Vectors2FeatureConstraints

cc.mallet.topics.TopicAssignment

cc.mallet.util.Randoms

uk.ac.cam.ha293.tweetlabel.topics.MalletLDA

varaha.topic.LDATopics

java.util.Formatter

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.