Examples of com.flaptor.hounder.classifier.bayes.BayesCalculator

Package com.flaptor.hounder.classifier.bayes

Examples of com.flaptor.hounder.classifier.bayes.BayesCalculator

com.flaptor.hounder.classifier.bayes.BayesCalculator
This class implements the bayes classifier formulae. There are two differents stages of the filter. Traning:
Use multiple calls to addData() to feed the classfier with document tokens, explicitly specifying if the document belongs to the category or not.
Call computeProbabilities() to calculate the probability of each token to belong to the category or not, and to persist the new data to disk, making it available to start classifying new documents.
@todo load some (now hardcoded parameters) from the configuration.TODO: NOT THREAD SAFE: WHoHas is static @author Flaptor Development Team

     * @param catName
     * @throws IOException 
     */
    public void calculate(String catName) throws IOException{
        loadIncludedNotIncludedUrls(catName);
        BayesCalculator calculator= new BayesCalculator(config.getBaseDir(), catName, cfg, maxTuple);        
        for (String url: includedUrlsList){
            String item=cache.getItem(url);
            if (null==item){
                LOGGER.warn("Page " + url + "is in included for " + catName + " but not in cache");
                continue;
            }
            calculator.addData(DocumentParser.parse(item, maxTuple),true, url);
        }


        for (String url: notIncludedUrlsList){
            String item=cache.getItem(url);
            if (null==item){
                LOGGER.warn("Page " + url + "is in notIncluded for " + catName + " but not in cache");                
                continue;
            }            
            calculator.addData(DocumentParser.parse(item, maxTuple),false, url);
        }
        calculator.computeProbabilities(); //and save the .probabilities to disk
    }

View Full Code Here

        }
        calculator.computeProbabilities(); //and save the .probabilities to disk
    }


    public void setMyProbs(String newProbs, String catName) throws IOException{
        BayesCalculator calculator= new BayesCalculator(config.getBaseDir(), catName, cfg, maxTuple);
        String[] tokVals= newProbs.split("\\s+;\\s+");
        String key=null;
        Double val=null;
        for (String tv: tokVals){
            String []s= tv.split("=");
            key= s[0];
            val= Double.valueOf(s[1]);
            LOGGER.warn("Writing " + key + " = " + val);
            calculator.updateMyProbabilities(key, val, false); //false ~ noflush            
        }
        LOGGER.warn("Writing " + key + " = " + val);
        calculator.updateMyProbabilities(key, val, true); // true ~ flush        
    }

View Full Code Here

        LOGGER.warn("Writing " + key + " = " + val);
        calculator.updateMyProbabilities(key, val, true); // true ~ flush        
    }


    public Map<String,Double> getMyProbabilities(String catName) throws IOException{
        BayesCalculator calculator= new BayesCalculator(config.getBaseDir(), catName, cfg, maxTuple);
        return calculator.getMyProbabilities();
    }

View Full Code Here

TOP

Related Classes of com.flaptor.hounder.classifier.bayes.BayesCalculator

com.flaptor.hounder.classifier.CacheCalculatorBean

com.flaptor.hounder.classifier.util.TokenCounterPersistence

com.flaptor.util.sort.RecordReader

java.util.Date

java.io.File

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.