Examples of org.archive.crawler.util.TopNSet

org.archive.crawler.util.TopNSet

Counting Set which only remembers the 'top N' of all String values reported (with counts) to it. Precise if counts reported for a certain String key only ever increase. Otherwise, the current top N might decrement to be smaller than other tallies, which won't be noticed until those other, previously ignored tallies, are re-reported. TODO: Histotable and TopNSet that they could possibly have a closer relationship and share some code (even though Histotable tracks 'all' keys, and handles increments, while TopNSet only remembers a small subset of keys, and requires a fresh full value on each update.) @contributor gojomo

        boolean isRecover = (recoveryCheckpoint != null); 
        try {
            this.processedSeedsRecords = bdb.getObjectCache("processedSeedsRecords",
                    isRecover, SeedRecord.class);
            
            this.hostsDistributionTop = new TopNSet(getLiveHostReportSize());
            this.hostsBytesTop = new TopNSet(getLiveHostReportSize());
            this.hostsLastFinishedTop = new TopNSet(getLiveHostReportSize());
            
            if(isRecover) {
                JSONObject json = recoveryCheckpoint.loadJson(beanName);
                
                crawlStartTime = json.getLong("crawlStartTime");

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of org.archive.crawler.util.TopNSet

Related Classes of org.archive.crawler.util.TopNSet