Examples of com.flaptor.hounder.crawler.pagedb.PageDB

Package com.flaptor.hounder.crawler.pagedb

Examples of com.flaptor.hounder.crawler.pagedb.PageDB

com.flaptor.hounder.crawler.pagedb.PageDB
The PageDB implements a database of pages. It can be read in url order or in md5 order (md5 hash of the url), which is close to random order. It also takes care of various statistics, like number of pages @author Flaptor Development Team

            FileUtil.deleteDir(catchDir);
            if (!catchdb.rename(catchDir)) {
                throw new IOException("Unable to rename the catchdb.tmp because I couldn't remove a previous catchdb");
            }
        }
        PageDB tmp = catchdb;
        newCatcher();
        return tmp;
    }

View Full Code Here

0 1 2

TOP

Related Classes of com.flaptor.hounder.crawler.pagedb.PageDB

com.flaptor.hounder.crawler.Crawler

com.flaptor.hounder.crawler.Crawler$InjectedFetchlistQueueMonitor

com.flaptor.hounder.crawler.CrawlerProgress

com.flaptor.hounder.crawler.CrawlerTest

com.flaptor.hounder.crawler.modules.IndexerModule

com.flaptor.hounder.crawler.pagedb.distributed.DPageDB

com.flaptor.hounder.crawler.pagedb.distributed.DPageDBTest

com.flaptor.hounder.crawler.pagedb.distributed.PageCatcher

com.flaptor.hounder.crawler.pagedb.distributed.PageCatcherTest

com.flaptor.hounder.crawler.PageDBTrimmer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.