Package us.codecraft.webmagic.scheduler

Examples of us.codecraft.webmagic.scheduler.QueueScheduler

32
33
34
35
36
37
38
39
40
        return site;

    }

    public static void main(String[] args) throws JMException {
        Spider spider = Spider.create(new OschinaBlogPageProcesser()).setScheduler(new QueueScheduler().setDuplicateRemover(new BloomFilterDuplicateRemover(2000)));
        SpiderMonitor.instance().register(spider);
        spider.run();
    }
View Full Code Here
TOP

Related Classes of us.codecraft.webmagic.scheduler.QueueScheduler

  • us.codecraft.webmagic.samples.OschinaBlogPageProcesser
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.