Package us.codecraft.webmagic.processor.example

Examples of us.codecraft.webmagic.processor.example.OschinaBlogPageProcessor

11
12
13
14
15
16
17
18
19
20
21
*/
public class MonitorExample {

    public static void main(String[] args) throws Exception {

        Spider oschinaSpider = Spider.create(new OschinaBlogPageProcessor())
                .addUrl("http://my.oschina.net/flashsword/blog");
        Spider githubSpider = Spider.create(new GithubRepoPageProcessor())
                .addUrl("https://github.com/code4craft");

        SpiderMonitor.instance().register(oschinaSpider);
View Full Code Here
18
19
20
21
22
23
24
25
26
27
28
            protected SpiderStatusMXBean getSpiderStatusMBean(Spider spider, MonitorSpiderListener monitorSpiderListener) {
                return new CustomSpiderStatus(spider, monitorSpiderListener);
            }
        };

        Spider oschinaSpider = Spider.create(new OschinaBlogPageProcessor())
                .addUrl("http://my.oschina.net/flashsword/blog").thread(2);
        Spider githubSpider = Spider.create(new GithubRepoPageProcessor())
                .addUrl("https://github.com/code4craft");

        spiderMonitor.register(oschinaSpider, githubSpider);
View Full Code Here
32
33
34
35
36
37
38
              System.out.println(resultItemse);
          }
          ooSpider.close();*/
         
         
          Spider.create(new OschinaBlogPageProcessor()).addUrl("http://my.oschina.net/flashsword/blog").run();
    }
View Full Code Here
TOP

Related Classes of us.codecraft.webmagic.processor.example.OschinaBlogPageProcessor

  • us.codecraft.webmagic.example.MonitorExample
  • us.codecraft.webmagic.monitor.SpiderMonitorTest
  • webmagic.App
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.