Package us.codecraft.webmagic.scheduler

Examples of us.codecraft.webmagic.scheduler.RedisScheduler

23
24
25
26
27
28
29
30
    public static void main(String[] args) {
        OOSpider.create(Site.me().setDomain("www.jokeji.cn").setCharset("gbk").setSleepTime(100).setTimeOut(3000)
                .setUserAgent("Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)")
                , new ConsolePageModelPipeline(), JokejiModel.class).addUrl("http://www.jokeji.cn/").thread(2)
                .scheduler(new RedisScheduler("127.0.0.1"))
                .run();
    }
View Full Code Here
71
72
73
74
75
76
77
                '}';
    }

    public static void main(String[] args) {
        OOSpider.create(Site.me(), News163.class).addUrl("http://news.163.com/13/0802/05/958I1E330001124J_2.html")
                .scheduler(new RedisScheduler("localhost")).addPipeline(new MultiPagePipeline()).addPipeline(new ConsolePipeline()).run();
    }
View Full Code Here
TOP

Related Classes of us.codecraft.webmagic.scheduler.RedisScheduler

  • redis.clients.jedis.Jedis
  • redis.clients.jedis.JedisPool
  • redis.clients.jedis.JedisPoolConfig
  • us.codecraft.webmagic.model.samples.JokejiModel
  • us.codecraft.webmagic.model.samples.News163
  • us.codecraft.webmagic.Request
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.