Package com.soulgalore.crawler.core.impl

Examples of com.soulgalore.crawler.core.impl.AhrefPageURLParser

42
43
44
45
46
47
48
49
50
        CrawlerConfiguration apiConfig =
                new CrawlerConfiguration.Builder().setStartUrl(url).setMaxLevels(linkDepth).setVerifyUrls(false)
                        .build();
        DefaultCrawler crawler =
                new DefaultCrawler(new ResponseFetcher(processor), Executors.newFixedThreadPool(10),
                        new CompositeURLParser(new FramePageURLParser(), new AhrefPageURLParser()));
        crawler.getUrls(apiConfig);
        crawler.shutdown();
    }
View Full Code Here
TOP

Related Classes of com.soulgalore.crawler.core.impl.AhrefPageURLParser

  • sagan.search.support.CrawlerService
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.