Package bixo.fetcher

Examples of bixo.fetcher.SimulationWebServerForTests

77
78
79
80
81
82
83
84
85
86
87
88
    @Test
    public void testCircularRedirect() throws Exception {
        BaseFetcher fetcher = RobotUtils.createFetcher(ConfigUtils.BIXO_TEST_AGENT, 1);
        BaseRobotsParser parser = new SimpleRobotRulesParser();
       
        SimulationWebServerForTests webServer = new SimulationWebServerForTests();
        Server server = webServer.startServer(new CircularRedirectResponseHandler(), 8089);
       
        try {
            BaseRobotRules rules = RobotUtils.getRobotRules(fetcher, parser, new URL("http://localhost:8089/robots.txt"));
            Assert.assertTrue(rules.isAllowAll());
        } finally {
View Full Code Here
93
94
95
96
97
98
99
100
101
102
103
104
    @Test
    public void testRedirectToHtml() throws Exception {
        BaseFetcher fetcher = RobotUtils.createFetcher(ConfigUtils.BIXO_TEST_AGENT, 1);
        BaseRobotsParser parser = new SimpleRobotRulesParser();
       
        SimulationWebServerForTests webServer = new SimulationWebServerForTests();
        Server server = webServer.startServer(new RedirectToTopResponseHandler(), 8089);
       
        try {
            BaseRobotRules rules = RobotUtils.getRobotRules(fetcher, parser, new URL("http://localhost:8089/robots.txt"));
            Assert.assertTrue(rules.isAllowAll());
        } finally {
View Full Code Here
TOP

Related Classes of bixo.fetcher.SimulationWebServerForTests

  • bixo.robots.RobotUtilsTest
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.