Package de.l3s.boilerpipe.sax

Examples of de.l3s.boilerpipe.sax.HTMLDocument

50
51
52
53
54
55
56
57
58
59
60
61
                String text;
                if(cache.contains(url.toString())){
                    text = cache.get(url.toString());
                    logger.debug("  Fetched from cache:"+url.toString());
                } else {
                    HTMLDocument htmlDoc = HTMLFetcher.fetch(url);
                    TextDocument doc = new BoilerpipeSAXInput(htmlDoc.toInputSource()).getTextDocument();
                    text = ArticleExtractor.INSTANCE.getText(doc);
                    cache.put(url.toString(), text);
                    logger.debug("Fetched from web:"+url.toString());
                }
                if(text.length()<100){
View Full Code Here
84
85
86
87
88
89
90
                }
                in.close();

                final byte[] data = bos.toByteArray();
               
                return new HTMLDocument(data, cs);
        }
View Full Code Here
TOP

Related Classes of de.l3s.boilerpipe.sax.HTMLDocument

  • org.mediameter.cliff.test.places.focus.GdeltFocusChecker
  • org.mediameter.cliff.test.util.HTMLFetcher
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.