Examples of org.archive.wayback.util.htmllex.ContextAwareLexer

Package org.archive.wayback.util.htmllex

Examples of org.archive.wayback.util.htmllex.ContextAwareLexer

org.archive.wayback.util.htmllex.ContextAwareLexer
The Lexer that comes with htmlparser does not handle non-escaped HTML entities within SCRIPT tags - by default, something like: Can cause the lexer to skip over a large part of the document. Technically, the above isn't legit HTML, but of course, folks do stuff like that all the time. So, this class uses a ParseContext object, passed in at construction, which observes the SCRIPT and STYLE tags, both setting properties on the ParseContext, and using that state information to perform a parseCDATA() call instead of a nextNode() call at the right time, to try to keep the SAX parsing in sync with the document. @author brad

    // and finally, parse, using the special lexer that knows how to
    // handle javascript blocks containing unescaped HTML entities:
    Page lexPage = new Page(bais,charSet);
    Lexer lexer = new Lexer(lexPage);
    Lexer.STRICT_REMARKS = false;
      ContextAwareLexer lex = new ContextAwareLexer(lexer, context);


      Node node;
      while ((node = lex.nextNode()) != null) {
          delegator.handleNode(context, node);
      }
      delegator.handleParseComplete(context);


    // At this point, baos contains the utf-8 encoded bytes of our result:

View Full Code Here

    // and finally, parse, using the special lexer that knows how to
    // handle javascript blocks containing unescaped HTML entities:
    Page lexPage = new Page(decodedResource,charSet);
    Lexer lexer = new Lexer(lexPage);
    Lexer.STRICT_REMARKS = false;
    ContextAwareLexer lex = new ContextAwareLexer(lexer, context);
    Node node;
    try {
      delegator.handleParseStart(context);
      while((node = lex.nextNode()) != null) {
        delegator.handleNode(context, node);
      }
      delegator.handleParseComplete(context);
    } catch (ParserException e) {
      e.printStackTrace();

View Full Code Here


    ParseContext context = new ParseContext();
     
      Node node;
      try {
          ContextAwareLexer lex = new ContextAwareLexer(
              new Lexer(new Page(is,charSet)),context);
      while((node = lex.nextNode()) != null) {
//        System.err.println("\nDEBUG-Node:js("+context.isInJS()+")css("+context.isInCSS()+"):");
//        System.err.println("-------------------/START");
//        System.err.println(node.toHtml(true));
//        System.err.println("-------------------/END");
        rules.handleNode(context, node);

View Full Code Here

TOP

Related Classes of org.archive.wayback.util.htmllex.ContextAwareLexer

org.archive.wayback.archivalurl.ArchivalUrlSAXRewriteReplayRenderer

org.archive.wayback.archivalurl.FastArchivalUrlReplayParseEventHandlerTest

org.archive.wayback.resourcestore.indexer.HTTPRecordAnnotater

org.htmlparser.Node

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.