Examples of antlr.TokenStreamSelector

antlr.TokenStreamSelector
A token stream MUX (multiplexor) knows about n token streams and can multiplex them onto the same channel for use by token stream consumer like a parser. This is a way to have multiple lexers break up the same input stream for a single parser. Or, you can have multiple instances of the same lexer handle multiple input streams; this works great for includes.

        try{
      //TODO: Are any of these classes thread safe, and would not have to be created for each query?
            TerrierLexer lexer = new TerrierLexer(new StringReader(query));
            TerrierFloatLexer flexer = new TerrierFloatLexer(lexer.getInputState());


            TokenStreamSelector selector = new TokenStreamSelector();
            selector.addInputStream(lexer, "main");
            selector.addInputStream(flexer, "numbers");
            selector.select("main");
            TerrierQueryParser parser = new TerrierQueryParser(selector);
            parser.setSelector(selector);
            rtr = parser.query();
        }catch (Exception e) {
           // throw new QueryParserException("Failed to process '"+query+"'",e);

View Full Code Here

        // Make a main lexer and a CALDoc lexer
        lexer = new CALLexer(this, reader);
        caldocLexer = new CALDocLexer(this, lexer.getInputState());


        // Create a lexical selector and add the two lexers
        streamSelector = new TokenStreamSelector();
        streamSelector.addInputStream(lexer, "main");
        streamSelector.addInputStream(caldocLexer, "caldoclexer");


        // Set the starting token stream
        streamSelector.select(lexer);

View Full Code Here

public static AbstractTCLNode parse( String data )
    throws ParseException
{
   try 
   {
      final TokenStreamSelector _selector = new TokenStreamSelector();


      // set up two Lexers
      final TCLLexer _tclLexer = new TCLLexer( new StringReader( data ) );
      _tclLexer.setTokenStreamSelector( _selector );


      final ComponentLexer _compLexer = 
         new ComponentLexer( _tclLexer.getInputState() );


      _compLexer.setTokenStreamSelector( _selector );


      _selector.addInputStream( _tclLexer, 
                                TCLLexer.LEXER_NAME );


      _selector.addInputStream( _compLexer, 
                                ComponentLexer.LEXER_NAME );


      _selector.select( TCLLexer.LEXER_NAME );


      // connect the Parser with the two Lexers
      final TCLParser _parser = new TCLParser( _selector );


      // begin parse

View Full Code Here

TOP

Related Classes of antlr.TokenStreamSelector

org.jacorb.notification.filter.etcl.TCLParser

org.openquark.cal.compiler.CALMultiplexedLexer

org.terrier.querying.parser.QueryParser

java.util.Hashtable

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.