Examples of it.unimi.dsi.mg4j.query.parser.SimpleParser$LookaheadSuccess

it.unimi.dsi.mg4j.query.parser.SimpleParser
A simple parser that transform a query string into a {@linkplain it.unimi.dsi.mg4j.query.nodes query}.
The parser supports multiple indices. You must provide a set of index aliases that the user will use to select indices, and the name of the default index alias to be used. After that, you parse a query by using the {@link #parse(String)} method.
The parser generated by JavaCC for this class will break terms using operators and nonspace-to-space transitions. Operators can be included in terms, if needed, using the backslash, which acts as an escape character, and makes the next character (usually an operator) a standard character. The syntax of the parser can be seen in action in the documentation of the package {@link it.unimi.dsi.mg4j.search}.
The parser returns a {@link it.unimi.dsi.mg4j.query.nodes.Query}—an abstract representation of the query string that can be turned later into a {@link it.unimi.dsi.mg4j.search.DocumentIterator}.
If a {@link TermProcessor} is specified, it will be applied to the terms found in thequery string. This can lead to transformations (e.g., downcasing) or generate an exception if the query string contains terms filtered by the term processor. @author Sebastiano Vigna @author Paolo Boldi @since 1.0.1

    Query.loadIndicesFromSpec( basenameWeight, loadSizes, documentCollection, indexMap, index2Weight );
    
    final Object2ObjectOpenHashMap<String,TermProcessor> termProcessors = new Object2ObjectOpenHashMap<String,TermProcessor>( indexMap.size() );
    for( String alias: indexMap.keySet() ) termProcessors.put( alias, indexMap.get( alias ).termProcessor );
    
    final SimpleParser simpleParser = new SimpleParser( indexMap.keySet(), indexMap.firstKey(), termProcessors );


    final Reference2ReferenceMap<Index, Object> index2Parser = new Reference2ReferenceOpenHashMap<Index, Object>();
    /*
    // Fetch parsers for payload-based fields.
    for( Index index: indexMap.values() ) if ( index.hasPayloads ) {

View Full Code Here

    /* To run a query in a simple way we need a query engine. The engine requires a parser
     * (which in turn requires the set of index names and a default index), a document iterator
     * builder, which needs the index map, a default index, and a limit on prefix query
     * expansion, and finally the index map. */
    QueryEngine engine = new QueryEngine(
      new SimpleParser( indexMap.keySet(), "text", termProcessors ),
      new DocumentIteratorBuilderVisitor( indexMap, text, 1000 ), 
      indexMap
      
    );

View Full Code Here

      public Query expand( Prefix prefix ) {
        return prefix;
      }
    };
    
    Query query = new SimpleParser().parse( "foo AND bar" );
    assertEquals( new And( new Term( "afoo"), new Term( "abar") ), addAqueryTransformer.transform( query ) ); 


    QueryTransformer expandQueryTransformer = new AbstractTermExpander() {
      /** The visitor used by this expander. */
      @Override

View Full Code Here


public class OrderedAndDocumentIteratorTest extends TestCase {


  public void testSkipBug() throws QueryParserException, QueryBuilderVisitorException, IOException, ConfigurationException, SecurityException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    BitStreamIndex index;
    SimpleParser simpleParser;
    String basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a a b c d e" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    simpleParser = new SimpleParser( index.termProcessor );


    Query query = simpleParser.parse( "(a|b|d)<(a|b|d)" );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    documentIterator.nextDocument();
    IntervalIterator intervalIterator = documentIterator.intervalIterator();
    assertEquals( Interval.valueOf( 0, 1 ), intervalIterator.nextInterval() );

View Full Code Here

  public void setUp() throws ConfigurationException, SecurityException, IOException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {


    String basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a", "b", "c" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    simpleParser = new SimpleParser( index.termProcessor );
  }

View Full Code Here

  @Override
  public void setUp() throws ConfigurationException, SecurityException, IOException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a", "b", "c" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    simpleParser = new SimpleParser( index.termProcessor );
  }

View Full Code Here

    indexFTitle = DiskBasedIndex.getInstance( titleFBasename + "-text", true, true );
    indexBodyBis = DiskBasedIndex.getInstance( bodyBasenameBis + "-text", true, true );


    new Paste( basenameFComb, new String[] { bodyFBasename + "-text", titleFBasename + "-text" }, true, true, 1024, null, 1024, CompressionFlags.DEFAULT_STANDARD_INDEX, true, false, -1, -1, -1, 10 ).run();
    immutableExternalPrefixMap = new ImmutableExternalPrefixMap( new FileLinesCollection( basenameFComb + ".terms", "UTF-8" ) );
    simpleParser = new SimpleParser( index0.termProcessor );
  }

View Full Code Here

  }


  private void assertScoresMultiIndex( String q, Object2ReferenceMap<String,it.unimi.dsi.mg4j.index.Index> indexMap, it.unimi.dsi.mg4j.index.Index defaultIndex, Scorer scorer, double[] expected ) throws QueryParserException, QueryBuilderVisitorException, IOException {
    Object2ReferenceOpenHashMap<String, TermProcessor> termProcessors = new Object2ReferenceOpenHashMap<String,TermProcessor>();
    for( String s: indexMap.keySet() ) termProcessors.put( s, indexMap.get( s ).termProcessor );
    SimpleParser simpleParser = new SimpleParser( 
      indexMap.keySet(),
      "t", // TODO: make it a parameter
      termProcessors
    );
        
    it.unimi.dsi.mg4j.query.nodes.Query query = simpleParser.parse( q );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( indexMap, defaultIndex, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    scorer.wrap( documentIterator );
    
    int i = 0;

View Full Code Here

TOP

Related Classes of it.unimi.dsi.mg4j.query.parser.SimpleParser$LookaheadSuccess

it.unimi.dsi.mg4j.examples.RunQuery

it.unimi.dsi.mg4j.index.MultiTermIndexIteratorTest

it.unimi.dsi.mg4j.query.nodes.AbstractTermExpanderTest

it.unimi.dsi.mg4j.query.Query

it.unimi.dsi.mg4j.search.OrderedAndDocumentIteratorTest

it.unimi.dsi.mg4j.search.OrDocumentIteratorTest

it.unimi.dsi.mg4j.search.score.ScorerTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.