Examples of it.unimi.dsi.mg4j.search.DocumentIterator

it.unimi.dsi.mg4j.search.DocumentIterator
An iterator over documents (pointers) and their intervals.
Each call to {@link #nextDocument()}will return a document pointer, or -1 if no more documents are available. Just after the call to {@link #nextDocument()}, {@link #intervalIterator(Index)} will return an interval iteratorenumerating intervals in the last returned document for the specified index. The latter method may return, as a special result, a special {@link it.unimi.dsi.mg4j.search.IntervalIterators#TRUE TRUE} value: this means that albeit the current document satisfies the query, there is only a generic empty witness to prove it (see {@link it.unimi.dsi.mg4j.search.IntervalIterators#TRUE TRUE} for some elaboration).
Note that this class implements {@link IntIterator}. Nonetheless, for performance reasons, the preferred access to the document pointers is {@link #nextDocument()}.
The {@link #iterator()} method must be an alias for {@link #intervalIterator()}, and shares the same limitations.
A document iterator is usually structured as composite, with operators as internal nodes and {@link it.unimi.dsi.mg4j.index.IndexIterator}s as leaves. The methods {@link #accept(DocumentIteratorVisitor)} and {@link #acceptOnTruePaths(DocumentIteratorVisitor)} implement the visitor pattern.
The {@link #dispose()} method is intended to recursively release all resources associatedto a composite document iterator. Note that this is not always what you want, as you might be, say, pooling {@linkplain it.unimi.dsi.mg4j.index.IndexReader index readers} to reduce the numberof file open/close operations. For this reason, we intentionally avoid calling the method “close”.
Warning: the interval enumeration can be carried out only just after a call to {@link #nextDocument()}. Subsequent calls to {@link #nextDocument()} or even to {@link java.util.Iterator#hasNext()} will reset the internal state of the iterator. In particular, trying to enumerate intervals after a call to {@link java.util.Iterator#hasNext()} will usually throw an {@link java.lang.IllegalStateException}.

        } );
    IntArrayDocumentIterator i2 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 4, 5 } }, 
        } );
    DocumentIterator consecutiveDocumentIterator = ConsecutiveDocumentIterator.getInstance( i0, i1, i2 );
    assertTrue( consecutiveDocumentIterator.hasNext() );
    assertEquals( 0, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 0, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 5 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertNull( consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.hasNext() );
    
  }

View Full Code Here

        } );
    IntArrayDocumentIterator i1 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 3, 4 } }, 
        } );
    DocumentIterator consecutiveDocumentIterator = ConsecutiveDocumentIterator.getInstance( i0, i1 );
    assertTrue( consecutiveDocumentIterator.hasNext() );
    assertEquals( 0, consecutiveDocumentIterator.nextDocument() );
    assertEquals( Interval.valueOf( 1, 4 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
  }

View Full Code Here

        { 2, 5 }, 
        { 0 }, 
        { 2 }, 
        { 4, 6 }, 
        } );
    DocumentIterator consecutiveDocumentIterator = ConsecutiveDocumentIterator.getInstance( i0, i1, i2 );
    assertTrue( consecutiveDocumentIterator.hasNext() );
    assertEquals( 0, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 0, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 2 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 3, 5 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 2, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 2, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 2 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 3, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 3, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 4, 6 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertFalse( consecutiveDocumentIterator.hasNext() );


    
    i0.reset();
    i1.reset();
    i2.reset();


    consecutiveDocumentIterator = ConsecutiveDocumentIterator.getInstance( i0, i1, i2 );
    assertEquals( 2, consecutiveDocumentIterator.skipTo( 1 ) );
    assertEquals( 2, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 2 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 3, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 3, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 4, 6 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertFalse( consecutiveDocumentIterator.hasNext() );


    i0.reset();
    i1.reset();
    i2.reset();


    consecutiveDocumentIterator = ConsecutiveDocumentIterator.getInstance( new DocumentIterator[] { i0, i1, i2 }, new int[] { 1, 0, 0 } );
    assertTrue( consecutiveDocumentIterator.hasNext() );
    assertEquals( 0, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 0, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 2, 5 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 3, consecutiveDocumentIterator.nextDocument() );
    assertEquals( 3, consecutiveDocumentIterator.document() );
    assertEquals( Interval.valueOf( 3, 6 ), consecutiveDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( consecutiveDocumentIterator.intervalIterator().hasNext() );
    assertFalse( consecutiveDocumentIterator.hasNext() );
  }

View Full Code Here

                "E B"
    ) ).run();
    Index index = DiskBasedIndex.getInstance( basename + "-text", true, true );


        /// String query = "A| B+C+G|W|S+J";
        DocumentIterator iterator = OrDocumentIterator.getInstance(
                index.documents("A"),
                MultiTermIndexIterator.getInstance(
                        index.documents("B"),
                        index.documents("C"),
                        index.documents("G")
                ),
                index.documents("W"),
                MultiTermIndexIterator.getInstance(
                        index.documents("S"),
                        index.documents("J")
                ));




        final int[] currDoc = new int[ 1 ];
        // A visitor invoking positionArray() on IndexIterators positioned on the current document.
        AbstractDocumentIteratorVisitor visitor = new AbstractDocumentIteratorVisitor() {
          public Boolean visit(IndexIterator indexIterator) throws IOException {
            if (indexIterator.count() > 0 && indexIterator.document() == currDoc[ 0 ] ) indexIterator.positionArray();
              return Boolean.TRUE;
          }
        };




        for (int document = 0; document < index.numberOfDocuments; document++) {
            currDoc[ 0 ] = iterator.skipTo(document);


            if (document == currDoc[ 0 ]) {
               iterator.accept(visitor); // see method visit below.
            }
        }


        
        while( iterator.hasNext() ) {
          currDoc[ 0 ] = iterator.nextDocument();
          iterator.accept(  visitor );
        }
    }

View Full Code Here

    assertEquals( Interval.valueOf( 0, 7 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
  }
  
  public void testLoopcounterPersistence() throws IOException {
    DocumentIterator i0 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 10, 20 }, { 30, 40 }, { 87, 88 } } 
    });
    DocumentIterator i1 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 25, 27 }, { 45, 47 } } 
        } );
    DocumentIterator i2 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 32, 40 }, { 35, 52 }, { 80, 90 }, { 120, 130 } }, 
        } );
    DocumentIterator i3 = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 49, 50 }, { 92, 105 }, { 140, 150 } }, 
        } );
    DocumentIterator orderedAndDocumentIterator = OrderedAndDocumentIterator.getInstance( i0, i1, i2, i3 );
    assertTrue( orderedAndDocumentIterator.hasNext() );
    assertEquals( 0, orderedAndDocumentIterator.nextDocument() );
    assertEquals( Interval.valueOf( 10, 50 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 30, 105 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
  }

View Full Code Here

    assertEquals( Interval.valueOf( 30, 105 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
  }


  public void testLoopcounterPersistenceSingleton() throws IOException {
    DocumentIterator i0 = new IntArrayIndexIterator( new int[] { 0 }, 
        new int[][] { 
        { 1, 4, 11 } 
    });
    DocumentIterator i1 = new IntArrayIndexIterator( new int[] { 0 }, 
        new int[][] { 
        {  2, 7 } 
        } );
    DocumentIterator i2 = new IntArrayIndexIterator( new int[] { 0 }, 
        new int[][] { 
        { 7, 10, 14, 20 }, 
        } );
    DocumentIterator i3 = new IntArrayIndexIterator( new int[] { 0 }, 
        new int[][] { 
        { 10, 15, 30 }, 
        } );
    DocumentIterator orderedAndDocumentIterator = OrderedAndDocumentIterator.getInstance( i0, i1, i2, i3 );
    assertTrue( orderedAndDocumentIterator.hasNext() );
    assertEquals( 0, orderedAndDocumentIterator.nextDocument() );
    assertEquals( Interval.valueOf( 1, 10 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 4, 15 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
  }

View Full Code Here

    assertEquals( Interval.valueOf( 4, 15 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
  }


  public void testIndexIntervalIterator() throws IOException {
    DocumentIterator i0 = new IntArrayIndexIterator( new int[] { 0, 1, 2, 3, 4, 5 }, 
        new int[][] { 
        { 0, 1, 2 }, 
        { 0 }, 
        { 3 }, 
        { 1, 2 }, 
        { 1, 2 }, 
        { 1, 3 },
        } );
    DocumentIterator i1 = new IntArrayIndexIterator( new int[] { 0, 1, 2, 3, 4, 5 }, 
        new int[][] { 
        { 0, 1, 2 }, 
        { 1 }, 
        { 4 }, 
        { 2, 4 }, 
        { 3 }, 
        { 2, 3 }
        } );
    DocumentIterator i2 = new IntArrayIndexIterator( new int[] { 0, 1, 2, 3, 4, 5 }, 
        new int[][] { 
        { 0, 1, 2 }, 
        { 2 }, 
        { 2 }, 
        { 4, 8 }, 
        { 4 },
        { 5 }
        } );
    DocumentIterator orderedAndDocumentIterator = OrderedAndDocumentIterator.getInstance( i0, i1, i2 );
    assertTrue( orderedAndDocumentIterator.hasNext() );
    assertEquals( 0, orderedAndDocumentIterator.nextDocument() );
    assertEquals( 0, orderedAndDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 2 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 1, orderedAndDocumentIterator.nextDocument() );
    assertEquals( 1, orderedAndDocumentIterator.document() );
    assertEquals( Interval.valueOf( 0, 2 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 3, orderedAndDocumentIterator.nextDocument() );
    assertEquals( 3, orderedAndDocumentIterator.document() );
    assertEquals( Interval.valueOf( 1, 4 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 2, 8 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 4, orderedAndDocumentIterator.nextDocument() );
    assertEquals( 4, orderedAndDocumentIterator.document() );
    assertEquals( Interval.valueOf( 2, 4 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
    assertEquals( 5, orderedAndDocumentIterator.nextDocument() );
    assertEquals( 5, orderedAndDocumentIterator.document() );
    assertEquals( Interval.valueOf( 1, 5 ), orderedAndDocumentIterator.intervalIterator().nextInterval() );
    assertFalse( orderedAndDocumentIterator.intervalIterator().hasNext() );
    assertFalse( orderedAndDocumentIterator.hasNext() );
  }

View Full Code Here


  
  public void testSkipBug() throws QueryParserException, QueryBuilderVisitorException, IOException {
    Query query = simpleParser.parse( "a | b | c" );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    assertEquals( 2, documentIterator.skipTo( 2 ) );
    documentIterator.dispose();
  }

View Full Code Here

    documentIterator.dispose();
  }
  


  public void testOr() throws IOException {
    DocumentIterator i0 = new IntArrayDocumentIterator( new int[] { 0, 2, 3, 4, 5, 6, 7 }, 
        new int[][][] { 
        { { 0, 1 } }, 
        { { 0, 1 }, { 1, 2 } },
        { { 0, 1 }, { 1, 2 } },
        { {} },
        { {} },
        {},
        {},
        } );
    DocumentIterator i1 = new IntArrayDocumentIterator( new int[] { 1, 2, 3, 4, 5, 6, 7 }, 
        new int[][][] { 
        { { 1 } },
        { { 1, 3 }, { 3, 4 } },
        { {} },
        { {} },

View Full Code Here

  }


  public void testWeights() throws QueryBuilderVisitorException, IOException {
    Query query = new Weight( 0.5, new And( new Term( "a" ), new Term( "b" ) ) );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    assertEquals( .5, documentIterator.weight(), 0 );
    documentIterator.dispose();


    query = new Weight( .1, new Weight( 0.5, new And( new Weight( .2, new Term( "a" ) ), new Term( "b" ) ) ) );
    documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );
    documentIterator = query.accept( documentIteratorBuilderVisitor );
    assertEquals( .5, documentIterator.weight(), 0 );
    documentIterator.dispose();
}

View Full Code Here

0 1 2 3

TOP

Related Classes of it.unimi.dsi.mg4j.search.DocumentIterator

it.unimi.dsi.mg4j.index.cluster.LexicalCluster

it.unimi.dsi.mg4j.index.MultiTermIndexIteratorTest

it.unimi.dsi.mg4j.query.QueryEngine

it.unimi.dsi.mg4j.search.ConsecutiveDocumentIteratorTest

it.unimi.dsi.mg4j.search.DifferenceDocumentIteratorTest

it.unimi.dsi.mg4j.search.NotDocumentIteratorTest

it.unimi.dsi.mg4j.search.OrderedAndDocumentIteratorTest

it.unimi.dsi.mg4j.search.OrDocumentIteratorTest

it.unimi.dsi.mg4j.search.score.ScorerTest

it.unimi.dsi.mg4j.search.WeightTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.