Examples of it.unimi.dsi.mg4j.search.DocumentIterator

it.unimi.dsi.mg4j.search.DocumentIterator
An iterator over documents (pointers) and their intervals.
Each call to {@link #nextDocument()}will return a document pointer, or -1 if no more documents are available. Just after the call to {@link #nextDocument()}, {@link #intervalIterator(Index)} will return an interval iteratorenumerating intervals in the last returned document for the specified index. The latter method may return, as a special result, a special {@link it.unimi.dsi.mg4j.search.IntervalIterators#TRUE TRUE} value: this means that albeit the current document satisfies the query, there is only a generic empty witness to prove it (see {@link it.unimi.dsi.mg4j.search.IntervalIterators#TRUE TRUE} for some elaboration).
Note that this class implements {@link IntIterator}. Nonetheless, for performance reasons, the preferred access to the document pointers is {@link #nextDocument()}.
The {@link #iterator()} method must be an alias for {@link #intervalIterator()}, and shares the same limitations.
A document iterator is usually structured as composite, with operators as internal nodes and {@link it.unimi.dsi.mg4j.index.IndexIterator}s as leaves. The methods {@link #accept(DocumentIteratorVisitor)} and {@link #acceptOnTruePaths(DocumentIteratorVisitor)} implement the visitor pattern.
The {@link #dispose()} method is intended to recursively release all resources associatedto a composite document iterator. Note that this is not always what you want, as you might be, say, pooling {@linkplain it.unimi.dsi.mg4j.index.IndexReader index readers} to reduce the numberof file open/close operations. For this reason, we intentionally avoid calling the method “close”.
Warning: the interval enumeration can be carried out only just after a call to {@link #nextDocument()}. Subsequent calls to {@link #nextDocument()} or even to {@link java.util.Iterator#hasNext()} will reset the internal state of the iterator. In particular, trying to enumerate intervals after a call to {@link java.util.Iterator#hasNext()} will usually throw an {@link java.lang.IllegalStateException}.

import junit.framework.TestCase;


public class DifferenceDocumentIteratorTest extends TestCase {


  public void testdifference() throws IOException {
    DocumentIterator minuendIterator = new IntArrayDocumentIterator( new int[] { 0, 1, 2, 3 }, 
        new int[][][] { 
        { { 0, 1 } }, 
        { { 0, 1 }, { 1, 2 }, { 2, 3 } },
        { { 0, 1 }, { 1, 2 }, { 2, 3 } },
        { { 0 } }
        } );
    DocumentIterator subtrahendIterator = new IntArrayDocumentIterator( new int[] { 1, 2 }, 
        new int[][][] { 
        { { 1 } },
        { { 1, 3 } } 
        } );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertTrue( differenceIterator.hasNext() );
    assertTrue( differenceIterator.hasNext() ); // To increase coverage
    assertEquals( 0, differenceIterator.nextDocument() );
    assertEquals( 0, differenceIterator.document() );
    assertEquals( Interval.valueOf( 0, 1 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertEquals( 1, differenceIterator.nextDocument() );
    assertEquals( 1, differenceIterator.document() );
    assertEquals( Interval.valueOf( 2, 3 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertEquals( 2, differenceIterator.nextDocument() );
    assertEquals( 2, differenceIterator.document() );
    assertEquals( Interval.valueOf( 0, 1 ), differenceIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 1, 2 ), differenceIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 2, 3 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertEquals( 3, differenceIterator.nextDocument() );
    assertEquals( 3, differenceIterator.document() );
    assertEquals( Interval.valueOf( 0 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertFalse( differenceIterator.hasNext() ); // To increase coverage
  }

View Full Code Here

    assertFalse( differenceIterator.hasNext() ); // To increase coverage
  } 




  public void testSubtrahendExhaustion() throws IOException {
    DocumentIterator minuendIterator = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 0, 1 }, { 1, 2 }, { 2, 3 }, { 3, 4 } }
        } );
    DocumentIterator subtrahendIterator = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 1 } },
        } );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertTrue( differenceIterator.hasNext() );
    assertEquals( 0, differenceIterator.nextDocument() );
    assertEquals( 0, differenceIterator.document() );
    assertEquals( Interval.valueOf( 2, 3 ), differenceIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 3, 4 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertFalse( differenceIterator.hasNext() ); // To increase coverage
  }

View Full Code Here

        { { 1 } },
        { { 1, 3 } },
        { {} },
        { {} },
        } );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertEquals( 2, differenceIterator.skipTo( 1 ) );
    assertEquals( 2, differenceIterator.skipTo( 1 ) ); // To increase coverage
    assertEquals( Integer.MAX_VALUE, differenceIterator.skipTo( 4 ) );


    minuendIterator.reset();
    subtrahendIterator.reset();
    differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertEquals( 2, differenceIterator.skipTo( 2 ) );
    
    
    minuendIterator.reset();
    subtrahendIterator.reset();
    differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertEquals( Integer.MAX_VALUE, differenceIterator.skipTo( 10 ) );
    assertEquals( Integer.MAX_VALUE, differenceIterator.skipTo( 10 ) );
    assertEquals( -1, differenceIterator.nextDocument() );
  }

View Full Code Here

  }
  
  private void assertScores( String q, it.unimi.dsi.mg4j.index.Index index, Scorer scorer, double[] expected ) throws QueryParserException, QueryBuilderVisitorException, IOException {
    it.unimi.dsi.mg4j.query.nodes.Query query = simpleParser.parse( q );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    scorer.wrap( documentIterator );
    
    int i = 0;
    while ( documentIterator.nextDocument() != -1 ) {
      final double score = scorer.score();
      assertEquals( "Item " + i + " (document " + documentIterator.document() + ")", expected[ i ], score, ASSERT_DIFF );
      i++;
    }
    
    documentIterator.dispose();
  }

View Full Code Here

      termProcessors
    );
        
    it.unimi.dsi.mg4j.query.nodes.Query query = simpleParser.parse( q );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( indexMap, defaultIndex, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    scorer.wrap( documentIterator );
    
    int i = 0;
    while ( documentIterator.hasNext() ) {
      documentIterator.next();


      final double score = scorer.score();
      assertEquals( "Item " + i + " (document " + documentIterator.document() + ")", expected[ i ], score, ASSERT_DIFF );
      i++;
    }
    
    assertEquals( i, expected.length );
    
    documentIterator.dispose();
  }

View Full Code Here

    assertEquals( Integer.MAX_VALUE, differenceIterator.skipTo( 10 ) );
    assertEquals( -1, differenceIterator.nextDocument() );
  }


  public void testtrueFalseDifference() throws IOException {
    DocumentIterator minuendIterator = new IntArrayDocumentIterator( new int[] { 0, 1, 2, 3 }, 
        new int[][][] { 
        { {} }, 
        {},
        { {} }, 
        {},
        } );
    DocumentIterator subtrahendIterator = new IntArrayDocumentIterator( new int[] { 0, 1, 2, 3 }, 
        new int[][][] { 
        { {} }, 
        { {} }, 
        {},
        {},
        } );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertTrue( differenceIterator.hasNext() );
    assertEquals( 2, differenceIterator.nextDocument() );
    assertEquals( IntervalIterators.TRUE, differenceIterator.intervalIterator() );
    assertFalse( differenceIterator.hasNext() );
  }

View Full Code Here

    CounterCollectionVisitor counterCollectionVisitor = new CounterCollectionVisitor( setupVisitor );


    it.unimi.dsi.mg4j.query.nodes.Query query = simpleParser.parse( "this" );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index0, Integer.MAX_VALUE );


    DocumentIterator indexIterator = query.accept( documentIteratorBuilderVisitor );


    indexIterator.nextDocument();


    termVisitor.prepare();
    indexIterator.accept( termVisitor );
    setupVisitor.prepare();
    indexIterator.accept( setupVisitor );


    // assertEquals( 3, indexIterator.frequency() );
    assertEquals( 3, setupVisitor.frequency[ 0 ] );


    assertEquals( 1, indexIterator.nextDocument() );
    counterCollectionVisitor.prepare();
    indexIterator.accept( counterCollectionVisitor );
    assertEquals( 1, setupVisitor.count[ 0 ] );
    assertEquals( 2, indexIterator.nextDocument() );
    counterCollectionVisitor.prepare();
    indexIterator.accept( counterCollectionVisitor );
    assertEquals( 1, setupVisitor.count[ 0 ] );
  }

View Full Code Here

    assertEquals( IntervalIterators.TRUE, differenceIterator.intervalIterator() );
    assertFalse( differenceIterator.hasNext() );
  }


  public void testTrueFalseOtherDifference() throws IOException {
    DocumentIterator minuendIterator = new IntArrayDocumentIterator( new int[] { 0, 1, 2, 3 }, 
        new int[][][] { 
        { { 0, 1 }, { 1, 2 } }, 
        { { 0, 1 }, { 1, 2 } },
        { {} }, 
        {},
        } );
    DocumentIterator subtrahendIterator = new IntArrayDocumentIterator( new int[] { 0, 1, 2, 3 }, 
        new int[][][] { 
        { {} }, 
        {},
        { { 0, 1 }, { 1, 2 } }, 
        { { 0, 1 }, { 1, 2 } },
        } );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertTrue( differenceIterator.hasNext() );
    assertEquals( 1, differenceIterator.nextDocument() );
    assertEquals( Interval.valueOf( 0, 1 ), differenceIterator.intervalIterator().nextInterval() );
    assertEquals( Interval.valueOf( 1, 2 ), differenceIterator.intervalIterator().nextInterval() );
    assertFalse( differenceIterator.intervalIterator().hasNext() );
    assertEquals( 2, differenceIterator.nextDocument() );
    assertEquals( IntervalIterators.TRUE, differenceIterator.intervalIterator() );
    assertFalse( differenceIterator.hasNext() );
  }

View Full Code Here


    Scorer scorer = new CountScorer();


    it.unimi.dsi.mg4j.query.nodes.Query query = simpleParser.parse( "C" );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, indexBody, Integer.MAX_VALUE );
    DocumentIterator documentIterator = query.accept( documentIteratorBuilderVisitor );
    scorer.wrap( documentIterator );
    
    final double expected[] = { 2, 1 };


    int i = 0;
    while ( documentIterator.hasNext() ) {
      documentIterator.next();
      final double score = scorer.score();
      assertEquals( "Item " + i, expected[ i ], score, 0.001 );
      i++;
    }
  }

View Full Code Here

    assertEquals( IntervalIterators.TRUE, differenceIterator.intervalIterator() );
    assertFalse( differenceIterator.hasNext() );
  }


  public void testvoidDifference() {
    DocumentIterator minuendIterator = new IntArrayDocumentIterator( new int[] { 0 }, 
        new int[][][] { 
        { { 0, 1 }, { 1, 2 } }, 
        } );
    DocumentIterator subtrahendIterator = new IntArrayDocumentIterator( new int[] {}, 
        new int[][][] {} );
    DocumentIterator differenceIterator = DifferenceDocumentIterator.getInstance( minuendIterator, subtrahendIterator );
    assertEquals( minuendIterator, differenceIterator );
  }

View Full Code Here

0 1 2 3

TOP

Related Classes of it.unimi.dsi.mg4j.search.DocumentIterator

it.unimi.dsi.mg4j.index.cluster.LexicalCluster

it.unimi.dsi.mg4j.index.MultiTermIndexIteratorTest

it.unimi.dsi.mg4j.query.QueryEngine

it.unimi.dsi.mg4j.search.ConsecutiveDocumentIteratorTest

it.unimi.dsi.mg4j.search.DifferenceDocumentIteratorTest

it.unimi.dsi.mg4j.search.NotDocumentIteratorTest

it.unimi.dsi.mg4j.search.OrderedAndDocumentIteratorTest

it.unimi.dsi.mg4j.search.OrDocumentIteratorTest

it.unimi.dsi.mg4j.search.score.ScorerTest

it.unimi.dsi.mg4j.search.WeightTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.