Examples of IndexBuilder

com.hp.hpl.jena.tdb.index.IndexBuilder
A policy holder for making indexes and range indexes.
index.IndexBuilder
Class responsible for creating an index for the given configuration @author rahul
it.unimi.di.big.mg4j.tool.IndexBuilder
it.unimi.dsi.mg4j.tool.IndexBuilder
An index builder.
An instance of this class exposes a {@link #run()} methodthat will index the {@link DocumentSequence} provided at construction timeby calling {@link Scan} and {@link Combine} in sequence.
Additionally, a main method provides easy access to index construction.
All indexing parameters are available either as chainable setters that can be called optionally before invoking {@link #run()}, or as public mutable collections and maps. For instance,
```
 new IndexBuilder( "foo", sequence ).skips( true ).run(); 
```
will build an index with basename foo using skips. If instead we want to index just the first field of the sequence, and use a {@link ShiftAddXorSignedStringMap}as a term map, we can use the following code:
```
 new IndexBuilder( "foo", sequence ) .termMapClass( ShiftAddXorSignedMinimalPerfectHash.class ) .indexedFields( 0 ).run(); 
```
More sophisticated modifications can be applied using public maps:
```
 IndexBuilder indexBuilder = new IndexBuilder( "foo", sequence ); indexBuilder.virtualDocumentGaps.put( 0, 30 ); indexBuilder.virtualDocumentResolver.put( 0, someVirtualDocumentResolver ); indexBuilder.run(); 
```

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder

public class ScanTest extends TestCase {


  public void testEverywhereTerms() throws IOException, ConfigurationException, SecurityException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    
    String basename = File.createTempFile( getClass().getSimpleName(), "everywhereTerms" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a a" ) ).keepBatches( true ).run();
    IndexIterator indexIterator = Index.getInstance( basename + "-text@0" ).documents( 0 );
    indexIterator.next();
    assertEquals( 2, indexIterator.count() );
    int[] position = new int[ 2 ];
    indexIterator.positions( position );

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder


  public void testSkipBug() throws QueryParserException, QueryBuilderVisitorException, IOException, ConfigurationException, SecurityException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    BitStreamIndex index;
    SimpleParser simpleParser;
    String basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a a b c d e" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    simpleParser = new SimpleParser( index.termProcessor );


    Query query = simpleParser.parse( "(a|b|d)<(a|b|d)" );
    DocumentIteratorBuilderVisitor documentIteratorBuilderVisitor = new DocumentIteratorBuilderVisitor( null, index, Integer.MAX_VALUE );

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder

  private String basename;


  @Override
  public void setUp() throws ConfigurationException, SecurityException, IOException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a", "b", "c" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    simpleParser = new SimpleParser( index.termProcessor );
  }

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder


public class BitStreamIndexReaderTest extends TestCase {


  public void testSkipToEndOfList() throws ConfigurationException, SecurityException, IOException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    String basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( new String[] { "a", "a", "c" } ) ).run();


    Index index = DiskBasedIndex.getInstance( basename + "-text", true, true );
    IndexIterator indexIterator = index.documents( "a" );
    assertEquals( Integer.MAX_VALUE, indexIterator.skipTo( Integer.MAX_VALUE ) );
    indexIterator.dispose();

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder

  public void testSkipToBeyondUsedClusters() throws Exception {
    /* We test what happens when we skip to a document belonging to a local index larger 
     * than any index in which the term appears. */
    
    final String basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
        new IndexBuilder( basename, new StringArrayDocumentCollection( "A B", "B", "A", "A" ) ).run();
    BinIO.storeObject( DocumentalStrategies.uniform( 2, 4 ), basename + "-strategy" );
    new PartitionDocumentally( basename + "-text", basename + "-cluster", DocumentalStrategies.uniform( 2, 4 ), basename + "-strategy", 0, 1024, CompressionFlags.DEFAULT_STANDARD_INDEX, true, false, 0, 0, 0, ProgressLogger.DEFAULT_LOG_INTERVAL ).run();
    FileLinesCollection flc;
    flc = new FileLinesCollection( basename + "-cluster-0.terms", "ASCII" );
    BinIO.storeObject( new ShiftAddXorSignedStringMap( flc.iterator(), new MWHCFunction<CharSequence>( flc , TransformationStrategies.utf16() ) ), basename + "-cluster-0.termmap" );

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder

  private BitStreamIndex index;
  private String basename;


  public void setUp() throws ConfigurationException, SecurityException, IOException, URISyntaxException, ClassNotFoundException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
    basename = File.createTempFile( getClass().getSimpleName(), "test" ).getCanonicalPath();
    new IndexBuilder( basename, new StringArrayDocumentCollection( "a", "b", "c" ) ).run();
    index = DiskBasedIndex.getInstance( basename + "-text", true, true );
  }

View Full Code Here

Examples of it.unimi.dsi.mg4j.tool.IndexBuilder

    // Global data for BM25F
    // Weight: coll2:3, coll1:1
    // Doc size: 6 5 6 9 4 (avg: 15/8)
    // Frequency: A:2 C:2 D:1 Q:3 X:2 Z:3
    
    new IndexBuilder( basename0, documentCollection0 ).run();
    new IndexBuilder( bodyBasename, bodyDocumentCollection ).run();
    new IndexBuilder( titleBasename, titleDocumentCollection ).run();
    new IndexBuilder( bodyFBasename, bodyFDocumentCollection ).run();
    new IndexBuilder( titleFBasename, titleFDocumentCollection ).run();
    new IndexBuilder( bodyBasenameBis, bodyDocumentCollection ).run();
    index0 = DiskBasedIndex.getInstance( basename0 + "-text", true, true );
    indexBody = DiskBasedIndex.getInstance( bodyBasename + "-text", true, true );
    indexTitle = DiskBasedIndex.getInstance( titleBasename + "-text", true, true );
    indexFBody = DiskBasedIndex.getInstance( bodyFBasename + "-text", true, true );
    indexFTitle = DiskBasedIndex.getInstance( titleFBasename + "-text", true, true );

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.