Examples of it.unimi.dsi.mg4j.index.cluster.DocumentalPartitioningStrategy

it.unimi.dsi.mg4j.index.cluster.DocumentalPartitioningStrategy
A way to associate a document with a local index out of a given set and a local document number in the local index.
When partitioning documentally an index, we need a way to associate to each global document pointer {@linkplain #localIndex(int) a local index} (the indexthat will contain the postings of that document) and {@linkplain #localPointer(int) a localdocument pointer} (the document pointer actually represented in the local index).
Usually, a documental partitioning strategy has a matching {@link it.unimi.dsi.mg4j.index.cluster.DocumentalClusteringStrategy}whose methods satisfy the following equations:
```
 globalPointer(localIndex(d), localPointer(d)) = d <localIndex(globalPointer(i, l)), localPointer(globalPointer(i, l))> = <i, l> 
```
@author Alessandro Arrabito @author Sebastiano Vigna

    JSAPResult jsapResult = jsap.parse( arg );
    if ( jsap.messagePrinted() ) return;
    String inputBasename = jsapResult.getString( "inputBasename" );
    String outputBasename = jsapResult.getString( "outputBasename" );
    String strategyFilename = jsapResult.getString( "strategy" );
    DocumentalPartitioningStrategy strategy = null;


    if ( jsapResult.userSpecified( "uniformStrategy" ) ) {
      strategy = DocumentalStrategies.uniform( jsapResult.getInt( "uniformStrategy" ), Index.getInstance( inputBasename ).numberOfDocuments );
      BinIO.storeObject( strategy, strategyFilename = outputBasename + IndexCluster.STRATEGY_DEFAULT_EXTENSION );
    }

View Full Code Here

    // Vanilla indexing
    new IndexBuilder( basename, getSequence() ).standardWriterFlags( flags ).skipBufferSize( 1024 ).pasteBufferSize( 1024 ).interleaved( interleaved ).skips( quantum != 0 ).quantum( quantum ).height( height )
        .virtualDocumentResolver( 3, RESOLVER ).run();


    // Now we use a crazy strategy moving around documents using modular arithmetic
    final DocumentalPartitioningStrategy modulo3 = new Modulo3DocumentalClusteringStrategy( NUMBER_OF_DOCUMENTS );
    BinIO.storeObject( modulo3, basename + "-strategy" );


    new PartitionDocumentally( basename + "-text", basename + "-text-part", modulo3, basename + "-strategy", 0, 1024, flags, interleaved, quantum != 0, Math.abs( quantum ), height,
        1024 * 1024, DEFAULT_LOG_INTERVAL ).run();
    new PartitionDocumentally( basename + "-int", basename + "-int-part", modulo3, basename + "-strategy", 0, 1024, DEFAULT_PAYLOAD_INDEX, interleaved, quantum != 0, Math.abs( quantum ), height, 1024 * 1024,

View Full Code Here

TOP

Related Classes of it.unimi.dsi.mg4j.index.cluster.DocumentalPartitioningStrategy

it.unimi.dsi.mg4j.tool.IndexTest

it.unimi.dsi.mg4j.tool.PartitionDocumentally

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.