Examples of it.unimi.dsi.mg4j.index.cluster.LexicalPartitioningStrategy

it.unimi.dsi.mg4j.index.cluster.LexicalPartitioningStrategy
A way to associate a term number with a local index out of a given set and a local term number in the local index.
When partitioning lexically an index (i.e., termwise), we need a way to associate to each term {@linkplain #localIndex(int) a local index} (the indexthat will contain the postings of that term) and {@linkplain #localNumber(int) a localterm number} (the number of the term in the local index).
Usually, a lexical partitioning strategy has a matching {@link it.unimi.dsi.mg4j.index.cluster.LexicalClusteringStrategy} whose methodssatisfy the following equations:
```
 globalNumber(localIndex(t), localNumber(t)) = t <localIndex(globalNumber(i, l)), localNumber(globalNumber(i, l))> = <i, l> 
```
@author Alessandro Arrabito @author Sebastiano Vigna

    JSAPResult jsapResult = jsap.parse( arg );
    if ( jsap.messagePrinted() ) return;
    String inputBasename = jsapResult.getString( "inputBasename" );
    String outputBasename = jsapResult.getString( "outputBasename" );
    String strategyFilename = jsapResult.getString( "strategy" );
    LexicalPartitioningStrategy strategy = null;


    if ( jsapResult.userSpecified( "uniformStrategy" ) ) {
      strategy = LexicalStrategies.uniform( jsapResult.getInt( "uniformStrategy" ), DiskBasedIndex.getInstance( inputBasename, false, false, true ) );
      BinIO.storeObject( strategy, strategyFilename = outputBasename + IndexCluster.STRATEGY_DEFAULT_EXTENSION );
    }

View Full Code Here

      Exception {
    // Vanilla indexing
    new IndexBuilder( basename, getSequence() ).standardWriterFlags( flags ).interleaved( interleaved ).skipBufferSize( 1024 ).pasteBufferSize( 1024 ).virtualDocumentResolver( 3, RESOLVER ).run();


    // Now we use a crazy strategy moving around documents using modular arithmetic
    final LexicalPartitioningStrategy uniform = LexicalStrategies.uniform( 3, DiskBasedIndex.getInstance( basename + "-text" ) );
    BinIO.storeObject( uniform, basename + "-strategy" );


    new PartitionLexically( basename + "-text", basename + "-text-part", uniform, basename + "-strategy", 1024, DEFAULT_LOG_INTERVAL ).run();
    new PartitionLexically( basename + "-virtual", basename + "-virtual-part", uniform, basename + "-strategy", 1024, DEFAULT_LOG_INTERVAL ).run();

View Full Code Here

TOP

Related Classes of it.unimi.dsi.mg4j.index.cluster.LexicalPartitioningStrategy

it.unimi.dsi.mg4j.tool.IndexTest

it.unimi.dsi.mg4j.tool.PartitionLexically

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.