Examples of DocLengthTable2B

ivory.core.data.stat.DocLengthTable2B

Object that keeps track of the length of each document in the collection as a two-byte integer (shorts). Document lengths are measured in number of terms.

Document length data is stored in a serialized data file, in the following format:

An integer that specifies the docno offset d, where d + 1 is the first docno in the collection.
An integer that specifies the number of documents in the collection n.
Exactly n shorts, one for each document in the collection.

Since the documents are numbered sequentially starting at d + 1, each short corresponds unambiguously to a particular document.

@author Jimmy Lin

        throw new RuntimeException("Error loading df table from " + localFiles[0]);
      }


      try {
        if (shortDocLengths)
          mDLTable = new DocLengthTable2B(pathMapping.get(dlFile), FileSystem.getLocal(conf));
        else
          mDLTable = new DocLengthTable4B(pathMapping.get(dlFile), FileSystem.getLocal(conf));
      } catch (IOException e1) {
        throw new RuntimeException("Error loading dl table from " + localFiles[2]);
      }


      LOG.info("Global Stats table loaded successfully.");


      try {
        if(shortDocLengths)
          mDLTable = new DocLengthTable2B(pathMapping.get(dlFile), FileSystem.getLocal(conf));
        else 
          mDLTable = new DocLengthTable4B(pathMapping.get(dlFile), FileSystem.getLocal(conf));
      } catch (IOException e1) {
        throw new RuntimeException("Error loading dl table from "+localFiles[4]);
      }

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of DocLengthTable2B

Examples of ivory.core.data.stat.DocLengthTable2B

Examples of ivory.core.data.stat.DocLengthTable2B