Examples of FSDirectory

org.apache.hadoop.hdfs.server.namenode.FSDirectory
FSDirectory stores the filesystem directory state. It handles writing/loading values to disk, and logging changes as we go. It keeps the filename->blockset mapping always-current and logged to disk.
org.apache.lucene.store.FSDirectory
sun.com/bugdatabase/view_bug.do?bug_id=6265734">Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link NIOFSDirectory} java docfor details.
{@link MMapDirectory} uses memory-mapped IO whenreading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's {@link IndexInput#close}is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link MMapDirectory}java doc for details.

Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the {@link #open} method, to allow Lucene to choosethe best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use {@link #open}. For all others, you should instantiate the desired implementation directly.

The locking implementation is by default {@link NativeFSLockFactory}, but can be changed by passing in a custom {@link LockFactory} instance. @see Directory

org.apache.nutch.indexer.FsDirectory

Reads a Lucene index stored in DFS.

org.jnode.fs.FSDirectory

FSDirectory interface provide methods related to directory operations in a file system. @author epr

Examples of org.apache.hadoop.hdfs.server.namenode.FSDirectory

    final FileSystem fs = cluster.getFileSystem();
    assertTrue("Not a HDFS: "+fs.getUri(),
                fs instanceof DistributedFileSystem);


    final DistributedFileSystem dfs = (DistributedFileSystem)fs;
    FSDirectory fsd = cluster.getNameNode().namesystem.dir;
    INodeDirectoryWithQuota rootDir = (INodeDirectoryWithQuota) (fsd
        .getExistingPathINodes("/")[0]);
    try {
      generateFiles(dfs, rootDir, 1024, 512);
      generateFiles(dfs, rootDir, 1019, 512);
    } finally {

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    }
    
    if(localdp == null)
      return null;
    
    FSDirectory dir = localdp.getDirectory();
    File indexRoot = dir.getFile();
    if (key == null) {
      File[] subIndex = indexRoot.listFiles();
      for (File sub : subIndex) {
        localDirectoryToMap(sub.getName());
      }
      return localdp;
    }
    DirectoryProvider dp = providerMap.get(key);
    if (dp != null)
      return dp;
    File indexFile = new File(indexRoot, key);
    String indexName = "";
    boolean create = !indexFile.exists();
    FSDirectory directory = null;
    try {
      indexName = indexFile.getCanonicalPath();
      directory = FSDirectory.getDirectory(indexName);
      if (create) {
        log.debug("Initialize index: '" + indexFile + "'");

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

   */
  public static void main(String[] args) throws Exception
  {
    File file = new File("/Users/jwang/dataset/facet_idx_2/beef");


    FSDirectory idxDir = FSDirectory.open(file);
    
    IndexReader reader = IndexReader.open(idxDir,true);
    
    long start =System.currentTimeMillis();
    BoboIndexReader boboReader = BoboIndexReader.getInstance(reader);

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    System.out.println(Arrays.toString(args));
    String filename = "/Users/xgu/lucene29test/caches/people-search-index";
    if (args.length>0) filename ="/Users/xgu/lucene29test/caches/people-search-index";
    System.out.println(filename);
    File file = new File(filename);
    FSDirectory directory = new SimpleFSDirectory(file);
//    FSDirectory directory = FSDirectory.getDirectory(file);
    System.out.println(directory.getClass().getName());
    IndexReader reader = IndexReader.open(directory, true);
    loadFile();
//    TermEnum termEnum = reader.terms(new Term("b", ""));
//    while(termEnum.next())
//    {

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    if (facetHandlers == null) // try to load from index
    {
      Directory idxDir = directory();
      if (idxDir != null && idxDir instanceof FSDirectory)
      {
        FSDirectory fsDir = (FSDirectory) idxDir;
        File file = fsDir.getFile();


        if (new File(file, SPRING_CONFIG).exists())
        {
          facetHandlers = loadFromIndex(file,_workArea);
        }

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

  //*-- otherwise, delete the existing index for the document
  //*-- should occur only for modified documents
  else
  { try  
     { synchronized(this)
       { FSDirectory fsd = FSDirectory.getDirectory(new File(Constants.getINDEXDIR()), false);
         IndexReader ir = IndexReader.open(fsd);
         ir.deleteDocuments(new Term("key", iDocument ) ); 
         ir.close(); 
       }
      }

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    } //*-- end of outer if
    ++count;
   } //*-- end of while


   //*-- clean up the Lucene index
   FSDirectory fsd = FSDirectory.getDirectory(new File(Constants.getINDEXDIR()), false);
   IndexReader ir = IndexReader.open(fsd);
   for (int i = 0; i < delFiles.size(); i++)
    ir.deleteDocuments(new Term("key", (String) delFiles.get(i)) ); 
   ir.close();
  }

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

  // The top numTerms will be displayed
  public static final int numTerms = 100;


  public static void main(String[] args) throws Exception {
    IndexReader reader = null;
    FSDirectory dir = null;
    String field = null;
    if (args.length == 1) {
      dir = FSDirectory.open(new File(args[0]));
      reader = IndexReader.open(dir, true);
    } else if (args.length == 2) {

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

  public static void main(String[] args) throws IOException {
    if (args.length < 3) {
      System.err.println("Usage: IndexMergeTool <mergedIndex> <index1> <index2> [index3] ...");
      System.exit(1);
    }
    FSDirectory mergedIndex = FSDirectory.open(new File(args[0]));


    IndexWriter writer = new IndexWriter(mergedIndex, new  SimpleAnalyzer(), true, IndexWriter.MaxFieldLength.LIMITED);


    Directory[] indexes = new Directory[args.length - 1];
    for (int i = 1; i < args.length; i++) {

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    {
      System.out.println(
                 "java org.apache.lucene.wordnet.SynExpand <index path> <query>");
    }


    FSDirectory directory = FSDirectory.open(new File(args[0]));
    IndexSearcher searcher = new IndexSearcher(directory, true);


    String query = args[1];
    String field = "contents";


    Query q = expand( query, searcher, new StandardAnalyzer(Version.LUCENE_CURRENT), field, 0.9f);
    System.out.println( "Query: " + q.toString( field));






    searcher.close();
    directory.close();
  }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.