Examples of FSDirectory

org.apache.hadoop.hdfs.server.namenode.FSDirectory
FSDirectory stores the filesystem directory state. It handles writing/loading values to disk, and logging changes as we go. It keeps the filename->blockset mapping always-current and logged to disk.
org.apache.lucene.store.FSDirectory
sun.com/bugdatabase/view_bug.do?bug_id=6265734">Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link NIOFSDirectory} java docfor details.
{@link MMapDirectory} uses memory-mapped IO whenreading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's {@link IndexInput#close}is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link MMapDirectory}java doc for details.

Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the {@link #open} method, to allow Lucene to choosethe best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use {@link #open}. For all others, you should instantiate the desired implementation directly.

The locking implementation is by default {@link NativeFSLockFactory}, but can be changed by passing in a custom {@link LockFactory} instance. @see Directory

org.apache.nutch.indexer.FsDirectory

Reads a Lucene index stored in DFS.

org.jnode.fs.FSDirectory

FSDirectory interface provide methods related to directory operations in a file system. @author epr

Examples of org.apache.lucene.store.FSDirectory

    public void testFalseDirectoryAlreadyClosed() throws Throwable {


      File indexDir = _TestUtil.getTempDir("lucenetestdiralreadyclosed");


      try {
        FSDirectory dir = FSDirectory.getDirectory(indexDir);
        IndexWriter w = new IndexWriter(indexDir, new WhitespaceAnalyzer(), IndexWriter.MaxFieldLength.LIMITED);
        w.setUseCompoundFile(false);
        Document doc = new Document();
        w.addDocument(doc);
        w.close();
        assertTrue(new File(indexDir, "_0.fnm").delete());


        try {
          IndexReader.open(indexDir);
          fail("did not hit expected exception");
        } catch (AlreadyClosedException ace) {
          fail("should not have hit AlreadyClosedException");
        } catch (FileNotFoundException ioe) {
          // expected
        }


        // Make sure we really did close the dir inside IndexReader.open
        dir.close();


        try {
          dir.fileExists("hi");
          fail("did not hit expected exception");
        } catch (AlreadyClosedException ace) {
          // expected
        }
      } finally {

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    {
        File indexDir = File.createTempFile( remoteIndexFile, ".dir" );
        indexDir.delete();
        indexDir.mkdirs();


        FSDirectory directory = FSDirectory.open( indexDir );


        BufferedInputStream is = null;


        try
        {
            is = new BufferedInputStream( fetcher.retrieve( remoteIndexFile ) );


            Date timestamp = null;


            if ( remoteIndexFile.endsWith( ".gz" ) )
            {
                timestamp = unpackIndexData( is, directory, //
                    updateRequest.getIndexingContext() );
            }
            else
            {
                // legacy transfer format
                timestamp = unpackIndexArchive( is, directory, //
                    updateRequest.getIndexingContext() );
            }


            if ( updateRequest.getDocumentFilter() != null )
            {
                filterDirectory( directory, updateRequest.getDocumentFilter() );
            }


            if ( merge )
            {
                updateRequest.getIndexingContext().merge( directory );
            }
            else
            {
                updateRequest.getIndexingContext().replace( directory );
            }
            if ( sideEffects != null && sideEffects.size() > 0 )
            {
                getLogger().info( IndexUpdateSideEffect.class.getName() + " extensions found: " + sideEffects.size() );
                for ( IndexUpdateSideEffect sideeffect : sideEffects )
                {
                    sideeffect.updateIndex( directory, updateRequest.getIndexingContext(), merge );
                }
            }


            return timestamp;
        }
        finally
        {
            IOUtil.close( is );


            if ( directory != null )
            {
                directory.close();
            }


            try
            {
                FileUtils.deleteDirectory( indexDir );

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory


        File indexDir = new File( indexArchive.getAbsoluteFile().getParentFile(), indexArchive.getName() + ".dir" );


        indexDir.mkdirs();


        FSDirectory fdir = FSDirectory.open( indexDir );


        try
        {
            unpackDirectory( fdir, is );
            copyUpdatedDocuments( fdir, directory, context );

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory


        File indexDir = new File( indexArchive.getAbsoluteFile().getParentFile(), indexArchive.getName() + ".dir" );


        indexDir.mkdirs();


        FSDirectory fdir = FSDirectory.open( indexDir );


        try
        {
            // force the timestamp update
            IndexUtils.updateTimestamp( context.getIndexDirectory(), context.getTimestamp() );

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

   @Test
   public void profileTestFSDirectory() throws InterruptedException, IOException {
      File indexDir = new File(new File("."), indexName);
      boolean directoriesCreated = indexDir.mkdirs();
      assert directoriesCreated : "couldn't create directory for FSDirectory test";
      FSDirectory dir = FSDirectory.open(indexDir);
      testDirectory(dir, "FSDirectory");
   }

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

   @Test
   public void profileTestFSDirectory() throws InterruptedException, IOException {
      File indexDir = new File(new File("."), indexName);
      boolean directoriesCreated = indexDir.mkdirs();
      assert directoriesCreated : "couldn't create directory for FSDirectory test";
      FSDirectory dir = FSDirectory.open(indexDir);
      stressTestDirectory(dir, "FSDirectory");
   }

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

      File subDir = new File(rootDir, indexName);
      boolean directoriesCreated = subDir.mkdir();
      assert directoriesCreated : "couldn't create directory for test";


      //We need at least one Directory to exist on filesystem to trigger the problem
      FSDirectory luceneDirectory = FSDirectory.open(subDir);
      luceneDirectory.close();
      ConfigurationBuilder builder = new ConfigurationBuilder();
      builder.persistence()
            .addStore(LuceneStoreConfigurationBuilder.class)
               .autoChunkSize(110)
               .location(rootDir.getAbsolutePath());

View Full Code Here

Examples of org.apache.lucene.store.FSDirectory

    * @param invert           flag which identifies which terms should be inserted which not.
    * @throws IOException
    */
   public static void createIndex(File rootDir, String indexName, int termsToAdd, boolean invert) throws IOException {
      File indexDir = new File(rootDir, indexName);
      FSDirectory directory = FSDirectory.open(indexDir);
      try {
         CacheTestSupport.initializeDirectory(directory);
         IndexWriter iwriter = LuceneSettings.openWriter(directory, 100000);
         try {
            for (int i = 0; i <= termsToAdd; i++) {
               Document doc = new Document();
               String term = String.valueOf(i);
               //For even values of i we add to "main" field
               if (i % 2 == 0 ^ invert) {
                  doc.add(new Field("main", term, Field.Store.NO, Field.Index.NOT_ANALYZED));
               }
               else {
                  doc.add(new Field("secondaryField", term, Field.Store.YES, Field.Index.NOT_ANALYZED));
               }
               iwriter.addDocument(doc);
            }
            iwriter.commit();
         }
         finally {
            iwriter.close();
         }
      }
      finally {
         directory.close();
      }
   }

View Full Code Here

Examples of org.apache.nutch.indexer.FsDirectory

    if ("file".equals(this.fs.getUri().getScheme())) {
      Path qualified = file.makeQualified(FileSystem.getLocal(conf));
      File fsLocal = new File(qualified.toUri());
      return FSDirectory.open(new File(fsLocal.getAbsolutePath()));
    } else {
      return new FsDirectory(this.fs, file, false, this.conf);
    }
  }

View Full Code Here

Examples of org.jnode.fs.FSDirectory

     * @param level
     * @throws IOException
     */
    public static void createDirAndFiles(FSDirectory parent, int level) throws IOException {
        for (int i = 0; i < FILES_DIR_PER_LEVEL; i++) {
            FSDirectory subDir = createDirectory(parent, "dir" + i);
            if (level > 0)
                createDirAndFiles(subDir, level - 1);


            createFile(parent, "file_" + level + "_" + i);
        }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.