Examples of org.apache.lucene.store.FSDirectory

org.apache.lucene.store.FSDirectory
sun.com/bugdatabase/view_bug.do?bug_id=6265734">Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link NIOFSDirectory} java docfor details.
{@link MMapDirectory} uses memory-mapped IO whenreading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's {@link IndexInput#close}is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using {@link Thread#interrupt()} or{@link Future#cancel(boolean)} should use{@link SimpleFSDirectory} instead. See {@link MMapDirectory}java doc for details.

Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the {@link #open} method, to allow Lucene to choosethe best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use {@link #open}. For all others, you should instantiate the desired implementation directly.

The locking implementation is by default {@link NativeFSLockFactory}, but can be changed by passing in a custom {@link LockFactory} instance. @see Directory

  public String getSigmentUniqKey()
  {
    StringBuffer buffer=new StringBuffer();
    buffer.append(si.name).append("@");
    if(core.dir instanceof FSDirectory){
        FSDirectory dddir=(FSDirectory)core.dir;
        buffer.append(dddir.getDirectory().getAbsolutePath()).append("@");
      }else if(core.dir instanceof FileSystemDirectory){
        FileSystemDirectory dddir=(FileSystemDirectory)core.dir;
        buffer.append("@hdfs@"+dddir.directory.toString()).append("@");
      }
      else if(core.dir instanceof RAMDirectory){

View Full Code Here

    Directory dir = commit.getDirectory();


    // For anything persistent, make something that will
    // be the same, regardless of the Directory instance.
    if (dir instanceof FSDirectory) {
      FSDirectory fsd = (FSDirectory) dir;
      File fdir = fsd.getDirectory();
      sb.append(fdir.getPath());
    } else {
      sb.append(dir);
    }

View Full Code Here

    private static void index(String indexDir, Map<String,List<String>> word2Nums, Map<String,List<String>> num2Words)
        throws Throwable
    {
        int row = 0;
        int mod = 1;
        FSDirectory dir = FSDirectory.open(new File(indexDir));
        try {


          // override the specific index if it already exists
          IndexWriter writer = new IndexWriter(dir, new IndexWriterConfig(
              Version.LUCENE_CURRENT, ana).setOpenMode(OpenMode.CREATE));
          ((TieredMergePolicy) writer.getConfig().getMergePolicy()).setUseCompoundFile(true); // why?
          Iterator<String> i1 = word2Nums.keySet().iterator();
          while (i1.hasNext()) // for each word
          {
              String g = i1.next();
              Document doc = new Document();


              int n = index(word2Nums, num2Words, g, doc);
              if (n > 0)
              {
          doc.add( new Field( F_WORD, g, Field.Store.YES, Field.Index.NOT_ANALYZED));
                  if ((++row % mod) == 0)
                  {
                      o.println("\trow=" + row + "/" + word2Nums.size() + " doc= " + doc);
                      mod *= 2;
                  }
                  writer.addDocument(doc);
              } // else degenerate
          }
          o.println( "Optimizing..");
          writer.optimize();
          writer.close();
        } finally {
          dir.close();
        }
    }

View Full Code Here

  public static final int DEFAULTnumTerms = 100;
  public static int numTerms = DEFAULTnumTerms;
  
  public static void main(String[] args) throws Exception {
    IndexReader reader = null;
    FSDirectory dir = null;
    String field = null;
    boolean IncludeTermFreqs = false; 
   
    if (args.length == 0 || args.length > 4) {
      usage();

View Full Code Here

    {
      System.out.println(
                 "java org.apache.lucene.wordnet.SynExpand <index path> <query>");
    }


    FSDirectory directory = FSDirectory.open(new File(args[0]));
    IndexSearcher searcher = new IndexSearcher(directory, true);


    String query = args[1];
    String field = "contents";


    Query q = expand( query, searcher, new StandardAnalyzer(Version.LUCENE_CURRENT), field, 0.9f);
    System.out.println( "Query: " + q.toString( field));






    searcher.close();
    directory.close();
  }

View Full Code Here

    if (args.length != 2) {
      System.out.println(
                 "java org.apache.lucene.wordnet.SynLookup <index path> <word>");
    }


    FSDirectory directory = FSDirectory.open(new File(args[0]));
    IndexSearcher searcher = new IndexSearcher(directory, true);


    String word = args[1];
    Query query = new TermQuery(new Term(Syns2Index.F_WORD, word));
    CountingCollector countingCollector = new CountingCollector();
    searcher.search(query, countingCollector);


    if (countingCollector.numHits == 0) {
      System.out.println("No synonyms found for " + word);
    } else {
      System.out.println("Synonyms found for \"" + word + "\":");
    }


    ScoreDoc[] hits = searcher.search(query, countingCollector.numHits).scoreDocs;
    
    for (int i = 0; i < hits.length; i++) {
      Document doc = searcher.doc(hits[i].doc);


      String[] values = doc.getValues(Syns2Index.F_SYN);


      for (int j = 0; j < values.length; j++) {
        System.out.println(values[j]);
      }
    }


    searcher.close();
    directory.close();
  }

View Full Code Here

    infos.commit(fsDir);
  }


  public void split(File destDir, String[] segs) throws IOException {
    destDir.mkdirs();
    FSDirectory destFSDir = FSDirectory.open(destDir);
    SegmentInfos destInfos = new SegmentInfos();
    destInfos.counter = infos.counter;
    for (String n : segs) {
      SegmentInfo info = getInfo(n);
      destInfos.add(info);

View Full Code Here

                url = new URL(a[++i]);
            }
        }


        PrintStream o = System.out;
        FSDirectory dir = FSDirectory.open(new File(indexName));
        IndexReader r = IndexReader.open(dir, true);
        o.println("Open index " + indexName + " which has " + r.numDocs() + " docs");


        MoreLikeThis mlt = new MoreLikeThis(r);

View Full Code Here

  }


  private static Directory newFSDirectoryImpl(
      Class<? extends FSDirectory> clazz, File file)
      throws IOException {
    FSDirectory d = null;
    try {
      // Assuming every FSDirectory has a ctor(File), but not all may take a
      // LockFactory too, so setting it afterwards.
      Constructor<? extends FSDirectory> ctor = clazz.getConstructor(File.class);
      d = ctor.newInstance(file);

View Full Code Here

    public void testFalseDirectoryAlreadyClosed() throws Throwable {


      File indexDir = _TestUtil.getTempDir("lucenetestdiralreadyclosed");


      try {
        FSDirectory dir = FSDirectory.getDirectory(indexDir);
        IndexWriter w = new IndexWriter(indexDir, new WhitespaceAnalyzer(), IndexWriter.MaxFieldLength.LIMITED);
        w.setUseCompoundFile(false);
        Document doc = new Document();
        w.addDocument(doc);
        w.close();
        assertTrue(new File(indexDir, "_0.fnm").delete());


        try {
          IndexReader.open(indexDir);
          fail("did not hit expected exception");
        } catch (AlreadyClosedException ace) {
          fail("should not have hit AlreadyClosedException");
        } catch (FileNotFoundException ioe) {
          // expected
        }


        // Make sure we really did close the dir inside IndexReader.open
        dir.close();


        try {
          dir.fileExists("hi");
          fail("did not hit expected exception");
        } catch (AlreadyClosedException ace) {
          // expected
        }
      } finally {

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of org.apache.lucene.store.FSDirectory

com.alimama.mdrill.buffer.CacheKeyBuffer

com.bericotech.clavin.index.IndexDirectoryBuilder

com.browseengine.bobo.api.BoboIndexReader

de.arago.lucene.api.Index

larq.cmdline.ModLARQindex

org.apache.blur.store.blockcache.BlockDirectoryTest

org.apache.cocoon.components.search.LuceneCocoonHelper

org.apache.ctakes.core.resource.LuceneIndexReaderResourceImpl

org.apache.lucene.ant.IndexTask

org.apache.lucene.benchmark.quality.trec.QueryDriver

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.