List of reusableTokenStream() Examples

Examples of reusableTokenStream()

org.apache.lucene.analysis.Analyzer.reusableTokenStream()
Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. Callers that do not need to use more than one TokenStream at the same time from this analyzer should use this method for better performance.
org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()
This method uses {@link #createComponents(String,Reader)} to obtain aninstance of {@link TokenStreamComponents}. It returns the sink of the components and stores the components internally. Subsequent calls to this method will reuse the previously stored components if and only if the {@link TokenStreamComponents#reset(Reader)} method returnedtrue. Otherwise a new instance of {@link TokenStreamComponents} is created. @param fieldName the name of the field the created TokenStream is used for @param reader the reader the streams source reads from

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

        // Use the analyzer to get all the tokens, and then build a TermQuery,
        // PhraseQuery, or nothing based on the term count


        TokenStream source = null;
        try {
          source = analyzer.reusableTokenStream(field, new StringReader(queryText));
          source.reset();
        } catch (IOException e) {
          throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e);  
        }
        ArrayList<Token> lst = new ArrayList<Token>();

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()


    if (!TokenizerChain.class.isInstance(analyzer)) {


      TokenStream tokenStream = null;
      try {
        tokenStream = analyzer.reusableTokenStream(context.getFieldName(), new StringReader(value));
        tokenStream.reset();
      } catch (IOException e) {
        throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e);
      }
      NamedList<List<NamedList>> namedList = new NamedList<List<NamedList>>();

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < 5000; i++) {
      sb.append("我购买了道具和服装。");
    }
    Analyzer analyzer = new SmartChineseAnalyzer(TEST_VERSION_CURRENT);
    TokenStream stream = analyzer.reusableTokenStream("", new StringReader(sb.toString()));
    stream.reset();
    while (stream.incrementToken()) {
    }
  }

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < 5000; i++) {
      sb.append("我购买了道具和服装");
    }
    Analyzer analyzer = new SmartChineseAnalyzer(TEST_VERSION_CURRENT);
    TokenStream stream = analyzer.reusableTokenStream("", new StringReader(sb.toString()));
    stream.reset();
    while (stream.incrementToken()) {
    }
  }

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    public final TokenStream reusableTokenStream(String fieldName, Reader reader)
            throws IOException {
        if (indexingConfig != null) {
            Analyzer propertyAnalyzer = indexingConfig.getPropertyAnalyzer(fieldName);
            if (propertyAnalyzer != null) {
                return propertyAnalyzer.reusableTokenStream(fieldName, reader);
            }
        }
        return defaultAnalyzer.reusableTokenStream(fieldName, reader);
    }
}

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    public TokenStream reusableTokenStream(String fieldName, Reader reader)
            throws IOException {
        if (indexingConfig != null) {
            Analyzer propertyAnalyzer = indexingConfig.getPropertyAnalyzer(fieldName);
            if (propertyAnalyzer != null) {
                return propertyAnalyzer.reusableTokenStream(fieldName, reader);
            }
        }
        return defaultAnalyzer.reusableTokenStream(fieldName, reader);
    }
}

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

          stringReader.init(stringValue);
          reader = stringReader;
        }
        
        // Tokenize field
        stream = analyzer.reusableTokenStream(field.name(), reader);
      }


      // reset the TokenStream to the first token
      stream.reset();

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

          stringReader.init(stringValue);
          reader = stringReader;
        }
        
        // Tokenize field
        stream = analyzer.reusableTokenStream(field.name(), reader);
      }


      // reset the TokenStream to the first token
      stream.reset();

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < 5000; i++) {
      sb.append("我购买了道具和服装。");
    }
    Analyzer analyzer = new SmartChineseAnalyzer(TEST_VERSION_CURRENT);
    TokenStream stream = analyzer.reusableTokenStream("", new StringReader(sb.toString()));
    stream.reset();
    while (stream.incrementToken()) {
    }
  }

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < 5000; i++) {
      sb.append("我购买了道具和服装");
    }
    Analyzer analyzer = new SmartChineseAnalyzer(TEST_VERSION_CURRENT);
    TokenStream stream = analyzer.reusableTokenStream("", new StringReader(sb.toString()));
    stream.reset();
    while (stream.incrementToken()) {
    }
  }

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.