Examples of org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()

Class org.apache.lucene.analysis.ReusableAnalyzerBase

Examples of org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()

org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()
This method uses {@link #createComponents(String,Reader)} to obtain aninstance of {@link TokenStreamComponents}. It returns the sink of the components and stores the components internally. Subsequent calls to this method will reuse the previously stored components if and only if the {@link TokenStreamComponents#reset(Reader)} method returnedtrue. Otherwise a new instance of {@link TokenStreamComponents} is created. @param fieldName the name of the field the created TokenStream is used for @param reader the reader the streams source reads from

                Tokenizer t = new WhitespaceTokenizer(Lucene.VERSION, reader);
                return new TokenStreamComponents(t, new UniqueTokenFilter(t));
            }
        };


        TokenStream test = analyzer.reusableTokenStream("test", new StringReader("this test with test"));
        CharTermAttribute termAttribute = test.addAttribute(CharTermAttribute.class);
        assertThat(test.incrementToken(), equalTo(true));
        assertThat(termAttribute.toString(), equalTo("this"));


        assertThat(test.incrementToken(), equalTo(true));

View Full Code Here

                Tokenizer t = new WhitespaceTokenizer(Lucene.VERSION, reader);
                return new TokenStreamComponents(t, new TruncateTokenFilter(t, 3));
            }
        };


        TokenStream test = analyzer.reusableTokenStream("test", new StringReader("a bb ccc dddd eeeee"));
        CharTermAttribute termAttribute = test.addAttribute(CharTermAttribute.class);
        assertThat(test.incrementToken(), equalTo(true));
        assertThat(termAttribute.toString(), equalTo("a"));


        assertThat(test.incrementToken(), equalTo(true));

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.