Examples of CharFilter

com.intellij.openapi.util.text.CharFilter
@author Dmitry Avdeev @see StringUtil#strip(String,CharFilter) @see StringUtil#findFirst(String,CharFilter)
org.apache.lucene.analysis.CharFilter
Subclasses of CharFilter can be chained to filter CharStream. They can be used as {@link java.io.Reader} with additional offsetcorrection. {@link Tokenizer}s will automatically use {@link #correctOffset}if a CharFilter/CharStream subclass is used. @version $Id$

Examples of org.apache.lucene.analysis.CharFilter

    assertCharFilterEquals(filter, "時々、おおのさんと一緒にお寿司が食べたいです。abcところどころ。");
  }


  public void testNone() throws IOException {
    // Test no repetition marks
    CharFilter filter = new JapaneseIterationMarkCharFilter(
        new StringReader("時々、おゝのさんと一緒にお寿司が食べたいです。abcところゞゝゝ。"),
        false, // no kanji
        false  // no kana
    );
    assertCharFilterEquals(filter, "時々、おゝのさんと一緒にお寿司が食べたいです。abcところゞゝゝ。");

View Full Code Here

Examples of org.apache.lucene.analysis.CharFilter

    List<String> mappingRules = new ArrayList<>();
    mappingRules.add( "\"&uuml;\" => \"ü\"" );
    NormalizeCharMap.Builder builder = new NormalizeCharMap.Builder();
    builder.add("&uuml;", "ü");
    NormalizeCharMap normMap = builder.build();
    CharFilter charStream = new MappingCharFilter( normMap, new StringReader( INPUT ) );


    // create PatternTokenizer
    Tokenizer stream = new PatternTokenizer(newAttributeFactory(), charStream, Pattern.compile("[,;/\\s]+"), -1);
    assertTokenStreamContents(stream,
        new String[] { "Günther", "Günther", "is", "here" },

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.