Examples of org.apache.lucene.analysis.cjk.CJKBigramFilter

org.apache.lucene.analysis.cjk.CJKBigramFilter

Forms bigrams of CJK terms that are generated from StandardTokenizer or ICUTokenizer.

CJK types are set by these tokenizers, but you can also use {@link #CJKBigramFilter(TokenStream,int)} to explicitly control whichof the CJK scripts are turned into bigrams.

By default, when a CJK character has no adjacent characters to form a bigram, it is output in unigram form. If you want to always output both unigrams and bigrams, set the outputUnigrams flag in {@link CJKBigramFilter#CJKBigramFilter(TokenStream,int,boolean)}. This can be used for a combined unigram+bigram approach.

In all cases, all non-CJK input is passed thru unmodified.

    outputUnigrams = getBoolean("outputUnigrams", false);
  }
  
  @Override
  public TokenStream create(TokenStream input) {
    return new CJKBigramFilter(input, flags, outputUnigrams);
  }

    }
  }
  
  @Override
  public TokenStream create(TokenStream input) {
    return new CJKBigramFilter(input, flags, outputUnigrams);
  }

        this.flags = flags;
    }


    @Override
    public TokenStream create(TokenStream tokenStream) {
        return new CJKBigramFilter(tokenStream, flags, outputUnigrams);
    }

Related Classes of org.apache.lucene.analysis.cjk.CJKBigramFilter

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.