Package org.apache.lucene.analysis.cn.smart

Examples of org.apache.lucene.analysis.cn.smart.HMMChineseTokenizer


        super(index, indexSettings, name, settings);
    }

    @Override
    public Tokenizer create(Reader reader) {
        return new HMMChineseTokenizer(reader);
    }
View Full Code Here


                return "smartcn_tokenizer";
            }

            @Override
            public Tokenizer create(Reader reader) {
                return new HMMChineseTokenizer(reader);
            }
        }));

        // Register smartcn_sentence tokenizer -- for backwards compat an alias to smartcn_tokenizer
        indicesAnalysisService.tokenizerFactories().put("smartcn_sentence", new PreBuiltTokenizerFactoryFactory(new TokenizerFactory() {
            @Override
            public String name() {
                return "smartcn_sentence";
            }

            @Override
            public Tokenizer create(Reader reader) {
                return new HMMChineseTokenizer(reader);
            }
        }));

        // Register smartcn_word token filter -- noop
        indicesAnalysisService.tokenFilterFactories().put("smartcn_word", new PreBuiltTokenFilterFactoryFactory(new TokenFilterFactory() {
View Full Code Here

TOP

Related Classes of org.apache.lucene.analysis.cn.smart.HMMChineseTokenizer

Copyright © 2018 www.massapicom. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.