Examples of HyphenationCompoundWordTokenFilter

org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter
A {@link TokenFilter} that decomposes compound words found in many Germanic languages.
"Donaudampfschiff" becomes Donau, dampf, schiff so that you can find "Donaudampfschiff" even when you only enter "schiff". It uses a hyphenation grammar and a word dictionary to achieve this.

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

      IOUtils.closeQuietly(stream);
    }
  }
  
  public HyphenationCompoundWordTokenFilter create(TokenStream input) {
    return new HyphenationCompoundWordTokenFilter(luceneMatchVersion, input, hyphenator, dictionary, minWordSize, minSubwordSize, maxSubwordSize, onlyLongestMatch);
  }

View Full Code Here

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

            throw new ElasticSearchIllegalArgumentException("Exception while reading hyphenation_patterns_path: " + e.getMessage());
        }
    }


    @Override public TokenStream create(TokenStream tokenStream) {
        return new HyphenationCompoundWordTokenFilter(version, tokenStream,
                hyphenationTree, wordList,
                minWordSize, minSubwordSize, maxSubwordSize, onlyLongestMatch);
    }

View Full Code Here

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

    }
  }
  
  @Override
  public HyphenationCompoundWordTokenFilter create(TokenStream input) {
    return new HyphenationCompoundWordTokenFilter(luceneMatchVersion, input, hyphenator, dictionary, minWordSize, minSubwordSize, maxSubwordSize, onlyLongestMatch);
  }

View Full Code Here

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

    }


    HyphenationTree hyphenator = HyphenationCompoundWordTokenFilter
        .getHyphenationTree(reader);


    HyphenationCompoundWordTokenFilter tf = new HyphenationCompoundWordTokenFilter(
        new WhitespaceTokenizer(new StringReader(
            "Rindfleischüberwachungsgesetz Drahtschere abba")), hyphenator,
        dict, CompoundWordTokenFilterBase.DEFAULT_MIN_WORD_SIZE,
        CompoundWordTokenFilterBase.DEFAULT_MIN_SUBWORD_SIZE,
        CompoundWordTokenFilterBase.DEFAULT_MAX_SUBWORD_SIZE, false);

View Full Code Here

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

    }


    HyphenationTree hyphenator = HyphenationCompoundWordTokenFilter
        .getHyphenationTree(reader);


    HyphenationCompoundWordTokenFilter tf = new HyphenationCompoundWordTokenFilter(
        new WhitespaceTokenizer(new StringReader(
            "Rindfleischüberwachungsgesetz")), hyphenator, dict,
        CompoundWordTokenFilterBase.DEFAULT_MIN_WORD_SIZE,
        CompoundWordTokenFilterBase.DEFAULT_MIN_SUBWORD_SIZE, 40, true);
    assertFiltersTo(tf, new String[] { "Rindfleischüberwachungsgesetz",

View Full Code Here

Examples of org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter

    }


    @Override
    public TokenStream create(TokenStream tokenStream) {
        if (version.onOrAfter(Version.LUCENE_4_4_0)) {
            return new HyphenationCompoundWordTokenFilter(tokenStream, hyphenationTree, wordList, minWordSize, 
                                                          minSubwordSize, maxSubwordSize, onlyLongestMatch);
        } else {
            return new Lucene43HyphenationCompoundWordTokenFilter(tokenStream, hyphenationTree, wordList, minWordSize, 
                    minSubwordSize, maxSubwordSize, onlyLongestMatch);
        }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.