Examples of org.languagetool.tokenizers.ca.CatalanWordTokenizer

org.languagetool.tokenizers.ca.CatalanWordTokenizer
Tokenizes a sentence into words. Punctuation and whitespace gets its own token. Special treatment for hyphens and apostrophes in Catalan. @author Jaume Ortolà

  }  
  
  @Override
  public final Tokenizer getWordTokenizer() {
    if (wordTokenizer == null) {
      wordTokenizer = new CatalanWordTokenizer();
    }
    return wordTokenizer;
  }

View Full Code Here

  }  
  
  @Override
  public final Tokenizer getWordTokenizer() {
    if (wordTokenizer == null) {
      wordTokenizer = new CatalanWordTokenizer();
    }
    return wordTokenizer;
  }

View Full Code Here

TOP

Related Classes of org.languagetool.tokenizers.ca.CatalanWordTokenizer

org.languagetool.language.Catalan

org.languagetool.rules.spelling.morfologik.MorfologikSpeller

org.languagetool.tagging.ca.CatalanTagger

java.util.regex.Matcher

java.util.StringTokenizer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.