Package opennlp.tools.tokenize

Examples of opennlp.tools.tokenize.SimpleTokenizer

87
88
89
90
91
92
93
94
95
96
97
98
  @Test
  public void testSingleTokeNameAtSentenceEnd() {
    String sentence = "a b c Max";

    SimpleTokenizer tokenizer = SimpleTokenizer.INSTANCE;
    String tokens[] = tokenizer.tokenize(sentence);

    Span names[] = mNameFinder.find(tokens);

    assertTrue(names.length == 1);
    assertTrue(names[0].getStart() == 3 && names[0].getEnd() == 4);
View Full Code Here
TOP

Related Classes of opennlp.tools.tokenize.SimpleTokenizer

  • opennlp.tools.lang.english.TreebankNameFinder
  • opennlp.tools.namefind.DictionaryNameFinderTest
  • opennlp.tools.util.Span
  • org.dbpedia.spotlight.spot.NESpotter
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.