The method {@link ParseTreePatternMatcher#split(String)} returns a list ofchunks in preparation for creating a token stream by {@link ParseTreePatternMatcher#tokenize(String)}. From there, we get a parse tree from with {@link ParseTreePatternMatcher#compile(String,int)}. These chunks are converted to {@link RuleTagToken}, {@link TokenTagToken}, or the regular tokens of the text surrounding the tags.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|