Package org.apache.lucene.analysis.charfilter

Examples of org.apache.lucene.analysis.charfilter.BaseCharFilter


    @Test
    public void IncrementsOffsetCorrectlyWithAnotherReader2() throws IOException {
        String input = "test1 <a href=\"foo\">testlink</a> test2 test3";

        BaseCharFilter filter = new HTMLStripCharFilter(new StringReader(input));
        Tokenizer t = new Tokenizer(filter, HSpellLoader.readDefaultPrefixes());

        Reference<String> token = new Reference<String>("");

        t.nextToken(token);
        assertEquals(0, filter.correctOffset(t.getOffset()));
        assertEquals(5, t.getLengthInSource());

        t.nextToken(token);
        assertEquals(20, filter.correctOffset(t.getOffset()));
        assertEquals(8, t.getLengthInSource());

        t.nextToken(token);
        assertEquals(33, filter.correctOffset(t.getOffset()));
        assertEquals(5, t.getLengthInSource());

        t.nextToken(token);
        assertEquals(39, filter.correctOffset(t.getOffset()));
        assertEquals(5, t.getLengthInSource());
    }
View Full Code Here

TOP

Related Classes of org.apache.lucene.analysis.charfilter.BaseCharFilter

Copyright © 2018 www.massapicom. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.