Examples of org.terrier.terms.SkipTermPipeline

org.terrier.terms.SkipTermPipeline
Class that identified tokens which should not be passed down the entire term pipeline, and instead passed onto a specified stage instead. Tokens are autmatically lowercased if lowercase is set (as it is by default). If no tokens are specified, then no tokens are omitted from the term pipeline.
Properties
- termpipelines.skip - list of tokens to skip
- lowercase - whether tokens should be lowercased
@author Rodrygo Santos and Craig Macdonald @since 2.2

      }
    }
    String skipTerms = null;
    //add SkipTermPipeline as the first pipeline step to allow for special terms to skip the pipeline processing sequence
    if ((skipTerms = ApplicationSetup.getProperty("termpipelines.skip", null)) != null && skipTerms.trim().length() > 0)
      pipeline_first = new SkipTermPipeline(next, last);
    else
      pipeline_first = next;
  }

View Full Code Here

TOP

Related Classes of org.terrier.terms.SkipTermPipeline

org.terrier.indexing.Indexer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.