Examples of URILocalnameFilter

org.sindice.siren.analysis.filter.URILocalnameFilter

Extract the localname of an URI, and break it into smaller components based on delimiters, such as uppercase or integers.

This filter returns the complete URI, the full localname of the URIs as well as the localname tokens.

This filter is less demanding than the {@link URINormalisationFilter}in term of CPU. In addition, it is also less costly in term of index size since it creates less tokens per URI.

Before tokenisation, check the length of the localname. If the localname is too large, it is not tokenised. By default, the maximum localname length is set to 64.

        return new URITrailingSlashFilter(in);


      // here, trailing slash filter is after localname filtering, in order to
      // avoid filtering subdirectory instead of localname
      case LOCALNAME:
        in = new URILocalnameFilter(in);
        return new URITrailingSlashFilter(in);


      // here, trailing slash filter is before localname filtering, in order to
      // avoid trailing slash checking on every tokens generated by the
      // URI normalisation filter

   maxLength = (maxArg != null ? Integer.parseInt(maxArg) : URILocalnameFilter.DEFAULT_MAX_LENGTH);
  }


  @Override
  public TokenStream create(final TokenStream input) {
    final URILocalnameFilter filter = new URILocalnameFilter(input);
    filter.setMaxLength(maxLength);
    return filter;
  }

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of URILocalnameFilter

Examples of org.sindice.siren.analysis.filter.URILocalnameFilter

Examples of org.sindice.siren.analysis.filter.URILocalnameFilter