Examples of FuzzyLikeThisQuery

org.apache.lucene.sandbox.queries.FuzzyLikeThisQuery
Fuzzifies ALL terms provided as strings and then picks the best n differentiating terms. In effect this mixes the behaviour of FuzzyQuery and MoreLikeThis but with special consideration of fuzzy scoring factors. This generally produces good results for queries where users may provide details in a number of fields and have no knowledge of boolean query syntax and also want a degree of fuzzy matching and a fast query. For each source term the fuzzy variants are held in a BooleanQuery with no coord factor (because we are not looking for matches on multiple variants in any one doc). Additionally, a specialized TermQuery is used for variants and does not use that variant term's IDF because this would favour rarer terms eg misspellings. Instead, all variants use the same IDF ranking (the one for the source query term) and this is factored into the variant's boost. If the source query term does not exist in the index the average IDF of the variants is used.
org.apache.lucene.search.FuzzyLikeThisQuery
Fuzzifies ALL terms provided as strings and then picks the best n differentiating terms. In effect this mixes the behaviour of FuzzyQuery and MoreLikeThis but with special consideration of fuzzy scoring factors. This generally produces good results for queries where users may provide details in a number of fields and have no knowledge of boolean query syntax and also want a degree of fuzzy matching and a fast query. For each source term the fuzzy variants are held in a BooleanQuery with no coord factor (because we are not looking for matches on multiple variants in any one doc). Additionally, a specialized TermQuery is used for variants and does not use that variant term's IDF because this would favour rarer terms eg misspellings. Instead, all variants use the same IDF ranking (the one for the source query term) and this is factored into the variant's boost. If the source query term does not exist in the index the average IDF of the variants is used.

Examples of org.apache.lucene.sandbox.queries.FuzzyLikeThisQuery


  @Override
  public Query getQuery(Element e) throws ParserException {
    NodeList nl = e.getElementsByTagName("Field");
    int maxNumTerms = DOMUtils.getAttribute(e, "maxNumTerms", DEFAULT_MAX_NUM_TERMS);
    FuzzyLikeThisQuery fbq = new FuzzyLikeThisQuery(maxNumTerms, analyzer);
    fbq.setIgnoreTF(DOMUtils.getAttribute(e, "ignoreTF", DEFAULT_IGNORE_TF));


    for (int i = 0; i < nl.getLength(); i++) {
      Element fieldElem = (Element) nl.item(i);
      float minSimilarity = DOMUtils.getAttribute(fieldElem, "minSimilarity", DEFAULT_MIN_SIMILARITY);
      int prefixLength = DOMUtils.getAttribute(fieldElem, "prefixLength", DEFAULT_PREFIX_LENGTH);
      String fieldName = DOMUtils.getAttributeWithInheritance(fieldElem, "fieldName");


      String value = DOMUtils.getText(fieldElem);
      fbq.addTerms(value, fieldName, minSimilarity, prefixLength);
    }


    fbq.setBoost(DOMUtils.getAttribute(e, "boost", 1.0f));
    return fbq;
  }

View Full Code Here

Examples of org.apache.lucene.sandbox.queries.FuzzyLikeThisQuery

            } else {
                return null;
            }
        }


        FuzzyLikeThisQuery fuzzyLikeThisQuery = new FuzzyLikeThisQuery(maxNumTerms, analyzer);
        fuzzyLikeThisQuery.addTerms(likeText, fieldName, fuzziness.asSimilarity(), prefixLength);
        fuzzyLikeThisQuery.setBoost(boost);
        fuzzyLikeThisQuery.setIgnoreTF(ignoreTF);


        // move to the next end object, to close the field name
        token = parser.nextToken();
        if (token != XContentParser.Token.END_OBJECT) {
            throw new QueryParsingException(parseContext.index(), "[flt_field] query malformed, no end_object");

View Full Code Here

Examples of org.apache.lucene.sandbox.queries.FuzzyLikeThisQuery


        if (analyzer == null) {
            analyzer = parseContext.mapperService().searchAnalyzer();
        }


        FuzzyLikeThisQuery query = new FuzzyLikeThisQuery(maxNumTerms, analyzer);
        if (fields == null) {
            fields = Lists.newArrayList(parseContext.defaultField());
        } else if (fields.isEmpty()) {
            throw new QueryParsingException(parseContext.index(), "fuzzy_like_this requires 'fields' to be non-empty");
        }
        for (Iterator<String> it = fields.iterator(); it.hasNext(); ) {
            final String fieldName = it.next();
            if (!Analysis.generatesCharacterTokenStream(analyzer, fieldName)) {
                if (failOnUnsupportedField) {
                    throw new ElasticsearchIllegalArgumentException("more_like_this doesn't support binary/numeric fields: [" + fieldName + "]");
                } else {
                    it.remove();
                }
            }
        }
        if (fields.isEmpty()) {
            return null;
        }
        float minSimilarity = fuzziness.asFloat();
        if (minSimilarity >= 1.0f && minSimilarity != (int)minSimilarity) {
            throw new ElasticsearchIllegalArgumentException("fractional edit distances are not allowed");
        }
        if (minSimilarity < 0.0f)  {
            throw new ElasticsearchIllegalArgumentException("minimumSimilarity cannot be less than 0");
        }
        for (String field : fields) {
            query.addTerms(likeText, field, minSimilarity, prefixLength);
        }
        query.setBoost(boost);
        query.setIgnoreTF(ignoreTF);


        if (queryName != null) {
            parseContext.addNamedQuery(queryName, query);
        }
        return query;

View Full Code Here

Examples of org.apache.lucene.search.FuzzyLikeThisQuery

        }
        if (analyzer == null) {
            analyzer = parseContext.mapperService().searchAnalyzer();
        }


        FuzzyLikeThisQuery query = new FuzzyLikeThisQuery(maxNumTerms, analyzer);
        query.addTerms(likeText, fieldName, minSimilarity, prefixLength);
        query.setBoost(boost);
        query.setIgnoreTF(ignoreTF);


        // move to the next end object, to close the field name
        token = parser.nextToken();
        assert token == XContentParser.Token.END_OBJECT;

View Full Code Here

Examples of org.apache.lucene.search.FuzzyLikeThisQuery


        if (analyzer == null) {
            analyzer = parseContext.mapperService().searchAnalyzer();
        }


        FuzzyLikeThisQuery query = new FuzzyLikeThisQuery(maxNumTerms, analyzer);
        if (fields == null) {
            // add the default _all field
            query.addTerms(likeText, AllFieldMapper.NAME, minSimilarity, prefixLength);
        } else {
            for (String field : fields) {
                query.addTerms(likeText, field, minSimilarity, prefixLength);
            }
        }
        query.setBoost(boost);
        query.setIgnoreTF(ignoreTF);


        return query;
    }

View Full Code Here

Examples of org.apache.lucene.search.FuzzyLikeThisQuery


  public Query getQuery(Element e) throws ParserException
  {
    NodeList nl = e.getElementsByTagName("Field");
    int maxNumTerms=DOMUtils.getAttribute(e,"maxNumTerms",defaultMaxNumTerms);
    FuzzyLikeThisQuery fbq=new FuzzyLikeThisQuery(maxNumTerms,analyzer);
    fbq.setIgnoreTF(DOMUtils.getAttribute(e,"ignoreTF",defaultIgnoreTF));
    for(int i=0;i<nl.getLength();i++)
    {
      Element fieldElem=(Element) nl.item(i);
      float minSimilarity=DOMUtils.getAttribute(fieldElem,"minSimilarity",defaultMinSimilarity);
      int prefixLength=DOMUtils.getAttribute(fieldElem,"prefixLength",defaultPrefixLength);
      String fieldName=DOMUtils.getAttributeWithInheritance(fieldElem,"fieldName");
      
      String value=DOMUtils.getText(fieldElem);
      fbq.addTerms(value,fieldName,minSimilarity,prefixLength);
    }
    fbq.setBoost(DOMUtils.getAttribute(e,"boost",1.0f));


    return fbq;
  }

View Full Code Here

Examples of org.apache.lucene.search.FuzzyLikeThisQuery


  public Query getQuery(Element e) throws ParserException
  {
    NodeList nl = e.getElementsByTagName("Field");
    int maxNumTerms=DOMUtils.getAttribute(e,"maxNumTerms",defaultMaxNumTerms);
    FuzzyLikeThisQuery fbq=new FuzzyLikeThisQuery(maxNumTerms,analyzer);
    fbq.setIgnoreTF(DOMUtils.getAttribute(e,"ignoreTF",defaultIgnoreTF));
    for(int i=0;i<nl.getLength();i++)
    {
      Element fieldElem=(Element) nl.item(i);
      float minSimilarity=DOMUtils.getAttribute(fieldElem,"minSimilarity",defaultMinSimilarity);
      int prefixLength=DOMUtils.getAttribute(fieldElem,"prefixLength",defaultPrefixLength);
      String fieldName=DOMUtils.getAttributeWithInheritance(fieldElem,"fieldName");
      
      String value=DOMUtils.getText(fieldElem);
      fbq.addTerms(value,fieldName,minSimilarity,prefixLength);
    }
    fbq.setBoost(DOMUtils.getAttribute(e,"boost",1.0f));


    return fbq;
  }

View Full Code Here

Examples of org.apache.lucene.search.FuzzyLikeThisQuery


  public Query getQuery(Element e) throws ParserException
  {
    NodeList nl = e.getElementsByTagName("Field");
    int maxNumTerms=DOMUtils.getAttribute(e,"maxNumTerms",defaultMaxNumTerms);
    FuzzyLikeThisQuery fbq=new FuzzyLikeThisQuery(maxNumTerms,analyzer);
    fbq.setIgnoreTF(DOMUtils.getAttribute(e,"ignoreTF",defaultIgnoreTF));
    for(int i=0;i<nl.getLength();i++)
    {
      Element fieldElem=(Element) nl.item(i);
      float minSimilarity=DOMUtils.getAttribute(fieldElem,"minSimilarity",defaultMinSimilarity);
      int prefixLength=DOMUtils.getAttribute(fieldElem,"prefixLength",defaultPrefixLength);
      String fieldName=DOMUtils.getAttributeWithInheritance(fieldElem,"fieldName");
      
      String value=DOMUtils.getText(fieldElem);
      fbq.addTerms(value,fieldName,minSimilarity,prefixLength);
    }
    fbq.setBoost(DOMUtils.getAttribute(e,"boost",1.0f));


    return fbq;
  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.