Examples of edu.stanford.nlp.stats.EquivalenceClassEval

edu.stanford.nlp.stats.EquivalenceClassEval

A class for calculating precision and recall statistics based on comparisons between two {@link Collection}s. Allows flexible specification of:

The criterion by which to evaluate whether two Objects are equivalent for purposes of precision and recall calculation (specified by an {@link EqualityChecker} instance)
The criterion by which Objects are grouped into equivalence classes for purposes of calculating subclass precision and recall (specified by an {@link EquivalenceClasser} instance)
Evaluation is set-based or bag-based (by default, it is set-based). For example, if a gold collection has {a,a,b} and a guess collection has {a,b}, then recall is 100% in set-based evaluation, but is 66.67% in bag-based evaluation.

Note that for set-based evaluation, sets are always constructed using object equality, NOT equality on the basis of an {@link EqualityChecker} if one is given. If set-based evaluationwere conducted on the basis of an EqualityChecker, then there would be indeterminacy when it did not subsume the {@link EquivalenceClasser}, if one was given. For example, if objects of the form X:y were equivalence-classed by the left criterion and evaluated for equality on the right, then set-based evaluation based on the equality checker would be indeterminate for a collection of {A:a,B:a} because it would be unclear whether to use the first or second element of the collection. @author Roger Levy @author Sarah Spikes (sdspikes@cs.stanford.edu) Attempt at templatization... this may be a failure.

    MemoryTreebank testTreebank = ctpp.memoryTreebank();
    testTreebank.loadPath(new File(args[0]), testFilt);
    PrintWriter pw = new PrintWriter(new OutputStreamWriter(new FileOutputStream("out.chi"), "GB18030"), true);
    WordCatEquivalenceClasser eqclass = new WordCatEquivalenceClasser();
    WordCatEqualityChecker eqcheck = new WordCatEqualityChecker();
    EquivalenceClassEval eval = new EquivalenceClassEval(eqclass, eqcheck);
    //    System.out.println("Preterminals:" + preterminals);
    System.out.println("Testing...");
    for (Tree gold : testTreebank) {
      Tree tree;
      try {
        tree = lp.parseTree(gold.yieldHasWord());
        if (tree == null) {
          System.out.println("Failed to parse " + gold.yieldHasWord());
          continue;
        }
      } catch (Exception e) {
        e.printStackTrace();
        continue;
      }
      gold = gold.firstChild();
      pw.println(Sentence.listToString(gold.preTerminalYield()));
      pw.println(Sentence.listToString(gold.yield()));
      gold.pennPrint(pw);


      pw.println(tree.preTerminalYield());
      pw.println(tree.yield());
      tree.pennPrint(pw);
      //      Collection allBrackets = WordCatConstituent.allBrackets(tree);
      //      Collection goldBrackets = WordCatConstituent.allBrackets(gold);
      //      eval.eval(allBrackets, goldBrackets);
      eval.displayLast();
    }
    System.out.println();
    System.out.println();
    eval.display();
  }

Examples of edu.stanford.nlp.stats.EquivalenceClassEval

Related Classes of edu.stanford.nlp.stats.EquivalenceClassEval