Examples of ivory.core.eval.Qrels

ivory.core.eval.Qrels

Representation of relevance judgments. In TREC parlance, qrels are judgments made by humans as to whether a document is relevant to an information need (i.e., topic). Typically, qrels are created by a process known as "pooling" in large-scale system evaluations such as those at TREC.
@author Jimmy Lin

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/wt10g/qrels.wt10g.all"));
  }

View Full Code Here

    qr.runQueries();
    long end = System.currentTimeMillis();
    LOG.info("Total query time: " + (end - start) + "ms");


    verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/trec/qrels.robust04.noCRFR.txt"));
  }

View Full Code Here


    Map<String, Map<String, Float>> AllModelsP10Scores = Maps.newHashMap();
    AllModelsP10Scores.put("robust04-dir-wsd-sd", loadScoresIntoMap(sDir_WSD_SD_RawP10));
    AllModelsP10Scores.put("robust04-dir-wsd-fd", loadScoresIntoMap(sDir_WSD_FD_RawP10));


    sQrels = new Qrels("data/trec/qrels.robust04.noCRFR.txt");


    String[] params = new String[] {
            "data/trec/run.robust04.wsd.xml",
            "data/trec/queries.robust04.xml" };

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Gov2_Basic.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/gov2/qrels.gov2.all"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Gov2_NonPositional_Baselines.verifyAllResults(qr.getModels(), qr.getAllResults(),
        qr.getDocnoMapping(), new Qrels("data/gov2/qrels.gov2.all"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Robust04_Basic.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/trec/qrels.robust04.noCRFR.txt"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Web09catB_All.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/clue/qrels.web09catB.txt"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Wt10g_Basic.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/wt10g/qrels.wt10g.all"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Web09catB_All.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/clue/qrels.web09catB.txt"));


    LOG.info("Done!");
  }

View Full Code Here

    long end = System.currentTimeMillis();


    LOG.info("Total query time: " + (end - start) + "ms");


    Robust04_Basic.verifyAllResults(qr.getModels(), qr.getAllResults(), qr.getDocnoMapping(),
        new Qrels("data/trec/qrels.robust04.noCRFR.txt"));


    LOG.info("Done!");
  }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of ivory.core.eval.Qrels

ivory.core.util.DelimitedValuesFileReader

ivory.integration.adhoc.VerifyClueNonPositionalIndexIP

ivory.integration.adhoc.VerifyCluePositionalIndexIP

ivory.integration.adhoc.VerifyGenomics05PositionalIndexIP

ivory.integration.adhoc.VerifyGov2NonPositionalIndexIP

ivory.integration.adhoc.VerifyGov2PositionalIndexIP

ivory.integration.adhoc.VerifyTrec45NonPositionalIndexIP

ivory.integration.adhoc.VerifyTrec45PositionalIndexIP

ivory.integration.adhoc.VerifyTrec45PositionalIndexLP

ivory.integration.adhoc.VerifyTrecNonPositionalIndexIP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.