Examples of joshua.decoder.NbestMinRiskReranker

joshua.decoder.NbestMinRiskReranker

this class implements: (1) nbest min risk (MBR) reranking using BLEU as a gain funtion.

This assume that the string is unique in the nbest list In Hiero, due to spurious ambiguity, a string may correspond to many possible derivations, and ideally the probability of a string should be the sum of all the derivataions leading to that string. But, in practice, one normally uses a Viterbi approximation: the probability of a string is its best derivation probability So, if one want to deal with spurious ambiguity, he/she should do that before calling this class @author Zhifei Li, @version $LastChangedDate: 2010-01-07 22:36:11 -0600 (Thu, 07 Jan 2010) $

  
    int ngramStateID = 0;
    SymbolTable symbolTbl = new BuildinSymbol(null);;
    
    NbestCrunching cruncher = new NbestCrunching(symbolTbl, scalingFactor, topN);
    NbestMinRiskReranker mbrReranker = new NbestMinRiskReranker(false, 1.0);
  
    BufferedWriter onebestWriter =  FileUtilityOld.getWriteFileStream(onebestFile);  
    
    System.out.println("############Process file  " + testItemsFile);
    DiskHyperGraph diskHG = new DiskHyperGraph(symbolTbl, ngramStateID, true, null); //have model costs stored
    diskHG.initRead(testItemsFile, testRulesFile,null);      
    for(int sentID=0; sentID < numSents; sentID ++){
      System.out.println("#Process sentence " + sentID);
      HyperGraph testHG = diskHG.readHyperGraph();
      
      List<String> nbest = cruncher.processOneSent(testHG, sentID, true);//produce the disorder nbest
      
      String bestHyp = mbrReranker.processOneSent(nbest, sentID);//nbest: list of unique strings
      
      FileUtilityOld.writeLzf(onebestWriter, bestHyp + "\n");
    }
    FileUtilityOld.closeWriteFile(onebestWriter);        
  }

Examples of joshua.decoder.NbestMinRiskReranker

Related Classes of joshua.decoder.NbestMinRiskReranker