Examples of joshua.decoder.hypergraph.DiskHyperGraph

joshua.decoder.hypergraph.DiskHyperGraph
this class implements functions of writting/reading hypergraph on disk. Limitations of this version (1) cannot recover each individual feature, notably the LM feature (2) assume we only have one stateful featuure, which must be a LM feature @author Zhifei Li, @author wren ng thornton @version $LastChangedDate: 2010-02-10 09:46:05 -0600 (Wed, 10 Feb 2010) $

  
   
   private void initDiskReading(){
    logger.info("initialize reading hypergraphss..............");
     
    diskHG = new DiskHyperGraph(symbolTbl, ngramStateID, true, null); //have model costs stored
        diskHG.initRead(diskHGFilePrefix+".hg.items", diskHGFilePrefix+".hg.rules", null);
        
        //=== references files, they are needed only when we want annote the hypergraph with risk   
        if(this.readReferences){
          refFileReaders = new BufferedReader[referenceFiles.length];

View Full Code Here

    NbestCrunching cruncher = new NbestCrunching(symbolTbl, insideOutsideScalingFactor, topN);    
  
    BufferedWriter onebestWriter =  FileUtilityOld.getWriteFileStream(onebestFile);  
    
    System.out.println("############Process file  " + testItemsFile);
    DiskHyperGraph diskHG = new DiskHyperGraph(symbolTbl, ngramStateID, true, null); //have model costs stored
    diskHG.initRead(testItemsFile, testRulesFile,null);      
    for(int sentID=0; sentID < numSents; sentID ++){
      System.out.println("#Process sentence " + sentID);
      HyperGraph testHG = diskHG.readHyperGraph();      
      List<String> oneBest = cruncher.processOneSent(testHG, sentID, false);//produce the reranked onebest
      FileUtilityOld.writeLzf(onebestWriter, oneBest.get(0) + "\n");
    }
    FileUtilityOld.closeWriteFile(onebestWriter);        
  }

View Full Code Here

    EstimateEntropyGapOnNbest cruncher = new EstimateEntropyGapOnNbest(p_symbol, inside_outside_scaling_factor, topN);    
  
    //BufferedWriter t_writer_1best =  FileUtilityOld.getWriteFileStream(f_1best);  
    
    System.out.println("############Process file  " + f_test_items);
    DiskHyperGraph dhg_test = new DiskHyperGraph(p_symbol, baseline_lm_feat_id, true, null); //have model costs stored
    dhg_test.initRead(f_test_items, f_test_rules,null);    
    double sumGap = 0;
    for(int sent_id=0; sent_id < num_sents; sent_id ++){
      System.out.println("#Process sentence " + sent_id);
      HyperGraph hg_test = dhg_test.readHyperGraph();      
      double gap = cruncher.processOneSent(hg_test, sent_id, false);//produce the reranked onebest
      sumGap += gap;
    }
    sumGap *= 1.44;
    System.out.println("sum of bits in gap is: " +  sumGap);

View Full Code Here

0 1 2

TOP

Related Classes of joshua.decoder.hypergraph.DiskHyperGraph

joshua.corpus.vocab.BuildinSymbol

joshua.corpus.vocab.SymbolTable

joshua.decoder.DecoderThread

joshua.decoder.ff.state_maintenance.NgramDPState

joshua.decoder.ff.tm.hiero.DiskHyperGraphFormatReader

joshua.decoder.ff.tm.Rule

joshua.discriminative.feature_related.FeatureIntersection

joshua.discriminative.feature_related.FeatureSelectionHG

joshua.discriminative.ranker.HGRanker

joshua.discriminative.ranker.RescorerHGSimple

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.