Examples of WikipediaDocnoMapping

edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping
Provides a mapping between Wikipedia internal ids (docids) and sequentially-numbered ints (docnos).

The main of this class provides a simple program for accessing docno mappings. Command-line arguments are as follows:
- getDocno, getDocid: get docno from docid or, get docid from docno
- [mappings-file]: the mappings file
- [docid/docno]: the docid or docno (optional)
@author Jimmy Lin

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }
      
      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

        sLogger.info("Loaded "+samplesMap.size()+" samples");
      }else{
        sLogger.info("No sample file read.");
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }
      
      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

        "-" + WikipediaDocnoMappingBuilder.LANGUAGE_OPTION + "=" + language
    };


    List<Integer> counts = IntegrationUtils.execWiki(Joiner.on(" ").join(args));


    WikipediaDocnoMapping mapping = new WikipediaDocnoMapping();
    mapping.loadMapping(new Path(mappingFile), fs);


    System.out.println("DISAMBIGUATION = " + numDisamb + "; ARTICLE = " + numArticles + "; TOTAL = " + total);
    System.out.println("DOCNO 0 = " + mapping.getDocid(1));
    System.out.println("DOCNO 100000 = " + mapping.getDocid(100000));


    // docno to docid
    assertEquals(docid1, mapping.getDocid(1));
    assertEquals(docid2, mapping.getDocid(100000));


    // docid to docno
    assertEquals(1, mapping.getDocno(docid1));
    assertEquals(100000, mapping.getDocno(docid2));
    
    // # of disamb pages
    assertEquals(numDisamb, (int) counts.get(0));
    // # of articles
    assertEquals(numArticles, (int) counts.get(1));

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

        sLogger.info("Loaded " + samplesMap.size() + " samples");
      } else {
        sLogger.info("No sample file read.");
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

        sLogger.info("Loaded " + samplesMap.size() + " samples");
      } else {
        sLogger.info("No sample file read.");
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

Examples of edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

          reader.close();
        } catch (IOException e1) {
        }
      }


      mDocMapping = new WikipediaDocnoMapping();
      FileSystem fs;
      try {
        fs = FileSystem.get(job);
        String indexPath = job.get("Ivory.IndexPath");
        RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.