Examples of org.archive.access.nutch.jobs.NutchwaxLinkDb

org.archive.access.nutch.jobs.NutchwaxLinkDb
Subclass of nutch indexer that writes out LinkDb keys that include the collection name. Bulk of code is a copy and paste from LinkDb. LinkDb is not amenable to subclassing. @author stack

  protected void doInvert(final OutputDirectories od, final Path [] segments)
    throws IOException
  {
    createLinkdb(od);
      
    new NutchwaxLinkDb(getJobConf()).
      invert(od.getLinkDb(), segments, true, true, false);
  }

View Full Code Here

  protected void doInvert(final OutputDirectories od)
    throws IOException
  {
    LOG.info("inverting links in " + od.getSegments());


    new NutchwaxLinkDb(getJobConf()).
      invert(od.getLinkDb(), getSegments(od), true, true, false);
  }

View Full Code Here

TOP

Related Classes of org.archive.access.nutch.jobs.NutchwaxLinkDb

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.Path

org.apache.hadoop.io.Text

org.apache.hadoop.mapred.JobConf

org.apache.nutch.crawl.Inlink

org.apache.nutch.crawl.Inlinks

org.apache.nutch.net.URLFilters

org.apache.nutch.net.URLNormalizers

org.apache.nutch.parse.Outlink

org.apache.nutch.parse.ParseData

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.