Examples of org.apache.nutch.db.Page

org.apache.nutch.db.Page

A row in the Page Database.

 type   name    description --------------------------------------------------------------- byte   VERSION  - A byte indicating the version of this entry. String URL      - The url of a page.  This is the primary key. 128bit ID       - The MD5 hash of the contents of the page. 64bit  DATE     - The date this page should be refetched. byte   RETRIES  - The number of times we've failed to fetch this page. byte   INTERVAL - Frequency, in days, this page should be refreshed. float  SCORE   - Multiplied into the score for hits on this page. float  NEXTSCORE   - Multiplied into the score for hits on this page.

@author Mike Cafarella @author Doug Cutting


  public ProtocolOutput getProtocolOutput(String urlString) {
    ProtocolOutput output = null;
    try {
      return getProtocolOutput(new FetchListEntry(true,
            new Page(urlString, 1.0f), new String[0]));
    } catch (MalformedURLException mue) {
      return new ProtocolOutput(null, new ProtocolStatus(mue));
    }
  }

View Full Code Here


  public ProtocolOutput getProtocolOutput(String urlString) {
    ProtocolOutput output = null;
    try {
      return getProtocolOutput(new FetchListEntry(true,
            new Page(urlString, 1.0f), new String[0]));
    } catch (MalformedURLException mue) {
      return new ProtocolOutput(null, new ProtocolStatus(mue));
    }
  }

View Full Code Here


  public ProtocolOutput getProtocolOutput(String urlString) {
    ProtocolOutput output = null;
    try {
      return getProtocolOutput(new FetchListEntry(true,
            new Page(urlString, 1.0f), new String[0]));
    } catch (MalformedURLException mue) {
      return new ProtocolOutput(null, new ProtocolStatus(mue));
    }
  }

View Full Code Here

    }
  }


  public ProtocolOutput getProtocolOutput(String urlString) {
    try {
      return getProtocolOutput(new FetchListEntry(true, new Page(urlString, 1.0f), new String[0]));
    } catch (MalformedURLException mue) {
      return new ProtocolOutput(null, new ProtocolStatus(mue));
    }
  }

View Full Code Here

      if (unique) {
        rnd = "/" + System.currentTimeMillis();
        url += rnd;
      }
      url += "/example.html";
      FetchListEntry fle = new FetchListEntry(true, new Page(url, 1.0f), new String[] { "test" + rnd });
      FetcherOutput fo = new FetcherOutput(fle, MD5Hash.digest(url), ProtocolStatus.STATUS_SUCCESS);
      StringBuffer content = new StringBuffer("<html><body><h1>Hello from Page " + i + "</h1>");
      if (unique) {
        content.append("<p>Created at epoch time: " + System.currentTimeMillis() + ", " + r.nextLong() + "</p>");
      }

View Full Code Here

TOP

Related Classes of org.apache.nutch.db.Page

org.apache.nutch.protocol.file.File

org.apache.nutch.protocol.ftp.Ftp

org.apache.nutch.protocol.http.Http

org.apache.nutch.protocol.httpclient.Http

org.apache.nutch.tools.TestSegmentMergeTool

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.