Examples of com.google.enterprise.connector.spi.DocumentList

Package com.google.enterprise.connector.spi

Examples of com.google.enterprise.connector.spi.DocumentList

com.google.enterprise.connector.spi.DocumentList
Interface that represents a list of documents to be traversed. Documents are accessed through an iterator-like method: {@link #nextDocument()}, which returns the next available {@link Document} or {@code null} if there are no more.
Important: a {@link Document} object obtainedby calling {@link #nextDocument()} is invalidated by the nextcall to {@link #nextDocument()}. Typically, the caller will store the current Document in a loop variable, so that it is clear that this rule is observed; see the example code below.
In addition, a this interface has a special method {@link #checkpoint()}, which produces a String that encapsulates the traversal state. The consumer may call {@link #checkpoint()} at any time. The implementation shouldreturn a non-null String that, if supplied to {@link TraversalManager#resumeTraversal(String)}, would cause traversal to resume from the next unprocessed document.

Boundary cases are important for {@link #checkpoint()}:

If {@link #checkpoint()} is called before any calls to{@link #nextDocument()}, then traversal should resume with the Document that would have been returned by the first call to {@link #nextDocument()}.
If {@link #checkpoint()} is called after a call to{@link #nextDocument()} that returns a valid document, thentraversal should resume with the Document that would have been returned by the next call to {@link #nextDocument()}.
If {@link #checkpoint()} is not called, then traversal shouldresume from the previous checkpoint, as if none of the documents in this {@code DocumentList} had been processed.

The typical pattern for consuming an object that implements this interface is as follows (disregarding exception handling):

 DocumentList docList = ... Document doc; while (doc = docList.nextDocument()) { handleDoc(doc); if (whatever reason) break; } String check = doclist.checkpoint();

Note: because of the restriction that the next call to {@link #nextDocument()} invalidates the previous Document, and thereare similar restrictions in the {@link Document} interface, it is possibleto provide a single stateful object that implements {@link DocumentList}, {@link Document} and {@link Property}, by returning {@code this}(or {@code null}) to all calls to {@link #nextDocument()} and{@link Document#findProperty(String)}. However, if preferred, the implementor may also use separate objects for each of those interfaces. @since 1.0

    traversalContext.setTraversalTimeLimitSeconds(1);
    mtm.setTraversalContext(traversalContext);
    String checkpoint = "{index:0,uuid:['one','another'],"
        + "lastModified=['2010-04-01 00:01:00','2010-10-27 22:55:03']}";


    DocumentList documentList = mtm.resumeTraversal(checkpoint);
    assertNotNull(documentList);
    assertNull(documentList.nextDocument());
    assertNotNull(documentList.checkpoint()); // It's not MockDocumentList.
    assertEquals(1, mtm.count); // Would have been 2 without the timeout.
  }

View Full Code Here

   * Test method for
   * 'com.google.enterprise.connector.dctm.DctmDocumentList.nextDocument()'
   */
  public void testNextDocument() throws RepositoryException {
    int counter = 0;
    DocumentList propertyMapList = qtm.startTraversal();
    Document pm = null;
    Property prop = null;


    while ((pm = propertyMapList.nextDocument()) != null) {


      assertTrue(pm instanceof DctmSysobjectDocument);
      prop = pm.findProperty(SpiConstants.PROPNAME_DOCID);


      assertNotNull(prop);

View Full Code Here

   */
  public void testCheckpoint() throws RepositoryException {
    String checkPoint = null;


    int counter = 0;
    DocumentList propertyMapList = qtm.startTraversal();


    while ((propertyMapList.nextDocument()) != null) {
      counter++;
    }
    assertEquals(DmInitialize.DM_RETURN_TOP_UNBOUNDED, counter);


    // XXX: This time comes back as if the timestamp value were in
    // seconds, but in MockDmTimeTest we get milliseconds. Also, this
    // time is local time. Should it be UTC?
    SimpleDateFormat iso8601 =
      new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
    String lastModified = iso8601.format(new Date(260 * 1000));


    checkPoint = propertyMapList.checkpoint();
    assertTrue(checkPoint,
        checkPoint.indexOf("\"uuid\":[\"doc26\"]") != -1);
    assertTrue(checkPoint,
        checkPoint.indexOf("\"lastModified\":[\"" + lastModified + "\"]") != -1);
  }

View Full Code Here

   * Tests document containing no parent ACL document, no inherit-from field.
   */
  @Test
  public void documentContainsDirectDefaultAcls() throws Exception {
    String[][] data = {{expectedId, timeStr}};
    @SuppressWarnings("unchecked") DocumentList doclist =
        getDocumentList(data, directAces, defaultAces);


    Document doc = doclist.nextDocument();
    assertTrue(doc instanceof FileDocument);
    Property inheritFrom =
        doc.findProperty(SpiConstants.PROPNAME_ACLINHERITFROM_DOCID);
    assertNull(inheritFrom);
    assertDocContainsDirectDefaultAces(doc, 0);


    assertNull("Document should not have TMPL or FLDR ACL document",
        doclist.nextDocument());
  }

View Full Code Here

   * existed.
   */
  @Test
  public void documentContainsDirectDefaultTemplateAcls() throws Exception {
    String[][] data = {{expectedId, timeStr}};
    @SuppressWarnings("unchecked") DocumentList doclist =
        getDocumentList(data, directAces, defaultAces, templateAces);


    Document doc = doclist.nextDocument();
    assertTrue(doc instanceof FileDocument);
    assertDocInheritFrom(doc, expectedId + AclDocument.SEC_POLICY_POSTFIX);
    assertDocContainsDirectDefaultAces(doc, 0);


    Document acl = doclist.nextDocument();
    assertTrue(acl instanceof AclDocument);
    assertEquals(expectedId + AclDocument.SEC_POLICY_POSTFIX,
        Value.getSingleValueString(acl, SpiConstants.PROPNAME_DOCID));
    assertAclContainsAces(acl, PermissionSource.SOURCE_TEMPLATE, 0);


    assertNull(doclist.nextDocument());
  }

View Full Code Here

   * existed.
   */
  @Test
  public void documentContainsDirectDefaultParentAcls() throws Exception {
    String[][] data = {{expectedId, timeStr}};
    @SuppressWarnings("unchecked") DocumentList doclist =
        getDocumentList(data, directAces, defaultAces, parentAces);


    Document doc = doclist.nextDocument();
    assertTrue(doc instanceof FileDocument);
    assertDocInheritFrom(doc, expectedId + AclDocument.SEC_FOLDER_POSTFIX);
    assertDocContainsDirectDefaultAces(doc, 0);


    Document acl = doclist.nextDocument();
    assertTrue(acl instanceof AclDocument);
    assertEquals(expectedId + AclDocument.SEC_FOLDER_POSTFIX,
        Value.getSingleValueString(acl, SpiConstants.PROPNAME_DOCID));
    assertAclContainsAces(acl, PermissionSource.SOURCE_PARENT, 0);


    assertNull(doclist.nextDocument());
  }

View Full Code Here

   */
  @Test
  public void documentContainsDirectDefaultTemplateParentAcls()
      throws Exception {
    String[][] data = {{expectedId, timeStr}};
    @SuppressWarnings("unchecked") DocumentList doclist =
        getDocumentList(data, directAces, defaultAces, templateAces,
            parentAces);


    Document doc = doclist.nextDocument();
    assertTrue(doc instanceof FileDocument);
    assertDocInheritFrom(doc, expectedId + AclDocument.SEC_POLICY_POSTFIX);
    assertDocContainsDirectDefaultAces(doc, 0);


    Document fldrAcl = doclist.nextDocument();
    assertTrue(fldrAcl instanceof AclDocument);
    assertAclContainsAces(fldrAcl, PermissionSource.SOURCE_PARENT, 0);


    Document tmplAcl = doclist.nextDocument();
    assertTrue(tmplAcl instanceof AclDocument);
    assertDocInheritFrom(tmplAcl, expectedId + AclDocument.SEC_FOLDER_POSTFIX);
    assertAclContainsAces(tmplAcl, PermissionSource.SOURCE_TEMPLATE, 0);


    assertNull(doclist.nextDocument());
  }

View Full Code Here

    conn.setIncludedLocationNodes(getStartNodes());


    TraversalManager mgr = sess.getTraversalManager();
    mgr.setBatchHint(3000);


    DocumentList rs = mgr.startTraversal();
    processResultSet(rs);
  }

View Full Code Here

  public void testPerformance(int dummy) throws Exception {
    // Traverse the repository for a bunch of ids.
    final int batchHint = 100000;
    TraversalManager tm = session.getTraversalManager();
    tm.setBatchHint(batchHint);
    DocumentList docList = tm.startTraversal();
    ArrayList<String> docids = new ArrayList<String>();
    Document doc;
    while ((doc = docList.nextDocument()) != null) {
      Property p = doc.findProperty(SpiConstants.PROPNAME_DOCID);
      docids.add(p.nextValue().toString());
    }


    long start = 0;

View Full Code Here

        objectId - 1);


    // Now push that one document
    TraversalManager mgr = sess.getTraversalManager();
    mgr.setBatchHint(1);
    DocumentList rs = mgr.resumeTraversal(checkpoint.toString() + ',');
    processResultSet(rs);
  }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of com.google.enterprise.connector.spi.DocumentList

com.google.enterprise.connector.adgroups.AdGroupsTraversalManagerTest

com.google.enterprise.connector.dctm.DctmMockDocumentListTest

com.google.enterprise.connector.dctm.DctmMockTraversalManagerTest

com.google.enterprise.connector.dctm.DctmTraversalManager

com.google.enterprise.connector.dctm.DctmTraversalManagerTest

com.google.enterprise.connector.dctm.DctmTraversalUtil

com.google.enterprise.connector.filenet4.AclDocumentTest

com.google.enterprise.connector.filenet4.FileDocumentListTest

com.google.enterprise.connector.filenet4.FileDocumentTraverserTest

com.google.enterprise.connector.filenet4.FileTraversalManagerTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.