Examples of org.exist.storage.dom.DOMFile

org.exist.storage.dom.DOMFile
This is the main storage for XML nodes. Nodes are stored in document order. Every document gets its own sequence of pages, which is bound to the writing thread to avoid conflicting writes. The page structure is as follows: | page header | (tid1 node-data, tid2 node-data, ..., tidn node-data) | node-data contains the raw binary data of the node. Within a page, a node is identified by a unique id, called tuple id (tuple id). Every node can thus be located by a virtual address pointer, which consists of the page id and the tid. Both components are encoded in a long value (with additional bits used for optional flags). The address pointer is used to reference nodes from the indexes. It should thus remain unchanged during the life-time of a document. However, XUpdate requests may insert new nodes in the middle of a page. In these cases, the page will be split and the upper portion of the page is copied to a split page. The record in the original page will be replaced by a forward link, pointing to the new location of the node data in the split page. As a consequence, the class has to distinguish three different types of data records: 1) Ordinary record: | tuple id | length | data | 3) Relocated record: | tuple id | length | address pointer to original location | data | 2) Forward link: | tuple id | address pointer | tuple id and length each use two bytes (short), address pointers 8 bytes (long). The upper two bits of the tuple id are used to indicate the type of the record (see {@link org.exist.storage.dom.ItemId}). @author Wolfgang Meier

     *
     * @param doc the document object to check
     * @return
     */
    public ErrorReport checkDocument(final DocumentImpl doc) {
        final DOMFile domDb = ( (NativeBroker)broker ).getDOMFile();
        return (ErrorReport)new DOMTransaction( this, domDb, Lock.WRITE_LOCK, doc ) {
            public Object start() {
                EmbeddedXMLStreamReader reader = null;
                try {
                    final ElementImpl root = (ElementImpl)doc.getDocumentElement();

View Full Code Here

     *
     * @return  null if the document is consistent, an error report otherwise.
     */
    public ErrorReport checkXMLTree( final DocumentImpl doc )
    {
        final DOMFile domDb = ( (NativeBroker)broker ).getDOMFile();
        return( (ErrorReport)new DOMTransaction( this, domDb, Lock.WRITE_LOCK, doc ) {
                    public Object start() {
                        EmbeddedXMLStreamReader reader = null;
                        try {
                            final ElementImpl             root            = (ElementImpl)doc.getDocumentElement();
                            reader = broker.getXMLStreamReader( root, true );
                            NodeId                  nodeId;
                            boolean                 attribsAllowed  = false;
                            int                     expectedAttribs = 0;
                            int                     attributeCount  = 0;


                            while( reader.hasNext() ) {
                                final int status = reader.next();


                                nodeId = (NodeId)reader.getProperty( EmbeddedXMLStreamReader.PROPERTY_NODE_ID );
                                ElementNode parent = null;


                                if( ( status != XMLStreamReader.END_ELEMENT ) && !elementStack.isEmpty() ) {
                                    parent = elementStack.peek();
                                    parent.childCount++;


                                    // test parent-child relation
                                    if( !nodeId.isChildOf( parent.elem.getNodeId() ) ) {
                                        return( new ErrorReport.ResourceError( ErrorReport.NODE_HIERARCHY, "Node " + nodeId + " is not a child of " + parent.elem.getNodeId() ) );
                                    }


                                    // test sibling relation
                                    if( ( parent.prevSibling != null ) && !( nodeId.isSiblingOf( parent.prevSibling ) && ( nodeId.compareTo( parent.prevSibling ) > 0 ) ) ) {
                                        return( new ErrorReport.ResourceError( ErrorReport.INCORRECT_NODE_ID, "Node " + nodeId + " is not a sibling of " + parent.prevSibling ) );
                                    }
                                    parent.prevSibling = nodeId;
                                }


                                switch( status ) {


                                    case XMLStreamReader.ATTRIBUTE: {
                                        attributeCount++;
                                        break;
                                    }


                                    case XMLStreamReader.END_ELEMENT: {
                                        if( elementStack.isEmpty() ) {
                                            return( new org.exist.backup.ErrorReport.ResourceError( ErrorReport.NODE_HIERARCHY, "Error in node hierarchy: received END_ELEMENT event " + "but stack was empty!" ) );
                                        }
                                        final ElementNode lastElem = elementStack.pop();
                                        if( lastElem.childCount != lastElem.elem.getChildCount() ) {
                                            return( new ErrorReport.ResourceError( org.exist.backup.ErrorReport.NODE_HIERARCHY, "Element reports incorrect child count: expected " + lastElem.elem.getChildCount() + " but found " + lastElem.childCount ) );
                                        }
                                        break;
                                    }


                                    case XMLStreamReader.START_ELEMENT: {
                                        if( nodeId.getTreeLevel() <= defaultIndexDepth ) {


                                            // check dom.dbx btree, which maps the node
                                            // id to the node's storage address
                                            // look up the node id and check if the
                                            // returned storage address is correct
                                            final NativeBroker.NodeRef nodeRef = new NativeBroker.NodeRef( doc.getDocId(), nodeId );


                                            try {
                                                final long p = domDb.findValue( nodeRef );


                                                if( p != reader.getCurrentPosition() ) {
                                                    final Value v = domDb.get( p );


                                                    if( v == null ) {
                                                        return( new ErrorReport.IndexError( ErrorReport.DOM_INDEX, "Failed to access node " + nodeId + " through dom.dbx index. Wrong storage address. Expected: " + p + "; got: " + reader.getCurrentPosition() + " - ", doc.getDocId() ) );
                                                    }
                                                }

View Full Code Here

      assertNotNull(mgr);
      Txn txn = mgr.beginTransaction();
      assertNotNull(txn);
      System.out.println("Transaction started ...");
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            assertNotNull(domDb);
            domDb.setOwnerObject(this);
            
            // put 1000 values into the btree
            long firstToRemove = -1;
            for (int i = 1; i <= 10000; i++) {
                byte[] data = ("Value" + i).getBytes();
                NodeId id = idFact.createInstance(i);
                long addr = domDb.put(txn, new NativeBroker.NodeRef(500, id), data);
//              TODO : test addr ?
                if (i == 1)
                    firstToRemove = addr;
            }


            domDb.closeDocument();
            
            // remove all
            NativeBroker.NodeRef ref = new NativeBroker.NodeRef(500);
            assertNotNull(ref);
            IndexQuery idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
            assertNotNull(idx);
            domDb.remove(txn, idx, null);
            domDb.removeAll(txn, firstToRemove);
            
            // put some more
            for (int i = 1; i <= 10000; i++) {
                byte[] data = ("Value" + i).getBytes();
                @SuppressWarnings("unused")
        long addr = domDb.put(txn, new NativeBroker.NodeRef(500, idFact.createInstance(i)), data);
//              TODO : test addr ?
            }
            
            domDb.closeDocument();
            mgr.commit(txn);
            System.out.println("Transaction commited ...");
            
            txn = mgr.beginTransaction();
            System.out.println("Transaction started ...");
            
            // put 1000 new values into the btree
            for (int i = 1; i <= 1000; i++) {
                byte[] data = ("Value" + i).getBytes();
                long addr = domDb.put(txn, new NativeBroker.NodeRef(501, idFact.createInstance(i)), data);
//              TODO : test addr ?                
                if (i == 1)
                    firstToRemove = addr;
            }            
            
            domDb.closeDocument();
            mgr.commit(txn);
            System.out.println("Transaction commited ...");
            
            // the following transaction is not committed and will be rolled back during recovery
            txn = mgr.beginTransaction();
            System.out.println("Transaction started ...");
            
            for (int i = 1; i <= 200; i++) {
                domDb.remove(txn, new NativeBroker.NodeRef(500, idFact.createInstance(i)));
            }


            idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, new NativeBroker.NodeRef(501));
            domDb.remove(txn, idx, null);
            domDb.removeAll(txn, firstToRemove);
            
//          Don't commit...
            mgr.commit(txn);
            mgr.getJournal().flushToLog(true);
            System.out.println("Transaction interrupted ...");
            
            Writer writer = new StringWriter();
            domDb.dump(writer);
            System.out.println(writer.toString());
      } catch (Exception e) {
        e.printStackTrace();
          fail(e.getMessage());               
    } finally {

View Full Code Here

            broker = pool.get(pool.getSecurityManager().getSystemSubject());
            assertNotNull(broker);
            TransactionManager mgr = pool.getTransactionManager();
            assertNotNull(mgr);
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            assertNotNull(domDb);
            domDb.setOwnerObject(this);
            
            IndexQuery query = new IndexQuery(IndexQuery.GT, new NativeBroker.NodeRef(500));
            assertNotNull(query);
            List<?> keys = domDb.findKeys(query);
            assertNotNull(keys);
            int count = 0;
            for (Iterator<?> i = keys.iterator(); i.hasNext(); count++) {
                Value key = (Value) i.next();
                assertNotNull(key);
                Value value = domDb.get(key);
                assertNotNull(value);
                System.out.println(new String(value.data(), value.start(), value.getLength()));
            }
            System.out.println("Values read: " + count);
            
            Writer writer = new StringWriter();
            domDb.dump(writer);
            System.out.println(writer.toString());
      } catch (Exception e) {            
          fail(e.getMessage());             
        } finally {
            pool.release(broker);

View Full Code Here

            is.close();
            data = new String(bdata);
            assertNotNull(data);
            System.out.println(data);
            
            DOMFile domDb = ((NativeBroker)broker).getDOMFile();
            assertNotNull(domDb);
            Writer writer = new StringWriter();
            domDb.dump(writer);
            //System.out.println(writer.toString());
            
            transact = pool.getTransactionManager();
            assertNotNull(transact);
            transaction = transact.beginTransaction();

View Full Code Here

                assertNotNull(modifications);
                modifications[0].process(transaction);
                proc.reset();
            }
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            assertNotNull(domDb);
            System.out.println(domDb.debugPages(info.getDocument(), false));
            
            mgr.commit(transaction);
            System.out.println("Transaction commited ...");
            
            // the following transaction will not be committed and thus undone during recovery

View Full Code Here

                assertNotNull(modifications);
                modifications[0].process(transaction);
                proc.reset();
            }
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            assertNotNull(domDb);
            System.out.println(domDb.debugPages(info.getDocument(), false));
            
            mgr.commit(transaction);
            System.out.println("Transaction commited ...");            
            
            // the following transaction will not be committed and thus undone during recovery

View Full Code Here

            broker = pool.get(pool.getSecurityManager().getSystemSubject());
            broker.flush();
            Txn txn = mgr.beginTransaction();
            System.out.println("Transaction started ...");
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            domDb.setOwnerObject(this);
            
            BrokerPool.FORCE_CORRUPTION = true;
            
            // put 1000 values into the btree
            NodeId id;
            for (int i = 1; i < 1001; i++) {
              id = idFact.createInstance(i);
                domDb.addValue(txn, new NativeBroker.NodeRef(500, id), i);
            }
            
            IndexQuery idx = new IndexQuery(IndexQuery.GT, new NativeBroker.NodeRef(500, idFact.createInstance(800)));
            domDb.remove(txn, idx, null);
            
            mgr.commit(txn);
            
            // start a dirty, uncommitted transaction. This will be rolled back by the recovery.
            txn = mgr.beginTransaction();
            
            for (int i = 801; i < 2001; i++) {
                domDb.addValue(txn, new NativeBroker.NodeRef(500, idFact.createInstance(i)), i);
            }
            
            for (int i = 101; i < 301; i++) {
                domDb.addValue(txn, new NativeBroker.NodeRef(500, idFact.createInstance(i)), i * 3);
            }
            
            idx = new IndexQuery(IndexQuery.GT, new NativeBroker.NodeRef(500, idFact.createInstance(600)));
            domDb.remove(txn, idx, null);
            
            mgr.getJournal().flushToLog(true);
            
            Writer writer = new StringWriter();
            domDb.dump(writer);
            System.out.println(writer.toString());
        } catch (Exception e) {
          e.printStackTrace();
            fail(e.getMessage());
        } finally {

View Full Code Here

        NodeIdFactory idFact = pool.getNodeFactory();
        DBBroker broker = null;
        try {
            broker = pool.get(pool.getSecurityManager().getSystemSubject());
            
            DOMFile domDb = ((NativeBroker) broker).getDOMFile();
            domDb.setOwnerObject(this);
            
            IndexQuery query = new IndexQuery(IndexQuery.GEQ, new NativeBroker.NodeRef(500, idFact.createInstance(1)));
            domDb.query(query, new IndexCallback());
            System.out.println("Found: " + count);
            assertEquals(count, 800);
            
            Writer writer = new StringWriter();
            domDb.dump(writer);
            System.out.println(writer.toString());
        } catch (Exception e) {
          e.printStackTrace();
            fail(e.getMessage());
        } finally {

View Full Code Here

            Collection test2 = broker.getOrCreateCollection(transaction, TestConstants.TEST_COLLECTION_URI2);
            assertNotNull(test2); 
            broker.saveCollection(transaction, test2);
            
            System.out.println("Contents of dom.dbx:\n\n");
            DOMFile domDb = ((NativeBroker)broker).getDOMFile();
            assertNotNull(domDb); 
            Writer writer = new StringWriter();
            domDb.dump(writer);
            System.out.println(writer.toString());
            
            File f;
            IndexInfo info;

View Full Code Here

0 1

TOP

Related Classes of org.exist.storage.dom.DOMFile

org.apache.commons.io.output.ByteArrayOutputStream

org.exist.backup.ConsistencyCheck

org.exist.dom.DocumentImpl

org.exist.dom.NodeProxy

org.exist.numbering.NodeId

org.exist.stax.EmbeddedXMLStreamReader

org.exist.storage.btree.Value

org.exist.storage.BTreeRecoverTest

org.exist.storage.BufferStats

org.exist.storage.cache.LRUCache

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.