Examples of com.sleepycat.je.tree.IN

com.sleepycat.je.tree.IN
An IN represents an Internal Node in the JE tree. Explanation of KD (KnownDeleted) and PD (PendingDelete) entry flags =================================================================== PD: set for all LN entries that are deleted, even before the LN is committed. Is used as an authoritative (transactionally correct) indication that an LN is deleted. PD will be cleared if the txn for the deleted LN is aborted. KD: set under special conditions for entries containing LNs which are known to be obsolete. Not used for entries in an active/uncommitted transaction. First notice that IN.fetchTarget will allow a FileNotFoundException when the PD or KD flag is set on the entry. And it will allow a NULL_LSN when the KD flag is set. KD was implemented first, and was originally used when the cleaner attempts to migrate an LN and discovers it is deleted (see Cleaner.migrateLN). We need KD because the INCompressor may not have run, and may not have compressed the BIN. There's the danger that we'll try to fetch that entry, and that the file was deleted by the cleaner. KD was used more recently when an unexpected exception occurs while logging an LN, after inserting the entry. Rather than delete the entry to clean up, we mark the entry KD so it won't cause a fetch error later. In this case the entry LSN is NULL_LSN. See Tree.insertNewSlot. PD is closely related to the first use of KD above (for cleaned deleted LNs) and came about because of a cleaner optimization we make. The cleaner considers all deleted LN log entries to be obsolete, without doing a tree lookup, and without any record of an obsolete offset. This makes the cost of cleaning of deleted LNs very low. For example, if the log looks like this: 100 LNA 200 delete of LNA then LSN 200 will be considered obsolete when this file is processed by the cleaner. After all, only two things can happen: (1) the txn commits, and we don't need LSN 200, because we can wipe this LN out of the tree, or (2) the txn aborts, and we don't need LSN 200, because we are going to revert to LSN 100/LNA. We set PD for the entry of a deleted LN at the time of the operation, and we clear PD if the transaction aborts. There is no real danger that this log entry will be processed by the cleaner before it's committed, because cleaning can only happen after the first active LSN. Just as in the first use of KD above, setting PD is necessary to avoid a fetch error, when the file is deleted by the cleaner but the entry containing the deleted LN has not been deleted by the INCompressor. PD is also set in replication rollback, when LNs are marked as invisible. When LSN locking was implemented (see CursorImpl.lockLN), the PD flag took on additional meaning. PD is used to determine whether an LN is deleted without fetching it, and therefore is relied on to be transactionally correct. In addition to the setting and use of the KD/PD flags described above, the situation is complicated by the fact that we must restore the state of these flags during abort, recovery, and set them properly during slot reuse. We have been meaning to consider whether PD and KD can be consolidated into one flag: simply the Deleted flag. The Deleted flag would be set in the same way as PD is currently set, as well as the second use of KD described above (when the LSN is NULL_LSN after an insertion error). The use of KD and PD for invisible entries and recovery rollback should also be considered. If we consolidate the two flags and set the Deleted flag during a delete operation (like PD), we'll have to remove optimizations (in CursorImpl for example) that consider a slot deleted when KD is set. Since KD is rarely set currently, this shouldn't have a noticeable performance impact.

            /* 
             * Get a new reference to the child, in case the reference
             * saved in the selection list became out of date because of
             * changes to that parent.
             */
            IN renewedChild = (IN) parent.getTarget(index);
            
            /*
             * See the evict() method in this class for an explanation for
             * calling latchNoWait(false).
             */
            if ((renewedChild != null) &&
                (renewedChild.getGeneration() <= oldGenerationCount) &&
                renewedChild.latchNoWait(false)) {


                try {
                    if (renewedChild.isEvictable()) {


                        /* 
                         * Log the child if dirty and env is not r/o. Remove
                         * from IN list.
                         */
                        long renewedChildLsn = DbLsn.NULL_LSN;
                        boolean newChildLsn = false;
                        if (renewedChild.getDirty()) {
                            if (!envIsReadOnly) {


        /* 
                                 * Determine whether provisional logging is
                                 * needed.  The checkpointer can be null if it
                                 * was shutdown or never started.
         */
        boolean logProvisional =
            (envImpl.getCheckpointer() != null &&
             (renewedChild.getLevel() < envImpl.
             getCheckpointer().
              getHighestFlushLevel()));


                                /*
                                 * Log a full version (no deltas) and with
                                 * cleaner migration allowed.
                                 */
                                renewedChildLsn = renewedChild.log
                                    (logManager,
                                     false, // allowDeltas
                                     logProvisional,
                                     true,  // proactiveMigration
                                     parent);
                                newChildLsn = true;
                            }
                        } else {
                            renewedChildLsn = parent.getLsn(index);
                        }


                        if (renewedChildLsn != DbLsn.NULL_LSN) {
                            /* Take this off the inlist. */
                            scanIter.mark();
                            inlist.removeLatchAlreadyHeld(renewedChild);
                            scanIter.resetToMark();


                            evictBytes = renewedChild.getInMemorySize();
                            if (newChildLsn) {


                                /* 
                                 * Update the parent so its reference is
                                 * null and it has the proper LSN.
                                 */
                                parent.updateEntry
                                    (index, null, renewedChildLsn);
                            } else {


                                /*
                                 * Null out the reference, but don't dirty
                                 * the node since only the reference
                                 * changed.
                                 */
                                parent.updateEntry(index, (Node) null);
                            }


                            /* Stats */
                            nNodesEvictedThisRun++;
                            nNodesEvicted++;
                        }
                    }
                } finally {
                    renewedChild.releaseLatch();
                }
            }
        } finally {
            parent.releaseLatch();
        }

View Full Code Here

                        cleaner.processPending();
                    }


                } else if (isIN) {


                    IN targetIN = reader.getIN();
                    DatabaseId dbId = reader.getDatabaseId();
                    DatabaseImpl db = dbMapTree.getDb
                        (dbId, cleaner.lockTimeout, dbCache);
                    targetIN.setDatabase(db);
                    
                    processIN(targetIN, db, lsn);
                    
                } else if (isRoot) {

View Full Code Here

            }


            Tree tree = db.getTree();
            assert tree != null;


            IN inInTree = findINInTree(tree, db, inClone, lsn);


            if (inInTree == null) {
                /* IN is no longer in the tree.  Do nothing. */
                nINsDeadThisRun++;
                obsolete = true;
            } else {


                /* 
                 * IN is still in the tree.  Dirty it.  Checkpoint or eviction
                 * will write it out.  Prohibit the next delta, since the
                 * original version must be made obsolete.
                 */
                nINsMigratedThisRun++;
                inInTree.setDirty(true);
                inInTree.setProhibitNextDelta();
                inInTree.releaseLatch();
                dirtied = true;
            }


            completed = true;
        } finally {

View Full Code Here

    private IN findINInTree(Tree tree, DatabaseImpl db, IN inClone, long lsn)
        throws DatabaseException {


        /* Check if inClone is the root. */
        if (inClone.isDbRoot()) {
            IN rootIN = isRoot(tree, db, inClone, lsn);
            if (rootIN == null) {


                /*
                 * inClone is a root, but no longer in use. Return now, because
                 * a call to tree.getParentNode will return something
                 * unexpected since it will try to find a parent.
                 */
                return null;  
            } else {
                return rootIN;
            }
        }       


        /* It's not the root.  Can we find it, and if so, is it current? */
        inClone.latch(Cleaner.UPDATE_GENERATION);
        SearchResult result = null;
        try {
            result = tree.getParentINForChildIN
                (inClone,
                 true,   // requireExactMatch
                 Cleaner.UPDATE_GENERATION,
                 inClone.getLevel(),
                 null);  // trackingList
            if (!result.exactParentFound) {
                return null;
            }
        
            int compareVal =
    DbLsn.compareTo(result.parent.getLsn(result.index), lsn);
            
            if (compareVal > 0) {
                /* Log entry is obsolete. */
                return null;
            } else {


                /*
                 * Log entry is same or newer than what's in the tree.  Dirty
                 * the IN and let checkpoint write it out.
                 */
                IN in;
                if (compareVal == 0) {
                    /* We can reuse the log entry if the LSNs are equal. */
                    in = (IN) result.parent.getTarget(result.index);
                    if (in == null) {
                        in = inClone;
                        in.postFetchInit(db, lsn);
                        result.parent.updateEntry(result.index, in);
                    }
                } else {
                    in = (IN) result.parent.fetchTarget(result.index);
                }
                in.latch(Cleaner.UPDATE_GENERATION);
                return in;
            }
        } finally {
            if ((result != null) && (result.exactParentFound)) {
                result.parent.releaseLatch();

View Full Code Here

                inClone.getNodeId()) {
                return null;
            }


            if (DbLsn.compareTo(root.getLsn(), lsn) <= 0) {
                IN rootIN = (IN) root.fetchTarget(db, null);
                rootIN.latch(Cleaner.UPDATE_GENERATION);
                return rootIN;
            } else {
                return null;
            }
        }

View Full Code Here

        MemoryBudget mb = envImpl.getMemoryBudget();


        try {
            Iterator iter = inMemINs.iterator();
            while (iter.hasNext()) {
                IN in = (IN) iter.next();
                in.latch(false);
                try {
                    totalSize = mb.accumulateNewUsage(in, totalSize);


                    if (in.getDirty()) {
                        Integer level = new Integer(in.getLevel());
                        Set dirtySet;
                        if (newDirtyMap.containsKey(level)) {
                            dirtySet = (Set) newDirtyMap.get(level);
                        } else {
                            dirtySet = new HashSet();
                            newDirtyMap.put(level, dirtySet);
                        }
                        dirtySet.add
                             (new CheckpointReference(in.getDatabase(),
                                                      in.getNodeId(),
                                                      in.containsDuplicates(),
                                                      in.isDbRoot(),
                                                      in.getMainTreeKey(),
                                                      in.getDupTreeKey()));
                    }
                } finally {
                    in.releaseLatch();
                }
            }


            /* Set the tree cache size. */
            mb.refreshTreeMemoryUsage(totalSize);

View Full Code Here


                        /* 
                         * If the child has already been evicted, don't
                         * refetch it.
                         */
                        IN renewedTarget =
                            (IN) result.parent.getTarget(result.index);


                        if (renewedTarget == null) {
                            /* nAlreadyEvictedThisRun++;  -- for future */
                            mustLogParent = true;

View Full Code Here


    public void dump() {
        System.out.println("size=" + getSize());
  Iterator iter = ins.iterator();
  while (iter.hasNext()) {
      IN theIN = (IN) iter.next();
      System.out.println("db=" + theIN.getDatabase().getId() +
                               " nid=: " + theIN.getNodeId() + "/" +
             theIN.getLevel());
  }
    }

View Full Code Here

            throws DatabaseException {


      if (root == null) {
    return null;
      }
            IN rootIN = (IN) root.fetchTarget(db, null);
            rootIN.latch(false);
            try {
                if (rootIN.getNodeId() == targetNodeId) {


                    /* 
         * stillRoot handles the situation where the root was split
         * after it was placed in the checkpointer's dirty set.
                     */
                    stillRoot = true;
                    if (rootIN.getDirty()) {
                        long newLsn = rootIN.log(logManager);
                        root.setLsn(newLsn);
                        flushed = true;
                    }
                }
            } finally {
                rootIN.releaseLatch();
            }                    
            return null;
        }

View Full Code Here

    public boolean positionFirstOrLast(boolean first, DIN duplicateRoot)
        throws DatabaseException {


        assert assertCursorState(false) : dumpToString(true);


        IN in = null;
        boolean found = false;
        try {
            if (duplicateRoot == null) {
                removeCursorBIN();
                if (first) {
                    in = database.getTree().getFirstNode();
                } else {
                    in = database.getTree().getLastNode();
                }


                if (in != null) {


                    assert (in instanceof BIN);


                    dupBin = null;
                    dupIndex = -1;
                    bin = (BIN) in;
                    index = (first ? 0 : (bin.getNEntries() - 1));
                    addCursor(bin);


                    TreeWalkerStatsAccumulator treeStatsAccumulator =
                        getTreeStatsAccumulator();


                    if (bin.getNEntries() == 0) {


                        /*
                         * An IN was found. Even if it's empty, let Cursor
                         * handle moving to the first non-deleted entry.
                         */
                        found = true;
                    } else {


                        /*
                         * See if we need to descend further.  If fetchTarget
                         * returns null, a deleted LN was cleaned.
                         */
                        Node n = null;
                        if (!in.isEntryKnownDeleted(index)) {
                            n = in.fetchTarget(index);
                        }


                        if (n != null && n.containsDuplicates()) {
                            DIN dupRoot = (DIN) n;
                            dupRoot.latch();
                            in.releaseLatch();
                            in = null;
                            found = positionFirstOrLast(first, dupRoot);
                        } else {


                            /* 
                             * Even if the entry is deleted, just leave our
                             * position here and return.
                             */
                            if (treeStatsAccumulator != null) {
                                if (n == null || ((LN) n).isDeleted()) {
                                    treeStatsAccumulator.
                                        incrementDeletedLNCount();
                                } else {
                                    treeStatsAccumulator.
                                        incrementLNCount();
                                }
                            }
                            found = true;
                        }
                    }
                }
            } else {
                removeCursorDBIN();
                if (first) {
                    in = database.getTree().getFirstNode(duplicateRoot);
                } else {
                    in = database.getTree().getLastNode(duplicateRoot);
                }


                if (in != null) {


        /* 
         * An IN was found. Even if it's empty, let Cursor handle
         * moving to the first non-deleted entry.
         */
        assert (in instanceof DBIN);


        dupBin = (DBIN) in;
                    dupIndex = (first ? 0 : (dupBin.getNEntries() - 1));
        addCursor(dupBin);
        found = true;
                }
            }
            status = CURSOR_INITIALIZED;
            return found;
        } catch (DatabaseException e) {
            /* Release latch on error. */
            if (in != null) {
                in.releaseLatch();
            }
            throw e;
        }
    }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of com.sleepycat.je.tree.IN

com.sleepycat.je.cleaner.Cleaner

com.sleepycat.je.cleaner.Cleaner$RootDoWork

com.sleepycat.je.cleaner.FileProcessor

com.sleepycat.je.cleaner.FileProcessor$RootDoWork

com.sleepycat.je.cleaner.INUtilizationTest

com.sleepycat.je.cleaner.PackedObsoleteInfo

com.sleepycat.je.dbi.CursorImpl

com.sleepycat.je.dbi.DatabaseId

com.sleepycat.je.dbi.DatabaseImpl

com.sleepycat.je.dbi.DatabaseImpl$PreloadLSNTreeWalker

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.