Examples of com.sleepycat.je.tree.IN

com.sleepycat.je.tree.IN
An IN represents an Internal Node in the JE tree. Explanation of KD (KnownDeleted) and PD (PendingDelete) entry flags =================================================================== PD: set for all LN entries that are deleted, even before the LN is committed. Is used as an authoritative (transactionally correct) indication that an LN is deleted. PD will be cleared if the txn for the deleted LN is aborted. KD: set under special conditions for entries containing LNs which are known to be obsolete. Not used for entries in an active/uncommitted transaction. First notice that IN.fetchTarget will allow a FileNotFoundException when the PD or KD flag is set on the entry. And it will allow a NULL_LSN when the KD flag is set. KD was implemented first, and was originally used when the cleaner attempts to migrate an LN and discovers it is deleted (see Cleaner.migrateLN). We need KD because the INCompressor may not have run, and may not have compressed the BIN. There's the danger that we'll try to fetch that entry, and that the file was deleted by the cleaner. KD was used more recently when an unexpected exception occurs while logging an LN, after inserting the entry. Rather than delete the entry to clean up, we mark the entry KD so it won't cause a fetch error later. In this case the entry LSN is NULL_LSN. See Tree.insertNewSlot. PD is closely related to the first use of KD above (for cleaned deleted LNs) and came about because of a cleaner optimization we make. The cleaner considers all deleted LN log entries to be obsolete, without doing a tree lookup, and without any record of an obsolete offset. This makes the cost of cleaning of deleted LNs very low. For example, if the log looks like this: 100 LNA 200 delete of LNA then LSN 200 will be considered obsolete when this file is processed by the cleaner. After all, only two things can happen: (1) the txn commits, and we don't need LSN 200, because we can wipe this LN out of the tree, or (2) the txn aborts, and we don't need LSN 200, because we are going to revert to LSN 100/LNA. We set PD for the entry of a deleted LN at the time of the operation, and we clear PD if the transaction aborts. There is no real danger that this log entry will be processed by the cleaner before it's committed, because cleaning can only happen after the first active LSN. Just as in the first use of KD above, setting PD is necessary to avoid a fetch error, when the file is deleted by the cleaner but the entry containing the deleted LN has not been deleted by the INCompressor. PD is also set in replication rollback, when LNs are marked as invisible. When LSN locking was implemented (see CursorImpl.lockLN), the PD flag took on additional meaning. PD is used to determine whether an LN is deleted without fetching it, and therefore is relied on to be transactionally correct. In addition to the setting and use of the KD/PD flags described above, the situation is complicated by the fact that we must restore the state of these flags during abort, recovery, and set them properly during slot reuse. We have been meaning to consider whether PD and KD can be consolidated into one flag: simply the Deleted flag. The Deleted flag would be set in the same way as PD is currently set, as well as the second use of KD described above (when the LSN is NULL_LSN after an insertion error). The use of KD and PD for invisible entries and recovery rollback should also be considered. If we consolidate the two flags and set the Deleted flag during a delete operation (like PD), we'll have to remove optimizations (in CursorImpl for example) that consider a slot deleted when KD is set. Since KD is rarely set currently, this shouldn't have a noticeable performance impact.

             * IN list.
             */
            while ((evictBytes < requiredEvictBytes) &&
                   (nNodesScannedThisRun <= inListStartSize)) {


                IN target = selectIN(inList, scanIter);


                if (target == null) {
                    break;
                } else {
                    assert evictProfile.count(target);//intentional side effect

View Full Code Here

     */
    private IN selectIN(INList inList, ScanIterator scanIter)
        throws DatabaseException {


        /* Find the best target in the next <nodesPerScan> nodes. */
        IN target = null;
        long targetGeneration = Long.MAX_VALUE;
        int targetLevel = Integer.MAX_VALUE;
        boolean targetDirty = true;
  boolean envIsReadOnly = envImpl.isReadOnly();
        int scanned = 0;
        boolean wrapped = false;
        while (scanned < nodesPerScan) {
            if (scanIter.hasNext()) {
                IN in = scanIter.next();
                nNodesScannedThisRun++;


                DatabaseImpl db = in.getDatabase();


                /* If the database was deleted, remove it from the inlist. */
                if (db == null || db.getIsDeleted()) {
                    scanIter.remove();
                    continue;
                }


                /* 
                 * Don't evict the DatabaseImpl Id Mapping Tree (db 0), both
                 * for object identity reasons and because the id mapping tree
                 * should stay cached.
                 */
                if (db.getId().equals(DbTree.ID_DB_ID)) {
                    continue;
                }


                /* 
                 * If this is a read only database and we have at least one
                 * target, skip any dirty INs (recovery dirties INs even in a
                 * read-only environment). We take at least one target so we
                 * don't loop endlessly if everything is dirty.
                 */
                if (envIsReadOnly && (target != null) && in.getDirty()) {
                    continue;
                }


                /*
                 * Only scan evictable or strippable INs.  This prevents higher
                 * level INs from being selected for eviction, unless they are
                 * part of an unused tree.
                 */
                int evictType = in.getEvictionType();
                if (evictType == IN.MAY_NOT_EVICT) {
                    continue;
                }


                /*
                 * This node is in the scanned node set.  Select according to
                 * the configured eviction policy.
                 */
                if (evictByLruOnly) {


                    /*
                     * Select the node with the lowest generation number,
                     * irrespective of tree level or dirtyness.
                     */
                    if (targetGeneration > in.getGeneration()) {
                        targetGeneration = in.getGeneration();
                        target = in;
                    }
                } else {


                    /*
                     * Select first by tree level, then by dirtyness, then by
                     * generation/LRU.
                     */
                    int level = normalizeLevel(in, evictType);
                    if (targetLevel != level) {
                        if (targetLevel > level) {
                            targetLevel = level;
                            targetDirty = in.getDirty();
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    } else if (targetDirty != in.getDirty()) {
                        if (targetDirty) {
                            targetDirty = false;
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    } else {
                        if (targetGeneration > in.getGeneration()) {
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    }
                }
                scanned++;

View Full Code Here

            /* 
             * Get a new reference to the child, in case the reference
             * saved in the selection list became out of date because of
             * changes to that parent.
             */
            IN renewedChild = (IN) parent.getTarget(index);
            
            /*
             * See the evict() method in this class for an explanation for
             * calling latchNoWait(false).
             */
            if ((renewedChild != null) &&
                (renewedChild.getGeneration() <= oldGenerationCount) &&
                renewedChild.latchNoWait(false)) {


                try {
                    if (renewedChild.isEvictable()) {


                        /* 
                         * Log the child if dirty and env is not r/o. Remove
                         * from IN list.
                         */
                        long renewedChildLsn = DbLsn.NULL_LSN;
                        boolean newChildLsn = false;
                        if (renewedChild.getDirty()) {
                            if (!envIsReadOnly) {


        /* 
                                 * Determine whether provisional logging is
                                 * needed.  The checkpointer can be null if it
                                 * was shutdown or never started.
         */
        boolean logProvisional =
            (envImpl.getCheckpointer() != null &&
             (renewedChild.getLevel() < envImpl.
             getCheckpointer().
              getHighestFlushLevel()));


                                /*
                                 * Log a full version (no deltas) and with
                                 * cleaner migration allowed.
                                 */
                                renewedChildLsn = renewedChild.log
                                    (logManager,
                                     false, // allowDeltas
                                     logProvisional,
                                     true,  // allowMigration
                                     parent);
                                newChildLsn = true;
                            }
                        } else {
                            renewedChildLsn = parent.getLsn(index);
                        }


                        if (renewedChildLsn != DbLsn.NULL_LSN) {
                            /* Take this off the inlist. */
                            scanIter.mark();
                            inlist.removeLatchAlreadyHeld(renewedChild);
                            scanIter.resetToMark();


                            evictBytes = renewedChild.getInMemorySize();
                            if (newChildLsn) {


                                /* 
                                 * Update the parent so its reference is
                                 * null and it has the proper LSN.
                                 */
                                parent.updateEntry
                                    (index, null, renewedChildLsn);
                            } else {


                                /*
                                 * Null out the reference, but don't dirty
                                 * the node since only the reference
                                 * changed.
                                 */
                                parent.updateEntry(index, (Node) null);
                            }


                            /* Stats */
                            nNodesEvictedThisRun++;
                            nNodesEvicted++;
                        }
                    }
                } finally {
                    renewedChild.releaseLatch();
                }
            }
        } finally {
            parent.releaseLatch();
        }

View Full Code Here

         * @return true if the in-memory root was replaced.
         */
        public IN doWork(ChildReference root) 
            throws DatabaseException {
            
            IN rootIN = (IN) root.fetchTarget(db, null);
            rootLevel = rootIN.getLevel();
            return null;
        }

View Full Code Here


  long targetTime = Long.MAX_VALUE;
  if (maxMillisecs > 0) {
      targetTime = System.currentTimeMillis() + maxMillisecs;
  }
  IN next = tree.getFirstNode();
  if (next == null) {
      return;
  }


  if (maxBytes == 0) {
            maxBytes = envImpl.getMemoryBudget().getCacheBudget();
  }


  try {
      while (true) {
    if (System.currentTimeMillis() > targetTime) {
        break;
    }
    next = tree.getNextBin((BIN) next, null);
    if (next == null) {
        break;
    }


    if (envImpl.getMemoryBudget().getCacheMemoryUsage() >
        maxBytes) {
        break;
    }
      }
  } finally {
      if (next != null) {
    next.releaseLatch();
      }
  }


        assert Latch.countLatchesHeld() == 0;
    }

View Full Code Here

       * pass the LSN of the current log entry and also the LSN of the IN
       * in question. The only time these differ is when the log entry is
       * a BINDelta -- then the IN's LSN is the last full version LSN,
       * and the log LSN is the current log entry.
             */
            IN in = reader.getIN();
            long inLsn = reader.getLsnOfIN();
            in.postRecoveryInit(db, inLsn);
            in.latch();
            replaceOrInsert(db, in, reader.getLastLsn(), inLsn,
                            requireExactMatch);
        }


        /*

View Full Code Here

        boolean inserted = false;
        boolean replaced = false;
        long originalLsn = DbLsn.NULL_LSN;


        byte[] mainTreeKey = inFromLog.getMainTreeKey();
        IN parent = null;
        int index = -1;
        boolean success = false;
        try {


            /*
             * Allow splits since the parent BIN of this DIN may be full.
             * [#13435]
             */
            parent = db.getTree().searchSplitsAllowed
                (mainTreeKey, -1, true /*updateGeneration*/);
            assert parent instanceof BIN;


      ChildReference newRef =
    new ChildReference(inFromLog, mainTreeKey, lsn);
      index = parent.insertEntry1(newRef);
      if ((index >= 0 &&
     (index & IN.EXACT_MATCH) != 0)) {


    index &= ~IN.EXACT_MATCH;


    /* 
     * Replace whatever's at this entry, whether it's an LN or an
     * earlier root DIN as long as one of the following is true:
     * 
     * - the entry is known deleted
     * - or the LSN is earlier than the one we've just read from
     *     the log.
     */
    if (parent.isEntryKnownDeleted(index)) {
        /* Be sure to clear the known deleted bit. */
        parent.setEntry(index, inFromLog, mainTreeKey,
            lsn, (byte) 0);
        replaced = true;
    } else {
        originalLsn = parent.getLsn(index);
        if (DbLsn.compareTo(originalLsn, lsn) < 0) {
      parent.setEntry(index, inFromLog, mainTreeKey, lsn,
          parent.getState(index));
      replaced = true;
        }
    }
      } else {
    found = false;
      }
      success = true;
        } finally {
            if (parent != null) {
                parent.releaseLatch();
            }
            trace(detailedTraceLevel,
                  db,
                  TRACE_DUP_ROOT_REPLACE, success, inFromLog,
                  lsn, parent, found,

View Full Code Here

    }


    private long getINFile(Cursor cursor)
        throws DatabaseException {


        IN in = TestUtils.getIN(TestUtils.getBIN(cursor));
        long lsn = in.getLastFullVersion();
        assertTrue(lsn != DbLsn.NULL_LSN);
        return DbLsn.getFileNumber(lsn);
    }

View Full Code Here


    public void dump() {
        System.out.println("size=" + getSize());
  Iterator iter = ins.iterator();
  while (iter.hasNext()) {
      IN theIN = (IN) iter.next();
      System.out.println("db=" + theIN.getDatabase().getId() +
                               " nid=: " + theIN.getNodeId() + "/" +
             theIN.getLevel());
  }
    }

View Full Code Here

        DatabaseId id = dbImpl.getId();
        majorLatch.acquire();


  Iterator iter = ins.iterator();
  while (iter.hasNext()) {
      IN theIN = (IN) iter.next();
      if (theIN.getDatabase().getId().equals(id)) {
    theIN.setGeneration(0);
      }
  }


        releaseMajorLatch();
    }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of com.sleepycat.je.tree.IN

com.sleepycat.je.cleaner.Cleaner

com.sleepycat.je.cleaner.Cleaner$RootDoWork

com.sleepycat.je.cleaner.FileProcessor

com.sleepycat.je.cleaner.FileProcessor$RootDoWork

com.sleepycat.je.cleaner.INUtilizationTest

com.sleepycat.je.cleaner.PackedObsoleteInfo

com.sleepycat.je.dbi.CursorImpl

com.sleepycat.je.dbi.DatabaseId

com.sleepycat.je.dbi.DatabaseImpl

com.sleepycat.je.dbi.DatabaseImpl$PreloadLSNTreeWalker

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.