Examples of com.sleepycat.je.dbi.DbTree

com.sleepycat.je.dbi.DbTree
DbTree represents the database directory for this environment. DbTree is itself implemented through two databases. The nameDatabase maps databaseName-> an internal databaseId. The idDatabase maps databaseId->DatabaseImpl. For example, suppose we have two databases, foo and bar. We have the following structure: nameDatabase idDatabase IN IN | | BIN BIN +-------------+--------+ +---------------+--------+ . | | . | | NameLNs NameLN NameLN MapLNs for MapLN MapLN for internal key=bar key=foo internal dbs key=53 key=79 dbs data= data= data= data= dbId79 dbId53 DatabaseImpl DatabaseImpl | | Tree for foo Tree for bar | | root IN root IN Databases, Cursors, the cleaner, compressor, and other entities have references to DatabaseImpls. It's important that object identity is properly maintained, and that all constituents reference the same DatabaseImpl for the same db, lest they develop disparate views of the in-memory database; corruption would ensue. To ensure that, all entities must obtain their DatabaseImpl by going through the idDatabase. DDL type operations such as create, rename, remove and truncate get their transactional semantics by transactionally locking the NameLN appropriately. A read-lock on the NameLN, called a handle lock, is maintained for all DBs opened via the public API (openDatabase). This prevents them from being renamed or removed while open. However, for internal database operations, no handle lock on the NameLN is acquired and MapLNs are locked with short-lived non-transactional Lockers. An entity that is trying to get a reference to the DatabaseImpl gets a short lived read lock just for the fetch of the MapLN, and a DatabaseImpl usage count is incremented to prevent eviction; see getDb and releaseDb. A write lock on the MapLN is taken when the database is created, deleted, or when the MapLN is evicted. (see DatabaseImpl.isInUse()) The nameDatabase operates pretty much as a regular application database in terms of eviction and recovery. The idDatabase requires special treatment for both eviction and recovery. The issues around eviction of the idDatabase center on the need to ensure that there are no other current references to the DatabaseImpl other than that held by the mapLN. The presence of a current reference would both make the DatabaseImpl not GC'able, and more importantly, would lead to object identify confusion later on. For example, if the MapLN is evicted while there is a current reference to its DatabaseImpl, and then refetched, there will be two in-memory versions of the DatabaseImpl. Since locks on the idDatabase are short lived, DatabaseImpl.useCount acts as a reference count of active current references. DatabaseImpl.useCount must be modified and read in conjunction with appropriate locking on the MapLN. See DatabaseImpl.isInUse() for details. This reference count checking is only needed when the entire MapLN is evicted. It's possible to evict only the root IN of the database in question, since that doesn't interfere with the DatabaseImpl object identity.

        }


        try {


            /* Read all non-provisional INs that are in the repeat set. */
            DbTree dbMapTree = envImpl.getDbTree();
            while (reader.readNextEntry()) {
                counter.incNumRead();
                DatabaseId dbId = reader.getDatabaseId();
                if (targetDbs.contains(dbId)) {
                    DatabaseImpl db = dbMapTree.getDb(dbId);
                    try {
                        if (db == null) {
                            /* This db has been deleted, ignore the entry. */
                            counter.incNumDeleted();
                        } else {
                            counter.incNumProcessed();
                            replayOneIN(reader,
                                        db,
                                        true,  // requireExactMatch,
                                        null); // level recorder
                        }
                    } finally {
                        dbMapTree.releaseDb(db);
                    }
                }
            }


            counter.setRepeatIteratorReads(reader.getNRepeatIteratorReads());

View Full Code Here

        throws DatabaseException {


        final Map<Long,DbFileSummary> grandTotalsMap =
            new HashMap<Long,DbFileSummary>();


        DbTree dbTree = envImpl.getDbTree();


        /* Add in the special id and name database. */
        addDbDerivedTotals(dbTree.getDb(DbTree.ID_DB_ID), grandTotalsMap);
        addDbDerivedTotals(dbTree.getDb(DbTree.NAME_DB_ID), grandTotalsMap);


        /* Walk through all the regular databases. */
        CursorImpl.traverseDbWithCursor(dbTree.getDb(DbTree.ID_DB_ID),
                                        LockType.NONE,
                                        true /*allowEviction*/,
                                        new CursorImpl.WithCursor() {
            public boolean withCursor(CursorImpl cursor,
                                      DatabaseEntry key,

View Full Code Here


        for (LogEntryType lt: logTypes) {
            reader.addTargetType(lt);
        }


        DbTree dbMapTree = envImpl.getDbTree();


        /*
         * See RollbackTracker.java for details on replication rollback
         * periods.  Standalone recovery must handle replication rollback at
         * recovery, because we might be opening a replicated environment in a

View Full Code Here


        for (LogEntryType lt : lnTypes) {
            reader.addTargetType(lt);
        }


        DbTree dbMapTree = envImpl.getDbTree();
        TreeLocation location = new TreeLocation();


        try {


            /*
             * Iterate over the target LNs and construct in-memory tree.
             */
            while (reader.readNextEntry()) {
                counter.incNumRead();
                RedoEligible eligible = eligibleForRedo(reader);


                if (!eligible.isEligible) {
                    continue;
                }


                /*
                 * We're doing a redo. Invoke the evictor in this loop to
                 * reduce memory consumption.
                 */
                envImpl.invokeEvictor();


                DatabaseId dbId = reader.getDatabaseId();
                DatabaseImpl db = dbMapTree.getDb(dbId);


                /*
                 * Database may be null if it's been deleted. Only redo for
                 * extant databases.
                 */
                if (db == null) {
                    counter.incNumDeleted();
                    continue;
                }
                try {
                    LNLogEntry lnEntry = reader.getLNLogEntry();
                    lnEntry.postFetchInit(db);
                    LN ln = lnEntry.getLN();


                    long logLsn = reader.getLastLsn();
                    long treeLsn = DbLsn.NULL_LSN;


                    counter.incNumProcessed();
                    treeLsn = redoOneLN(reader, ln, lnEntry.getKey(), logLsn,
                                        dbId, db, eligible, location);


                    /*
                     * Redo utilization info irregardless of whether the db
                     * is deleted or not.
                     */
                    redoUtilizationInfo
                        (logLsn, treeLsn, eligible.commitLsn,
                         eligible.isCommitted(),
                         reader.getAbortLsn(),
                         reader.getAbortKnownDeleted(),
                         reader.getLastEntrySize(),
                         ln, db);
                } finally {
                    dbMapTree.releaseDb(db);
                }
            }
            counter.setRepeatIteratorReads(reader.getNRepeatIteratorReads());
        } catch (Exception e) {
            traceAndThrowException(reader.getLastLsn(), "redoLns", e);

View Full Code Here


        startupTracker.start(Phase.REMOVE_TEMP_DBS);
        startupTracker.setProgress(RecoveryProgress.REMOVE_TEMP_DBS);
        Counter counter = startupTracker.getCounter(Phase.REMOVE_TEMP_DBS);
        
        DbTree dbMapTree = envImpl.getDbTree();
        BasicLocker locker =
            BasicLocker.createBasicLocker(envImpl, false /*noWait*/);
        boolean operationOk = false;
        try {
            Iterator<DatabaseId> removeDbs = tempDbIds.iterator();
            while (removeDbs.hasNext()) {
                counter.incNumRead();
                DatabaseId dbId = removeDbs.next();
                DatabaseImpl db = dbMapTree.getDb(dbId);
                dbMapTree.releaseDb(db); // Decrement use count.
                if (db != null) {
                    assert db.isTemporary();
                    if (!db.isDeleted()) {
                        try {
                            counter.incNumProcessed();

View Full Code Here

         */
        if (logVersion8UpgradeDbs.isEmpty()) {
            return;
        }


        final DbTree dbTree = envImpl.getDbTree();
        for (DatabaseId dbId : logVersion8UpgradeDbs) {
            final DatabaseImpl db = dbTree.getDb(dbId);
            try {
                if (db.getSortedDuplicates()) {
                    throw EnvironmentFailureException.unexpectedState
                        ("Before upgrading to JE 5.0, the following utility " +
                         "must be run using JE 4.1: " +
                         (envImpl.isReplicated() ? 
                          "DbRepPreUpgrade_4_1 " : "DbPreUpgrade_4_1 ") +
                         "using the JE old version. See the release notes.");
                }
            } finally {
                dbTree.releaseDb(db);
            }
        }
    }

View Full Code Here

     * often enough to prevent the pending LN set from growing too large.
     */
    void processPending()
        throws DatabaseException {


        DbTree dbMapTree = env.getDbTree();


        Map<Long, LNInfo> pendingLNs = fileSelector.getPendingLNs();
        if (pendingLNs != null) {
            TreeLocation location = new TreeLocation();


            for (Map.Entry<Long, LNInfo> entry : pendingLNs.entrySet()) {
                long originalLsn = entry.getKey();
                LNInfo info = entry.getValue();
                DatabaseId dbId = info.getDbId();
                DatabaseImpl db = dbMapTree.getDb(dbId, lockTimeout);
                try {
                    byte[] key = info.getKey();
                    LN ln = info.getLN();


                    /* Evict before processing each entry. */
                    if (DO_CRITICAL_EVICTION) {
                        env.daemonEviction(true /*backgroundIO*/);
                    }


                    processPendingLN(originalLsn, ln, db, key, location);
                } finally {
                    dbMapTree.releaseDb(db);
                }


                /* Sleep if background read/write limit was exceeded. */
                env.sleepAfterBackgroundIO();
            }
        }


        DatabaseId[] pendingDBs = fileSelector.getPendingDBs();
        if (pendingDBs != null) {
            for (DatabaseId dbId : pendingDBs) {
                DatabaseImpl db = dbMapTree.getDb(dbId, lockTimeout);
                try {
                    if (db == null || db.isDeleteFinished()) {
                        fileSelector.removePendingDB(dbId);
                    }
                } finally {
                    dbMapTree.releaseDb(db);
                }
            }
        }
    }

View Full Code Here

    void removePerDbMetadata(final Collection<Long> fileNums,
                             final Set<DatabaseId> databases)
        throws DatabaseException {


        final LogManager logManager = env.getLogManager();
        final DbTree dbTree = env.getDbTree();
        /* Only call logMapTreeRoot once for ID and NAME DBs. */
        DatabaseImpl idDatabase = dbTree.getDb(DbTree.ID_DB_ID);
        DatabaseImpl nameDatabase = dbTree.getDb(DbTree.NAME_DB_ID);
        boolean logRoot = false;
        if (logManager.removeDbFileSummaries(idDatabase, fileNums)) {
            logRoot = true;
        }
        if (logManager.removeDbFileSummaries(nameDatabase, fileNums)) {
            logRoot = true;
        }
        if (logRoot) {
            env.logMapTreeRoot();
        }
        /* Use DB ID set if available to avoid full scan of ID DB. */
        if (databases != null) {
            for (DatabaseId dbId : databases) {
                if (!dbId.equals(DbTree.ID_DB_ID) &&
                    !dbId.equals(DbTree.NAME_DB_ID)) {
                    DatabaseImpl db = dbTree.getDb(dbId);
                    try {
                        if (db != null &&
                            logManager.removeDbFileSummaries(db, fileNums)) {
                            dbTree.modifyDbRoot(db);
                        }
                    } finally {
                        dbTree.releaseDb(db);
                    }
                }
            }
        } else {


            /*
             * Use LockType.NONE for traversing the ID DB so that a lock is not
             * held when calling modifyDbRoot, which must release locks to
             * handle deadlocks.
             */
            CursorImpl.traverseDbWithCursor(idDatabase,
                                            LockType.NONE,
                                            true /*allowEviction*/,
                                            new CursorImpl.WithCursor() {
                public boolean withCursor(CursorImpl cursor,
                                          DatabaseEntry key,
                                          DatabaseEntry data)
                    throws DatabaseException {


                    MapLN mapLN = (MapLN) cursor.getCurrentLN(LockType.NONE);
                    if (mapLN != null) {
                        DatabaseImpl db = mapLN.getDatabase();
                        if (logManager.removeDbFileSummaries(db, fileNums)) {


                            /*
                             * Because we're using dirty-read, silently do
                             * nothing if the DB does not exist
                             * (mustExist=false).
                             */
                            dbTree.modifyDbRoot
                                (db, DbLsn.NULL_LSN /*ifBeforeLsn*/,
                                 false /*mustExist*/);
                        }
                    }
                    return true;

View Full Code Here

        throws DatabaseException {


        if (fileSummaryDb != null) {
            return true;
        }
        DbTree dbTree = env.getDbTree();
        Locker autoTxn = null;
        boolean operationOk = false;
        try {
            autoTxn = Txn.createLocalAutoTxn(env, new TransactionConfig());


            /*
             * releaseDb is not called after this getDb or createDb because we
             * want to prohibit eviction of this database until the environment
             * is closed.
             */
            DatabaseImpl db = dbTree.getDb
                (autoTxn, DbType.UTILIZATION.getInternalName(), null);
            if (db == null) {
                if (env.isReadOnly()) {
                    return false;
                }
                DatabaseConfig dbConfig = new DatabaseConfig();
                DbInternal.setReplicated(dbConfig, false);
                db = dbTree.createInternalDb
                    (autoTxn, DbType.UTILIZATION.getInternalName(),
                     dbConfig);
            }
            fileSummaryDb = db;
            operationOk = true;

View Full Code Here


        final Locker locker =
            Txn.createLocalAutoTxn(envImpl, new TransactionConfig());


        try {
            DbTree dbTree = envImpl.getDbTree();
            DatabaseImpl db = dbTree.getDb(locker,
                                           mappingDbName,
                                           null /* databaseHandle */);
            if (db == null) {
                if (envImpl.isReadOnly()) {
                    /* This should have been caught earlier. */
                    throw EnvironmentFailureException.unexpectedState
                       ("A replicated environment can't be opened read only.");
                }
                DatabaseConfig dbConfig = new DatabaseConfig();
                DbInternal.setReplicated(dbConfig, false);
                db = dbTree.createInternalDb(locker, mappingDbName, dbConfig);
            }
            mappingDbImpl = db;
        } finally {
            locker.operationEnd(true);
        }

View Full Code Here

0 1 2 3 4 5 6 7

TOP

Related Classes of com.sleepycat.je.dbi.DbTree

com.sleepycat.je.cleaner.Cleaner

com.sleepycat.je.cleaner.FileProcessor

com.sleepycat.je.cleaner.UtilizationProfile

com.sleepycat.je.cleaner.VerifyUtils

com.sleepycat.je.DatabaseConfig

com.sleepycat.je.DatabaseEntry

com.sleepycat.je.evictor.Evictor

com.sleepycat.je.incomp.INCompressor

com.sleepycat.je.log.DbOpReplicationContext

com.sleepycat.je.log.LoggableTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.