Examples of com.sleepycat.je.dbi.DbTree

com.sleepycat.je.dbi.DbTree
DbTree represents the database directory for this environment. DbTree is itself implemented through two databases. The nameDatabase maps databaseName-> an internal databaseId. The idDatabase maps databaseId->DatabaseImpl. For example, suppose we have two databases, foo and bar. We have the following structure: nameDatabase idDatabase IN IN | | BIN BIN +-------------+--------+ +---------------+--------+ . | | . | | NameLNs NameLN NameLN MapLNs for MapLN MapLN for internal key=bar key=foo internal dbs key=53 key=79 dbs data= data= data= data= dbId79 dbId53 DatabaseImpl DatabaseImpl | | Tree for foo Tree for bar | | root IN root IN Databases, Cursors, the cleaner, compressor, and other entities have references to DatabaseImpls. It's important that object identity is properly maintained, and that all constituents reference the same DatabaseImpl for the same db, lest they develop disparate views of the in-memory database; corruption would ensue. To ensure that, all entities must obtain their DatabaseImpl by going through the idDatabase. DDL type operations such as create, rename, remove and truncate get their transactional semantics by transactionally locking the NameLN appropriately. A read-lock on the NameLN, called a handle lock, is maintained for all DBs opened via the public API (openDatabase). This prevents them from being renamed or removed while open. However, for internal database operations, no handle lock on the NameLN is acquired and MapLNs are locked with short-lived non-transactional Lockers. An entity that is trying to get a reference to the DatabaseImpl gets a short lived read lock just for the fetch of the MapLN, and a DatabaseImpl usage count is incremented to prevent eviction; see getDb and releaseDb. A write lock on the MapLN is taken when the database is created, deleted, or when the MapLN is evicted. (see DatabaseImpl.isInUse()) The nameDatabase operates pretty much as a regular application database in terms of eviction and recovery. The idDatabase requires special treatment for both eviction and recovery. The issues around eviction of the idDatabase center on the need to ensure that there are no other current references to the DatabaseImpl other than that held by the mapLN. The presence of a current reference would both make the DatabaseImpl not GC'able, and more importantly, would lead to object identify confusion later on. For example, if the MapLN is evicted while there is a current reference to its DatabaseImpl, and then refetched, there will be two in-memory versions of the DatabaseImpl. Since locks on the idDatabase are short lived, DatabaseImpl.useCount acts as a reference count of active current references. DatabaseImpl.useCount must be modified and read in conjunction with appropriate locking on the MapLN. See DatabaseImpl.isInUse() for details. This reference count checking is only needed when the entire MapLN is evicted. It's possible to evict only the root IN of the database in question, since that doesn't interfere with the DatabaseImpl object identity.

            UtilizationTracker tracker = new UtilizationTracker(env);


            /* Use local caching to reduce DbTree.getDb overhead. */
            Map dbCache = new HashMap();


            DbTree dbTree = env.getDbMapTree();
            BINSearch binSearch = new BINSearch();
            try {
                Iterator it = queueSnapshot.values().iterator();
                while (it.hasNext()) {
                    if (env.isClosed()) {

View Full Code Here

        /* Validate all entries in at least one full recovery pass. */
        reader.setAlwaysValidateChecksum(true);


        try {
            info.numMapINs = 0;
            DbTree dbMapTree = env.getDbMapTree();


            /* 
       * Process every IN, INDeleteInfo, and INDupDeleteInfo in the
       * mapping tree.
       */
            while (reader.readNextEntry()) {
                DatabaseId dbId = reader.getDatabaseId();
                if (dbId.equals(DbTree.ID_DB_ID)) {
                    DatabaseImpl db = dbMapTree.getDb(dbId);
                    replayOneIN(reader, db, false);
                    info.numMapINs++;
                }
            }

View Full Code Here


            /*
             * Read all non-provisional INs, and process if they don't belong
             * to the mapping tree.
             */
            DbTree dbMapTree = env.getDbMapTree();
            while (reader.readNextEntry()) {
                DatabaseId dbId = reader.getDatabaseId();
                boolean isMapDb = dbId.equals(DbTree.ID_DB_ID);
                boolean isTarget = false;


                if (mapDbOnly && isMapDb) {
                    isTarget = true;
                } else if (!mapDbOnly && !isMapDb) {
                    isTarget = true;
                }
                if (isTarget) {
                    DatabaseImpl db = dbMapTree.getDb(dbId);
                    if (db == null) {
                        // This db has been deleted, ignore the entry.
                    } else {
                        replayOneIN(reader, db, requireExactMatch);
                        numINsSeen++;

View Full Code Here

        }


        Map countedFileSummaries = new HashMap(); // TxnNodeId -> file number
        Set countedAbortLsnNodes = new HashSet(); // set of TxnNodeId


        DbTree dbMapTree = env.getDbMapTree();
        TreeLocation location = new TreeLocation();
        try {


            /*
             * Iterate over the target LNs and commit records, constructing
             * tree.
             */
            while (reader.readNextEntry()) {
                if (reader.isLN()) {


                    /* Get the txnId from the log entry. */
                    Long txnId = reader.getTxnId();


                    /*
                     * If this node is not in a committed txn, examine it to
                     * see if it should be undone.
                     */
                    if (txnId != null &&
      !committedTxnIds.contains(txnId)) {


      /*
       * Invoke the evictor to reduce memory consumption.
       */
      env.invokeEvictor();


      LN ln = reader.getLN();
      long logLsn = reader.getLastLsn();
      long abortLsn = reader.getAbortLsn();
      boolean abortKnownDeleted =
          reader.getAbortKnownDeleted();
      DatabaseId dbId = reader.getDatabaseId();
      DatabaseImpl db = dbMapTree.getDb(dbId);
                        
      /* Database may be null if it's been deleted. */
      if (db != null) {
          ln.postFetchInit(db, logLsn);
          try {

View Full Code Here

            reader.addTargetType(lnType);
        }


        Set countedAbortLsnNodes = new HashSet(); // set of TxnNodeId


        DbTree dbMapTree = env.getDbMapTree();
        TreeLocation location = new TreeLocation();
        try {


            /* Iterate over the target LNs and construct in- memory tree. */
            while (reader.readNextEntry()) {
                if (reader.isLN()) {


                    /* Get the txnId from the log entry. */
                    Long txnId = reader.getTxnId();
                
                    /* 
                     * If this LN is in a committed txn, or if it's a
                     * non-transactional LN, redo it.
                     */
        boolean processThisLN = false;
        boolean lnIsCommitted = false;
        boolean lnIsPrepared = false;
        Txn preparedTxn = null;
        if (txnId == null) {
      processThisLN = true;
        } else {
      lnIsCommitted = committedTxnIds.contains(txnId);
      if (!lnIsCommitted) {
          preparedTxn = (Txn) preparedTxns.get(txnId);
          lnIsPrepared = preparedTxn != null;
      }
      if (lnIsCommitted || lnIsPrepared) {
          processThisLN = true;
      }
        }
        if (processThisLN) {


                        /* Invoke the evictor to reduce memory consumption. */
                        env.invokeEvictor();


                        LN ln = reader.getLN();
                        DatabaseId dbId = reader.getDatabaseId();
                        DatabaseImpl db = dbMapTree.getDb(dbId);
                        long logLsn = reader.getLastLsn();
                        long treeLsn = DbLsn.NULL_LSN;


                        /* Database may be null if it's been deleted. */
                        if (db != null) {

View Full Code Here

                 true); // doFetch


            if (result.parent == null) {
                /* It's null -- we actually deleted the root. */
                tree.withRootLatchedExclusive(new RootDeleter(tree));
                DbTree dbTree = db.getDbEnvironment().getDbMapTree();
                dbTree.modifyDbRoot(db);
                traceRootDeletion(Level.FINE, db);
                deleted = true;
            } else if (result.exactParentFound) {
                /* Exact match was found -- delete the parent entry. */
                found = true;

View Full Code Here

        throws DatabaseException {


        if (fileSummaryDb != null) {
            return true;
        }
        DbTree dbTree = env.getDbMapTree();
        Locker autoTxn = null;
        boolean operationOk = false;
        try {
            autoTxn = new AutoTxn(env, new TransactionConfig());
            DatabaseImpl db = dbTree.getDb
                (autoTxn, DbTree.UTILIZATION_DB_NAME, null,
                 true /* Eviction allowed. */);
            if (db == null) {
                if (env.isReadOnly()) {
                    return false;
                }
                db = dbTree.createDb
                    (autoTxn, DbTree.UTILIZATION_DB_NAME,
                     new DatabaseConfig(), null,
                     true /* Eviction allowed */);
            }
            fileSummaryDb = db;

View Full Code Here

     * often enough to prevent the pending LN set from growing too large.
     */
    void processPending()
        throws DatabaseException {


        DbTree dbMapTree = env.getDbMapTree();


        LNInfo[] pendingLNs = fileSelector.getPendingLNs();
        if (pendingLNs != null) {
            TreeLocation location = new TreeLocation();


            for (int i = 0; i < pendingLNs.length; i += 1) {
                LNInfo info = pendingLNs[i];


                DatabaseId dbId = info.getDbId();
                DatabaseImpl db = dbMapTree.getDb(dbId, lockTimeout);


                byte[] key = info.getKey();
                byte[] dupKey = info.getDupKey();
                LN ln = info.getLN();


                /* Evict before processing each entry. */
                if (DO_CRITICAL_EVICTION) {
                    env.getEvictor().doCriticalEviction();
                }


                processPendingLN
                    (ln, db, key, dupKey, location);
            }
        }


        DatabaseId[] pendingDBs = fileSelector.getPendingDBs();
        if (pendingDBs != null) {
            for (int i = 0; i < pendingDBs.length; i += 1) {
                DatabaseId dbId = pendingDBs[i];
                DatabaseImpl db = dbMapTree.getDb(dbId, lockTimeout);
                if (db == null || db.isDeleteFinished()) {
                    fileSelector.removePendingDB(dbId);
                }
            }
        }

View Full Code Here

            if (rootNeedsUpdating) {
                /*
                 * modifyDbRoot will grab locks and we can't have the INList
                 * latches or root latch held while it tries to acquire locks.
                 */
                DbTree dbTree = envImpl.getDbMapTree();
                dbTree.modifyDbRoot(database);
                RecoveryManager.traceRootDeletion(Level.FINE, database);
            } 


            /* 
             * Count obsolete nodes after logging the delete. We can do

View Full Code Here

        EnvironmentImpl envImpl = DbInternal.getEnvironmentImpl(env);


        /* If no database is specified, verify all. */
        List<String> dbNameList = null;
        List<String> internalDbs = null;
        DbTree dbMapTree = envImpl.getDbTree();


        if (dbName == null) {
            dbNameList = env.getDatabaseNames();


            dbNameList.addAll(dbMapTree.getInternalNoRepDbNames());
            if (envImpl.isReplicated()) {
                dbNameList.addAll(dbMapTree.getInternalRepDbNames());
            }
            internalDbs = dbMapTree.getInternalNoLookupDbNames();
        } else {
            dbNameList = new ArrayList<String>();
            dbNameList.add(dbName);
            internalDbs = new ArrayList<String>();
        }


        /* Check application data. */
        Iterator<String> iter = dbNameList.iterator();
        while (iter.hasNext()) {
            String targetDb = iter.next();
            LoggerUtils.envLogMsg(Level.INFO, envImpl,
                                  "DbVerify.verify of " + targetDb + " starting");


            DatabaseConfig dbConfig = new DatabaseConfig();
            dbConfig.setReadOnly(true);
            dbConfig.setAllowCreate(false);
            DbInternal.setUseExistingConfig(dbConfig, true);
            Database db;
            try {
                db = env.openDatabase(null, targetDb, dbConfig);
            } catch (DatabaseNotFoundException e) {
                /* DB was removed after getting names -- ignore it. */
                continue;
            } catch (DatabaseExistsException e) {
                /* Should never happen, ExclusiveCreate is false. */
                throw EnvironmentFailureException.unexpectedException(e);
            }


            try {
                if (!verifyOneDbImpl(DbInternal.getDatabaseImpl(db),
                                     targetDb,
                                     verifyConfig,
                                     out)) {
                    ret = false;
                }
            } finally {
                if (db != null) {
                    db.close();
                }
                LoggerUtils.envLogMsg(Level.INFO, envImpl,
                                   "DbVerify.verify of " + targetDb + 
                                   " ending");
            }
        }


        /*
         * Check internal databases, which don't have to be opened
         * through a Database handle.
         */
        iter = internalDbs.iterator();
        while (iter.hasNext()) {
            String targetDb = iter.next();
            LoggerUtils.envLogMsg(Level.INFO, envImpl,
                               "DbVerify.verify of " + targetDb + " starting");


            try {
                DatabaseImpl dbImpl = dbMapTree.getDb(null, targetDb,
                                                      null);
                try {
                    if (!verifyOneDbImpl(dbImpl,  targetDb,
                                         verifyConfig, out)) {
                        ret = false;
                    }
                } finally {
                    dbMapTree.releaseDb(dbImpl);
                }
            } finally {
                LoggerUtils.envLogMsg(Level.INFO, envImpl,
                                   "DbVerify.verify of " + targetDb + 
                                   " ending");

View Full Code Here

0 1 2 3 4 5 6 7

TOP

Related Classes of com.sleepycat.je.dbi.DbTree

com.sleepycat.je.cleaner.Cleaner

com.sleepycat.je.cleaner.FileProcessor

com.sleepycat.je.cleaner.UtilizationProfile

com.sleepycat.je.cleaner.VerifyUtils

com.sleepycat.je.DatabaseConfig

com.sleepycat.je.DatabaseEntry

com.sleepycat.je.evictor.Evictor

com.sleepycat.je.incomp.INCompressor

com.sleepycat.je.log.DbOpReplicationContext

com.sleepycat.je.log.LoggableTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.