Examples of com.sleepycat.je.tree.FileSummaryLN

com.sleepycat.je.tree.FileSummaryLN
A FileSummaryLN represents a Leaf Node in the UtilizationProfile database.
The contents of the FileSummaryLN are not fixed until the moment at which the LN is added to the log. A base summary object contains the summary last added to the log. A tracked summary object contains live summary info being updated in real time. The tracked summary is added to the base summary just before logging it, and then the tracked summary is reset. This ensures that the logged summary will accurately reflect the totals calculated at the point in the log where the LN is added.

This is all done in the writeToLog method, which operates under the log write latch. All utilization tracking must be done under the log write latch.

In record version 1, obsolete offset tracking was added and multiple records are stored for a single file rather than a single record. Each record contains the offsets that were tracked since the last record was written.
The key is 8 bytes: 4 bytes for the file number followed by 4 bytes for the sequence number. The lowest valued key for a given file contains the most recent summary information, while to get a complete list of obsolete offsets all records for the file must be read. A range search using just the first 4 bytes can be used to find the most recent record -- this is possible because the sequence number values are decreasing over time for a given file. Here are example keys for three summary records in file 1:
```
 (file=1, sequence=Integer.MAX_VALUE - 300) (file=1, sequence=Integer.MAX_VALUE - 200) (file=1, sequence=Integer.MAX_VALUE - 100) 
```
The sequence number is the number of obsolete entries counted so far, subtracted from Integer.MAX_VALUE to cause the latest written record to have the lowest key.

Log version information

Version 0: Keys are old format strings. No obsolete detail is present.

Version 1: Keys are two 4 byte integers: {file, sequence}. Obsolete detail is present. Some offsets may be invalid if RMW was used.

Version 2: The RMW problem with invalid offsets was corrected. There is no data format change; all versions of JE 2.0.x can read version 1.
@see com.sleepycat.je.cleaner.UtilizationProfile

             * the file during this recovery pass.
             */
            if (LogEntryType.LOG_FILESUMMARYLN.equals(fromLogType)) {
                lnEntry.postFetchInit(false /*isDupDb*/);
                byte[] keyBytes = lnEntry.getKey();
                FileSummaryLN fsln = (FileSummaryLN) lnEntry.getMainItem();
                long fileNum = fsln.getFileNumber(keyBytes);
                fileNumToReset = fileNum;


                /* Save the LSN of the FileSummaryLN for use by undo/redo. */
                tracker.saveLastLoggedFileSummaryLN(fileNum, getLastLsn());

View Full Code Here

        tmp.add(summary);
        tmp.add(tfs);
        int sequence = tmp.getEntriesCounted();


        /* Insert an LN with the existing and tracked summary info. */
        FileSummaryLN ln = new FileSummaryLN(summary);
        ln.setTrackedSummary(tfs);
        insertFileSummary(ln, fileNum, sequence);


        /* Cache the updated summary object.  */
        summary = ln.getBaseSummary();
        if (fileSummaryMap.put(fileNumLong, summary) == null) {
            MemoryBudget mb = env.getMemoryBudget();
            mb.updateAdminMemoryUsage
                (MemoryBudget.UTILIZATION_PROFILE_ENTRY);
        }


        return ln.getObsoleteOffsets();
    }

View Full Code Here

                while (status == OperationStatus.SUCCESS) {


                    /* Perform eviction once per operation. */
                    env.daemonEviction(true /*backgroundIO*/);


                    final FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);
                    if (ln != null) {
                        /* Stop if the file number changes. */
                        if (fileNumVal !=
                            ln.getFileNumber(keyEntry.getData())) {
                            break;
                        }


                        final PackedOffsets offsets = ln.getObsoleteOffsets();
                        if (offsets != null) {
                            list.add(offsets.toArray());
                        }


                        /* Always evict after using a file summary LN. */

View Full Code Here

                     * backgroundIO because this is done during recovery and
                     * there is no reason to sleep.
                     */
                    env.daemonEviction(false /*backgroundIO*/);


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln == null) {
                        /* Advance past a cleaned record. */
                        status = cursor.getNext
                            (keyEntry, dataEntry, LockType.NONE,
                             true /*forward*/, false /*alreadyLatched*/,
                             null /*rangeConstraint*/);
                        continue;
                    }


                    byte[] keyBytes = keyEntry.getData();
                    boolean isOldVersion = ln.hasStringKey(keyBytes);
                    long fileNum = ln.getFileNumber(keyBytes);
                    Long fileNumLong = Long.valueOf(fileNum);


                    if (Arrays.binarySearch(existingFiles, fileNumLong) >= 0) {
                        counter.incNumProcessed();


                        /* File exists, cache the FileSummaryLN. */
                        FileSummary summary = ln.getBaseSummary();
                        fileSummaryMap.put(fileNumLong, summary);


                        /*
                         * Update old version records to the new version.  A
                         * zero sequence number is used to distinguish the

View Full Code Here

                while (status == OperationStatus.SUCCESS) {


                    /* Perform eviction once per operation. */
                    env.daemonEviction(true /*backgroundIO*/);


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln != null) {
                        long fileNumVal = ln.getFileNumber(key.getData());
                        PackedOffsets offsets = ln.getObsoleteOffsets();


                        /*
                         * Check every offset in the fsln to make sure it's
                         * truely obsolete.
                         */

View Full Code Here


            /* Delete all LNs for this file number. */
            OperationStatus status = OperationStatus.SUCCESS;
            while (status == OperationStatus.SUCCESS) {


                FileSummaryLN ln = (FileSummaryLN)
                    cursor.getCurrentLN(LockType.NONE);


                if (ln != null) {
                    /* Stop if the file number changes. */
                    if (fileNumVal != ln.getFileNumber(keyEntry.getData())) {
                        break;
                    }


                    TrackedFileSummary tfs =
                        tracker.getTrackedFile(fileNumVal);
                    /* Associate the tracked summary so it will be cleared. */
                    if (tfs != null) {
                        ln.setTrackedSummary(tfs);
                    }


                    /*
                     * Do not evict after deleting since the compressor would
                     * have to fetch it again.

View Full Code Here

        tmp.add(summary);
        tmp.add(tfs);
        int sequence = tmp.getEntriesCounted();


        /* Insert an LN with the existing and tracked summary info. */
        FileSummaryLN ln = new FileSummaryLN(summary);
        ln.setTrackedSummary(tfs);
        insertFileSummary(ln, fileNum, sequence);


        /* Cache the updated summary object.  */
        summary = ln.getBaseSummary();
        if (fileSummaryMap.put(fileNumLong, summary) == null) {
            MemoryBudget mb = env.getMemoryBudget();
            mb.updateMiscMemoryUsage
                (MemoryBudget.UTILIZATION_PROFILE_ENTRY);
        }


        return ln.getObsoleteOffsets();
    }

View Full Code Here

            }


            /* Read all LNs for this file number. */
            while (status == OperationStatus.SUCCESS) {


                FileSummaryLN ln = (FileSummaryLN)
                    cursor.getCurrentLN(LockType.NONE);
                if (ln != null) {
                    /* Stop if the file number changes. */
                    if (fileNumVal != ln.getFileNumber(keyEntry.getData())) {
                        break;
                    }


                    PackedOffsets offsets = ln.getObsoleteOffsets();
                    if (offsets != null) {
                        list.add(offsets.toArray());
                    }


                    /* Always evict after using a file summary LN. */

View Full Code Here

                                            false); // do need to latch
                }


                while (status == OperationStatus.SUCCESS) {


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln == null) {
                        /* Advance past a cleaned record. */
                        status = cursor.getNext
                            (keyEntry, dataEntry, LockType.NONE,
                             true,   // go forward
                             false); // do need to latch
                        continue;
                    }


                    byte[] keyBytes = keyEntry.getData();
                    boolean isOldVersion = ln.hasStringKey(keyBytes);
                    long fileNum = ln.getFileNumber(keyBytes);
                    Long fileNumLong = new Long(fileNum);


                    if (Arrays.binarySearch(existingFiles, fileNumLong) >= 0) {


                        /* File exists, cache the FileSummaryLN. */
                        fileSummaryMap.put(fileNumLong, ln.getBaseSummary());


                        /*
                         * Update old version records to the new version.  A
                         * zero sequence number is used to distinguish the
                         * converted records and to ensure that later records

View Full Code Here

                    (key, data, LockType.NONE, true);


                /* Iterate over all file summary lns. */
                while (status == OperationStatus.SUCCESS) {


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln != null) {
                        long fileNumVal = ln.getFileNumber(key.getData());
                        PackedOffsets offsets = ln.getObsoleteOffsets();


                        /*
                         * Check every offset in the fsln to make sure it's
                         * truely obsolete.
                         */

View Full Code Here

0 1 2

TOP

Related Classes of com.sleepycat.je.tree.FileSummaryLN

com.sleepycat.je.cleaner.FileSummary

com.sleepycat.je.cleaner.PackedOffsets

com.sleepycat.je.cleaner.SR13061Test

com.sleepycat.je.cleaner.UtilizationProfile

com.sleepycat.je.log.INFileReader

com.sleepycat.je.log.LoggableTest

java.nio.ByteBuffer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of com.sleepycat.je.tree.FileSummaryLN

Log version information

Related Classes of com.sleepycat.je.tree.FileSummaryLN