Examples of com.sleepycat.je.tree.FileSummaryLN

com.sleepycat.je.tree.FileSummaryLN
A FileSummaryLN represents a Leaf Node in the UtilizationProfile database.
The contents of the FileSummaryLN are not fixed until the moment at which the LN is added to the log. A base summary object contains the summary last added to the log. A tracked summary object contains live summary info being updated in real time. The tracked summary is added to the base summary just before logging it, and then the tracked summary is reset. This ensures that the logged summary will accurately reflect the totals calculated at the point in the log where the LN is added.

This is all done in the writeToLog method, which operates under the log write latch. All utilization tracking must be done under the log write latch.

In record version 1, obsolete offset tracking was added and multiple records are stored for a single file rather than a single record. Each record contains the offsets that were tracked since the last record was written.
The key is 8 bytes: 4 bytes for the file number followed by 4 bytes for the sequence number. The lowest valued key for a given file contains the most recent summary information, while to get a complete list of obsolete offsets all records for the file must be read. A range search using just the first 4 bytes can be used to find the most recent record -- this is possible because the sequence number values are decreasing over time for a given file. Here are example keys for three summary records in file 1:
```
 (file=1, sequence=Integer.MAX_VALUE - 300) (file=1, sequence=Integer.MAX_VALUE - 200) (file=1, sequence=Integer.MAX_VALUE - 100) 
```
The sequence number is the number of obsolete entries counted so far, subtracted from Integer.MAX_VALUE to cause the latest written record to have the lowest key.

Log version information

Version 0: Keys are old format strings. No obsolete detail is present.

Version 1: Keys are two 4 byte integers: {file, sequence}. Obsolete detail is present. Some offsets may be invalid if RMW was used.

Version 2: The RMW problem with invalid offsets was corrected. There is no data format change; all versions of JE 2.0.x can read version 1.
@see com.sleepycat.je.cleaner.UtilizationProfile

            LN lnFromLog = new LN();
            writeAndRead(buffer, ln, lnFromLog);
            lnFromLog.verify(null);
            assertTrue(ln.marshallOutsideWriteLatch());


            FileSummaryLN fsLN = new FileSummaryLN(new FileSummary());
            FileSummaryLN fsLNFromLog = new FileSummaryLN();
            writeAndRead(buffer, fsLN, fsLNFromLog);
            assertFalse(fsLN.marshallOutsideWriteLatch());


            /*
             * INs

View Full Code Here

                 * When a FileSummaryLN is encountered, reset the tracked
                 * summary for that file to replay what happens when a
                 * FileSummaryLN log entry is written.
                 */
                byte[] keyBytes = fsTrackingEntry.getKey();
                FileSummaryLN fsln =
                    (FileSummaryLN) fsTrackingEntry.getMainItem();
                long fileNum = fsln.getFileNumber(keyBytes);
                TrackedFileSummary trackedLN = tracker.getTrackedFile(fileNum);
                if (trackedLN != null) {
                    trackedLN.reset();
                }

View Full Code Here

    }


    public void testSR13061()
  throws DatabaseException {


        FileSummaryLN ln = new FileSummaryLN(new FileSummary());


        /*
         * All of these tests failed before checking that the byte array must
         * be eight bytes for integer keys.
         */
        assertTrue(ln.hasStringKey(stringKey(0)));
        assertTrue(ln.hasStringKey(stringKey(1)));
        assertTrue(ln.hasStringKey(stringKey(12)));
        assertTrue(ln.hasStringKey(stringKey(123)));
        assertTrue(ln.hasStringKey(stringKey(1234)));
        assertTrue(ln.hasStringKey(stringKey(12345)));
        assertTrue(ln.hasStringKey(stringKey(123456)));
        assertTrue(ln.hasStringKey(stringKey(1234567)));
        assertTrue(ln.hasStringKey(stringKey(123456789)));
        assertTrue(ln.hasStringKey(stringKey(1234567890)));


        /*
         * These tests failed before checking that the first byte of the
         * sequence number (in an eight byte key) must not be '0' to '9' for
         * integer keys.
         */
        assertTrue(ln.hasStringKey(stringKey(12345678)));
        assertTrue(ln.hasStringKey(stringKey(12340000)));


        /* These tests are just for good measure. */
        assertTrue(!ln.hasStringKey(intKey(0, 1)));
        assertTrue(!ln.hasStringKey(intKey(1, 1)));
        assertTrue(!ln.hasStringKey(intKey(12, 1)));
        assertTrue(!ln.hasStringKey(intKey(123, 1)));
        assertTrue(!ln.hasStringKey(intKey(1234, 1)));
        assertTrue(!ln.hasStringKey(intKey(12345, 1)));
        assertTrue(!ln.hasStringKey(intKey(123456, 1)));
        assertTrue(!ln.hasStringKey(intKey(1234567, 1)));
        assertTrue(!ln.hasStringKey(intKey(12345678, 1)));
        assertTrue(!ln.hasStringKey(intKey(123456789, 1)));
        assertTrue(!ln.hasStringKey(intKey(1234567890, 1)));
    }

View Full Code Here

                 * When a FileSummaryLN is encountered, reset the tracked
                 * summary for that file to replay what happens when a
                 * FileSummaryLN log entry is written.
                 */
                byte[] keyBytes = fsTrackingEntry.getKey();
                FileSummaryLN fsln =
                    (FileSummaryLN) fsTrackingEntry.getMainItem();
                long fileNum = fsln.getFileNumber(keyBytes);
                TrackedFileSummary trackedLN = tracker.getTrackedFile(fileNum);
                if (trackedLN != null) {
                    trackedLN.reset();
                }

View Full Code Here

            while (status == OperationStatus.SUCCESS) {


                /* Perform eviction once per operation. */
                env.getEvictor().doCriticalEviction();


                FileSummaryLN ln = (FileSummaryLN)
                    cursor.getCurrentLN(LockType.NONE);


                if (ln != null) {
                    /* Stop if the file number changes. */
                    if (fileNumVal != ln.getFileNumber(keyEntry.getData())) {
                        break;
                    }


                    TrackedFileSummary tfs =
                        tracker.getTrackedFile(fileNumVal);
                    /* Associate the tracked summary so it will be cleared. */
                    if (tfs != null) {
                        ln.setTrackedSummary(tfs);
                    }


                    /*
                     * Do not evict after deleting since the compressor would
                     * have to fetch it again.

View Full Code Here

        tmp.add(summary);
        tmp.add(tfs);
        int sequence = tmp.getEntriesCounted();


        /* Insert an LN with the existing and tracked summary info. */
        FileSummaryLN ln = new FileSummaryLN(summary);
        ln.setTrackedSummary(tfs);
        insertFileSummary(ln, fileNum, sequence);


        /* Cache the updated summary object.  */
        summary = ln.getBaseSummary();
        if (fileSummaryMap.put(fileNumLong, summary) == null) {
            MemoryBudget mb = env.getMemoryBudget();
            mb.updateMiscMemoryUsage
                (MemoryBudget.UTILIZATION_PROFILE_ENTRY);
        }


        return ln.getObsoleteOffsets();
    }

View Full Code Here

            while (status == OperationStatus.SUCCESS) {


                /* Perform eviction once per operation. */
                env.getEvictor().doCriticalEviction();


                FileSummaryLN ln = (FileSummaryLN)
                    cursor.getCurrentLN(LockType.NONE);
                if (ln != null) {
                    /* Stop if the file number changes. */
                    if (fileNumVal != ln.getFileNumber(keyEntry.getData())) {
                        break;
                    }


                    PackedOffsets offsets = ln.getObsoleteOffsets();
                    if (offsets != null) {
                        list.add(offsets.toArray());
                    }


                    /* Always evict after using a file summary LN. */

View Full Code Here

                while (status == OperationStatus.SUCCESS) {


                    /* Perform eviction once per operation. */
                    env.getEvictor().doCriticalEviction();


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln == null) {
                        /* Advance past a cleaned record. */
                        status = cursor.getNext
                            (keyEntry, dataEntry, LockType.NONE,
                             true,   // go forward
                             false); // do need to latch
                        continue;
                    }


                    byte[] keyBytes = keyEntry.getData();
                    boolean isOldVersion = ln.hasStringKey(keyBytes);
                    long fileNum = ln.getFileNumber(keyBytes);
                    Long fileNumLong = new Long(fileNum);


                    if (Arrays.binarySearch(existingFiles, fileNumLong) >= 0) {


                        /* File exists, cache the FileSummaryLN. */
                        fileSummaryMap.put(fileNumLong, ln.getBaseSummary());


                        /*
                         * Update old version records to the new version.  A
                         * zero sequence number is used to distinguish the
                         * converted records and to ensure that later records

View Full Code Here

                while (status == OperationStatus.SUCCESS) {


                    /* Perform eviction once per operation. */
                    env.getEvictor().doCriticalEviction();


                    FileSummaryLN ln = (FileSummaryLN)
                        cursor.getCurrentLN(LockType.NONE);


                    if (ln != null) {
                        long fileNumVal = ln.getFileNumber(key.getData());
                        PackedOffsets offsets = ln.getObsoleteOffsets();


                        /*
                         * Check every offset in the fsln to make sure it's
                         * truely obsolete.
                         */

View Full Code Here

            while (status == OperationStatus.SUCCESS) {


                /* Perform eviction once per operation. */
                env.criticalEviction(true /*backgroundIO*/);


                FileSummaryLN ln = (FileSummaryLN)
                    cursor.getCurrentLN(LockType.NONE);


                if (ln != null) {
                    /* Stop if the file number changes. */
                    if (fileNumVal != ln.getFileNumber(keyEntry.getData())) {
                        break;
                    }


                    TrackedFileSummary tfs =
                        tracker.getTrackedFile(fileNumVal);
                    /* Associate the tracked summary so it will be cleared. */
                    if (tfs != null) {
                        ln.setTrackedSummary(tfs);
                        clearedTrackedFile = true;
                    }


                    /*
                     * Do not evict after deleting since the compressor would

View Full Code Here

0 1 2

TOP

Related Classes of com.sleepycat.je.tree.FileSummaryLN

com.sleepycat.je.cleaner.FileSummary

com.sleepycat.je.cleaner.PackedOffsets

com.sleepycat.je.cleaner.SR13061Test

com.sleepycat.je.cleaner.UtilizationProfile

com.sleepycat.je.log.INFileReader

com.sleepycat.je.log.LoggableTest

java.nio.ByteBuffer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of com.sleepycat.je.tree.FileSummaryLN

Log version information

Related Classes of com.sleepycat.je.tree.FileSummaryLN