Examples of org.apache.cassandra.utils.obs.OpenBitSet

org.apache.cassandra.utils.obs.OpenBitSet
An "open" BitSet implementation that allows direct access to the arrays of words storing the bits. Derived from Lucene's OpenBitSet, but with a paged backing array (see bits delaration, below).
Unlike java.util.bitset, the fact that bits are packed into an array of longs is part of the interface. This allows efficient implementation of other algorithms by someone other than the author. It also allows one to efficiently implement alternate serialization or interchange formats.
OpenBitSet is faster than java.util.BitSet in most operations and *much* faster at calculating cardinality of sets and results of set operations. It can also handle sets of larger cardinality (up to 64 * 2**32-1)
The goals of OpenBitSet are the fastest implementation possible, and maximum code reuse. Extra safety and encapsulation may always be built on top, but if that's built in, the cost can never be removed (and hence people re-implement their own version in order to get better performance). If you want a "safe", totally encapsulated (and slower and limited) BitSet class, use java.util.BitSet.

        int hashes = dis.readInt();
        int bitLength = dis.readInt();
        long[] bits = new long[bitLength];
        for (int i = 0; i < bitLength; i++)
            bits[i] = dis.readLong();
        OpenBitSet bs = new OpenBitSet(bits, bitLength);
        return new BloomFilter(hashes, bs);
    }

View Full Code Here

        return serializer_;
    }


    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        return new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here


    public BloomFilter deserialize(DataInput dis) throws IOException
    {
        int hashes = dis.readInt();
        long bitLength = dis.readInt();
        OpenBitSet bs = new OpenBitSet(bitLength << 6);
        int pageSize = bs.getPageSize();
        int pageCount = bs.getPageCount();


        for (int p = 0; p < pageCount; p++)
        {
            long[] bits = bs.getPage(p);
            for (int i = 0; i < pageSize && bitLength-- > 0; i++)
                bits[i] = dis.readLong();
        }


        return new BloomFilter(hashes, bs);

View Full Code Here

    }
    
    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        long numBits = numElements * bucketsPer + EXCESS; //TODO overflow?
        return new OpenBitSet((long)Math.min(Long.MAX_VALUE, numBits));
    }

View Full Code Here

    }


    private static IFilter createFilter(int hash, long numElements, int bucketsPer, boolean offheap)
    {
        long numBits = (numElements * bucketsPer) + BITSET_EXCESS;
        IBitSet bitset = offheap ? new OffHeapBitSet(numBits) : new OpenBitSet(numBits);
        return new Murmur3BloomFilter(hash, bitset);
    }

View Full Code Here

        return serializer_;
    }


    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        return new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here

        return serializer_;
    }


    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        return new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here

        int hashes = dis.readInt();
        int bitLength = dis.readInt();
        long[] bits = new long[bitLength];
        for (int i = 0; i < bitLength; i++)
            bits[i] = dis.readLong();
        OpenBitSet bs = new OpenBitSet(bits, bitLength);
        return new BloomFilter(hashes, bs);
    }

View Full Code Here

    public void testBitSetOperations()
    {
        long size_to_test = Integer.MAX_VALUE / 40;
        long size_and_excess = size_to_test + 20;
        OffHeapBitSet offbs = new OffHeapBitSet(size_and_excess);
        OpenBitSet obs = new OpenBitSet(size_and_excess);
        for (long i = 0; i < size_to_test; i++)
            populateRandom(offbs, obs, i);


        for (long i = 0; i < size_to_test; i++)
            compare(offbs, obs, i);

View Full Code Here

    public void timeit()
    {
        long size_to_test = Integer.MAX_VALUE / 10; // about 214 million
        long size_and_excess = size_to_test + 20;


        OpenBitSet obs = new OpenBitSet(size_and_excess);
        OffHeapBitSet offbs = new OffHeapBitSet(size_and_excess);
        logger.info("||Open BS set's|Open BS get's|Open BS clear's|Offheap BS set's|Offheap BS get's|Offheap BS clear's|");
        // System.out.println("||Open BS set's|Open BS get's|Open BS clear's|Offheap BS set's|Offheap BS get's|Offheap BS clear's|");
        loopOnce(obs, offbs, size_to_test);
    }

View Full Code Here

0 1 2 3

TOP

Related Classes of org.apache.cassandra.utils.obs.OpenBitSet

org.apache.cassandra.utils.BitSetTest

org.apache.cassandra.utils.BloomFilter

org.apache.cassandra.utils.BloomFilterSerializer

org.apache.cassandra.utils.FilterFactory

org.apache.cassandra.utils.LongBitSetTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.