Examples of com.oltpbenchmark.distributions.ZipfianGenerator

com.oltpbenchmark.distributions.ZipfianGenerator
A generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of items from min to max inclusive). After you construct the instance, you can change the number of items by calling nextInt(itemcount) or nextLong(itemcount). Note that the popular items will be clustered together, e.g. item 0 is the most popular, item 1 the second most popular, and so on (or min is the most popular, min+1 the next most popular, etc.) If you don't want this clustering, and instead want the popular items scattered throughout the item space, then use ScrambledZipfianGenerator instead. Be aware: initializing this generator may take a long time if there are lots of items to choose from (e.g. over a minute for 100 million objects). This is because certain mathematical values need to be computed to properly generate a zipfian skew, and one of those values (zeta) is a sum sequence from 1 to n, where n is the itemcount. Note that if you increase the number of items in the set, we can compute a new zeta incrementally, so it should be fast unless you have added millions of items. However, if you decrease the number of items, we recompute zeta from scratch, so this can take a long time. The algorithm used here is from "Quickly Generating Billion-Record Synthetic Databases", Jim Gray et al, SIGMOD 1994.

        final PreparedStatement followersInsert = this.conn.prepareStatement(SQLUtil.getInsertSQL(catalog_tbl));


        int total = 1;
        int batchSize = 0;
        
        ZipfianGenerator zipfFollowee = new ZipfianGenerator(this.num_users,1.75);
        ZipfianGenerator zipfFollows = new ZipfianGenerator(this.num_follows,1.75);
        List<Integer> followees = new ArrayList<Integer>();
        for (int follower = 0; follower < this.num_users; follower++) {
            followees.clear();
            int time = zipfFollows.nextInt();
            if(time==0) time=1; // At least this follower will follow 1 user 
            for (int f = 0; f < time; ) {
                int followee = zipfFollowee.nextInt();
                if (follower != followee && !followees.contains(followee)) {
                    followsInsert.setInt(1, follower);

View Full Code Here

        assert (catalog_tbl != null);
        String sql = SQLUtil.getInsertSQL(catalog_tbl);
        PreparedStatement reviewInsert = this.conn.prepareStatement(sql);
        
        //
        ZipfianGenerator numReviews = new ZipfianGenerator(num_reviews, 1.8);
        ZipfianGenerator reviewer = new ZipfianGenerator(num_users);
        int total = 0;
        int batch = 0;
        for (int i = 0; i < num_items; i++) {
            List<Integer> reviewers = new ArrayList<Integer>();
            int review_count = numReviews.nextInt();
            if (review_count == 0)
                review_count = 1; // make sure at least each item has a review
            for (int rc = 0; rc < review_count;) {
                int u_id = reviewer.nextInt();
                if (!reviewers.contains(u_id)) {
                    rc++;
                    reviewInsert.setInt(1, total);
                    reviewInsert.setInt(2, u_id);
                    reviewInsert.setInt(3, i);

View Full Code Here

        PreparedStatement trustInsert = this.conn.prepareStatement(sql);
        
        //
        int total = 0;
        int batch = 0;
        ZipfianGenerator numTrust = new ZipfianGenerator(num_trust, 1.95);
        ScrambledZipfianGenerator reviewed = new ScrambledZipfianGenerator(num_users);
        Random isTrusted = new Random(System.currentTimeMillis());
        for (int i = 0; i < num_users; i++) {
            List<Integer> trusted = new ArrayList<Integer>();
            int trust_count = numTrust.nextInt();
            for (int tc = 0; tc < trust_count;) {
                int u_id = reviewed.nextInt();
                if (!trusted.contains(u_id)) {
                    tc++;
                    trustInsert.setInt(1, i);

View Full Code Here


    private final Map<Integer, String> m = new HashMap<Integer, String>();
    
    public YCSBWorker(int id, BenchmarkModule benchmarkModule, int init_record_count) {
        super(benchmarkModule, id);
        readRecord = new ZipfianGenerator(init_record_count);// pool for read keys
        randScan = new ZipfianGenerator(YCSBConstants.MAX_SCAN);
        
        synchronized (YCSBWorker.class) {
            // We must know where to start inserting
            if (insertRecord == null) {
                insertRecord = new CounterGenerator(init_record_count);

View Full Code Here

TOP

Related Classes of com.oltpbenchmark.distributions.ZipfianGenerator

com.oltpbenchmark.benchmarks.epinions.EpinionsLoader

com.oltpbenchmark.benchmarks.twitter.TwitterLoader

com.oltpbenchmark.benchmarks.ycsb.YCSBWorker

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.