Examples of HyperLogLogCollector

io.druid.query.aggregation.hyperloglog.HyperLogLogCollector
inria.fr/flajolet/Publications/FlFuGaMe07.pdf Run this code to see a simple indication of expected errors based on different m values: for (int i = 1; i < 20; ++i) { System.out.printf("i[%,d], val[%,d] => error[%f%%]%n", i, 2 << i, 104 / Math.sqrt(2 << i)); } This class is *not* multi-threaded. It can be passed among threads, but it is written with the assumption that only one thread is ever calling methods on it. If you have multiple threads calling methods on this concurrently, I hope you manage to get correct behavior

Examples of io.druid.query.aggregation.hyperloglog.HyperLogLogCollector

        LongWritable key,
        Iterable<BytesWritable> values,
        Context context
    ) throws IOException, InterruptedException
    {
      HyperLogLogCollector aggregate = HyperLogLogCollector.makeLatestCollector();
      for (BytesWritable value : values) {
        aggregate.fold(ByteBuffer.wrap(value.getBytes(), 0, value.getLength()));
      }
      Interval interval = config.getGranularitySpec().getSegmentGranularity().bucket(new DateTime(key.get()));
      intervals.add(interval);
      final Path outPath = config.makeSegmentPartitionInfoPath(interval);
      final OutputStream out = Utils.makePathAndOutputStream(
          context, outPath, config.isOverwriteFiles()
      );


      try {
        HadoopDruidIndexerConfig.jsonMapper.writerWithType(
            new TypeReference<Long>()
            {
            }
        ).writeValue(
            out,
            new Double(aggregate.estimateCardinality()).longValue()
        );
      }
      finally {
        Closeables.close(out, false);
      }

View Full Code Here

Examples of io.druid.query.aggregation.hyperloglog.HyperLogLogCollector

  }


  @Override
  public void aggregate(ByteBuffer buf, int position)
  {
    final HyperLogLogCollector collector = HyperLogLogCollector.makeCollector(
        (ByteBuffer) buf.duplicate().position(position).limit(
            position
            + HyperLogLogCollector.getLatestNumBytesForDenseStorage()
        )
    );

View Full Code Here

Examples of io.druid.query.aggregation.hyperloglog.HyperLogLogCollector

        .setDimensions(Arrays.<DimensionSpec>asList(new DefaultDimensionSpec("a", "a")))
        .setAggregatorSpecs(aggsWithUniques)
        .setPostAggregatorSpecs(POST_AGGS)
        .setContext(CONTEXT);


    final HyperLogLogCollector collector = HyperLogLogCollector.makeLatestCollector();
    collector.add(hashFn.hashString("abc123", Charsets.UTF_8).asBytes());
    collector.add(hashFn.hashString("123abc", Charsets.UTF_8).asBytes());


    testQueryCaching(
        client,
        builder.build(),
        new Interval("2011-01-01/2011-01-02"),

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.