Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus$SerializationHolder

Package com.clearspring.analytics.stream.cardinality

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus$SerializationHolder

com.clearspring.analytics.stream.cardinality.HyperLogLogPlus
This class exists to support Externalizable semantics for HyperLogLog objects without having to expose a public constructor, public write/read methods, or pretend final fields aren't final. In short, Externalizable allows you to skip some of the more verbose meta-data default Serializable gets you, but still includes the class name. In that sense, there is some cost to this holder object because it has a longer class name. I imagine people who care about optimizing for that have their own work-around for long class names in general, or just use a custom serialization framework. Therefore we make no attempt to optimize that here (eg. by raising this from an inner class and giving it an unhelpful name).

    
    Object result = udafEvaluator.terminate(agg);
    Assert.assertNotNull(result);
    
    byte[] b = ((JavaBinaryObjectInspector) finalOutputOi).getPrimitiveJavaObject(result);
    HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( b );
    Long cardEst = hll.cardinality();
    
    LOG.info("cardEst = " + cardEst);
    
    int actualUniques = h.keySet().size();
    LOG.info("actualUniques = " + actualUniques);

View Full Code Here

    
      byte[] bref1 = this.binary1Inspector.getPrimitiveJavaObject(blobObj1);
      byte[] bref2 = this.binary2Inspector.getPrimitiveJavaObject(blobObj2);
    
      if(bref1 != null && bref2 != null) {
        HyperLogLogPlus hll1 = HyperLogLogPlus.Builder.build( bref1 );
        HyperLogLogPlus hll2 = HyperLogLogPlus.Builder.build( bref2 );
      
        ICardinality merged = hll1.merge(hll2);
      return merged.getBytes();
      } else {
        return null;

View Full Code Here

    return precision != 0;
  }


  public void init(int precision) {
    this.precision = precision;
    hll = new HyperLogLogPlus(precision);
  }

View Full Code Here

      ///HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( bref.getData() );
      
      byte[] bref = this.binaryInspector.getPrimitiveJavaObject(blobObj);
      if(bref == null)
        return null;
      HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( bref );
      
      
      return hll.cardinality();
    } catch(Exception e) {
      LOG.error("Error", e);
      throw new HiveException(e);
    }

View Full Code Here

    this.outputOption = outputOption;
  }


  @Override
  public ICardinality initialize() {
    return new HyperLogLogPlus(precision);
  }

View Full Code Here

TOP

Related Classes of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus$SerializationHolder

brickhouse.udf.hll.CombineHyperLogLogUDF

brickhouse.udf.hll.EstimateCardinalityUDF

brickhouse.udf.hll.HLLBuffer

brickhouse.udf.hll.HyperLogLogUDAFTest

com.liveramp.cascading_ext.combiner.lib.HyperLogLogExactAggregator

java.io.ByteArrayInputStream

java.io.DataOutputStream

java.io.ByteArrayOutputStream

java.io.DataInputStream

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.