Package org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners

Examples of org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.DiscreteProbabilitySampleGenerator

66
67
68
69
70
71
72
73
74
75
76
            convertToArray(quantilesList);
            for (Entry<Object, Object> ent : weightedPartsData.entrySet()) {
                Tuple key = (Tuple) ent.getKey(); // sample item which repeats
                float[] probVec = getProbVec((Tuple) ent.getValue());
                weightedParts.put(getPigNullableWritable(key),
                        new DiscreteProbabilitySampleGenerator(probVec));
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
        comparator = ConfigUtils.getIntermediateInputKeyComparator(job);
View Full Code Here
79
80
81
82
83
84
85
86
87
88
89
        Map<String, Object> res = fq.exec(in);
       
        InternalMap weightedPartsData = (InternalMap) res.get(FindQuantiles.WEIGHTED_PARTS);
        Iterator<Object> it = weightedPartsData.values().iterator();
        float[] probVec = getProbVec((Tuple)it.next());
        new DiscreteProbabilitySampleGenerator(probVec);
        float sum = 0.0f;
        for (float f : probVec) {
            sum += f;
        }
        return sum;
View Full Code Here
133
134
135
136
137
138
139
140
141
142
143
        Map<String, Object> res = getFindQuantilesResult(samples, numReduceres);

        InternalMap weightedPartsData = (InternalMap) res.get(FindQuantiles.WEIGHTED_PARTS);
        Iterator<Object> it = weightedPartsData.values().iterator();
        float[] probVec = getProbVec((Tuple)it.next());
        new DiscreteProbabilitySampleGenerator(probVec);
        float sum = 0.0f;
        for (float f : probVec) {
            sum += f;
        }
        return sum;
View Full Code Here
TOP

Related Classes of org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.DiscreteProbabilitySampleGenerator

  • org.apache.pig.backend.hadoop.executionengine.tez.runtime.WeightedRangePartitionerTez
  • org.apache.pig.test.TestFindQuantiles
  • java.util.Random
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.