Examples of com.netflix.lipstick.util.OutputSampler

com.netflix.lipstick.util.OutputSampler
Output sampler of intermediate results of Pig jobs. @author nbates


        if(enableSampleOutput) {
            // Get sample output for the job
            try {
                P2jSampleOutputList sampleOutputList = new P2jSampleOutputList();
                OutputSampler os = new OutputSampler(jobStats);
                // The 10 & 1024 params (maxRows and maxBytes)
                // should be configurable via properties
                for (SampleOutput schemaOutputPair : os.getSampleOutputs(10, 1024)) {
                    P2jSampleOutput sampleOutput = new P2jSampleOutput();
                    sampleOutput.setSchemaString(schemaOutputPair.getSchema());
                    sampleOutput.setSampleOutput(schemaOutputPair.getOutput());
                    sampleOutputList.add(sampleOutput);
                }

View Full Code Here

        posList.add(pos);


        JobStats js = getJobStats();
        addStoresToJobStats(js, posList);


        OutputSampler os = getOutputSamplerWithOverriddenLoader(is, js);
        return os.getSampleOutputs(recordCount, byteCount);
    }

View Full Code Here

        }
        return recordList;
    }


    private OutputSampler getOutputSamplerWithOverriddenLoader(final InterStorage is, JobStats js) {
        OutputSampler os = new OutputSampler(js) {
            @Override
            protected LoadFunc getLoader(POStore pos) {
                return is;
            }
        };

View Full Code Here


    class ChildClass extends BaseClass {};


    @Test
    public void testGetInheritedFieldValue() throws Exception {
        OutputSampler sampler = new OutputSampler(null);
        ChildClass obj = new ChildClass();
        String value = (String)sampler.getInheritedFieldValue(obj, "BaseClsAttr");
        Assert.assertEquals("foo", value);
    }

View Full Code Here

TOP

Related Classes of com.netflix.lipstick.util.OutputSampler

com.netflix.lipstick.pigtolipstick.BasicP2LClient

com.netflix.lipstick.util.OutputSamplerTest

org.apache.pig.data.Tuple

org.apache.pig.impl.io.ReadToEndLoader

org.apache.pig.impl.PigContext

org.apache.pig.LoadFunc

java.lang.reflect.Field

java.lang.Class

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.