Examples of org.apache.pig.LoadFunc.bindTo()

org.apache.pig.LoadFunc.bindTo()

Specifies a portion of an InputStream to read tuples. Because the starting and ending offsets may not be on record boundaries it is up to the implementor to deal with figuring out the actual starting and ending offsets in such a way that an arbitrarily sliced up file will be processed in its entirety.

A common way of handling slices in the middle of records is to start at the given offset and, if the offset is not zero, skip to the end of the first record (which may be a partial record) before reading tuples. Reading continues until a tuple has been read that ends at an offset past the ending offset.

The load function should not do any buffering on the input stream. Buffering will cause the offsets returned by is.getPos() to be unreliable. @param fileName the name of the file to be read @param is the stream representing the file to be processed, and which can also provide its position. @param offset the offset to start reading tuples. @param end the ending offset for reading. @throws IOException

        sfunc.putNext(f1);
        sfunc.finish();
        
        FakeFSInputStream is = new FakeFSInputStream(buf);
        LoadFunc lfunc = new PigStorage();
        lfunc.bindTo(null, new BufferedPositionedInputStream(is), 0, buf.length);
        Tuple f2 = lfunc.getNext();
        
        assertTrue(f1.equals(f2));        
    }

        try{
             p = (LoadFunc)PigContext.instantiateFuncFromSpec(outFileSpec.getFuncSpec());


             InputStream is = FileLocalizer.open(outFileSpec.getFileName(), pigContext);


             p.bindTo(outFileSpec.getFileName(), new BufferedPositionedInputStream(is), 0, Long.MAX_VALUE);


        }catch (Exception e){
            int errCode = 2088;
            String msg = "Unable to get results for: " + outFileSpec;
            throw new ExecException(msg, errCode, PigException.BUG, e);

        String input1 = "this:is:delimited:by:a:colon\n";
        int arity1 = 6;


        LoadFunc p1 = new PigStorage(":");
        FakeFSInputStream ffis1 = new FakeFSInputStream(input1.getBytes());
        p1.bindTo(null, new BufferedPositionedInputStream(ffis1), 0, input1.getBytes().length);
        Tuple f1 = p1.getNext();
        assertTrue(f1.size() == arity1);


        LoadFunc p15 = new PigStorage();
        StringBuilder sb = new StringBuilder();

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.