Examples of org.apache.pig.StoreConfig

org.apache.pig.StoreConfig
A Class which will encapsulate metadata information that a OutputFormat (or possibly StoreFunc) may want to know about the data that needs to be stored.

        Path outputDir = new Path(sFile.getFileName()).makeQualified(FileSystem.get(outputConf));
        outputConf.set("mapred.output.dir", outputDir.toString());


        // Set the schema
        outputConf.set(PIG_STORE_CONFIG, 
                       ObjectSerializer.serialize(new StoreConfig(outputDir.toString(), schema, sortInfo)));


        // The workpath is set to a unique-per-store subdirectory of
        // the current working directory.
        String workPath = outputConf.get("mapred.work.output.dir");
        outputConf.set("mapred.work.output.dir",

View Full Code Here

                // get encoded as regular characters. Otherwise any control characters
                // in the store funcspec would break the job.xml which is created by
                // hadoop from the jobconf.
                jobConf.set("pig.storeFunc", ObjectSerializer.serialize(outputFuncSpec.toString()));
                jobConf.set(PIG_STORE_CONFIG, 
                            ObjectSerializer.serialize(new StoreConfig(outputPath, st.getSchema())));


                jobConf.set("pig.streaming.log.dir", 
                            new Path(outputPath, LOG_DIR).toString());
                jobConf.set("pig.streaming.task.output.dir", outputPath);
            }

View Full Code Here

        Path outputDir = new Path(sFile.getFileName()).makeQualified(FileSystem.get(outputConf));
        outputConf.set("mapred.output.dir", outputDir.toString());


        // Set the schema
        outputConf.set(PIG_STORE_CONFIG, 
                       ObjectSerializer.serialize(new StoreConfig(outputDir.toString(), schema)));


        // The workpath is set to a unique-per-store subdirectory of
        // the current working directory.
        String workPath = outputConf.get("mapred.work.output.dir");
        outputConf.set("mapred.work.output.dir",

View Full Code Here

0 1

TOP

Related Classes of org.apache.pig.StoreConfig

org.apache.hadoop.zebra.pig.TableOutputFormat

org.apache.hadoop.zebra.pig.TableRecordWriter

org.apache.hadoop.zebra.pig.TableStorer

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat

org.apache.pig.test.TestMultiQuery$DummyOutputFormat

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.