Examples of org.apache.pig.CommittableStoreFunc.commit()

Class org.apache.pig.CommittableStoreFunc

Examples of org.apache.pig.CommittableStoreFunc.commit()

org.apache.pig.CommittableStoreFunc.commit()
This method is called when all mappers (when the storefunc is part of map) or reducers (when the storefunc is part of reduce) are finished. This allows the storeFunc to do any global commit actions - only called when all mappers/reducers successfully complete. If the StoreFunc needs to get hold of StoreConfig object for the store it can call {@link MapRedUtil#getStoreConfig(org.apache.hadoop.mapred.JobConf)} whereconf is the Configuration object passed in the commit() call. @param conf Configuration object for the job

            confCopy.set("pig.storeFunc", ObjectSerializer.serialize(
                    sFuncString));
            confCopy.set(JobControlCompiler.PIG_STORE_CONFIG, 
                    ObjectSerializer.serialize(storeConfig));
            
            csFunc.commit(confCopy);
        }
    }
    
    private void commit(StoreFunc sFunc, Configuration conf,
            String storeConfigSerializedString, String sFuncString) throws IOException {

View Full Code Here

            confCopy.set("pig.storeFunc", ObjectSerializer.serialize(
                    sFuncString));
            confCopy.set(JobControlCompiler.PIG_STORE_CONFIG, 
                    storeConfigSerializedString);
            
            csFunc.commit(confCopy);
        }
    }
    
    private void commitStores(List<POStore> stores, Configuration conf)
    throws IOException {

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.