Examples of org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin

Package org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Examples of org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin
This operator implements merge join algorithm to do map side joins. Currently, only two-way joins are supported. One input of join is identified as left and other is identified as right. Left input tuples are the input records in map. Right tuples are read from HDFS by opening right stream. This join doesn't support outer join. Data is assumed to be sorted in ascending order. It will fail if data is sorted in descending order.

            if(usePOMergeJoin){
                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,inp,joinPlans,keyTypes);
                }
                catch (PlanException e) {
                    int errCode = 2042;
                    String msg = "Merge Join creation failed";

View Full Code Here

            logToPhyMap.put(loj, pfrj);
        }
        
        else if (loj.getJoinType() == LOJoin.JOINTYPE.MERGE && validateMergeJoin(loj)) {
            
            POMergeJoin smj;
            try {
                smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),loj.getRequestedParallelisam(),inp,joinPlans,keyTypes);
            }
            catch (Exception e) {
                int errCode = 2042;
                String msg = "Merge Join creation failed";
                throw new LogicalToPhysicalTranslatorException(msg, errCode, PigException.BUG, e);
            }


            smj.setResultType(DataType.TUPLE);
            currentPlan.add(smj);


            for (Operator op : inputs) {
                try {
                    currentPlan.connect(logToPhyMap.get(op), smj);

View Full Code Here

            if(usePOMergeJoin){
                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,inp,joinPlans,keyTypes);
                }
                catch (PlanException e) {
                    int errCode = 2042;
                    String msg = "Merge Join creation failed";

View Full Code Here


                // inner join on two sorted inputs. We have less restrictive
                // implementation here in a form of POMergeJoin which doesn't
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,
                                            inp,
                                            joinPlans,
                                            keyTypes,
                                            loj.getJoinType(),

View Full Code Here


                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,
                                            inp,
                                            joinPlans,
                                            keyTypes,
                                            loj.getJoinType(),

View Full Code Here


                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,
                                            inp,
                                            joinPlans,
                                            keyTypes,
                                            loj.getJoinType(),

View Full Code Here


                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,
                                            inp,
                                            joinPlans,
                                            keyTypes,
                                            loj.getJoinType(),

View Full Code Here

            if(usePOMergeJoin){
                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,inp,joinPlans,keyTypes, loj.getJoinType());
                }
                catch (PlanException e) {
                    int errCode = 2042;
                    String msg = "Merge Join creation failed";

View Full Code Here

            if(usePOMergeJoin){
                // inner join on two sorted inputs. We have less restrictive 
                // implementation here in a form of POMergeJoin which doesn't 
                // require loaders to implement collectable interface.
                try {
                    smj = new POMergeJoin(new OperatorKey(scope,nodeGen.getNextNodeId(scope)),
                                            parallel,inp,joinPlans,keyTypes);
                }
                catch (PlanException e) {
                    int errCode = 2042;
                    String msg = "Merge Join creation failed";

View Full Code Here

TOP

Related Classes of org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin

org.apache.hadoop.conf.Configuration

org.apache.hadoop.mapreduce.Job

org.apache.pig.backend.hadoop.executionengine.physicalLayer.Result

org.apache.pig.data.Tuple

org.apache.pig.experimental.logical.relational.LogToPhyTranslationVisitor

org.apache.pig.impl.builtin.DefaultIndexableLoader

org.apache.pig.impl.io.BufferedPositionedInputStream

org.apache.pig.impl.io.FileSpec

org.apache.pig.impl.plan.OperatorKey

org.apache.pig.newplan.logical.relational.LogToPhyTranslationVisitor

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.