Examples of org.apache.pig.newplan.PredicatePushDownFilterExtractor

org.apache.pig.newplan.PredicatePushDownFilterExtractor
We traverse the expression plan bottom up and separate it into two plans - pushdownExprPlan, plan that can be pushed down to the loader and - filterExprPlan, remaining plan that needs to be evaluated by pig If the predicate is not removable then filterExprPlan will not have the pushdownExprPlan removed.

        public void transform(OperatorPlan matched) throws FrontendException {
            subPlan = new OperatorSubPlan( currentPlan );


            setupColNameMaps();


            PredicatePushDownFilterExtractor filterFinder = new PredicatePushDownFilterExtractor(
                    loFilter.getFilterPlan(), getMappedKeys( predicateFields ), loadPredPushdown.getSupportedExpressionTypes() );
            filterFinder.visit();
            Expression pushDownPredicate = filterFinder.getPushDownExpression();


            if(pushDownPredicate != null) {
                // the column names in the filter may be the ones provided by
                // the user in the schema in the load statement - we may need
                // to replace them with partition column names as given by

View Full Code Here


    private Expression getExpressionForTest(String query, List<String> predicateCols) throws Exception {
        LogicalPlan newLogicalPlan = Util.buildLp(pigServer, query);
        Operator op = newLogicalPlan.getSinks().get(0);
        LOFilter filter = (LOFilter) newLogicalPlan.getPredecessors(op).get(0);
        PredicatePushDownFilterExtractor filterExtractor = new PredicatePushDownFilterExtractor(filter.getFilterPlan(), predicateCols, supportedOpTypes);
        filterExtractor.visit();
        return filterExtractor.getPushDownExpression();
    }

View Full Code Here

TOP

Related Classes of org.apache.pig.newplan.PredicatePushDownFilterExtractor

org.apache.pig.builtin.TestOrcStoragePushdown

org.apache.pig.newplan.logical.rules.PredicatePushdownOptimizer$PredicatePushDownTransformer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.