Examples of UnionProcessor

org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor
Implementation of the union processor. This can be enhanced later on. Currently, it does the following: Identify if both the subqueries of UNION are map-only. Store that fact in the unionDesc/UnionOperator. If either of the sub-query involves a map-reduce job, a FS is introduced on top of the UNION. This can be later optimized to clone all the operators above the UNION. The parse Context is not changed.

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

   * create the list of transformations
   */
  public void initialize() {
    transformations = new ArrayList<Transform>();
    transformations.add(new ColumnPruner());
    transformations.add(new UnionProcessor());
  }

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

      transformations.add(new GroupByOptimizer());
    }


    transformations.add(new SamplePruner());
    transformations.add(new MapJoinProcessor());
    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
  }

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

      transformations.add(new BucketMapJoinOptimizer());
      if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTSORTMERGEBUCKETMAPJOIN)) {
        transformations.add(new SortedMergeBucketMapJoinOptimizer());
      }
    }
    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTREDUCEDEDUPLICATION)) {
      transformations.add(new ReduceSinkDeDuplication());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVELIMITOPTENABLE)) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

      transformations.add(new PredicatePushDown());
      if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTPPR)) {
        transformations.add(new PartitionPruner());
      }
    }
    transformations.add(new UnionProcessor());
    transformations.add(new MapJoinProcessor());
  }

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

      transformations.add(new BucketMapJoinOptimizer());
      if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTSORTMERGEBUCKETMAPJOIN)) {
        transformations.add(new SortedMergeBucketMapJoinOptimizer());
      }
    }
    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTREDUCEDEDUPLICATION)) {
      transformations.add(new ReduceSinkDeDuplication());
    }
  }

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor


    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEBUCKETINGSORTING)) {
      transformations.add(new BucketingSortingReduceSinkOptimizer());
    }


    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.DYNAMICPARTITIONING) &&
        HiveConf.getVar(hiveConf, HiveConf.ConfVars.DYNAMICPARTITIONINGMODE).equals("nonstrict") &&
        HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTSORTDYNAMICPARTITION) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTLISTBUCKETING)) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor


    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEBUCKETINGSORTING)) {
      transformations.add(new BucketingSortingReduceSinkOptimizer());
    }


    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.DYNAMICPARTITIONING) &&
        HiveConf.getVar(hiveConf, HiveConf.ConfVars.DYNAMICPARTITIONINGMODE).equals("nonstrict") &&
        HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTSORTDYNAMICPARTITION) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTLISTBUCKETING)) {

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor


    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEBUCKETINGSORTING)) {
      transformations.add(new BucketingSortingReduceSinkOptimizer());
    }


    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTREDUCEDEDUPLICATION)) {
      transformations.add(new ReduceSinkDeDuplication());
    }
    transformations.add(new NonBlockingOpDeDupProc());

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor


    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEBUCKETINGSORTING)) {
      transformations.add(new BucketingSortingReduceSinkOptimizer());
    }


    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTREDUCEDEDUPLICATION)) {
      transformations.add(new ReduceSinkDeDuplication());
    }
    transformations.add(new NonBlockingOpDeDupProc());

View Full Code Here

Examples of org.apache.hadoop.hive.ql.optimizer.unionproc.UnionProcessor

      transformations.add(new BucketMapJoinOptimizer());
      if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTSORTMERGEBUCKETMAPJOIN)) {
        transformations.add(new SortedMergeBucketMapJoinOptimizer());
      }
    }
    transformations.add(new UnionProcessor());
    transformations.add(new JoinReorder());
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTREDUCEDEDUPLICATION)) {
      transformations.add(new ReduceSinkDeDuplication());
    }
  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.