Examples of org.apache.hadoop.hive.ql.optimizer.correlation.CorrelationOptimizer

Package org.apache.hadoop.hive.ql.optimizer.correlation

Examples of org.apache.hadoop.hive.ql.optimizer.correlation.CorrelationOptimizer

org.apache.hadoop.hive.ql.optimizer.correlation.CorrelationOptimizer
se.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf). Correlation Optimizer detects if ReduceSinkOperators share same keys. Then, it will transform the query plan tree (operator tree) by exploiting detected correlations. For details, see the original paper of YSmart. Test queries associated with this optimizer are correlationoptimizer1.q to correlationoptimizer14.q

      transformations.add(new GlobalLimitOptimizer());
    }
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTCORRELATION) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEGROUPBYSKEW) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new CorrelationOptimizer());
    }
    if (HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVELIMITPUSHDOWNMEMORYUSAGE) > 0) {
      transformations.add(new LimitPushdownOptimizer());
    }
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEMETADATAQUERIES)) {

View Full Code Here

    }
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTCORRELATION) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEGROUPBYSKEW) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME) &&
        !isTezExecEngine) {
      transformations.add(new CorrelationOptimizer());
    }
    if (HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVELIMITPUSHDOWNMEMORYUSAGE) > 0) {
      transformations.add(new LimitPushdownOptimizer());
    }
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTIMIZEMETADATAQUERIES)) {

View Full Code Here

      transformations.add(new GlobalLimitOptimizer());
    }
    if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTCORRELATION) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEGROUPBYSKEW) &&
        !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new CorrelationOptimizer());
    }
    if (HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVELIMITPUSHDOWNMEMORYUSAGE) > 0) {
      transformations.add(new LimitPushdownOptimizer());
    }
    transformations.add(new SimpleFetchOptimizer());  // must be called last

View Full Code Here

TOP

Related Classes of org.apache.hadoop.hive.ql.optimizer.correlation.CorrelationOptimizer

org.apache.hadoop.fs.ContentSummary

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.Path

org.apache.hadoop.hive.ql.exec.GroupByOperator

org.apache.hadoop.hive.ql.exec.JoinOperator

org.apache.hadoop.hive.ql.exec.ReduceSinkOperator

org.apache.hadoop.hive.ql.exec.TableScanOperator

org.apache.hadoop.hive.ql.exec.UnionOperator

org.apache.hadoop.hive.ql.lib.DefaultGraphWalker

org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.