Examples of org.apache.hadoop.hive.ql.optimizer.index.RewriteGBUsingIndex

Package org.apache.hadoop.hive.ql.optimizer.index

Examples of org.apache.hadoop.hive.ql.optimizer.index.RewriteGBUsingIndex

org.apache.hadoop.hive.ql.optimizer.index.RewriteGBUsingIndex
RewriteGBUsingIndex is implemented as one of the Rule-based Optimizations. Implements optimizations for GroupBy clause rewrite using aggregate index. This optimization rewrites GroupBy query over base table to the query over simple table-scan over index table, if there is index on the group by key(s) or the distinct column(s). E.g. select count(key) from table group by key; to select sum(_count_of_key) from idx_table group by key; The rewrite supports following queries:
- Queries having only those col refs that are in the index key.
- Queries that have index key col refs
  - in SELECT
  - in WHERE
  - in GROUP BY
- Queries with agg func COUNT(index key col ref) in SELECT
- Queries with SELECT DISTINCT index_key_col_refs
- Queries having a subquery satisfying above condition (only the subquery is rewritten)
@see AggregateIndexHandler @see IndexUtils @see RewriteCanApplyCtx @see RewriteCanApplyProcFactory @see RewriteParseContextGenerator @see RewriteQueryUsingAggregateIndexCtx @see RewriteQueryUsingAggregateIndex For test cases, @see ql_rewrite_gbtoidx.q

    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new SkewJoinOptimizer());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGROUPBY) ||
        HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_MAP_GROUPBY_SORT)) {
      transformations.add(new GroupByOptimizer());
    }

View Full Code Here

      transformations.add(new PredicatePushDown());
      transformations.add(new PartitionPruner());
      transformations.add(new PartitionConditionRemover());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGROUPBY)) {
      transformations.add(new GroupByOptimizer());
    }
    transformations.add(new SamplePruner());

View Full Code Here

    transformations.add(new ColumnPruner());
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new SkewJoinOptimizer());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    transformations.add(new SamplePruner());
    transformations.add(new MapJoinProcessor());
    boolean bucketMapJoinOptimizer = false;
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTBUCKETMAPJOIN)) {

View Full Code Here

      } else {
        LOG.warn("Skew join is currently not supported in tez! Disabling the skew join optimization.");
      }
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    transformations.add(new SamplePruner());
    transformations.add(new MapJoinProcessor());


    if ((HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTBUCKETMAPJOIN)) && !isTezExecEngine) {

View Full Code Here

    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new SkewJoinOptimizer());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGROUPBY) ||
        HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_MAP_GROUPBY_SORT)) {
      transformations.add(new GroupByOptimizer());
    }

View Full Code Here

    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME)) {
      transformations.add(new SkewJoinOptimizer());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGBYUSINGINDEX)) {
      transformations.add(new RewriteGBUsingIndex());
    }
    if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTGROUPBY) ||
        HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVE_MAP_GROUPBY_SORT)) {
      transformations.add(new GroupByOptimizer());
    }

View Full Code Here

TOP

Related Classes of org.apache.hadoop.hive.ql.optimizer.index.RewriteGBUsingIndex

org.apache.hadoop.hive.metastore.api.Index

org.apache.hadoop.hive.metastore.api.StorageDescriptor

org.apache.hadoop.hive.ql.exec.TableScanOperator

org.apache.hadoop.hive.ql.metadata.Hive

org.apache.hadoop.hive.ql.metadata.Table

org.apache.hadoop.hive.ql.optimizer.Optimizer

org.apache.hadoop.hive.ql.parse.SemanticException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.