Examples of PactCompiler

eu.stratosphere.compiler.PactCompiler
The optimizer that takes the user specified program plan and creates an optimized plan that contains exact descriptions about how the physical execution will take place. It first translates the user program into an internal optimizer representation and then chooses between different alternatives for shipping strategies and local strategies.
The basic principle is taken from optimizer works in systems such as Volcano/Cascades and Selinger/System-R/DB2. The optimizer walks from the sinks down, generating interesting properties, and ascends from the sources generating alternative plans, pruning against the interesting properties.
The optimizer also assigns the memory to the individual tasks. This is currently done in a very simple fashion: All sub-tasks that need memory (e.g. reduce or join) are given an equal share of memory.
org.apache.flink.compiler.PactCompiler
The optimizer that takes the user specified program plan and creates an optimized plan that contains exact descriptions about how the physical execution will take place. It first translates the user program into an internal optimizer representation and then chooses between different alternatives for shipping strategies and local strategies.
The basic principle is taken from optimizer works in systems such as Volcano/Cascades and Selinger/System-R/DB2. The optimizer walks from the sinks down, generating interesting properties, and ascends from the sources generating alternative plans, pruning against the interesting properties.
The optimizer also assigns the memory to the individual tasks. This is currently done in a very simple fashion: All sub-tasks that need memory (e.g. reduce or join) are given an equal share of memory.

Examples of org.apache.flink.compiler.PactCompiler

    output.setInput(testReducer);
    testReducer.setInput(input);


    Plan plan = new Plan(output);


    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);


    NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);


  }

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

  // ------------------------------------------------------------------------  
  
  @Before
  public void setup() {
    this.dataStats = new DataStatistics();
    this.withStatsCompiler = new PactCompiler(this.dataStats, new DefaultCostEstimator());
    this.withStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    this.noStatsCompiler = new PactCompiler(null, new DefaultCostEstimator());
    this.noStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
  }

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

    
    
    private OptimizedPlan compileProgram(String jobName) {
      Plan p = createProgramPlan(jobName);
      
      PactCompiler pc = new PactCompiler(new DataStatistics());
      return pc.compile(p);
    }

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

        // we use the existing session
        shutDownAtEnd = false;
      }


      try {
        PactCompiler pc = new PactCompiler(new DataStatistics());
        OptimizedPlan op = pc.compile(plan);
        
        NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
        JobGraph jobGraph = jgg.compileJobGraph(op);
        
        JobClient jobClient = this.nephele.getJobClient(jobGraph);

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

   *            The program's plan.
   * @return JSON dump of the optimized plan.
   * @throws Exception
   */
  public String getOptimizerPlanAsJSON(Plan plan) throws Exception {
    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);
    PlanJSONDumpGenerator gen = new PlanJSONDumpGenerator();
  
    return gen.getOptimizerPlanAsJSON(op);
  }

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

   */
  public static String optimizerPlanAsJSON(Plan plan) throws Exception {
    LocalExecutor exec = new LocalExecutor();
    try {
      exec.start();
      PactCompiler pc = new PactCompiler(new DataStatistics());
      OptimizedPlan op = pc.compile(plan);
      PlanJSONDumpGenerator gen = new PlanJSONDumpGenerator();


      return gen.getOptimizerPlanAsJSON(op);
    } finally {
      exec.stop();

View Full Code Here

Examples of org.apache.flink.compiler.PactCompiler

    Plan p = getTestJob();
    if (p == null) {
      Assert.fail("Error: Cannot obtain Pact plan. Did the thest forget to override either 'getPactPlan()' or 'getJobGraph()' ?");
    }
    
    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(p);
    
    if (printPlan) {
      System.out.println(new PlanJSONDumpGenerator().getOptimizerPlanAsJSON(op)); 
    }

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.