Examples of org.apache.flink.compiler.PactCompiler$GraphCreatingVisitor

org.apache.flink.compiler.PactCompiler
This utility class performs the translation from the user specified program to the optimizer plan. It works as a visitor that walks the user's job in a depth-first fashion. During the descend, it creates an optimizer node for each operator, respectively data source or -sink. During the ascend, it connects the nodes to the full graph.
This translator relies on the setInputs method in the nodes. As that method implements the size estimation and the awareness for optimizer hints, the sizes will be properly estimated and the translated plan already respects all optimizer hints.

    this.configuration = config;
    configuration.setString(ConfigConstants.JOB_MANAGER_IPC_ADDRESS_KEY, jobManagerAddress.getAddress().getHostAddress());
    configuration.setInteger(ConfigConstants.JOB_MANAGER_IPC_PORT_KEY, jobManagerAddress.getPort());
    
    this.userCodeClassLoader = userCodeClassLoader;
    this.compiler = new PactCompiler(new DataStatistics(), new DefaultCostEstimator());
  }

View Full Code Here

    if (port < 0) {
      throw new CompilerException("Cannot find port to job manager's RPC service in the global configuration.");
    }


    this.userCodeClassLoader = userCodeClassLoader;
    this.compiler = new PactCompiler(new DataStatistics(), new DefaultCostEstimator());
  }

View Full Code Here

    // generate plan
    Plan plan = new Plan(output);
    plan.setDefaultParallelism(DOP);


    // optimize and compile plan 
    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);
    
    // return job graph of failing job
    NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);
  }

View Full Code Here

    // generate plan
    Plan plan = new Plan(output);
    plan.setDefaultParallelism(4);


    // optimize and compile plan
    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);


    // return job graph of working job
    NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);
  }

View Full Code Here

    output.addInput(testMapper2);
    
    Plan plan = new Plan(output);
    plan.setDefaultParallelism(DOP);


    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);


    NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);
  }

View Full Code Here

      return;
    }
  }


  private JobGraph getJobGraph(final Plan plan) throws Exception {
    final PactCompiler pc = new PactCompiler(new DataStatistics());
    final OptimizedPlan op = pc.compile(plan);
    final NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);
  }

View Full Code Here

    output.setInput(testReducer);
    testReducer.setInput(input);


    Plan plan = new Plan(output);


    PactCompiler pc = new PactCompiler(new DataStatistics());
    OptimizedPlan op = pc.compile(plan);


    NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
    return jgg.compileJobGraph(op);


  }

View Full Code Here

  // ------------------------------------------------------------------------  
  
  @Before
  public void setup() {
    this.dataStats = new DataStatistics();
    this.withStatsCompiler = new PactCompiler(this.dataStats, new DefaultCostEstimator());
    this.withStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    this.noStatsCompiler = new PactCompiler(null, new DefaultCostEstimator());
    this.noStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
  }

View Full Code Here

    
    
    private OptimizedPlan compileProgram(String jobName) {
      Plan p = createProgramPlan(jobName);
      
      PactCompiler pc = new PactCompiler(new DataStatistics());
      return pc.compile(p);
    }

View Full Code Here

        // we use the existing session
        shutDownAtEnd = false;
      }


      try {
        PactCompiler pc = new PactCompiler(new DataStatistics());
        OptimizedPlan op = pc.compile(plan);
        
        NepheleJobGraphGenerator jgg = new NepheleJobGraphGenerator();
        JobGraph jobGraph = jgg.compileJobGraph(op);
        
        JobClient jobClient = this.nephele.getJobClient(jobGraph);

View Full Code Here

0 1

TOP

Related Classes of org.apache.flink.compiler.PactCompiler$GraphCreatingVisitor

eu.stratosphere.compiler.dag.BinaryUnionNode

eu.stratosphere.compiler.dag.BulkIterationNode

eu.stratosphere.compiler.dag.BulkPartialSolutionNode

eu.stratosphere.compiler.dag.CoGroupNode

eu.stratosphere.compiler.dag.CollectorMapNode

eu.stratosphere.compiler.dag.CrossNode

eu.stratosphere.compiler.dag.DataSinkNode

eu.stratosphere.compiler.dag.DataSourceNode

eu.stratosphere.compiler.dag.FilterNode

eu.stratosphere.compiler.dag.FlatMapNode

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.