Examples of eu.stratosphere.compiler.plan.OptimizedPlan

eu.stratosphere.compiler.plan.OptimizedPlan
The optimizer representation of a plan. The optimizer creates this from the user defined PACT job plan. It works on this representation during its optimization. Finally, this plan is translated to a schedule for the nephele runtime system.

        public void reduce(Iterator<Tuple2<String, Double>> values, Collector<Tuple2<String, Double>> out) {}
      }).name("reducer")
      .print().name("sink");
      
      Plan p = env.createProgramPlan();
      OptimizedPlan op = compileNoStats(p);
      
      OptimizerPlanNodeResolver resolver = getOptimizerPlanNodeResolver(op);
      
      // get the original nodes
      SourcePlanNode sourceNode = resolver.getNode("source");

View Full Code Here

      
      reduced.setCombinable(true);
      reduced.print().name("sink");
      
      Plan p = env.createProgramPlan();
      OptimizedPlan op = compileNoStats(p);
      
      OptimizerPlanNodeResolver resolver = getOptimizerPlanNodeResolver(op);
      
      // get the original nodes
      SourcePlanNode sourceNode = resolver.getNode("source");

View Full Code Here

        public void reduce(Iterator<Tuple2<String, Double>> values, Collector<Tuple2<String, Double>> out) {}
      }).name("reducer")
      .print().name("sink");
      
      Plan p = env.createProgramPlan();
      OptimizedPlan op = compileNoStats(p);
      
      OptimizerPlanNodeResolver resolver = getOptimizerPlanNodeResolver(op);
      
      // get the original nodes
      SourcePlanNode sourceNode = resolver.getNode("source");

View Full Code Here

      
      reduced.setCombinable(true);
      reduced.print().name("sink");
      
      Plan p = env.createProgramPlan();
      OptimizedPlan op = compileNoStats(p);
      
      OptimizerPlanNodeResolver resolver = getOptimizerPlanNodeResolver(op);
      
      // get the original nodes
      SourcePlanNode sourceNode = resolver.getNode("source");

View Full Code Here

      }
  
      // return the PACT plan
      Plan plan = new Plan(sinks, "Plans With Multiple Data Sinks");
  
      OptimizedPlan oPlan = compileNoStats(plan);
  
      // ---------- compile plan to nephele job graph to verify that no error is thrown ----------
  
      NepheleJobGraphGenerator jobGen = new NepheleJobGraphGenerator();
      jobGen.compileJobGraph(oPlan);

View Full Code Here

      sinks.add(sinkC);
      
      // return the PACT plan
      Plan plan = new Plan(sinks, "Plans With Multiple Data Sinks");
      
      OptimizedPlan oPlan = compileNoStats(plan);
      
      // ---------- check the optimizer plan ----------
      
      // number of sinks
      Assert.assertEquals("Wrong number of data sinks.", 3, oPlan.getDataSinks().size());
      
      // sinks contain all sink paths
      Set<String> allSinks = new HashSet<String>();
      allSinks.add(out1Path);
      allSinks.add(out2Path);
      allSinks.add(out3Path);
      
      for (SinkPlanNode n : oPlan.getDataSinks()) {
        String path = ((FileDataSink) n.getSinkNode().getPactContract()).getFilePath();
        Assert.assertTrue("Invalid data sink.", allSinks.remove(path));
      }
      
      // ---------- compile plan to nephele job graph to verify that no error is thrown ----------

View Full Code Here

      FileDataSink sink = new FileDataSink(new DummyOutputFormat(), OUT_FILE, co);
      
      // return the PACT plan
      Plan plan = new Plan(sink, "Branching Source Multiple Times");
      
      OptimizedPlan oPlan = compileNoStats(plan);
      
      NepheleJobGraphGenerator jobGen = new NepheleJobGraphGenerator();
      
      //Compile plan to verify that no error is thrown
      jobGen.compileJobGraph(oPlan);

View Full Code Here

      sinks.add(sinkC);
      
      // return the PACT plan
      Plan plan = new Plan(sinks, "Branching Plans With Multiple Data Sinks");
      
      OptimizedPlan oPlan = compileNoStats(plan);
      
      // ---------- check the optimizer plan ----------
      
      // number of sinks
      Assert.assertEquals("Wrong number of data sinks.", 3, oPlan.getDataSinks().size());
      
      // sinks contain all sink paths
      Set<String> allSinks = new HashSet<String>();
      allSinks.add(out1Path);
      allSinks.add(out2Path);
      allSinks.add(out3Path);
      
      for (SinkPlanNode n : oPlan.getDataSinks()) {
        String path = ((FileDataSink) n.getSinkNode().getPactContract()).getFilePath();
        Assert.assertTrue("Invalid data sink.", allSinks.remove(path));
      }
      
      // ---------- compile plan to nephele job graph to verify that no error is thrown ----------

View Full Code Here

  //    sink.addInput(co1);
      
      // return the PACT plan
      Plan plan = new Plan(sink, "Branching of each contract type");
      
      OptimizedPlan oPlan = compileNoStats(plan);
      
      NepheleJobGraphGenerator jobGen = new NepheleJobGraphGenerator();
      
      //Compile plan to verify that no error is thrown
      jobGen.compileJobGraph(oPlan);

View Full Code Here

      
      
      // return the PACT plan
      Plan plan = new Plan(sink, "Branching Union");
      
      OptimizedPlan oPlan = compileNoStats(plan);
      
      NepheleJobGraphGenerator jobGen = new NepheleJobGraphGenerator();
      
      //Compile plan to verify that no error is thrown
      jobGen.compileJobGraph(oPlan);

View Full Code Here

0 1 2 3 4 5 6

TOP

Related Classes of eu.stratosphere.compiler.plan.OptimizedPlan

eu.stratosphere.client.LocalExecutor

eu.stratosphere.client.program.ContextEnvironment

eu.stratosphere.client.RemoteExecutor

eu.stratosphere.client.web.JobSubmissionServlet

eu.stratosphere.compiler.PactCompiler

eu.stratosphere.compiler.PactCompiler$PlanFinalizer

eu.stratosphere.pact.compiler.AdditionalOperatorsTest

eu.stratosphere.pact.compiler.BranchingPlansCompilerTest

eu.stratosphere.pact.compiler.DOPChangeTest

eu.stratosphere.pact.compiler.GroupOrderTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.