Examples of LogicalNode

com.cloudera.flume.agent.LogicalNode
This is a logical node. This contains state about a node, it does not have any specific RPC mechanism associated with it. TODO (jon) this is a slight lie, FlumeConfigData is a thrift generate struct. Here is how a configuration is loaded, and where errors are handled: A call to checkConfig triggers a logicalNode update check based on info provided by FlumeConfigData for the logical node. It is assumed that there are not multiple concurrent checkConfig calls. If the config needs to be updated, the logical node updates itself via loadConfig. If there is a previous configuration the driver, its source and its sink is first closed. Configuration for a source and a sink are instantiated and then instantiated into a new Driver. Any parsing/instantiation failures exit by throwing exceptions. Currently a separate thread is used to reconfigure a node. A previous patch made the logicalNode lazily open the source and sinks. This defers real open errors into the main driver thread. This actually differs any open exceptions into the driver thread which is actually a simplifies error handling. An instantiated driver that pulls events out of the sources and into the sink. Run this until an unhandled exception occurs or the source exits with null signaling that it has been completely drained. TODO (jon) This class is not consistently synchronized currently. Another look at this code may be necessary.
org.apache.tajo.engine.planner.logical.LogicalNode

Examples of com.cloudera.flume.agent.LogicalNode

    BenchmarkHarness.cleanupLocalWriteDir();
  }


  LogicalNode setupAgent(long count, String agentSink) throws IOException,
      RuntimeException, FlumeSpecException {
    LogicalNode agent = new LogicalNode(
        new LogicalNodeContext("phys", "agent"), "agent");
    FlumeConfigData fcd = new FlumeConfigData(0, "asciisynth(" + count + ")",
        agentSink, 1, 1, "flow");
    agent.loadConfig(fcd);
    return agent;
  }

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

    Set<String> alreadyJoinedRelNames = new HashSet<String>();


    // Get candidates from all relations to be joined
    List<RelationNode> candidates = getSmallerRelations(relationMap, remainRelNames);


    LogicalNode lastOne = candidates.get(0); // Get the first candidate relation (i.e., smallest)
    remainRelNames.remove(((RelationNode)lastOne).getCanonicalName()); // Remove the first candidate relation


    // Add the first candidate to the set of joined relations
    alreadyJoinedRelNames.add(((RelationNode)lastOne).getCanonicalName());

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

    TaskAttemptContext
        ctx = new TaskAttemptContext(conf, LocalTajoTestingUtility.newQueryUnitAttemptId(),
        new Fragment[] {frags[0]}, testDir);
    Expr expr = analyzer.parse(SORT_QUERY[0]);
    LogicalPlan plan = planner.createPlan(expr);
    LogicalNode rootNode = optimizer.optimize(plan);


    PhysicalPlanner phyPlanner = new PhysicalPlannerImpl(conf,sm);
    PhysicalExec exec = phyPlanner.createPlan(ctx, rootNode);


    ProjectionExec proj = (ProjectionExec) exec;

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

    TaskAttemptContext
        ctx = new TaskAttemptContext(conf, LocalTajoTestingUtility.newQueryUnitAttemptId(),
        new Fragment[] {frags[0]}, testDir);
    Expr expr = analyzer.parse(SORT_QUERY[1]);
    LogicalPlan plan = planner.createPlan(expr);
    LogicalNode rootNode = optimizer.optimize(plan);


    PhysicalPlanner phyPlanner = new PhysicalPlannerImpl(conf,sm);
    PhysicalExec exec = phyPlanner.createPlan(ctx, rootNode);


    ProjectionExec proj = (ProjectionExec) exec;

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

  };


  @Test
  public final void testFullOuterMergeJoin0() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[0]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode




  @Test
  public final void testFullOuterMergeJoin1() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[1]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

  }


  @Test
  public final void testFullOuterMergeJoin2() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[2]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode

  }


  @Test
  public final void testFullOuterMergeJoin3() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[3]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode




  @Test
  public final void testFullOuterMergeJoin4() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[4]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

Examples of org.apache.tajo.engine.planner.logical.LogicalNode




  @Test
  public final void testFullOuterMergeJoin5() throws IOException, PlanningException {
    Expr expr = analyzer.parse(QUERIES[5]);
    LogicalNode plan = planner.createPlan(expr).getRootBlock().getRoot();
    JoinNode joinNode = PlannerUtil.findTopNode(plan, NodeType.JOIN);
    Enforcer enforcer = new Enforcer();
    enforcer.enforceJoinAlgorithm(joinNode.getPID(), JoinAlgorithm.MERGE_JOIN);


    Fragment[] emp3Frags = StorageManager.splitNG(conf, "emp3", emp3.getMeta(), emp3.getPath(), Integer.MAX_VALUE);

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.