Examples of com.cloudera.flume.core.connector.DirectDriver

Package com.cloudera.flume.core.connector

Examples of com.cloudera.flume.core.connector.DirectDriver

com.cloudera.flume.core.connector.DirectDriver
This connector hooks a source to a sink and allow this connection to be stopped and started. This assumes that sources and sinks are opened before the start() method is called

    EventSink snk = new ConsoleEventSink();
    EventSink snk2 = new BenchmarkReportDecorator<EventSink>("report", snk);
    EventSink snk3 = new BenchmarkInjectDecorator<EventSink>(snk2);
    EventSink snk4 = new InMemoryDecorator<EventSink>(snk3);


    DirectDriver connect = new DirectDriver(src, snk4);
    src.open();
    snk4.open();
    connect.start();
    connect.join(Long.MAX_VALUE);
    src.close();
    snk4.close();
    snk2.getReport().toText(new OutputStreamWriter(System.err));
  }

View Full Code Here

    EventSource src = new NoNlASCIISynthSource(25, 100, 1);
    EventSink snk = new ConsoleEventSink();
    EventSink snk2 = new BenchmarkReportDecorator<EventSink>("report", snk);
    EventSink snk3 = new BenchmarkInjectDecorator<EventSink>(snk2);


    DirectDriver connect = new DirectDriver(src, snk3);
    src.open();
    snk3.open();
    connect.start();
    connect.join(Long.MAX_VALUE);
    src.close();
    snk3.close();
    snk2.getReport().toText(new OutputStreamWriter(System.err));
  }

View Full Code Here

    // one driver for each choke
    DirectDriver[] directDriverArray = new DirectDriver[numDrivers];


    for (int i = 0; i < numDrivers; i++) {
      // Driver i is mapped to ith choke, simple 1 to 1 mapping.
      directDriverArray[i] = new DirectDriver("TestDriver" + i,
          new SynthSourceRndSize(0, minMsgSize, maxMsgSize), tchokeArray[i]);
    }


    // check if all the ChokeIDs are present in the chokeMap
    LOG.info("Running the Individual Test Now!");

View Full Code Here

    int randChokeIndex = 0;
    for (int i = 0; i < numDrivers; i++) {
      randChokeIndex = rand.nextInt(numChokes);
      // DirectDriverArray[i] = new DirectDriver(new SynthSourceRndSize(0,
      // minMsgSize, maxMsgSize), tchokeArray[randChokeIndex]);
      directDriverArray[i] = new DirectDriver(new SynthSourceRndSize(0,
          minMsgSize, maxMsgSize), tchokeArray[randChokeIndex]);


      // adds this choke to the set of chokesUsed
      chokesUsed.add(tchokeArray[randChokeIndex]);
    }

View Full Code Here

    snk.open();


    EventSource src = FlumeBuilder.buildSource(SOURCE);
    src.open();


    DirectDriver conn = new DirectDriver(src, snk);
    conn.start();


    conn.join(Long.MAX_VALUE);


    snk.close();
    src.close();
    assertNull(conn.getError());
  }

View Full Code Here

    input.open();


    drainStarted = new CountDownLatch(1);
    drainCompleted = new CountDownLatch(1);


    drainDriver = new DirectDriver("FileFailover", drainSource, sink);
    /**
     * Don't synchronize on DiskFailoverDeco.this in the ConnectorListener
     * otherwise you might get a deadlock.
     */
    drainDriver.registerListener(new DriverListener.Base() {

View Full Code Here

    input.open();


    started = new CountDownLatch(1);
    completed = new CountDownLatch(1);


    conn = new DirectDriver("naive file wal transmit", drainSource, drainSink);
    /**
     * Do not take the lock on NaiveFileWALDeco.this in the connector listener,
     * as it's possible to deadlock within e.g. close()
     */
    conn.registerListener(new DriverListener.Base() {

View Full Code Here

      }
    }


    // this will be replaceable with multi-threaded queueing versions or other
    // mechanisms
    driver = new DirectDriver("logicalNode " + nodeName, src, snk);
    driver.registerListener(new DriverListener() {
      @Override
      public void fireError(Driver conn, Exception ex) {
        LOG.info("Connector " + nodeName + " exited with error "
            + ex.getMessage());

View Full Code Here

0 1 2 3 4

TOP

Related Classes of com.cloudera.flume.core.connector.DirectDriver

com.cloudera.flume.agent.diskfailover.DiskFailoverDeco

com.cloudera.flume.agent.durability.NaiveFileWALDeco

com.cloudera.flume.agent.LogicalNode

com.cloudera.flume.conf.TestFactories

com.cloudera.flume.conf.TestFlumeBuilderFunctional

com.cloudera.flume.core.Event

com.cloudera.flume.core.EventSink

com.cloudera.flume.core.EventSource

com.cloudera.flume.handlers.debug.TestBenchmarkDeco

com.cloudera.flume.handlers.debug.TestChokeDecos

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.