Examples of com.cloudera.flume.conf.Context

com.cloudera.flume.conf.Context
This context is an abstraction for scopes. Prior to this there were one scopes the physical node (essentially conf file). This allows for new subscopes such as logical node contexts. This allows information to be passed to specific logical nodes (such as logicalnode name). This interface will likely subsume FlumeConfiguration (physical node scope), and configuration information grabbed from the master such as as logical node specific failover chains (master scope). With scope chaining and shadowing we can in the future introduce node specific configuration settings transparently.

    node.openLoadNode(new CloseExnSource(), new NullSink());
  }


  @Test
  public void testRestartNextException() throws Exception {
    LogicalNode node = new LogicalNode(new Context(), "test");


    final AtomicInteger count = new AtomicInteger();


    class NextExnSource extends EventSource.Base {
      @Override

View Full Code Here

  }


  @Test
  public void testFailfastOutException() throws IOException,
      InterruptedException {
    LogicalNode node = new LogicalNode(new Context(), "test");


    class OpenExnSource extends EventSource.Base {
      @Override
      public void close() throws IOException {
      }

View Full Code Here

        isOpen = true;
      }
    }
    ;


    LogicalNode node = new LogicalNode(new Context(), "test");
    IsOpenSource prev = new IsOpenSource();
    node.openLoadNode(prev, new NullSink());
    node.getSource().next(); // force lazy source to open
    for (int i = 0; i < 10; i++) {

View Full Code Here

   * @throws InterruptedException
   */
  @Test
  public void testOpenCloseSyslogTcpSourceThreads() throws IOException,
      InterruptedException {
    LogicalNode node = new LogicalNode(new Context(), "test");
    EventSource prev = new SyslogTcpSourceThreads(6789);
    node.openLoadNode(prev, new NullSink());
    for (int i = 0; i < 20; i++) {
      EventSource cur = new SyslogTcpSourceThreads(6789);
      node.openLoadNode(cur, new NullSink());

View Full Code Here

    FlumeConfiguration.get().set(
        FlumeConfiguration.OUTPUT_FORMAT_PLUGIN_CLASSES, "java.lang.String");
    FlumeNode.loadOutputFormatPlugins();


    try {
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(),
          "console(\"raw\")"));
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(),
          "console(\"avrojson\")"));
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(),
          "console(\"avrodata\")"));
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(),
          "console(\"syslog\")"));
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(),
          "console(\"log4j\")"));
      Assert.assertNotNull(FlumeBuilder.buildSink(new Context(), "console()"));
    } catch (FlumeSpecException e) {
      LOG
          .error(
              "Unable to create a console sink with one of the built in output formats. Exception follows.",
              e);

View Full Code Here

        FlumeConfiguration.OUTPUT_FORMAT_PLUGIN_CLASSES,
        "com.cloudera.flume.agent.TestFlumeNode$TestOutputFormatPlugin");
    FlumeNode.loadOutputFormatPlugins();


    try {
      FlumeBuilder.buildSink(new Context(), "console(\"testformat\")");
    } catch (FlumeSpecException e) {
      LOG
          .error(
              "Caught exception building console sink with testformat output format. Exception follows.",
              e);

View Full Code Here

  @Test
  public void testIndependentEvents() throws FlumeSpecException, IOException,
      InterruptedException {


    String spec = "[ { value(\"foo\",\"bar\") => null}, { value(\"foo\",\"bar\" ) => null } ] ";
    EventSink snk = FlumeBuilder.buildSink(new Context(), spec);
    snk.open();
    snk.append(new EventImpl("event body".getBytes()));


  }

View Full Code Here

    final CountDownLatch start = new CountDownLatch(threads);
    final CountDownLatch done = new CountDownLatch(threads);
    final NaiveFileWALManager wal = new NaiveFileWALManager(dir);
    wal.open();


    Context ctx = new ReportTestingContext();
    EventSink cntsnk = new CompositeSink(ctx, "counter(\"total\")");
    // use the same wal, but different counter.
    final EventSink snk = new NaiveFileWALDeco(ctx, cntsnk, wal,
        new TimeTrigger(new ProcessTagger(), 1000000), new AckListener.Empty(),
        1000000);

View Full Code Here

        @Override
        public void run() {
          start.countDown();
          try {
            EventSource src = new NoNlASCIISynthSource(count, 100);
            Context ctx = new ReportTestingContext();
            EventSink snk = new CompositeSink(ctx, "counter(\"total." + idx
                + "\")");
            // use the same wal, but different counter.
            snk = new NaiveFileWALDeco(ctx, snk, wal, new TimeTrigger(
                new ProcessTagger(), 1000000), new AckListener.Empty(), 1000000);

View Full Code Here

        @Override
        public void run() {
          start.countDown();
          try {
            EventSource src = new NoNlASCIISynthSource(count, 100);
            Context ctx = new ReportTestingContext();
            EventSink snk = new CompositeSink(ctx, "counter(\"total." + idx
                + "\")");
            // use the same wal, but different counter.


            snk = new NaiveFileWALDeco(ctx, snk, wal, new TimeTrigger(

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of com.cloudera.flume.conf.Context

com.cloudera.flume.agent.AgentFailChainSink

com.cloudera.flume.agent.AgentSink

com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager

com.cloudera.flume.agent.diskfailover.TestDiskFailoverBehavior

com.cloudera.flume.agent.diskfailover.TestDiskFailoverDeco

com.cloudera.flume.agent.durability.TestConcurrentWALMan

com.cloudera.flume.agent.durability.TestNaiveFileWALDeco

com.cloudera.flume.agent.durability.TestNaiveFileWALManagerConcurrently

com.cloudera.flume.agent.FlumeNode

com.cloudera.flume.agent.LogicalNodeManager

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.