Examples of FlumeConfiguration

com.cloudera.flume.conf.FlumeConfiguration
This is the configuration file for flume. It uses hadoop's xml schema for properties. The default configuration is in $FLUME_HOME/flume-conf.xml and is a singleton. NOTE: The logging system gets configuration information from this file and thus depends on this being initialized. So no logging events are allowed when this is being initialized. If logger is used, an infinite recursion happens and causes a stack overflow.
org.apache.flume.conf.FlumeConfiguration

FlumeConfiguration is an in memory representation of the hierarchical configuration namespace required by the PropertiesFileConfigurationProvider. This class is instantiated with a properties object which is parsed to construct the hierarchy in memory. Once the entire set of properties have been parsed and populated, a validation routine is run that identifies and removes invalid components.
@see org.apache.flume.conf.properties.PropertiesFileConfigurationProvider

Examples of com.cloudera.flume.conf.FlumeConfiguration

      @Override
      public EventSinkDecorator<EventSink> build(Context context,
          String... argv) {
        Preconditions.checkArgument(argv.length <= 3,
            "usage: ackedWriteAhead[(maxMillis,walnode,checkMs)]");
        FlumeConfiguration conf = FlumeConfiguration.get();
        long delayMillis = conf.getAgentLogMaxAge();


        if (argv.length >= 1) {
          delayMillis = Long.parseLong(argv[0]);
        }

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

    }


    case DISK_FAILOVER: {


      // Move these options to the builder.
      FlumeConfiguration conf = FlumeConfiguration.get();
      long maxSingleBo = conf.getFailoverMaxSingleBackoff();
      long initialBo = conf.getFailoverInitialBackoff();
      long maxCumulativeBo = conf.getFailoverMaxCumulativeBackoff();
      String rpc = String.format("rpcSink(\"%s\", %d)", dsthost, port);


      String snk = String.format("< %s ? { diskFailover => { insistentAppend "
          + "=> { stubbornAppend => { insistentOpen(%d,%d,%d) => %s} } } } >",
          rpc, maxSingleBo, initialBo, maxCumulativeBo, rpc);

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

  public static SinkBuilder e2eBuilder() {
    return new SinkBuilder() {
      @Override
      public EventSink build(Context context, String... argv) {
        Preconditions.checkArgument(argv.length <= 2);
        FlumeConfiguration conf = FlumeConfiguration.get();
        String collector = conf.getCollectorHost();
        int port = conf.getCollectorPort();
        if (argv.length >= 1) {
          collector = argv[0];
        }


        if (argv.length >= 2) {

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

    return new SinkBuilder() {
      @Override
      public EventSink build(Context context, String... argv) {
        Preconditions.checkArgument(argv.length >= 2);
        String format = argv[0];
        FlumeConfiguration conf = FlumeConfiguration.get();
        List<String> list = Arrays.asList(argv);
        list = list.subList(1, list.size());


        try {
          return new FailoverChainSink(context, format, list, conf
              .getFailoverInitialBackoff(), conf.getFailoverMaxSingleBackoff());
        } catch (FlumeSpecException e) {
          LOG.error("Bad spec or format", e);
          throw new IllegalArgumentException(e);
        }
      }

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

  public static SinkBuilder dfoBuilder() {
    return new SinkBuilder() {
      @Override
      public EventSink build(Context context, String... argv) {
        Preconditions.checkArgument(argv.length <= 2);
        FlumeConfiguration conf = FlumeConfiguration.get();
        String collector = conf.getCollectorHost();
        int port = conf.getCollectorPort();
        if (argv.length >= 1) {
          collector = argv[0];
        }


        if (argv.length >= 2) {

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

  public static SinkBuilder beBuilder() {
    return new SinkBuilder() {
      @Override
      public EventSink build(Context context, String... argv) {
        Preconditions.checkArgument(argv.length <= 2);
        FlumeConfiguration conf = FlumeConfiguration.get();
        String collector = conf.getCollectorHost();
        int port = conf.getCollectorPort();
        if (argv.length >= 1) {
          collector = argv[0];
        }


        if (argv.length >= 2) {

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

    if (argv.length < 2) {
      System.out.println("need to specify target output file, and source dir");
      System.exit(-1);
    }


    FlumeConfiguration conf = FlumeConfiguration.get();
    Path dst = new Path(argv[0]);
    String src = argv[1];
    FileSystem fs = FileSystem.getLocal(conf);


    SequenceFile.Writer writer = SequenceFile.createWriter(fs, conf, dst,

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

  public void testOpenClose() throws IOException, Exception {
    // Set directory of webapps to build-specific dir
    FlumeConfiguration.get().set(FlumeConfiguration.WEBAPPS_PATH,
        "build/webapps");


    FlumeConfiguration conf = FlumeConfiguration.get();
    String webPath = FlumeNode.getWebPath(conf);
    int port = FlumeConfiguration.get().getNodeStatusPort();
    StatusHttpServer http = new StatusHttpServer("flumeagent", webPath,
        "0.0.0.0", port, false);

View Full Code Here

Examples of com.cloudera.flume.conf.FlumeConfiguration

  public void testAutoFindPort() throws IOException, Exception {
    // Set directory of webapps to build-specific dir
    FlumeConfiguration.get().set(FlumeConfiguration.WEBAPPS_PATH,
        "build/webapps");


    FlumeConfiguration conf = FlumeConfiguration.get();
    String webPath = FlumeNode.getWebPath(conf);
    int port = FlumeConfiguration.get().getNodeStatusPort();
    StatusHttpServer http = new StatusHttpServer("flumeagent", webPath,
        "0.0.0.0", port, true);
    http.start();

View Full Code Here

Examples of org.apache.flume.conf.FlumeConfiguration

      String fileContent = new String(configData, Charsets.UTF_8);
      Properties properties = new Properties();
      properties.load(new StringReader(fileContent));
      configMap = toMap(properties);
    }
    return new FlumeConfiguration(configMap);
  }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.