Package org.apache.storm.hdfs.bolt.rotation

Examples of org.apache.storm.hdfs.bolt.rotation.FileSizeRotationPolicy

58
59
60
61
62
63
64
65
66
67
68
        // sync the filesystem after every 1k tuples
        SyncPolicy syncPolicy = new CountSyncPolicy(1000);

        // rotate files when they reach 5MB
        FileRotationPolicy rotationPolicy = new FileSizeRotationPolicy(5.0f, Units.MB);

        FileNameFormat fileNameFormat = new DefaultFileNameFormat()
                .withPath("/source/")
                .withExtension(".seq");
View Full Code Here
48
49
50
51
52
53
54
55
56
57
58
    fileNameFormat.withPath("/tmp/storm");
    fileNameFormat.withPrefix("karma");
    sequenceFileBolt.withFileNameFormat(fileNameFormat);
    sequenceFileBolt.withFsUrl("file:///");
    sequenceFileBolt.withSyncPolicy(new CountSyncPolicy(1));
    sequenceFileBolt.withRotationPolicy(new FileSizeRotationPolicy(1, Units.KB));
    Set<String> sources = new HashSet<String>();
    sources.add(source);
    KarmaReducerBolt reducerBolt = new KarmaReducerBolt(sources);
    builder.setBolt("karma-reducer-json", reducerBolt).fieldsGrouping("karma-generate-json", new Fields("id"));
    builder.setBolt("karma-output-json", sequenceFileBolt).shuffleGrouping("karma-reducer-json");
View Full Code Here
TOP

Related Classes of org.apache.storm.hdfs.bolt.rotation.FileSizeRotationPolicy

  • edu.isi.karma.storm.TestBasicKarmaTopology
  • org.apache.storm.hdfs.bolt.SequenceFileTopology
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.