Examples of SeqfileEventSink

com.cloudera.flume.handlers.hdfs.SeqfileEventSink
This is an event sink that dumps to a hadoop sequence file on the local file system..

Examples of com.cloudera.flume.handlers.hdfs.SeqfileEventSink

    txt.close();
    b.mark("disk_loaded");


    File tmp = File.createTempFile("test", "tmp");
    tmp.deleteOnExit();
    SeqfileEventSink sink = new SeqfileEventSink(tmp);
    sink.open();
    b.mark("receiver_started");


    EventUtil.dumpAll(mem, sink);


    b.mark("seqfile_disk_write");


    sink.close();
    b.mark("seqfile size", tmp.length());
    b.done();


    // //////// second phase using the file written in previous phase.
    Benchmark b2 = new Benchmark("seqfile_disk_read");

View Full Code Here

Examples of com.cloudera.flume.handlers.hdfs.SeqfileEventSink

    txt.close();
    b.mark("disk_loaded");


    File tmp = File.createTempFile("test", "tmp");
    tmp.deleteOnExit();
    SeqfileEventSink sink = new SeqfileEventSink(tmp);
    sink.open();
    b.mark("localdisk_write_started");


    EventUtil.dumpAll(mem, sink);


    b.mark("local_disk_write done");


    sink.close();


    FlumeConfiguration conf = FlumeConfiguration.get();
    Path src = new Path(tmp.getAbsolutePath());
    Path dst = new Path("hdfs://localhost/testfile");
    FileSystem hdfs = dst.getFileSystem(conf);

View Full Code Here

Examples of com.cloudera.flume.handlers.hdfs.SeqfileEventSink

   */
  synchronized public EventSink newWritingSink(final Tagger tagger)
      throws IOException {
    File dir = getDir(State.WRITING);
    final String tag = tagger.newTag();
    EventSink curSink = new SeqfileEventSink(new File(dir, tag)
        .getAbsoluteFile());
    writingQ.add(tag);
    DFOData data = new DFOData(tag);
    table.put(tag, data);
    writingCount.incrementAndGet();

View Full Code Here

Examples of com.cloudera.flume.handlers.hdfs.SeqfileEventSink

  synchronized public EventSink newAckWritingSink(final Tagger tagger,
      AckListener al) throws IOException {
    File dir = getDir(State.WRITING);
    final String tag = tagger.newTag();


    EventSink bareSink = new SeqfileEventSink(new File(dir, tag)
        .getAbsoluteFile());
    EventSink curSink = new AckChecksumInjector<EventSink>(bareSink, tag
        .getBytes(), al);


    writingQ.add(tag);

View Full Code Here

Examples of com.cloudera.flume.handlers.hdfs.SeqfileEventSink

   */
  synchronized public EventSink newWritingSink(final Tagger tagger)
      throws IOException {
    File dir = getDir(State.WRITING);
    final String tag = tagger.newTag();
    EventSink curSink = new SeqfileEventSink(new File(dir, tag)
        .getAbsoluteFile());
    writingQ.add(tag);
    WALData data = new WALData(tag);
    table.put(tag, data);

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.