Examples of com.google.appengine.tools.mapreduce.outputs.LevelDbOutputWriter

Package com.google.appengine.tools.mapreduce.outputs

Examples of com.google.appengine.tools.mapreduce.outputs.LevelDbOutputWriter

com.google.appengine.tools.mapreduce.outputs.LevelDbOutputWriter
google.com/p/leveldb/"}This implementation deviates from the specification above, in that it allows blocks to be zero padded regardless of how much data is in the block, rather than only if the block is within 6 bytes of full.

    }


  }


  public void writeData(GcsFilename filename, ByteBufferGenerator gen) throws IOException {
    LevelDbOutputWriter writer = new GoogleCloudStorageLevelDbOutputWriter(
        new GoogleCloudStorageFileOutputWriter(filename, "TestData"));
    writer.beginShard();
    writer.beginSlice();
    while (gen.hasNext()) {
      writer.write(gen.next());
    }
    writer.endSlice();
    writer.endShard();
  }

View Full Code Here

      Marshaller<ByteBuffer> identity = Marshallers.getByteBufferMarshaller();
      // Uses LevelDbOutputWriter wrapping GoogleCloudStorageFileOutputWriter rather than
      // GoogleCloudStorageLevelDbOutputWriter because the padding at the end of the slice is
      // unneeded as the file is being finalized.
      return new MarshallingOutputWriter<>(
          new LevelDbOutputWriter(new GoogleCloudStorageFileOutputWriter(
              new GcsFilename(bucket, fileName), MapReduceConstants.REDUCE_INPUT_MIME_TYPE)),
          Marshallers.getKeyValuesMarshaller(identity, identity));
    }

View Full Code Here

TOP

Related Classes of com.google.appengine.tools.mapreduce.outputs.LevelDbOutputWriter

com.google.appengine.tools.mapreduce.impl.GoogleCloudStorageSortOutput$SlicingOutputWriterImpl

com.google.appengine.tools.mapreduce.impl.util.Crc32c

com.google.appengine.tools.mapreduce.impl.util.LevelDbConstants.RecordType

com.google.appengine.tools.mapreduce.inputs.GoogleCloudStorageLevelDbInputReaderTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.