Examples of com.amazonaws.services.kinesis.connectors.s3.S3ManifestEmitter

Package com.amazonaws.services.kinesis.connectors.s3

Examples of com.amazonaws.services.kinesis.connectors.s3.S3ManifestEmitter

com.amazonaws.services.kinesis.connectors.s3.S3ManifestEmitter
This implementaion of IEmitter inserts records into Amazon S3 and emits filenames into a separate Amazon Kinesis stream. The separate Amazon Kinesis stream is to be used by another Amazon Kinesis enabled application that utilizes RedshiftManifestEmitters to insert the records into Amazon Redshift via a manifest copy. This class requires the configuration of an Amazon S3 bucket and endpoint, as well as Amazon Kinesis endpoint and output stream.
When the buffer is full, this Emitter:
1. Puts all records into a single file in S3
2. Puts the single file name into the manifest stream
NOTE: the Amazon S3 bucket and Amazon Redshift cluster must be in the same region.

 */
public class S3Pipeline implements IKinesisConnectorPipeline<KinesisMessageModel, byte[]> {


    @Override
    public IEmitter<byte[]> getEmitter(KinesisConnectorConfiguration configuration) {
        return new S3Emitter(configuration);
    }

View Full Code Here

 */
public class S3ManifestPipeline implements IKinesisConnectorPipeline<KinesisMessageModel, byte[]> {


    @Override
    public IEmitter<byte[]> getEmitter(KinesisConnectorConfiguration configuration) {
        return new S3ManifestEmitter(configuration);
    }

View Full Code Here

TOP

Related Classes of com.amazonaws.services.kinesis.connectors.s3.S3ManifestEmitter

com.amazonaws.services.kinesis.AmazonKinesisClient

com.amazonaws.services.kinesis.model.PutRecordRequest

com.amazonaws.services.s3.AmazonS3Client

samples.redshiftmanifest.S3ManifestPipeline

samples.s3.S3Pipeline

java.nio.ByteBuffer

java.io.ByteArrayOutputStream

java.io.ByteArrayInputStream

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.