Examples of org.jets3t.service.multithread.S3ServiceSimpleMulti

org.jets3t.service.multithread.S3ServiceSimpleMulti
S3 service wrapper that performs multiple S3 requests at a time using multi-threading and an underlying thread-safe {@link S3Service} implementation.
This class provides a simplified interface to the {@link S3ServiceMulti} service. It will block while doing its work, return the results of an operation when it is finished, and throw an exception if anything goes wrong.
For a non-blocking multi-threading service that is more powerful, but also more complicated, see {@link S3ServiceMulti}. @author James Murty


        // To use the S3ServiceSimpleMulti service you construct it by providing an existing 
        // S3Service object.


        // Create a simple multi-threading service based on our existing S3Service
        S3ServiceSimpleMulti simpleMulti = new S3ServiceSimpleMulti(s3Service);
        
        /*
         * Upload multiple objects at once 
         */
        
        // To demonstrate multiple uploads, let's create some small text-data objects and a bucket to put them in.


        // First, create a bucket.
        S3Bucket bucket = new S3Bucket(awsCredentials.getAccessKey() + ".TestMulti");
        bucket = s3Service.createBucket(bucket);


        // Create an array of data objects to upload.
        S3Object[] objects = new S3Object[5];
        objects[0] = new S3Object(bucket, "object1.txt", "Hello from object 1");
        objects[1] = new S3Object(bucket, "object2.txt", "Hello from object 2");
        objects[2] = new S3Object(bucket, "object3.txt", "Hello from object 3");
        objects[3] = new S3Object(bucket, "object4.txt", "Hello from object 4");
        objects[4] = new S3Object(bucket, "object5.txt", "Hello from object 5");


        // Now we have some sample objects, we can upload them.


        // Upload multiple objects.
        S3Object[] createdObjects = simpleMulti.putObjects(bucket, objects);        
        System.out.println("Uploaded " + createdObjects.length + " objects");


        /*
         * Retrieve the HEAD information of multiple objects 
         */


        // Perform a Details/HEAD query for multiple objects.
        S3Object[] objectsWithHeadDetails = simpleMulti.getObjectsHeads(bucket, objects);


        // Print out details about all the objects.
        System.out.println("Objects with HEAD Details...");
        for (int i = 0; i < objectsWithHeadDetails.length; i++) {
            System.out.println(objectsWithHeadDetails[i]);
        }


        /*
         * Download objects to local files 
         */


        // The multi-threading services provide a method to download multiple objects at a time, but 
        // to use this you must first prepare somewhere to put the data associated with each object. 
        // The most obvious place to put this data is into a file, so let's go through an example of 
        // downloading object data into files.


        // To download our objects into files we first must create a S3ObjectAndOutputStream class for 
        // each object. This class is a simple container which merely associates an object with an 
        // output stream, to which the object's data will be written.
        
        // Create a DownloadPackage for each object, to associate the object with an output file.
        DownloadPackage[] downloadPackages = new DownloadPackage[5];
        downloadPackages[0] = new DownloadPackage(objects[0], 
            new File(objects[0].getKey()));
        downloadPackages[1] = new DownloadPackage(objects[1], 
            new File(objects[1].getKey()));
        downloadPackages[2] = new DownloadPackage(objects[2], 
            new File(objects[2].getKey()));
        downloadPackages[3] = new DownloadPackage(objects[3], 
            new File(objects[3].getKey()));
        downloadPackages[4] = new DownloadPackage(objects[4], 
            new File(objects[4].getKey()));
        
        // Download the objects.
        simpleMulti.downloadObjects(bucket, downloadPackages);
        System.out.println("Downloaded objects to current working directory");


        /*
         * Delete multiple objects 
         */
        
        // It's time to clean up, so let's get rid of our multiple objects and test bucket.


        // Delete multiple objects, then the bucket too.
        simpleMulti.deleteObjects(bucket, objects);
        s3Service.deleteBucket(bucket);
        System.out.println("Deleted bucket: " + bucket);


        /* *****************
         * Advanced Examples

View Full Code Here


        // To use the S3ServiceSimpleMulti service you construct it by providing an existing 
        // S3Service object.


        // Create a simple multi-threading service based on our existing S3Service
        S3ServiceSimpleMulti simpleMulti = new S3ServiceSimpleMulti(s3Service);
        
        /*
         * Upload multiple objects at once 
         */
        
        // To demonstrate multiple uploads, let's create some small text-data objects and a bucket to put them in.


        // First, create a bucket.
        S3Bucket bucket = new S3Bucket(awsCredentials.getAccessKey() + ".TestMulti");
        bucket = s3Service.createBucket(bucket);


        // Create an array of data objects to upload.
        S3Object[] objects = new S3Object[5];
        objects[0] = new S3Object(bucket, "object1.txt", "Hello from object 1");
        objects[1] = new S3Object(bucket, "object2.txt", "Hello from object 2");
        objects[2] = new S3Object(bucket, "object3.txt", "Hello from object 3");
        objects[3] = new S3Object(bucket, "object4.txt", "Hello from object 4");
        objects[4] = new S3Object(bucket, "object5.txt", "Hello from object 5");


        // Now we have some sample objects, we can upload them.


        // Upload multiple objects.
        S3Object[] createdObjects = simpleMulti.putObjects(bucket, objects);        
        System.out.println("Uploaded " + createdObjects.length + " objects");


        /*
         * Retrieve the HEAD information of multiple objects 
         */


        // Perform a Details/HEAD query for multiple objects.
        S3Object[] objectsWithHeadDetails = simpleMulti.getObjectsHeads(bucket, objects);


        // Print out details about all the objects.
        System.out.println("Objects with HEAD Details...");
        for (int i = 0; i < objectsWithHeadDetails.length; i++) {
            System.out.println(objectsWithHeadDetails[i]);
        }


        /*
         * Download objects to local files 
         */


        // The multi-threading services provide a method to download multiple objects at a time, but 
        // to use this you must first prepare somewhere to put the data associated with each object. 
        // The most obvious place to put this data is into a file, so let's go through an example of 
        // downloading object data into files.


        // To download our objects into files we first must create a DownloadPackage class for  
        // each object. This class is a simple container which merely associates an object with a  
        // file, to which the object's data will be written.
        
        // Create a DownloadPackage for each object, to associate the object with an output file.
        DownloadPackage[] downloadPackages = new DownloadPackage[5];
        downloadPackages[0] = new DownloadPackage(objects[0], 
            new File(objects[0].getKey()));
        downloadPackages[1] = new DownloadPackage(objects[1], 
            new File(objects[1].getKey()));
        downloadPackages[2] = new DownloadPackage(objects[2], 
            new File(objects[2].getKey()));
        downloadPackages[3] = new DownloadPackage(objects[3], 
            new File(objects[3].getKey()));
        downloadPackages[4] = new DownloadPackage(objects[4], 
            new File(objects[4].getKey()));
        
        // Download the objects.
        simpleMulti.downloadObjects(bucket, downloadPackages);
        System.out.println("Downloaded objects to current working directory");
        
        /*
         * Delete multiple objects 
         */
        
        // It's time to clean up, so let's get rid of our multiple objects and test bucket.


        // Delete multiple objects, then the bucket too.
        simpleMulti.deleteObjects(bucket, objects);
        s3Service.deleteBucket(bucket);
        System.out.println("Deleted bucket: " + bucket);


        /* *****************
         * Advanced Examples

View Full Code Here


        // To use the S3ServiceSimpleMulti service you construct it by providing an existing 
        // S3Service object.


        // Create a simple multi-threading service based on our existing S3Service
        S3ServiceSimpleMulti simpleMulti = new S3ServiceSimpleMulti(s3Service);
        
        /*
         * Upload multiple objects at once 
         */
        
        // To demonstrate multiple uploads, let's create some small text-data objects and a bucket to put them in.


        // First, create a bucket.
        S3Bucket bucket = new S3Bucket(awsCredentials.getAccessKey() + ".TestMulti");
        bucket = s3Service.createBucket(bucket);


        // Create an array of data objects to upload.
        S3Object[] objects = new S3Object[5];
        objects[0] = new S3Object(bucket, "object1.txt", "Hello from object 1");
        objects[1] = new S3Object(bucket, "object2.txt", "Hello from object 2");
        objects[2] = new S3Object(bucket, "object3.txt", "Hello from object 3");
        objects[3] = new S3Object(bucket, "object4.txt", "Hello from object 4");
        objects[4] = new S3Object(bucket, "object5.txt", "Hello from object 5");


        // Now we have some sample objects, we can upload them.


        // Upload multiple objects.
        S3Object[] createdObjects = simpleMulti.putObjects(bucket, objects);        
        System.out.println("Uploaded " + createdObjects.length + " objects");


        /*
         * Retrieve the HEAD information of multiple objects 
         */


        // Perform a Details/HEAD query for multiple objects.
        S3Object[] objectsWithHeadDetails = simpleMulti.getObjectsHeads(bucket, objects);


        // Print out details about all the objects.
        System.out.println("Objects with HEAD Details...");
        for (int i = 0; i < objectsWithHeadDetails.length; i++) {
            System.out.println(objectsWithHeadDetails[i]);
        }


        /*
         * Download objects to local files 
         */


        // The multi-threading services provide a method to download multiple objects at a time, but 
        // to use this you must first prepare somewhere to put the data associated with each object. 
        // The most obvious place to put this data is into a file, so let's go through an example of 
        // downloading object data into files.


        // To download our objects into files we first must create a DownloadPackage class for  
        // each object. This class is a simple container which merely associates an object with a  
        // file, to which the object's data will be written.
        
        // Create a DownloadPackage for each object, to associate the object with an output file.
        DownloadPackage[] downloadPackages = new DownloadPackage[5];
        downloadPackages[0] = new DownloadPackage(objects[0], 
            new File(objects[0].getKey()));
        downloadPackages[1] = new DownloadPackage(objects[1], 
            new File(objects[1].getKey()));
        downloadPackages[2] = new DownloadPackage(objects[2], 
            new File(objects[2].getKey()));
        downloadPackages[3] = new DownloadPackage(objects[3], 
            new File(objects[3].getKey()));
        downloadPackages[4] = new DownloadPackage(objects[4], 
            new File(objects[4].getKey()));
        
        // Download the objects.
        simpleMulti.downloadObjects(bucket, downloadPackages);
        System.out.println("Downloaded objects to current working directory");
        
        /*
         * Delete multiple objects 
         */
        
        // It's time to clean up, so let's get rid of our multiple objects and test bucket.


        // Delete multiple objects, then the bucket too.
        simpleMulti.deleteObjects(bucket, objects);
        s3Service.deleteBucket(bucket);
        System.out.println("Deleted bucket: " + bucket);


        /* *****************
         * Advanced Examples

View Full Code Here


        // To use the S3ServiceSimpleMulti service you construct it by providing an existing
        // S3Service object.


        // Create a simple multi-threading service based on our existing S3Service
        S3ServiceSimpleMulti simpleMulti = new S3ServiceSimpleMulti(s3Service);


        /*
         * Upload multiple objects at once
         */


        // To demonstrate multiple uploads, let's create some small text-data objects and a bucket to put them in.


        // First, create a bucket.
        S3Bucket bucket = new S3Bucket(awsCredentials.getAccessKey() + ".TestMulti");
        bucket = s3Service.createBucket(bucket);


        // Create an array of data objects to upload.
        S3Object[] objects = new S3Object[5];
        objects[0] = new S3Object("object1.txt", "Hello from object 1");
        objects[1] = new S3Object("object2.txt", "Hello from object 2");
        objects[2] = new S3Object("object3.txt", "Hello from object 3");
        objects[3] = new S3Object("object4.txt", "Hello from object 4");
        objects[4] = new S3Object("object5.txt", "Hello from object 5");


        // Now we have some sample objects, we can upload them.


        // Upload multiple objects.
        S3Object[] createdObjects = simpleMulti.putObjects(bucket, objects);
        System.out.println("Uploaded " + createdObjects.length + " objects");


        /*
         * Retrieve the HEAD information of multiple objects
         */


        // Perform a Details/HEAD query for multiple objects.
        S3Object[] objectsWithHeadDetails = simpleMulti.getObjectsHeads(bucket, objects);


        // Print out details about all the objects.
        System.out.println("Objects with HEAD Details...");
        for (int i = 0; i < objectsWithHeadDetails.length; i++) {
            System.out.println(objectsWithHeadDetails[i]);
        }


        /*
         * Download objects to local files
         */


        // The multi-threading services provide a method to download multiple objects at a time, but
        // to use this you must first prepare somewhere to put the data associated with each object.
        // The most obvious place to put this data is into a file, so let's go through an example of
        // downloading object data into files.


        // To download our objects into files we first must create a DownloadPackage class for
        // each object. This class is a simple container which merely associates an object with a
        // file, to which the object's data will be written.


        // Create a DownloadPackage for each object, to associate the object with an output file.
        DownloadPackage[] downloadPackages = new DownloadPackage[5];
        downloadPackages[0] = new DownloadPackage(objects[0],
            new File(objects[0].getKey()));
        downloadPackages[1] = new DownloadPackage(objects[1],
            new File(objects[1].getKey()));
        downloadPackages[2] = new DownloadPackage(objects[2],
            new File(objects[2].getKey()));
        downloadPackages[3] = new DownloadPackage(objects[3],
            new File(objects[3].getKey()));
        downloadPackages[4] = new DownloadPackage(objects[4],
            new File(objects[4].getKey()));


        // Download the objects.
        simpleMulti.downloadObjects(bucket, downloadPackages);
        System.out.println("Downloaded objects to current working directory");


        /*
         * Delete multiple objects
         */


        // It's time to clean up, so let's get rid of our multiple objects and test bucket.


        // Delete multiple objects, then the bucket too.
        simpleMulti.deleteObjects(bucket, objects);
        s3Service.deleteBucket(bucket);
        System.out.println("Deleted bucket: " + bucket);


        /* *****************
         * Bucket Versioning
         * *****************
         * S3 Buckets have a versioning feature which allows you to keep prior versions of
         * your objects when they are updated or deleted. This feature means you can be much
         * more confident that vital data will not be lost even if it is accidentally
         * overwritten or deleted.
         *
         * Versioning is not enabled for a bucket by default, you must explicitly enable
         * it. Once it is enabled you access and mange object versions using unique version
         * identifiers.
         */
        // Create a bucket to test versioning
        S3Bucket versioningBucket = s3Service.getOrCreateBucket(
            "test-versioning");
        String vBucketName = versioningBucket.getName();


        // Check bucket versioning status for the bucket
        S3BucketVersioningStatus versioningStatus =
            s3Service.getBucketVersioningStatus(vBucketName);
        System.out.println("Versioning enabled ? "
            + versioningStatus.isVersioningEnabled());


        // Suspend (disable) versioning for a bucket -- will have no
        // effect if bucket versioning is not yet enabled.
        // This will not delete any existing object versions.
        s3Service.suspendBucketVersioning(vBucketName);


        // Enable versioning for a bucket.
        s3Service.enableBucketVersioning(vBucketName);


        // Once versioning is enabled you can GET, PUT, copy and
        // delete objects as normal. Every change to an object will
        // cause a new version to be created.


        // Store and update and delete an object in the versioning bucket
        S3Object versionedObject = new S3Object("versioned-object", "Initial version");
        s3Service.putObject(vBucketName, versionedObject);
        versionedObject = new S3Object("versioned-object", "Second version");
        s3Service.putObject(vBucketName, versionedObject);
        versionedObject = new S3Object("versioned-object", "Final version");
        s3Service.putObject(vBucketName, versionedObject);


        // If you retrieve an object with the standard method you will
        // get the latest version, and if the object is in a versioned
        // bucket its Version ID will be available
        versionedObject = s3Service.getObject(vBucketName, "versioned-object");
        String finalVersionId = versionedObject.getVersionId();
        System.out.println("Version ID: " + finalVersionId);


        // If you delete a versioned object it is no longer available using
        // standard methods...
        s3Service.deleteObject(vBucketName, "versioned-object");
        try {
            s3Service.getObject(vBucketName, "versioned-object");
        } catch (S3ServiceException e) {
            if (e.getResponseCode() == 404) {
                System.out.println("Is deleted object versioned? "
                    + e.getResponseHeaders().get(Constants.AMZ_DELETE_MARKER));
                System.out.println("Delete marker version ID: "
                    + e.getResponseHeaders().get(Constants.AMZ_VERSION_ID));
            }
        }
        // ... but you can use a versioning-aware method to retrieve any of
        // the prior versions by Version ID.
        versionedObject = s3Service.getVersionedObject(finalVersionId,
            vBucketName, "versioned-object");
        String versionedData = ServiceUtils.readInputStreamToString(
            versionedObject.getDataInputStream(), "UTF-8");
        System.out.println("Data from prior version of deleted document: "
            + versionedData);


        // List all the object versions in the bucket, with no prefix
        // or delimiter restrictions. Each result object will be one of
        // S3Version or S3DeleteMarker.
        BaseVersionOrDeleteMarker[] versions =
            s3Service.listVersionedObjects(vBucketName, null, null);
        for (int i = 0; i < versions.length; i++) {
            System.out.println(versions[i]);
        }


        // List versions of objects that match a prefix.
        String versionPrefix = "versioned-object";
        versions = s3Service.listVersionedObjects(vBucketName, versionPrefix, null);


        // JetS3t includes a convenience method to list only the versions
        // for a specific object, even if it shares a prefix with other objects.
        versions = s3Service.getObjectVersions(vBucketName, "versioned-object");


        // There are versioning-aware methods corresponding to all S3 operations
        versionedObject = s3Service.getVersionedObjectDetails(
            finalVersionId, vBucketName, "versioned-object");
        // Confirm that S3 returned the versioned object you requested
        if (!finalVersionId.equals(versionedObject.getVersionId())) {
            throw new Exception("Incorrect version!");
        }


        s3Service.copyVersionedObject(finalVersionId,
            vBucketName, "versioned-object",
            "destination-bucket", new S3Object("copied-from-version"),
            false, null, null, null, null);


        AccessControlList versionedObjectAcl =
            s3Service.getVersionedObjectAcl(finalVersionId,
                vBucketName, "versioned-object");


        s3Service.putVersionedObjectAcl(finalVersionId,
            vBucketName, "versioned-object", versionedObjectAcl);


        // To delete an object version once-and-for-all you must use the
        // versioning-specific delete operation, and you can only do so
        // if you are the owner of the bucket containing the version.
        s3Service.deleteVersionedObject(finalVersionId,
            vBucketName, "versioned-object");


        // You can easily delete all the versions of an object using
        // one of JetS3t's multi-threaded services.
        versions = s3Service.getObjectVersions(vBucketName, "versioned-object");
        // Convert version and delete marker objects into versionId strings.
        String[] versionIds = BaseVersionOrDeleteMarker.toVersionIds(versions);
        (new S3ServiceSimpleMulti(s3Service)).deleteVersionsOfObject(
            versionIds, vBucketName, "versioned-object");




        //////////////////////////////////////////////////////////////
        // For additional data protection you can require multi-factor

View Full Code Here

TOP

Related Classes of org.jets3t.service.multithread.S3ServiceSimpleMulti

org.jets3t.samples.CodeSamples

org.jets3t.service.model.S3Object

java.io.File

org.jets3t.service.S3ServiceException

java.util.ArrayList

java.util.List

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.