Examples of org.apache.hadoop.io.compress.bzip2.CBZip2OutputStream

Package org.apache.hadoop.io.compress.bzip2

Examples of org.apache.hadoop.io.compress.bzip2.CBZip2OutputStream

org.codehaus.plexus.archiver.bzip2.CBZip2OutputStream

An output stream that compresses into the BZip2 format (without the file header chars) into another stream.

The compression requires large amounts of memory. Thus you should call the {@link #close() close()} method as soon as possible, to forceCBZip2OutputStream to release the allocated memory.

You can shrink the amount of allocated memory and maybe raise the compression speed by choosing a lower blocksize, which in turn may cause a lower compression ratio. You can avoid unnecessary memory allocation by avoiding using a blocksize which is bigger than the size of the input.

You can compute the memory usage for compressing by the following formula:

 <code>400k + (9 * blocksize)</code>.

To get the memory required for decompression by {@link CBZip2InputStream CBZip2InputStream} use

 <code>65k + (5 * blocksize)</code>.

Memory usage by blocksize
Blocksize	Compression memory usage	Decompression memory usage
100k	1300k	565k
200k	2200k	1065k
300k	3100k	1565k
400k	4000k	2065k
500k	4900k	2565k
600k	5800k	3065k
700k	6700k	3565k
800k	7600k	4065k
900k	8500k	4565k

For decompression CBZip2InputStream allocates less memory if the bzipped input is smaller than one block.

Instances of this class are not threadsafe.

TODO: Update to BZip2 1.0.1

                return new GZIPOutputStream(ostream);
            } else {
                if (BZIP2.equals(value)) {
                    ostream.write('B');
                    ostream.write('Z');
                    return new CBZip2OutputStream(ostream);
                }
            }
            return ostream;
        }

View Full Code Here

            public OutputStream createArchiveOutputStream(File destination) {
                try {
                    OutputStream outStr = new FileOutputStream(destination);
                    outStr.write('B');
                    outStr.write('Z');
                    return new CBZip2OutputStream(outStr);
                } catch (Exception e) {
                    String message = String.format("Unable to create bzip2 output stream for file %s", destination);
                    throw new RuntimeException(message, e);
                }
            }

View Full Code Here

            dOut = new DeflaterOutputStream(pkOut, new Deflater(compression));
            break;
        case BZIP2:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA);
            pkOut.write(PGPCompressedData.BZIP2);
            dOut = new CBZip2OutputStream(pkOut);
            break;
        case PGPCompressedData.UNCOMPRESSED:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA);
            pkOut.write(PGPCompressedData.UNCOMPRESSED);
            dOut = pkOut;

View Full Code Here

            dOut = new DeflaterOutputStream(pkOut, new Deflater(compression));
            break;
        case PGPCompressedData.BZIP2:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA, buffer);
            pkOut.write(PGPCompressedData.BZIP2);
            dOut = new CBZip2OutputStream(pkOut);
            break;
        case PGPCompressedData.UNCOMPRESSED:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA, buffer);
            pkOut.write(PGPCompressedData.UNCOMPRESSED);
            dOut = pkOut;

View Full Code Here

    
                dfOut.finish();
            }
            else if (dOut instanceof CBZip2OutputStream)
            {
                CBZip2OutputStream cbOut = (CBZip2OutputStream)dOut;
    
                cbOut.finish();
            }
    
            dOut.flush();
    
            pkOut.finish();

View Full Code Here

            dOut = new DeflaterOutputStream(pkOut, new Deflater(compression));
            break;
        case BZIP2:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA);
            pkOut.write(PGPCompressedData.BZIP2);
            dOut = new CBZip2OutputStream(pkOut);
            break;
        case PGPCompressedData.UNCOMPRESSED:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA);
            pkOut.write(PGPCompressedData.UNCOMPRESSED);
            dOut = pkOut;

View Full Code Here

            dOut = new DeflaterOutputStream(pkOut, new Deflater(compression));
            break;
        case PGPCompressedData.BZIP2:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA, buffer);
            pkOut.write(PGPCompressedData.BZIP2);
            dOut = new CBZip2OutputStream(pkOut);
            break;
        case PGPCompressedData.UNCOMPRESSED:
            pkOut = new BCPGOutputStream(out, PacketTags.COMPRESSED_DATA, buffer);
            pkOut.write(PGPCompressedData.UNCOMPRESSED);
            dOut = pkOut;

View Full Code Here

    
                dfOut.finish();
            }
            else if (dOut instanceof CBZip2OutputStream)
            {
                CBZip2OutputStream cbOut = (CBZip2OutputStream)dOut;
    
                cbOut.finish();
            }
    
            dOut.flush();
    
            pkOut.finish();

View Full Code Here

            {
                if ( BZIP2.equals( value ) )
                {
                    ostream.write( 'B' );
                    ostream.write( 'Z' );
                    return new CBZip2OutputStream( ostream );
                }
            }
            return ostream;
        }

View Full Code Here

            }
            else if ( BZIP2.equals( value ) )
            {
                ostream.write( 'B' );
                ostream.write( 'Z' );
                return new CBZip2OutputStream( ostream );
            }
            return ostream;
        }

View Full Code Here

0 1 2 3 4 5 6 7 8

TOP

Related Classes of org.apache.hadoop.io.compress.bzip2.CBZip2OutputStream

at.molindo.utils.io.StreamUtils

com.endgame.binarypig.util.BuildSequenceFileFromArchive

com.packetloop.packetpig.loaders.pcap.PcapRecordReader

com.packetloop.packetpig.loaders.pcap.StreamingPcapRecordReader

edu.mit.simile.vicino.distances.BZip2Distance

org.apache.commons.vfs.provider.bzip2.Bzip2FileObject

org.apache.hadoop.io.compress.bzip2.Bzip2Compressor

org.apache.hadoop.io.compress.bzip2.Bzip2Decompressor

org.apache.hadoop.io.compress.bzip2.BZip2DummyCompressor

org.apache.hadoop.io.compress.bzip2.BZip2DummyDecompressor

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.