Examples of org.commoncrawl.compressors.gzip.GzipCompressorInputStream

org.commoncrawl.compressors.gzip.GzipCompressorInputStream

Input stream that decompresses .gz files. This supports decompressing concatenated .gz files which is important when decompressing standalone .gz files.

{@link java.util.zip.GZIPInputStream} doesn't decompress concatenated .gzfiles: it stops after the first member and silently ignores the rest. It doesn't leave the read position to point to the beginning of the next member, which makes it difficult workaround the lack of concatenation support.

Instead of using GZIPInputStream, this class has its own .gz container format decoder. The actual decompression is done with {@link java.util.zip.Inflater}.

    FileSystem fs = file.getFileSystem(job);


    this._fsin = fs.open(file);


    // create a GZIP stream that *does not* automatically read through members
    this._gzip = new GzipCompressorInputStream(this._fsin, false);


    this._fileLength = fs.getFileStatus(file).getLen();


    // First record should be an ARC file header record.  Skip it.
    this._skipRecord();

Examples of org.commoncrawl.compressors.gzip.GzipCompressorInputStream

Related Classes of org.commoncrawl.compressors.gzip.GzipCompressorInputStream