Class SnappyCompressorOutputStream
- All Implemented Interfaces:
Closeable
,Flushable
,AutoCloseable
This implementation uses an internal buffer in order to handle the back-references that are at the heart of the LZ77 algorithm. The size of the buffer must be at least as big as the biggest offset used in the compressed stream. The current version of the Snappy algorithm as defined by Google works on 32k blocks and doesn't contain offsets bigger than 32k which is the default block size used by this class.
The raw Snappy format requires the uncompressed size to be
written at the beginning of the stream using a varint
representation, i.e. the number of bytes needed to write the
information is not known before the uncompressed size is
known. We've chosen to make the uncompressedSize a parameter of the
constructor in favor of buffering the whole output until the size
is known. When using the FramedSnappyCompressorOutputStream
this limitation is taken care of by the warpping framing
format.
- Since:
- 1.14
- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final LZ77Compressor
private final ByteUtils.ByteConsumer
private boolean
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private static final int
private final byte[]
private final OutputStream
private static final int
private static final int
private static final int
-
Constructor Summary
ConstructorsConstructorDescriptionSnappyCompressorOutputStream
(OutputStream os, long uncompressedSize) Constructor using the default block size of 32k.SnappyCompressorOutputStream
(OutputStream os, long uncompressedSize, int blockSize) Constructor using a configurable block size.SnappyCompressorOutputStream
(OutputStream os, long uncompressedSize, Parameters params) Constructor providing full control over the underlying LZ77 compressor. -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
static Parameters.Builder
createParameterBuilder
(int blockSize) Returns a builder correctly configured for the Snappy algorithm using the gven block size.void
finish()
Compresses all remaining data and writes it to the stream, doesn't close the underlying stream.void
write
(byte[] data, int off, int len) void
write
(int b) private void
private void
writeBackReferenceWithFourOffsetBytes
(int len, int offset) private void
writeBackReferenceWithLittleEndianOffset
(int tag, int offsetBytes, int len, int offset) private void
writeBackReferenceWithOneOffsetByte
(int len, int offset) private void
writeBackReferenceWithTwoOffsetBytes
(int len, int offset) private void
private void
writeLiteralBlockFourSizeBytes
(LZ77Compressor.LiteralBlock block, int len) private void
writeLiteralBlockNoSizeBytes
(LZ77Compressor.LiteralBlock block, int len) private void
writeLiteralBlockOneSizeByte
(LZ77Compressor.LiteralBlock block, int len) private void
writeLiteralBlockThreeSizeBytes
(LZ77Compressor.LiteralBlock block, int len) private void
writeLiteralBlockTwoSizeBytes
(LZ77Compressor.LiteralBlock block, int len) private void
writeLiteralBlockWithSize
(int tagByte, int sizeBytes, int len, LZ77Compressor.LiteralBlock block) private void
writeLittleEndian
(int numBytes, int num) private void
writeUncompressedSize
(long uncompressedSize) Methods inherited from class java.io.OutputStream
flush, write
-
Field Details
-
compressor
-
os
-
consumer
-
oneByte
private final byte[] oneByte -
finished
private boolean finished -
MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES
private static final int MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES- See Also:
-
MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE
private static final int MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE- See Also:
-
MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES
private static final int MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES- See Also:
-
MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES
private static final int MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES- See Also:
-
ONE_SIZE_BYTE_MARKER
private static final int ONE_SIZE_BYTE_MARKER- See Also:
-
TWO_SIZE_BYTE_MARKER
private static final int TWO_SIZE_BYTE_MARKER- See Also:
-
THREE_SIZE_BYTE_MARKER
private static final int THREE_SIZE_BYTE_MARKER- See Also:
-
FOUR_SIZE_BYTE_MARKER
private static final int FOUR_SIZE_BYTE_MARKER- See Also:
-
MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
private static final int MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE- See Also:
-
MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
private static final int MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE- See Also:
-
MAX_OFFSET_WITH_ONE_OFFSET_BYTE
private static final int MAX_OFFSET_WITH_ONE_OFFSET_BYTE- See Also:
-
MAX_OFFSET_WITH_TWO_OFFSET_BYTES
private static final int MAX_OFFSET_WITH_TWO_OFFSET_BYTES- See Also:
-
ONE_BYTE_COPY_TAG
private static final int ONE_BYTE_COPY_TAG- See Also:
-
TWO_BYTE_COPY_TAG
private static final int TWO_BYTE_COPY_TAG- See Also:
-
FOUR_BYTE_COPY_TAG
private static final int FOUR_BYTE_COPY_TAG- See Also:
-
MIN_MATCH_LENGTH
private static final int MIN_MATCH_LENGTH- See Also:
-
MAX_MATCH_LENGTH
private static final int MAX_MATCH_LENGTH- See Also:
-
-
Constructor Details
-
SnappyCompressorOutputStream
Constructor using the default block size of 32k.- Parameters:
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of data- Throws:
IOException
- if writing of the size fails
-
SnappyCompressorOutputStream
public SnappyCompressorOutputStream(OutputStream os, long uncompressedSize, int blockSize) throws IOException Constructor using a configurable block size.- Parameters:
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of datablockSize
- the block size used - must be a power of two- Throws:
IOException
- if writing of the size fails
-
SnappyCompressorOutputStream
public SnappyCompressorOutputStream(OutputStream os, long uncompressedSize, Parameters params) throws IOException Constructor providing full control over the underlying LZ77 compressor.- Parameters:
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of dataparams
- the parameters to use by the compressor - note that the format itself imposes some limits like a maximum match length of 64 bytes- Throws:
IOException
- if writing of the size fails
-
-
Method Details
-
write
- Specified by:
write
in classOutputStream
- Throws:
IOException
-
write
- Overrides:
write
in classOutputStream
- Throws:
IOException
-
close
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Overrides:
close
in classOutputStream
- Throws:
IOException
-
finish
Compresses all remaining data and writes it to the stream, doesn't close the underlying stream.- Throws:
IOException
- if an error occurs
-
writeUncompressedSize
- Throws:
IOException
-
writeLiteralBlock
- Throws:
IOException
-
writeLiteralBlockNoSizeBytes
private void writeLiteralBlockNoSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws IOException - Throws:
IOException
-
writeLiteralBlockOneSizeByte
private void writeLiteralBlockOneSizeByte(LZ77Compressor.LiteralBlock block, int len) throws IOException - Throws:
IOException
-
writeLiteralBlockTwoSizeBytes
private void writeLiteralBlockTwoSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws IOException - Throws:
IOException
-
writeLiteralBlockThreeSizeBytes
private void writeLiteralBlockThreeSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws IOException - Throws:
IOException
-
writeLiteralBlockFourSizeBytes
private void writeLiteralBlockFourSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws IOException - Throws:
IOException
-
writeLiteralBlockWithSize
private void writeLiteralBlockWithSize(int tagByte, int sizeBytes, int len, LZ77Compressor.LiteralBlock block) throws IOException - Throws:
IOException
-
writeLittleEndian
- Throws:
IOException
-
writeBackReference
- Throws:
IOException
-
writeBackReferenceWithOneOffsetByte
- Throws:
IOException
-
writeBackReferenceWithTwoOffsetBytes
- Throws:
IOException
-
writeBackReferenceWithFourOffsetBytes
- Throws:
IOException
-
writeBackReferenceWithLittleEndianOffset
private void writeBackReferenceWithLittleEndianOffset(int tag, int offsetBytes, int len, int offset) throws IOException - Throws:
IOException
-
createParameterBuilder
Returns a builder correctly configured for the Snappy algorithm using the gven block size.- Parameters:
blockSize
- the block size.- Returns:
- a builder correctly configured for the Snappy algorithm using the gven block size
-