I have to insert uncompressed data in between the compressed data bytes. Type 0 header in zlib allows me to do that. But How can i do that ? any clues ?
There is no type 0 allowed in a zlib header. There is a stored block type in the deflate format used within a zlib stream. zlib will automatically use stored blocks if they are smaller than the same data compressed.
Related
Is there a way to specify a certain number of inflated clear text in zlib? The sample code zpipe.c reads of size CHUNK (16384 bytes) but I do not see where to specify an inflated size. For example, if I only want to read the first 10 bytes before determining if I want to continue the inflation. Is that possible?
Updating question per request:
I am parsing out some zlib deflated content, which has a header. The header determines the type of content (binary or otherwise). In some cases, I just want to deflate the header. At other times, I need to deflate the entire thing. I was wondering if I could write a deflation function that will return X bytes of inflated text, irrespective of the count of deflated text.
If you set avail_out to n, then inflate() will return at most n bytes of uncompressed data.
I am currently trying to write a compressor and decompressor with the same purpose as the RFC Deflate specification.
I'm not able to understand the difference between how blocks are composed in the compression with fixed tables and dynamic tables. The file is processed by LZ77 generating (distance, length) + literal.
How do I know the type of block?
Do I have to compress this data?
Given that I use a fixed compression and don't have to send the tables, how would the encoder know how to encode data?
Moreover, do I have to send data before the actual compression executes?
I am confused on the difference between fixed tables and the table we send in the dynamic mode, and how the two blocks use them to encode data.
I'm currently reading Data Compression: The Complete Reference. Any advice will be helpful.
Since you are trying to compress, you would pick the smaller of the two. zlib's deflate computes what the size of a fixed block, a dynamic block, and a stored block would be, and emits the smallest of the three.
If you are encoding a fixed block, you encode using the fixed code for literal/lengths and distances. This code is provided in the RFC.
Is it possible to compress a stream with some blocks compressed with static Huffman encoding, and some blocks compressed with dynamic Huffman encoding? If yes, is it decompressible ?
Yes, and yes. A stream can contain a mix of all three block types, with the third type being stored.
When calculating the Adler-32 checksum of uncompressed data in zlib format, should it include Type 0 (uncompressed data) data as well?
The zlib format does not support "type 0". The only type supported by the zlib format is type 8, deflate. Since purely stored data does not have a means to detect when it ends, it cannot be used as a zlib data type. The type used must be self-terminating.
The deflate format internally supports a stored mode, which precedes chunks of uncompressed data with counts.
If the zlib format ever supports compression types other than 8, then yes, the Adler-32 would be computed over the uncompressed result of those methods of compression.
I have multiple blocks of data compressed with zlib. I want to concatenate these blocks of data and store that in one file.
Obviously, I could use something like JSON or XML to separate the zlib data blocks, but I'm wondering if, to save space, I can just search for the next 78 01, 78 9C or 78 DA?
Basically my question is, can, theoretically, these byte combinations exist in a zlib data stream, or can I be sure that when I find one of these byte combinations, a new zlib data block is started, and the end is at the found position minus one?
I know the uncompressed data blocks are always 1024 bytes or less in length, so the compressed stream will never be > 1024 bytes.
No, you can't. Any byte sequence can appear in the compressed data. At any byte position, there is a probability of 1/1024 of finding a valid zlib header. So you will find a lot of valid zlib headers in a long compressed stream that are not actually zlib headers.
You could create your own byte stuffing scheme that wraps around arbitrary data, including zlib streams or anything else, that assures that certain sequences cannot occur unless they really are delimiters. Such schemes can incur an arbitrarily small expansion of the data. For example if you find three 0xff's in a row in the data, then insert a 0x00 byte. Then 0xff 0xff 0xff 0xff can be a delimiter, since it will never appear in the data. This will only expand the stream, on average, by about 0.000006%.