I am struggling with decompression of a Zlib compressed Mime body of AS2 request coming from BizTalk Server.
The thing is:
The HTTP Body I receive looks as expected. I can read the ASCII encoded Mime Header:
"Content-type: application/pkcs7-mime; smime-type=compressed-data; name=smime.p7m\r\nContent-Transfer-Encoding: binary\r\n\r\n"
Ending with two line breaks, I am expecting the compressed body after.
But when I use Ionic.Zlib ZlibStream.UncompressBuffer() to decompress the following bytes it throws an error.
Zlib Header can be identified for example by hex coded bytes "78da". When I start decompressing it from there on, it works fine.
What are the bytes between the two line breaks ending mime header and "78da" starting zlib compressed data?
"3080060b2a864886f70d0109100109a0803080020100300f060b2a864886f70d01091003080500308006092a864886f70d010701a080248004820769"
Next problem, if I read all bytes to the end, the last bytes can not be decompressed.
AS far as I Understood the zLib data should end with adler32 checksum, but how can I identify the end or length of the compressed data without trying to decompress.
I see some trailing bytes after the sucessfully decompressed data:
"1f9b1f1fcbc51f0482000445a59371"
What is that?
Thanks for your ideas!
You cannot find the end of the compressed data without decompressing. You don't need to save the result of the decompression, but you at least need to decode all of the compressed data in order to find where it self terminates.
Related
I am using NODERED to read a serial port. The data coming in is 8 bit per byte. reading is working fine. Now, I want to write that buffer to a file.
When I use buffer.toString() it generates a string but with formatting. Even the 'binary' option escapes the data when 8th bit is set.
Howto write the raw data to a file without it getting changed?
Mission:
I want to generate a bitmap file, adding a fixed length image to the header.
Comparing the file with an hex editor with an existing bitmap shows the escaped bytes in the new file. Obviously that is wrong.
I am trying to encrypt and decrypt files using PowerShell. In this case, I am working with .docx files. After encrypting the file, I passed it onto the decrypt function, and after decrypting it, the file is corrupted when trying to open.
However, after using a hex editor to compare both original and decrypted .docx files, the only difference is that the decrypted .docx file has 2 trailing bytes of "0d 0a".
I think this is the result of PowerShell's "Set-Content" command.
(The Out-File command produces a far worse result.)
However, I am not able to just replace all the carriage return and line feed bytes as I would like to preserve the line feeds and carriage returns of the word document.
Is there a way I am able to remove only the trailing bytes of "0d 0a" of the decrypted and already-written .docx file?
Without seeing your example its impossible to to tell where the extra CRLF is coming from. I would recommend examining your code to determine where it is coming from and then use an alternate route like the System.IO.File class. If you are just looking for a quick solution you could read in the file and strip the last four bytes then output the byte array back to the same file overwriting it. This is a bandaid but should work.
#read in all contents
$bytes = [system.io.file]::ReadAllBytes("somefile.docx")
#write out all bytes except the last 4
#0 based so the last byte is at position length-1 then an additional 4 bytes
[System.IO.File]::WriteAllBytes("somefile.docx",$bytes[0..($bytes.length-5)])
I was having problem in storing .png images , fetched from HTTP server using a C code . I must mention that the images are successfully fetched from server as i analysed wireshark while running the code and content length in header matches buffer size. When i write the buffer data to a .png file using fwrite like this:
Your logic of writing upto *pt != '\0' is not correct in this loop
while((c=*pt)!='\0')
{
fwrite(&c, sizeof(c),1, fp);
pt++;;
}
There can be '\0' character anywhere in the binary data of .png file. So you will not write data after that, hence you are seeing your file size smaller than file size on server.
You should parse the HTTP headers and get value for Content-Length, which gives size of data in bytes and read those many from server and write in local file.
Look for HTTP RFC for more details about the protocol.
There are other problems in your code like
char *pt=malloc(100000);
pt=strstr(rep,"\r\n\r\n");
With this malloc() to pt is not required and leaks the memory.
Try to open the file for output in binary mode , give FILE *fp the flag wb.
Also, why do you try to write byte by byte ? try fwrite(pt,10000,1,fp)
I have some file, there's some random bytes, and multiple gzip files. How can i find start and end of gzip stream inside the some file? there's many random bytes between gzip streams. So, basically i need to find any gzip file and get it from there.
Reading from the RFC 1952 - GZIP :
Each GZIP file is just a bunch of data chunks (called members), one for each file contained.
Each member starts with the following bytes:
0x1F (ID1)
0x8B (ID2)
compression method. 0x08 for a DEFLATEd file. 0-7 are reserved values.
flags. The top three bits are reserved and must be zero.
(4 bytes) last modified time. May be set to 0.
extra flags, defined by the compression method.
operating system, actually the file system. 0=FAT, 3=UNIX, 11=NTFS
The end of a member is not delimited. You have to actually walk the entire member. Note that concatenating multiple valid GZIP files creates a valid GZIP file. Also note that overshooting a member may still result in a successful reading of the member (unless the decompressing library is fail-eagerly-and-completely).
Search for a three-byte gzip signature, 0x1f 0x8b 0x08. When you find it, try to decode a gzip stream starting with the 0x1f. If you succeed, then that was a gzip stream, and it ended where it ended. Continue the search from after that gzip stream if it is one, or after the 0x08 if it isn't. Then you will find all of them and you will know their location and span.
I'm working on a simple server that have to work with a browser. When I give it some command, it has to reply me with some html code so to reply my answer. For example I can see the list of the files of a folder etc. I can access my server using localhost:port/somecommand
Now I'm working on donwloading a file from the local hard-disk. What I want to do is enter an url like localhost:port/download/filepath and make the browser download it. When I create the reply I put all things html need to understand that there is a file to download, and infact I have the classical pop-up window that ask me to download the file, but what I receive is bigger than the original file located to the hard-disk, and infact the file is corrupted.
Here the code:
This is the html I send back
HTTP/1.0 200 OK
Date: Tue Apr 10 16:23:55 2012
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=mypic.jpg
Content-Length: 2574359
//empty line here
(I followed The Content-Disposition Header Field text )
then I read from file and I send back html first and what I read from file then:
int file=open(name,O_RDONLY);
void * data=malloc(buf.st_size+1); //buf.st_size is the correct file size
size_t readed=read(file,data,buf.st_size);
send(sd_current, html, DIRSIZE, 0); //html is a string containing what you I showed you
send(sd_current, data, readed);
This result the file I can download using localhost:port/download/filepath to be bigger than original then corrupted, but I can't get rid of why. Can someone help me?
Things to check:
is DIRSIZE really the size of the http header? (Since the size should vary, and capitals normally means constant).
Is the read successful?
How is the file corrupted?
Are the line endings on the http header correct? They should be \r\n
EDIT:
If DIRSIZE is not the size of the header, then the rest of the buffer (containing NULLs or junk) will be sent to the other size.
So the other side sees the HTTP header, then starts receiving data - starting with the rest of the html buffer, then the contents of the file.
Then depending on the receiver, it either stops at the Content-Length header size, or carries on while the stream is still delivering data.
Does that match your result contents: Some junk at the beginning, followed by the expected file contents?
Content-Length: 2574359, 2574359 should be the size of your jpg file, but I see nothing in your code to put that in the html portion. buf.st_size == 2574359 ?