I'm trying to use Environments in Paw to help with testing a web API that uses a multipart/form-data submission of an image file. Paw will do this fine if I have a Body part name with a Value of the "File" type and drag my image file in.
file in multipart form
This produces a request part like:
--Emm5HQuI6PlP2Jo1k3KW284fY8yeluRO
Content-Disposition: form-data; name="persfront"; filename="sample_front.jpg"
Content-Type: image/jpeg
ÿØÿá ExifMM*i&;...
But when I try to abstract that file out of the normal Multipart Body window and into an Environment variable and then use that, it still puts the file contents in the request, but leaves the file information out of the part header.
This:
environment variables for file in form
produces this request:
--2gEAMPgSTNfB0dsLImFDHYMRLjqag7Hu
Content-Disposition: form-data; name="persfront"
ÿØÿá ExifMM*i&;...
The lack of file header, normally supplied in there, causes my request to fail.
It's as if the environment variable is being evaluated prior to it being used and so the multipart form is just seeing a bunch of binary instead of a file. Is this a bug or am I doing something incorrectly?
Is there any other way to use the utility of the Environments in this situation?
Thanks!
Probably very late for an answer, but yes, Paw has a special case where if the content of a multipart request is a file, it will include the filename in the multpart headers. Though, if you abstract the file in an environment variable, this special case isn't honoured anymore...
Related
Need a syntax example for sending input file using wget command.
the input file has an URL and a string argument.
Examples will be helpful.
Thanks in advance.
It is possible to do this with wget only. At least with version 1.13.4 and maybe others. The --post-file option allows you to specify a file to send, so long as the postdata file is constructed properly.
I have also tested this with binary files and it works as expected. You do NOT need to base64 encode the file, but you do need to ensure your file does not contain the boundary.
The minimum command required to make this work would be:
wget --header="Content-type: multipart/form-data boundary=FILEUPLOAD" --post-file postfile http://domain/uploadform
and the postdata file would need to contain something like:
--FILEUPLOAD Content-Disposition: form-data; name="comment"
I love uploading files!
--FILEUPLOAD Content-Disposition: form-data; name="uploadFile"; filename="myfile.bin"; Content-Type: application/octet-stream Media
Type: application/octet-stream
Give me some automated file upload action!
--FILEUPLOAD--
A number of details are important here:
Lines in the post data file are terminated with \r\n. The only exception is data inside the file context.
Every BOUNDARY attribute in the postdata must match the BOUNDARY value in the call to wget. (FILEUPLOAD in the example)
All boundaries are prefixed with two hyphens "--" and terminated with \r\n
The last boundary is suffixed with two extra hyphens "--" and terminated with \r\n
Each piece of data, file content or parameter value, is surrounded by an empty line "\r\n"
I'm working on a simple server that have to work with a browser. When I give it some command, it has to reply me with some html code so to reply my answer. For example I can see the list of the files of a folder etc. I can access my server using localhost:port/somecommand
Now I'm working on donwloading a file from the local hard-disk. What I want to do is enter an url like localhost:port/download/filepath and make the browser download it. When I create the reply I put all things html need to understand that there is a file to download, and infact I have the classical pop-up window that ask me to download the file, but what I receive is bigger than the original file located to the hard-disk, and infact the file is corrupted.
Here the code:
This is the html I send back
HTTP/1.0 200 OK
Date: Tue Apr 10 16:23:55 2012
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=mypic.jpg
Content-Length: 2574359
//empty line here
(I followed The Content-Disposition Header Field text )
then I read from file and I send back html first and what I read from file then:
int file=open(name,O_RDONLY);
void * data=malloc(buf.st_size+1); //buf.st_size is the correct file size
size_t readed=read(file,data,buf.st_size);
send(sd_current, html, DIRSIZE, 0); //html is a string containing what you I showed you
send(sd_current, data, readed);
This result the file I can download using localhost:port/download/filepath to be bigger than original then corrupted, but I can't get rid of why. Can someone help me?
Things to check:
is DIRSIZE really the size of the http header? (Since the size should vary, and capitals normally means constant).
Is the read successful?
How is the file corrupted?
Are the line endings on the http header correct? They should be \r\n
EDIT:
If DIRSIZE is not the size of the header, then the rest of the buffer (containing NULLs or junk) will be sent to the other size.
So the other side sees the HTTP header, then starts receiving data - starting with the rest of the html buffer, then the contents of the file.
Then depending on the receiver, it either stops at the Content-Length header size, or carries on while the stream is still delivering data.
Does that match your result contents: Some junk at the beginning, followed by the expected file contents?
Content-Length: 2574359, 2574359 should be the size of your jpg file, but I see nothing in your code to put that in the html portion. buf.st_size == 2574359 ?
So I'm working on a networking assignment to produce a basic HTTP/1.0 web server in C. I have most of it figured out, but one of the requirements is that it properly populate the Content-Type field in the header, and I can't seem to find any way to do this automatically.
I'm already using fstat() to get the file size and when it was last modified, and I noticed that also includes a "st_objtype" field, but after some research it looks like this is just the AS/400 object type (which is obviously not what I need) and stat() & lstat() appear to do essentially the same thing as fstat().
Is there any way in C to automatically generate a string with the HTTP-style file type for a given file, or do I just need to make a big list of types and plug the correct value into the header based on the ending of the requested file (.txt, .html, .png, etc)?
Some examples of the Content-Type field for various files I checked:
Content-Type: text/html; charset=ISO-8859-1
Content-Type: image/png
Content-Type: application/x-gzip
Content-Type: application/pdf
Some systems contain a file called /etc/mime.types which contains a bunch of extensions and MIME type pairs.
See the documentation for such a file.
Probably the best approach is a lookup table based on extensions. The idea that a given file has a single "type" associated with it is just wrong. At least using extensions gives you a bit of power to control how the file contents are interpreted. For example if you wanted to show an example of how html source works, you could rename example.html to example.html.txt and have the client treat it as text/plain. If you just used a heuristic to determine that the file contents "are html", you'd be stuck.
A file is only a bag of bytes, so there is no way to know for sure that a .html file doesn't contain C code, for example. You could delegate to the file command, which contains a lot of heuristics for determining these kinds of things based on the first few bytes of a file (scripts start with #!, for instance), but file still isn't foolproof. I would recommend the lookup table based on filenames.
I've made a servlet which uses the org.apache.commons.fileupload api to upload a CSV file which it should then load into a MySQL table. This works perfectly when posting to the servlet from a form in a browser. However, it fails when trying to post a form via a Tcl script http://www.tcl.tk/man/tcl/TclCmd/http.htm#M20.
The servlet throws the following MalformedStreamException:
org.apache.commons.fileupload.MultipartStream$MalformedStreamException:
Stream ended unexpectedly
The servlet is hosted on Tomcat version 6.0.16.
The connection is made successfully by the Tcl script as it receives an HTTP/1.1 200 OK response and the servlet does return some print statements back to the client Tcl script. However it fails when trying to read the input stream.
Tcl script:
package require http
proc upload { url csv } {
set boundary "-----WebKitFormBoundary[clock seconds][pid]"
set fid [open $csv r]
if {[catch {read $fid [file size $csv]} data]} {
return -code error $data
}
close $fid
set content {}
append content "--${boundary}\n"
append content "Content-Disposition: form-data; name=\"db\"\n\n"
append content "test\n"
append content "--${boundary}\n"
append content "Content-Disposition: form-data; name=\"table\"\n\n"
append content "testing\n"
append content "--${boundary}\n"
append content "Content-Disposition: form-data; name=\"file\"; filename=\"$csv\"\n"
append content "Content-Type: text/csv\n\n"
append content "$data\n"
append content "${boundary}--"
set headers "connection keep-alive"
set token [::http::geturl $url -keepalive 1 -headers $headers -query $content -type "multipart/form-data; boundary=$boundary"]
upvar 0 $token state
if {$state(http) == "HTTP/1.1 200 OK"} {
# no error reported in http headers
puts stdout $state(http)
puts stdout $state(body)
return 1
} else {
# error reported in http headers
puts stdout $state(http)
puts stdout $state(body)
return 0
}
}
set csv "data.csv"
set url "http://ecat:8080/MySqlImport/MySqlImportServlet"
set retVal [upload $url $csv]
Generating a proper multipart message can be highly aggravating. The mime package in Tcllib can help. The top example on this page looks highly relevant.
While Donal Fellows most probably got it right, I want to point out one (typical) mistake with the Daniel's approach: when one uses a high-level language which has the concept of a "string of characters"—as opposed to a "string of bytes"—one is oblidged to take special care when preparing the data being processed to be placed on "the wire".
In this case "the wire" is the HTTP protocol and so the data prepared should most probably go through something like [encoding convertto utf-8 ...] or something like this depending on the intent. (By the way, Tcl has rather extensive support for binary strings so that they can be appended to, concatenated etc without losing their property of being binary.)
The next thing which can be recommended is to get accustomed with a network sniffer (or a specific HTTP sniffer). A great cross-platform solution is Wireshark, on Windows platforms there's Network Monitor (a generic sniffer) and Fiddler (HTTP-specific). The idea is to first capture and study closely the "reference" session performed by a browser then to capture and analyze the misbehaving session performed by your program. So you just compare the sessions and see where the generated traffic differ when it comes to HTTP payload.
Thank you for the help on this one but I have found my specific issue and it was a simple one in the end. It seems as though the ServletFileUpload.parseRequest(request) method in apache.commons.fileupload requires the line endings to be in windows format.
As you can see in the question the $content line endings was \n changing this to \r\n corrected the problem.
Using just \r meant that the error
org.apache.commons.fileupload.MultipartStream$MalformedStreamException:
Stream ended unexpectedly
no longer occured, however it failed to recognise the different items or parts in the request. The combination of \r\n line endings and it works correctly.
Wonder if anyone has any experience posting image files with CURL in C..?
I am writing a program to post to a facebook Type web service, and everything is going fine, except when I attempt to post image files...
There's a special format that the server needs or it will not accept the post...
something like this:
---webformkitXXXXXXXX\r\n
filename"somefile.jpg"\r\n
JPEG or IMAGE FILE HERE (in binary)
---webformkitXXXXXXXX\r\n
END----
So when I am finally able to to memcpy together the different pieces I need,
I can save it to file, and it looks just fine, but I can see from the packet captures, that CURL doesn't like taking the binary, it appears that it's truncating the buffer at the first sign of a '\0' because, it only sends like 300 bytes, when it should be sending 80K...
I've been using this: curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
Thank You!
You need to use CURLOPT_WRITEDATA and plug in a function you have written to write the POST data.