How to handle FineUploader POST request in Java in PlayFramework? - multipartform-data

I am trying to integrate FineUploader in my play framework code. View part is set up properly and it is working fine.
I am not clear how to retrieve file in my controllers upload method. And other query parameters, qquuid, qqfilename, content-type e.tc.
Following is the dump of file upload request sent by FileUploader:-
Request Headers
Accept:application/json
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Content-Length:1021645
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryk8BZISBKKjjqiWQ6
DNT:1
Host:localhost:9000
Origin:http://localhost:9000
Referer:http://localhost:9000/computers
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.152 Safari/537.36
X-Requested-With:XMLHttpRequest
Request Payload
------WebKitFormBoundaryk8BZISBKKjjqiWQ6
Content-Disposition: form-data; name="id"
593
------WebKitFormBoundaryk8BZISBKKjjqiWQ6
Content-Disposition: form-data; name="qquuid"
bafb6448-1a87-40cc-abae-9624f37131cc
------WebKitFormBoundaryk8BZISBKKjjqiWQ6
Content-Disposition: form-data; name="qqfilename"
Test.png
------WebKitFormBoundaryk8BZISBKKjjqiWQ6
Content-Disposition: form-data; name="qqtotalfilesize"
1020995
------WebKitFormBoundaryk8BZISBKKjjqiWQ6
Content-Disposition: form-data; name="qqfile"; filename="Test.png"
Content-Type: image/png
------WebKitFormBoundaryk8BZISBKKjjqiWQ6--

I eventually found out by exploring play apis, sample code below.
Http.MultipartFormData body = request().body().asMultipartFormData();
Http.MultipartFormData.FilePart uploadFilePart = body.getFile("qqfile");
String fileName = uploadFilePart.getFilename();
File file = uploadFilePart.getFile();
Map<String,String[]> dataPart = request().body().asMultipartFormData().asFormUrlEncoded();
Iterator it = dataPart.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
String[] values = (String[]) pair.getValue();
System.out.println(pair.getKey() + " = " + values[0]);
}

Previous comment by me indicated there might have been something wrong with the http post fineuploader was doing, but of course that was wrong. I was trying to adapt traditional endpoint example code into a Spring 4.x MVC controller and was getting empty FileItem list from Apache Commons FileUpload libraries. After downloading the source for Spring and debugging through it, found that Spring MVC was using same Apache Commons FileUpload libraries to consume the http post request BEFORE it reached my controller code. A second call to Apache's FileUpload.parseRequest returns an empty FileItem list because it was already consumed. I had to repurpose the controller code to make use of correct signatures for request params. I found it easiest to let the MVC framework to the parsing work for me. I was still able to use much of the example code for traditional endpoints after refactoring for this. For example, my controller
#RequestMapping(value="/assetUpload", method = RequestMethod.POST)
public void assetUpload(HttpServletRequest request, HttpServletResponse response, #RequestParam("qqfile") MultipartFile file) throws ServletException, IOException { ....
Modification of the rest of the code to produce my own class to replace FileItem (in my case called the class "FineUploaderItem"), refactor RequestParser and how the Multipart parsing was working got my server endpoint working. I was able to use all the private methods in the UploadReceiver class in my code with very little modification after this and it all worked.

Related

Workato - Hash containing an array when sent in a multipart form request is considered as a file

I am trying to send an array as value for a hash key in a POST multipart request.
input = {"attribute"=> ["Countries:India", "Category:Can"]}
post("url")
.request_format_multipart_form.payload(input)
This works when using Ruby's HTTP and also POSTMAN. I can also see the differences between how Postman and Workato handles the form data.
Postman:
----------------------------428750340837882951989223
Content-Disposition: form-data; name="attribute"
Category:Test
----------------------------428750340837882951989223
Content-Disposition: form-data; name="attribute"
Countries:India
----------------------------428750340837882951989223--
Workato:
------RubyFormBoundaryhokO7A27Xb6cdSEz
Content-Disposition: form-data; name="attribute"; filename="attribute"
Content-Type: Category:Can
Countries:India
------RubyFormBoundaryhokO7A27Xb6cdSEz--
Why does Workato consider an array as a file? or am I wrong with the call?
I solved this by changing the request type to application/x-www-form-urlencoded and encode the input parameters as form data in the body (Using encode_www_form).
input = {"attribute"=> ["Countries:India", "Category:Can"]}
input = input.encode_www_form
post("url",input)
.request_format_www_form_urlencoded

BizTalk incorrect request (multipart form data)

We need to send a file of xbrl format (a type of xml) to a receiving system. However, the receiving system requires that we are sending some form data as well. Regarding the xbrl file we need to send with it a form data, name="data".
Regarding the second part, we need to send an email to the system via form data. And that is name="email".
(I'll be the first to admit that I really do not understand form data at all so I cant explain much better than this).
I have a custom pipeline and in the encode stage I have put my pipeline component (the entire Execute method is down below) and after that a MIME/SMIME component (standard Microsoft).
Take a look at how the request looks like when it is being sent from Postman (and fully accepted by the receiving system):
Content-Type: multipart/form-data; boundary=--------------------------035085236562027409735938
Content-Length: 299420
----------------------------035085236562027409735938
Content-Disposition: form-data; name="data"; filename="Dk_{5AA9547A-E103-40CD-A2F4-6B77F010E570}.xbrl"
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<!- ---- Lots of removed xbrl ---- ->
</xbrli:xbrl>
----------------------------035085236562027409735938
Content-Disposition: form-data; name="email"
Content-Type: text/plain
test#test.com
----------------------------035085236562027409735938--
Now what you see below is what we have managed to achieve this far:
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="_E9E8D52C-4722-488D-9FEC-713446D4A5C4_"
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_
MIME-Version: 1.0
Content-Type: application/xml; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Content-ID: {E8CA3F1A-F953-4FFB-A054-DEB8466A5D86}
Content-Description: body
<?xml version="1.0" encoding="utf-8"?>
<!- ---- Lots of removed xbrl ---- ->
</xbrli:xbrl>
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-ID: {3B35ECAC-990D-458C-8E7E-9C178CA99988}
Content-Description: email
Content-Disposition: attachment; filename="=?utf-8?B?QXR0YWPobWVudDF=?="
test#test.com
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_--
Now, a quick comparison reveals that we are closing in on the solution. The first important issue must be that the Content-Type (first part below MIME-version: 1.0) is becoming multipart/mixed when we need multipart/form-data.
Second issue is in the Content-Disposition which for both parts does not, at all, look like expected.
Please take a look at my pipeline component. What you see is the entire code from the Execute. Since formDataPart.Data only takes a stream I have a method that translates a string to a stream (MemoryStream).
// Variables
var outMsg = pContext.GetMessageFactory().CreateMessage();
outMsg.Context = PipelineUtil.CloneMessageContext(pInMsg.Context);
outMsg.Context.Promote("ContentType", "http://schemas.microsoft.com/BizTalk/2003/http-properties", "multipart/form-data;");
// Process xbrl file
var msgPart = pContext.GetMessageFactory().CreateMessagePart();
msgPart.PartProperties.Write("FileName", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "SomeID.xbrl");
msgPart.PartProperties.Write("ContentDescription", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "xml");
//msgPart.PartProperties.Write("ContentTransferEncoding", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "7bit");
msgPart.Data = pInMsg.BodyPart.Data;
msgPart.ContentType = "application/xml";
msgPart.Charset = "UTF-8";
outMsg.AddPart("data", msgPart, true);
// Form data (email)
var formDataPart = pContext.GetMessageFactory().CreateMessagePart();
formDataPart.ContentType = "text/plain";
formDataPart.Charset = "utf-8";
formDataPart.PartProperties.Write("ContentType", "http://schemas.microsoft.com/BizTalk/2003/http-properties", "form-data;");
formDataPart.Data = GenerateStreamFromString("test#test.com");
outMsg.AddPart("email", formDataPart, false);
return outMsg;
Does anybody know whow we can achieve this? I can also mention that we are using the MIME/SMIME component after this custom pipeline component.
I really think that the solution from Greg Forsythe would be the best solution since I would have total control over the message but I have no idea how to translate that into actual code.
Greg_suggestion
UPDATE: After so many trials and not coming anywhere we decided to ditch the above solution and go with Gregs solution instead. There is an issue there so I hope some C# guru can help us out. Please take a look at the new question if you are struggling with the form data request issue.
Second attempt here

How do I create a request in Swagger Inspector that accepts multiple data types?

I am currently trying to submit a request using the Swagger Inspector using multipart/form-data header to allow the submission of a file in conjunction with json data.
The JSON body of my request looks like:
And the headers with file upload look like:
For requests where I am just sending json to the server the Content-Type header is set to application/json and it is able to read from the body box. However I do not understand how this interface allows me to specify that the information coming from the body field is json and despite there being files on on the request.
I have seen requests that define multiple data types using the Conetent-Disspostion header, that look this this (reffenced from this Stack Overflow Post):
POST / HTTP/1.1
[[ Less interesting headers ... ]]
Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150
Content-Length: 834
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text1"
text default
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text2"
aωb
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file1"; filename="a.txt"
Content-Type: text/plain
Content of a.txt.
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file2"; filename="a.html"
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file3"; filename="binary"
Content-Type: application/octet-stream
aωb
-----------------------------735323031399963166993862150--
My question would be how do I create a request in swagger inspector that accepts multiple data types? It seems like I would need to set mulple sections in the body separated by boundaries with multiple Content-Disposition and Content-Type's for each. Would there be a cleaner way to do that through the Swagger Inspector interface? Or am I going about this in the wrong way?
Thanks!
Swagger Inspector currently supports multipart/form-data requests containing one or more files. It does not support arbitrary body parts in multipart requests (e.g. a file + JSON or text data). You'll need to use another HTTP client to test such requests.
You can submit feature request for Swagger Inspector here:
https://community.smartbear.com/t5/Swagger-Inspector-Feature/idb-p/SwaggerInspectorFeatureRequests

Azure blob put - image corrupt

I'm trying to use the azure storage blob to put an item on it.
It works fine with PDF, MP3 but, when I upload an IMAGE file (or TXT file) the file has changed.
When I download the pdf or the mp3, the file is readable. But for the image, it says that it's corrupt ...
However the TXT file is readable but the content is surrounded by the webkitfromboundary
I thinks this is the reason why image don't work...
Exemple :
TXT file with hello word only ->
------WebKitFormBoundary3rxc8zHbnz4expeP
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain
hello word
------WebKitFormBoundary3rxc8zHbnz4expeP--
So I guess this is the reason why images don't work
so this is my upload (from angularJS) --- I Use https://github.com/danialfarid/ng-file-upload
Upload.upload
url: url
method: 'PUT'
headers: head
file: file
.success (data, status, headers, config) ->
console.log("SUCCES!!!")
value of head - ( for an text file)
{"x-ms-version":"2014-02-14","x-ms-blob-type":"BlockBlob","x-ms-date":"2015-11-06 10:02:24 GMT","Authorization":"SharedKey generate_key","Content-Type":"text/plain"}
ofcourse, the content-type change when this is an image (it takes file.type )
If it can help, this is the request headers from the network page ->
Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:fr,en-US;q=0.8,en;q=0.6
Connection:keep-alive
Content-Length:214
Content-Type:application/json;charset=utf-8
Host:myaccount.blob.core.windows.net
Origin:http://localhost:9000
Referer:http://localhost:9000/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36
and the request payload ->
------WebKitFormBoundary3rxc8zHbnz4expeP
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain
------WebKitFormBoundary3rxc8zHbnz4expeP--
So, anyone has an idea ?
The anwser was that the Content-Type sent in the http request was never the good type.
It's because
Upload.upload
doesn't allow you to change it.
BUT you can use the http method of the same library, and it'll work like that :
Upload.http
url: uploadInfo.url
method: 'PUT'
headers: {
"Content-Type": file.type
[other headers ...]
}
data: file
and there you go ! it works !

Why "form POST" stay in status "PENDING" when trying to uplaod image to blobstore

When I go to https://appengine.google.com/blobstore/ I see that images are uploaded (cf. blob viewer).
In prod mode :
In chrome dev tool, when I submit the form in order to upload the image, I see that the form stay in a "PENDING" status. The purpose of this mail is to help me to understand what should fail. In the Network tab, I hae the following Header :
Request
URL:http://www.mananaseguro.com/_ah/upload/AMmfu6aImWsEdAeiy_FVrscqQiRoRSvjK2QSX6thgKTaMk4nKLbiJg86RrocrzAqWj2X2vi1gKrY_Yvr2kSQNpFMwxBiUFa1Tk5oEVZGjhMm_9SavhOAjNoteylbfLT7aZ5dUYMaDR2N/ALBNUaYAAAAAUeJL86BP_9txQLF54r96AvYfm1Nuw90l/
Request Headersview source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Content-Type:multipart/form-data; boundary=----WebKitFormBoundarymTEOMdBbtHBxoFJb
Origin:http://www.mananaseguro.com
Referer:http://www.mananaseguro.com/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36
Request Payload
------WebKitFormBoundarymTEOMdBbtHBxoFJb
Content-Disposition: form-data; name="myFile"; filename="trailer-8.webp"
Content-Type: image/webp
------WebKitFormBoundarymTEOMdBbtHBxoFJb--
And in the response tab :
**This request has no data response**
I have done the following servlet configuration (GUICE)
serve("/_ah/upload").with(BlobstoreUploadFinishedServlet.class); // post
serve("/_ah/serve").with(Servlet_Serve.class);
And some related line of code I call :
resp.sendRedirect("/_ah/serve?blob-key=" + url);
String url = blobstoreService.createUploadUrl("/_ah/upload");
Question : Can you explain me why "I" do not have a response (form POST is always pending) ?
I also do not know if I should use "/_ah/" or not (I have decided to put it everywhere) ?
I have another issue in dev mode, I cannot test upload to blobstore, because I have the following error :
WARNING: /_ah/upload/ag1tYW5hbmFzZWd1cm8xch0LEhVfX0Jsb2JVcGxvYWRTZXNzaW9uX18Y2aQCDA
java.lang.NullPointerException
at com.google.appengine.api.blobstore.dev.UploadBlobServlet.getSessionId(UploadBlobServlet.java:134)
Question : What is happening ? Is it a problem related to cookies ?
Thanks you,
I remember dealing with this issue a while back - sadly this happens because the blobstore implementation in devmode is different from the production one.
you don't need the /_ah/ prefix and if I'm not mistaken than /_ah/upload is reserved for the blobservice so you shouldn't use it. (don't take my word on this)
This is far from optimal, but you can understand if you are in devmode on the server side
by calling a utility function:
public static boolean isDevelopmentMode() {
return ( SystemProperty.environment.value() ==
SystemProperty.Environment.Value.Development );
}
and implementing a different logic by calling resp.sendRedirect("devmodeServe?blob-key="+blobkey); and binding that servlet to a DevmodeServeServlet class:
class DevmodeServeServlet extends HttpServlet
{
BlobstoreService blobstoreService = BlobstoreServiceFactory
.getBlobstoreService();
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
BlobKey blobKey = new BlobKey(req.getParameter("blob-key"));
blobstoreService.serve(blobKey, resp);
}
}
I also don't remember why, but I think you have to use your computer name instead of 127.0.0.1 or localhost in the browser address bar (you'll might need to add it to the allowed lists in the gwt devmode plugin in your browser)

Resources