httpClient, problem to do a POST of a Multipart in Chunked mode... - file

Well I am wondering how I can achieve to post a multipart in chunked mode. I have 3 parts, and the files which can be big so must be sent in chunks.
Here what I do :
MultipartEntity multipartEntity = new MultipartEntity() {
#Override
public boolean isChunked() {
return true;
}
};
multipartEntity.addPart("theText", new StringBody("some text", Charset.forName("UTF-8")));
FileBody fileBody1 = new FileBody(file1);
multipartEntity.addPart("theFile1", fileBody1);
FileBody fileBody2 = new FileBody(file2);
multipartEntity.addPart("theFile2", fileBody2);
httppost.setEntity(multipartEntity);
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpClient httpClient = new DefaultHttpClient(params);
HttpResponse httpResponse = httpClient.execute(httppost);
On the server side, I do receive the 3 parts but the files for example are not chunked, they are received as one piece... basically total I see 4 boundaries appearing only : 3 --xxx, 1 at the end --xxx-- .
I thought the override of isChunked would do the trick but no... ;(
Is what I am trying to do feasible ? How could I make that work ?
Thanks a lot.
Fab

To generate a multipart body chunked, one of the part must have it size unavailable. Like a part that is streaming.
For example let assume your file2 is a really big video. You could replace the part of your code:
FileBody fileBody2 = new FileBody(file2);
multipartEntity.addPart("theFile2", fileBody2);
wtih that code:
final InputStreamBody binVideo = new InputStreamBody(new FileInputStream(file2), "video/mp4", file2.getName());
multipartEntity.addPart("video", binVideo);
since now the third part is an InputStream instead of File, your multipart HTTP request will have the header Transfer-Encoding: chunked.

Usually any decent server-side HTTP framework (such as Java EE Servlet API) would hide transport details such as transfer coding from the application code. just because you are not seeing chunk delimiters by reading from the content stream does not mean the chunk coding was not used by the underlying HTTP transport.
You can see exactly what kind of HTTP packets HttpClient generates by activating the wire logging as described here:
http://hc.apache.org/httpcomponents-client-ga/logging.html

Related

CXF wsdl2java, GZip compression, and stub reutilization

I´m using CXF to consume a WebService and, as the responses are quite large, I´m requesting with a gzip "Accept-Encoding" and using GZIPInInterceptor to handle the gziped response. Also my WSDL is very large (360kb) and it takes a long time(+10 seconds) to create the stub, because it has to read and parse the WSDL, so I´m creating the stub once and reusing it.
The problem is, whenever I try to use two different methods the second request gives me an error saying it is expecting the previous request.
To illustrate my problem I created a simple example with this public WebService:
http://www.webservicex.net/BibleWebservice.asmx?WSDL
Without the GZip compression it works fine:
BibleWebserviceSoap bibleService = new BibleWebservice().getBibleWebserviceSoap();
String title = bibleService.getBookTitles();
response.getWriter().write(title);
String johnResponse = bibleService.getBibleWordsbyKeyWord("John");
response.getWriter().write(johnResponse);
I´m able to receive both responses.
Enabling Gzip compression:
BibleWebserviceSoap bibleService = new BibleWebservice().getBibleWebserviceSoap();
//GZIP compression on bibleService
Client client = ClientProxy.getClient(bibleService);
client.getInInterceptors().add(new GZIPInInterceptor());
client.getInFaultInterceptors().add(new GZIPInInterceptor());
// Creating HTTP headers
Map<String, List<String>> headers = new HashMap<String, List<String>>();
headers.put("Accept-Encoding", Arrays.asList("gzip"));
// Add HTTP headers to the web service request
client.getRequestContext().put(Message.PROTOCOL_HEADERS, headers);
String title = bibleService.getBookTitles();
response.getWriter().write(title);
String johnResponse = bibleService.getBibleWordsbyKeyWord("John");
response.getWriter().write(johnResponse);
When I try to receive the second response I´m getting this exception:
org.apache.cxf.interceptor.Fault: Unexpected wrapper element {http://www.webserviceX.NET}GetBookTitlesResponse found. Expected {http://www.webserviceX.NET}GetBibleWordsbyKeyWordResponse.
On my real application I´m getting an exception with the request:
org.apache.cxf.binding.soap.SoapFault: OperationFormatter encountered an invalid Message body. Expected to find node type 'Element' with name 'GetAvailabilityRequest' and namespace 'http://schemas.navitaire.com/WebServices/ServiceContracts/BookingService'. Found node type 'Element' with name 'ns4:PriceItineraryRequest' and namespace 'http://schemas.navitaire.com/WebServices/ServiceContracts/BookingService'
My sample project can be downloaded here:
http://www.sendspace.com/file/plt0m4
Thank you
Instead of setting the protocol headers directly like that, use CXF's GZIPOutInterceptor to handle that.
Either that or reset the PROTOCOL headers for each request. When set like that, the headers map gets updated as the request goes through the chain. In this case, the soapaction gets set. This then gets resent on the second request.

Returning multiple items with Servlet

Good day, I'm working on a Servlet that must return a PDF file and the message log for the processing done with that file.
So far I'm passing a boolean which I evaluate and return either the log or the file, depending on the user selection, as follows:
//If user Checked the Download PDF
if (isDownload) {
byte[] oContent = lel;
response.setContentType("application/pdf");
response.addHeader("Content-disposition", "attachment;filename=test.pdf");
out = response.getOutputStream();
out.write(oContent);
} //If user Unchecked Download PDF and only wants to see logs
else {
System.out.println("idCompany: "+company);
System.out.println("code: "+code);
System.out.println("date: "+dateValid);
System.out.println("account: "+acct);
System.out.println("documentType: "+type);
String result = readFile("/home/gianksp/Desktop/Documentos/Logs/log.txt");
System.setOut(System.out);
// Get the printwriter object from response to write the required json object to the output stream
PrintWriter outl = response.getWriter();
// Assuming your json object is **jsonObject**, perform the following, it will return your json object
outl.print(result);
outl.flush();
}
Is there an efficient way to return both items at the same time?
Thank you very much
HTTP protocol doesn't allow you to send more than one HTTP response per one HTTP request. With this restriction in mind you can think of the following alternatives:
Let client fire two HTTP requests, for example by specifyingonclick event handler, or, if you returned HTML page in the first response, you could fire another request on window.load or page.ready;
Provide your for an opportunity of choosing what he'd like to download and act in a servlet accordingly: if he chose PDF - return PDF; if he chose text - return text and if he chose both - pack them in an archive and return it.
Note that the first variant is both clumsy and not user friendly and as far as I'm concerned should be avoided at all costs. A page where user controls what he gets is a much better alternative.
You could wrap them in a DTO object or place them in the session to reference from a JSP.

UTF-8 Strings getting scrambled by Restlet on GAE

I have a simple Restlet service hosted on AppEngine. This performs basic CRUD operations with strings and is working well with all sorts of UTF-8 characters when I test it with curl (for all the verbs).
This is consumed by a simple restlet client hosted in a servlet on another AppEngine app:
// set response type
resp.setContentType("application/json");
// Create the client resource
ClientResource resource = new ClientResource(Messages.SERVICE_URL + "myentity/id");
// Customize the referrer property
resource.setReferrerRef("myapp");
// Write the response
resource.get().write(resp.getWriter());
The above is pretty much all I have in the servlet. Very plain.
The servlet is invoked via jquery ajax, and the json that I get back is well formed and everything, but the problem is that UTF-8 encoded strings are coming back scrambled, for example:
Université de Montréal becomes Universit?? de Montr??al.
I tried adding this line in the servlet (before everything else):
resp.setCharacterEncoding("UTF-8");
But the only diference is that instead of getting ?? I get Universitᅢᄅ de Montrᅢᄅal (I don't even know what kind of characters those are, asian I suppose).
I am 100% sure the restlet service is OK, because other than debugging it line by line I am able to test it from cmd line with curl and it's returning well formed strings.
By looking at the http header of the response from firefox (when calling the servlet via javascript) I can see the encoding is indeed UTF-8, as expected. After hours of struggling reading every possible related article I came across this restlet discussion and noticed that indeed I do have Transfer-Encoding: chunked on the http header of the response. I tried the proposed solutions (override ClientResource.toRepresentation, didn't do any good so I tried restlet 2.1 as susggested with ClientResource.setRe​questEntityBuffering​(true), no luck there either) but I am not convinced my issue is related to Transfer-Encoding: chunked at all.
At this point I am out of ideas, and I would really appreciate any suggestions! O_o
UPDATE:
I tried doing a manual GET with a classic UrlConnection and the string is coming back alright:
URL url = new URL(Messages.SERVICE_URL + "myentity/id");
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
resp.getWriter().print(writer.toString());
So much for being all RESTful and fancy ...but still I have no clue why the original version doesn't work! :/
I tried doing a manual GET with a classic UrlConnection and the string is coming back alright:
URL url = new URL(Messages.SERVICE_URL + "myentity/id");
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
resp.getWriter().print(writer.toString());
So much for being all RESTful and fancy ...but still I have no clue why the original version doesn't work! :/
Does your response contain the appropriate "Content-Type" header? It should be something like "Content-Type: application/json; charset=UTF-8" (note the charset).
Try starting your development server and retrieving your resource from the command line using cURL and inspecting the headers, e.g. curl -i http://localhost:8080/myentity/id. In theory browsers should assume UTF-8 for JSON, but I wouldn't trust on that.

Silverlight Upload file to MVC3 controller endpoint (Server Respose NotFound )

I'm developing a recorder in silverlight and I need to upload data from stream to the web server after recording process is completed.
On server side I'm using ASP.NET MVC 3, and I have created a Controller with method FileUpload.
public class FileUploaderController : Controller
{
[HttpPost]
public ActionResult FileUpload(string fileName)
{
....
}
}
In silverlight applet, the upload is made by parts, about 20000 bytes at time. Servers web config is configured to accept larger amount of data.
Server returns an exception "The remote server returned an error: NotFound.".
In this case the request have not reached the action and I can't understand why.
Example of code that is used to start upload:
UriBuilder httpHandlerUrlBuilder = new UriBuilder("http://localhost:37386/FileUploader/FileUpload/?fileName=" + Guid.NewGuid() + ".wav");
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(httpHandlerUrlBuilder.Uri);
webRequest.Method = "POST";
webRequest.ContentType = "multipart/form-data"; // This solved my problem
webRequest.BeginGetRequestStream(new AsyncCallback(WriteToStreamCallback), webRequest);
EDIT
My route configuration is by default:
routes.MapRoute(
"Default", // Route name
"{controller}/{action}/{id}", // URL with parameters
new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
);
When the small amount of data is sent, everything goes well and server receives the requested data. But when data to be send is larger I'm just getting NotFound response. This doesn't make any sense to me, what I'm doing is:
HttpWebRequest to send 20000 bytes
close request stream (obtained from request.EndGetRequestStream)
wait for server response (from webRequest.EndGetResponse) This is where error occurs.
In my case, I never send more than 20000 bytes, which is strange this to work sometimes and others not.
I don't know a better way to explain this problem. If you need I can provide more code and more information.
Any help is very much appreciated.
With filddler I was able to get more detailed information regarding to the error. It was "upload file potentially dangerous Request.Form value was detected from the client...".
To solve this I've specified content-type of the webRequest to "multipart/form-data"

Getting file size in Silverlight 4

I'm downloading (from server to client) a file using a WebClient object:
WebClient wc = new WebClient();
wc.OpenReadCompleted += Load_TransferCompleted;
wc.OpenReadAsync(uriAddress, Filename);
I would like to know the file size before starting the download operation. Is there a way to do this in SL4?
thanks for your help.
Gilad.
Here is some air code for you to play with (I haven't tested it myself)
WebRequest req = WebRequestCreator.ClientHttp.Create(yourUri);
req.Method = "HEAD";
req.BeginGetResponse(ar =>
{
WebResponse resp = req.EndGetResponse(ar);
int length = resp.ContentLength;
// Do stuff with length
}, null);
By using the ClientHttp stack you can use "HEAD" request which will return the same set of headers as a "GET" but not the actual entity body.
There is at least one thing to look out for though, none of the existing cookies for the uri will be sent in the request. If the response is sensitive to cookies (for example because it needs a session id) then things will get a whole lot more complicated.
The only way I see to make this possible is by publishing the size. It could be coded but also obtainable through a web service.

Resources