Objective: Suppose the client submits a string or text file to the server (Google App Engine) using a web form. I want the server to modify the original file and serve it back to the client.
I think the only way to serve files from GAE is using the Blobstore, right? Then, as we cannot modify blobs, I believe a solution would be:
Client uploads a file using HttpRequest
Server reads the uploaded file and copies it to a temp buffer (not sure if is there a method to do this)
Server deletes original blob
Server modifies data in the temp buffer
Server writes the modified buffer to the Blobstore
Server serves the new blob to the client
Would this work? Could you think about any other solution?
Thanks
I think the only way to serve files from GAE is using the Blobstore, right?
Wrong. A 'file' is just a way of storing data on disk; there's nothing about serving them from a webserver that requires the data come from an actual, writable disk file. You can simply accept the user's data via a form upload, modify it, and serve it back to them, without it having to ever touch disk, the blobstore, or any other permanent storage medium.
This only becomes a problem if the user's data is too large to fit in memory, in which case you will have to store the data somewhere while you work on it, such as in the blobstore.
http://code.google.com/appengine/kb/java.html#fileforms
shows you how to do it for file upload, which has to be performed thro multipart form-data.
Similarly for non-file data, where you read straight from the request stream.
You don't even have to store the file/input stream. Just spit out the processed data into the output response stream, while reading the input FileItemStream or request inputstream.
If your file/input processing requires look-forward, determine the maximum distance of look-forward and use that distance as your buffer size.
Further Edits
To respond to the client with a file type, set the response content-type or mime-type.
e.g., I've had apps which dynamically generated gifs, jpgs, xls, cvs, etc.
There isn't any difference whether source of response stream is a file you read or a stream that you generate dynamically. Because, even if you had a stored file that needs to be sent as response to client, you could still have to convert it into a response stream and flag the content-type appropriately.
For dynamically generated content, unless you need to cache the output, you need not generate the file into a web URL-visible location and then generate a new html page with the link, and send that html page to the browser. You don't need the user's browser to have to refresh itself just to get that link.
You would simply send the "file" directly with the response stream. You could design your GWT client to accept the "file", perhaps in a named frame, where the named frame src url is the app that performs the dynamic generation of the file.
Read http://en.wikipedia.org/wiki/Mime-type to find the content-type you need.
If the target client's browser does not have the content handler set-up for the response's content-type, it would as for a treatment or be treated as a file download.
I had frequently used jsp or jspx to generate dynamically generated charts or spreadsheets. No stored files involved. The response is written to while the request is being read. Let's look at the jsp page directive to set the content-type to invoke MS Excel on a CSV.
<%# page language="java" contentType="application/vnd-ms-excel; charset=UTF-8"
pageEncoding="UTF-8"%>
For a servlet, ServletResponse.setContentType(String)
is method to set the content-type.
Related
One Gateway application is going to send a GET request to my Vert.x application. For this request, I need to read a large zip file from Amazon S3 server which I am able to read in BufferedInputStream. I don't want to download this file but rather I need to send this stream data to the gateway application (NOT a downloadable file with application/zip content type but stream data or byte chunks) which it will further send to end application where this stream data will be downloaded as a zip file. So the sending the zip file in form of stream to the gateway application I need to achieve using Vert.x. Already gone through lot of documentation and blogs but everywhere it is given to download the zip file which is not my intention. Could anyone please suggest how I could achieve this streaming of zip file in http response of calling request using Vert.x? Do I need to use Java NIO? If yes, could you please put details? Sorry, but I have nothing to put here as part of code. Thanks in advance!
I'm trying to upload files to Google Drive with ProgressListener and ChunkSize enabled (thus with DirectUploadEnabled disabled). This way i have a more reliable upload and the possibility for a progress indication to the user.
I transfer the files from the GWT website to the GAE with a FormPanel and a FileUploadField which POSTS the file to GAE on submit(). On the GAE i receive the file with an UploadServlet which uses org.apache.commons.fileupload to receive the documents as a stream. I don't want to receive the complete documents on the GAE because the documents are to big. Therefore i start the upload (insert) to Google Drive with the received stream from the incoming request.
Now there's a problem; for the insert i need to know the size of the stream;
int lContentLength = getRequest().getContentLength();
FileItemStream lFileItemStream = getFileItemStream();
InputStream lInputStream = lFileItemStream.openStream();
BufferedInputStream lBufferedInputStream = new BufferedInputStream(lInputStream);
InputStreamContent lInputStreamContent = new InputStreamContent(pContentType, lBufferedInputStream);
lInputStreamContent.setLength(lContentLength);
My first guess was the ContentLengt from the incoming Servlet request. But this is not correct because this concerns the complete request (which also contains other fields which are used as parameters). Without the Drive option DirectUploadEnabled i need the exact stream size from the uploaded document, otherwise the upload stall's at the end...
How do i grap this document size? The Google example is stupid because it uses a local file;
https://code.google.com/p/google-api-java-client/wiki/MediaUpload
Yes from a local file it is easy to get the file size (mediaFile.length()). But from a website ... Several sites specify it is not possible to grab the file size before submit() from the website, and it seems also impossible to determine the stream-size on GAE without loading the complete file...
How do i determine this streamsize? Is there another solution for this problem?
Is it possible to send a http upload request a file to a Apache or IIS that will have a fileName with "../" or ".." that wouldn't be rejected and would be passed to php or ASP.Net engine?
Not really the way you are asking. By the time it gets to the server the browser has read the file and delivered it as a chunk of content with no information about where it came from other than the original file name which you can choose to use or discard.
Generally file uploads go into a temporary storage place (e.g. /tmp) and then need to be moved out of there to somewhere which you can control and name.
This storage is configured on the server, and so any attempt to put path info into the filename should also be blocked by the file upload implementation of the server which should sanitise the filenames again if the browser didn't already do so.
If there's a bug then all bets are off though.
I have a ~2MB file that my Google AppEngine server must use (not serve) as part of a computation for a service request.
That is, a client makes a particular request, my GAE server must first get the data from this ~2MB file, do some computations using this data, then serve a small response back to the client.
Where best do I store this data so that it can be quickly read and used by the server in the computation?
If the following assumptions hold true
the file is not going to require updates outside of appengine code updates
that the file is read only
Then deploy the file with your code and read the file into memory during startup (ideally using warmup requests) and just operate on it from memory. If you code has to have file based semantics to access the data (read,seek, etc) then read the file contents and wrap it in StringIO.
You will need to assign the value read from the file to a module level variable, that way whenever you get a new request you can just get the files contents by importing the module and referencing the name. ie. mymodule.filecontents
I'm working on a side project right now for an email client. I'm using a library to handle the retrieval of the messages from the server. However, I have a question on caching.
I don't want to fetch the entire list of headers everytime I load the client. Ideally, what I'd like to do is cache them and then update the list with what is on the server.
What's the best way to go about this? Should I store all the header information (including the server's message ID #) in a database, load the headers from that DB. Then as a background task sync up with the server...
Or is there a better way?
Look at the webmail sample of this open source project that use local caching:
http://mailsystem.codeplex.com/
If I remember well, he used a combination of local RFC822 plain text email storing with the message id as the filename and an index file with high level data.
Maybe the message itself where zipped to save disc space.
That's just a sample for the library, so don't expect code art there, but that's a start.