Apache Camel server app receiving a multipart form POST (file upload) - apache-camel

I'm using Camel servlet component in order to receive xml documents and now I also need to receive files (jpegs, gifs, etc). So here is how my client app is sending a file:
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.DefaultHttpClient;
public class HttpClientUploadHelper {
public boolean upload(final File file, final String url) {
boolean wasSent = false ;
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
entity.addPart(file.getName(), new FileBody(file));
post.setEntity(entity);
try {
HttpResponse response = client.execute(post);
wasSent = response.getStatusLine().getStatusCode()==200;
} catch (Exception e) {
}
return wasSent;
}
}
my Camel Processor then extracts the HttpServletRequest this way:
HttpServletRequest req = exchange.getIn().getHeader(Exchange.HTTP_SERVLET_REQUEST, HttpServletRequest.class);
then I have this method to finally parse and save the file:
import org.apache.commons.fileupload.FileItemFactory;
import org.apache.commons.fileupload.FileItemIterator;
import org.apache.commons.fileupload.FileItemStream;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.io.IOUtils
... class declaration, body, etc...
void parseAndSaveFile(final HttpServletRequest req) throws Exception {
// Check that we have a file upload request
boolean isMultipart = ServletFileUpload.isMultipartContent(req);
// Create a factory for disk-based file items
FileItemFactory factory = new DiskFileItemFactory();
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload(factory);
// Parse the request
FileItemIterator receivedFiles = upload.getItemIterator(req);
while (receivedFiles.hasNext()) {
FileItemStream file = receivedFiles.next();
if (file.isFormField()) {
System.out.println("WTF?");
} else {
String fileName = file.getName();
File uploadedFile = new File("/home/myuser/" + fileName);
FileOutputStream out = new FileOutputStream(uploadedFile);
IOUtils.copy(file.openStream(), out);
}
}
}
when I use above code within Camel, that isMultipart flag is "true" but that receivedFiles iterator doesn't contains any element. When I use above code within another project with just a plain servlet, the code works. In both ways I'm using jetty as the web container.
So is there any other way to extract the file name and it's content within my camel processor ?
Thanks!

Since you're using Jetty, have you considered using the included MultipartFilter instead of the FileUpload project? Super clean and easy to use.
From the javadoc:
"This class decodes the multipart/form-data stream sent by a HTML form that uses a file input item. Any files sent are stored to a temporary file and a File object added to the request as an attribute. All other values are made available via the normal getParameter API and the setCharacterEncoding mechanism is respected when converting bytes to Strings."

Does this help?
public void process(Exchange exchange) throws Exception {
Message in = exchange.getIn();
Set names = in.getAttachmentNames();
for(String n: names) {
System.out.println("attachment "+n);
DataHandler h = in.getAttachment(n);
if(h!=null) {
try {
Object o = h.getContent();
System.out.println(o);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
if(!names.isEmpty())
return;
}

Related

CsrfFilter always fires exception in Spring 6.0+

In the new Spring Boot 3.0.1 I'm adding support for CSRF protection by adding this simple configuration as per Spring documentation:
http.csrf().csrfTokenRepository(tokenRepository);
and
#Bean
public CsrfTokenRepository tokenRepository() {
var tokenRepo = CookieCsrfTokenRepository.withHttpOnlyFalse();
tokenRepo.setCookiePath("/");
return tokenRepo;
}
On the front-end side, there is an Angular app with import of the standard library
import {HTTP_INTERCEPTORS, HttpClientModule, HttpClientXsrfModule} from '#angular/common/http';
However I cannot make it work with the standard workflow, like
Execute request to get XCSRF-TOKEN cookie from the server.
Extract the token value from the cookie.
Add X-XSRF-TOKEN header with the extract value from the cookie.
The front-end sends both cookie and header with the same token value and it fails in check:
public final class XorCsrfTokenRequestAttributeHandler ...
private static String getTokenValue(String actualToken, String token) {
byte[] actualBytes;
try {
actualBytes = Base64.getUrlDecoder().decode(actualToken);
}
catch (Exception ex) {
return null;
}
byte[] tokenBytes = Utf8.encode(token);
int tokenSize = tokenBytes.length;
if (actualBytes.length < tokenSize) {
return null;
}
The methods always returns null for
if (actualBytes.length < tokenSize) {
return null;
}
The only way to make it work is to return encoded string to be pasted in the header directly by adding endpoint, like this:
DeferredCsrfToken deferredCsrfToken = repository.loadDeferredToken(request, response);
requestHandler.handle(request, response, deferredCsrfToken::get);
CsrfToken csrfToken = (CsrfToken) request.getAttribute("_csrf");
return csrfToken.getToken();
So the returned value to be used in header looks like:
ab5tfqabXomGPuLjDQk96mVMZHNOh_JnpPVM4F_hQU1sOMyWW9sLSsCtb72rWNPabyQJ01EpSUsvtctKwcQphWeAJX9ZDPmg
instead of
5adaf830-40e6-43c9-b42a-e36fd713c1a6
Any advice on what I'm missing here?

Injecting serverside data when using ISpaBuilder.UseReactDevelopmentServer

When using ASP.NET (Core, .NET 5) MVC's IApplicationBuilder.UseSpa / ISpaBuilder.UseReactDevelopmentServer (in development), is there a way to postprocess the index HTML before it's sent to the browser? I need to inject a script tag holding data about the currently auth'd user to be consumed by the React app.
I want to avoid having to do an extra call from inside my React app just to get the currently logged on user at startup.
You can use custom middleware to do that. Assuming you're on .net core 3.1, the middleware would look something along these lines:
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using System.IO;
using System.IO.Compression;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
namespace TestReactDevServer.Middleware
{
public class ScriptInjectorMiddleware
{
private readonly RequestDelegate _next;
public ScriptInjectorMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task InvokeAsync(HttpContext context)
{
//Save pointer to the original response body stream
var originalBodyStream = context.Response.Body;
//Create new memory stream
using (var responseBody = new MemoryStream())
{
//...and use it for subsequent requests so we can peek the contents
context.Response.Body = responseBody;
//Continue down the Middleware pipeline, eventually returning to this class
await _next(context);
//inspect response, inject script
await InjectScript(responseBody, "window.myUser='123';");
//copy the contents of the new memory stream to the original place
await responseBody.CopyToAsync(originalBodyStream);
}
}
private async Task<Stream> InjectScript(Stream input, string script)
{
input.Seek(0, SeekOrigin.Begin);
var decompressed = new MemoryStream();
using (var tmp = new GZipStream(input, CompressionMode.Decompress, true))
{
tmp.CopyTo(decompressed);
}
var html = await decompressed.StreamToString();
var modifiedHtml = Regex.Replace(html, "</body>[\\n\\r]+</html>", $"<script type=\"text/javascript\">{script}</script></body></html>", RegexOptions.IgnoreCase | RegexOptions.Multiline); // any way to locate closing tags will work here, you probably can be more efficient
input.Seek(0, SeekOrigin.Begin);
using (var modifiedHtmlStream = modifiedHtml.ToStream())
using (var tmp = new GZipStream(input, CompressionMode.Compress, true)) // might be optional
{
modifiedHtmlStream.CopyTo(tmp);
}
return input;
}
}
public static class ScriptInjectorMiddlewareExtensions
{
public static IApplicationBuilder UseScriptInjectorMiddleware(
this IApplicationBuilder builder)
{
return builder.UseMiddleware<ScriptInjectorMiddleware>();
}
}
public static class StreamExtensions {
public static async Task<string> StreamToString(this Stream stream) {
stream.Seek(0, SeekOrigin.Begin);
return await new StreamReader(stream).ReadToEndAsync();
}
public static Stream ToStream(this string str)
{
var stream = new MemoryStream();
var writer = new StreamWriter(stream);
writer.Write(str);
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
return stream;
}
}
}
a couple of things to point out:
Testing this with Chrome, I ended up having to decompress the proxied response and compress it back after modification - I think compression step might be optional.
Depending on your user agent you might need to handle more compression cases (see more examples on Github)
You will need to inject this middleware before your call to .UseSpa() in Startup.cs: adding app.UseUserInjectorMiddleware(); should pick up the included extension method
I suspect this example is far from being complete, especially in terms of handling different encodings and content types - I am hoping you'd be able to adapt the idea to your use case.

Send a PDF as an attachment through Sendgrid

I'm using Sendgrid to send emails through an app on GAE. It's working fine, but I also want to be able to send PDFs as an attachment.
I'm not using Sendgrid.jar file in my project. I've just used Sendgrid.java. And this class has no methods by which i can add attachments. Can someone help me?
public static boolean sendEmail(String fromMail, String title, String toMail, String message) throws IOException {
Email from = new Email(fromMail);
String subject = title;
Email to = new Email(toMail);
Content content = new Content("text/html", message);
Mail mail = new Mail(from, subject, to, content);
Path file = Paths.get("file path");
Attachments attachments = new Attachments();
attachments.setFilename(file.getFileName().toString());
attachments.setType("application/pdf");
attachments.setDisposition("attachment");
byte[] attachmentContentBytes = Files.readAllBytes(file);
String attachmentContent = Base64.getMimeEncoder().encodeToString(attachmentContentBytes);
String s = Base64.getEncoder().encodeToString(attachmentContentBytes);
attachments.setContent(s);
mail.addAttachments(attachments);
SendGrid sg = new SendGrid("sendgrid api key");
Request request = new Request();
request.setMethod(Method.POST);
request.setEndpoint("mail/send");
request.setBody(mail.build());
Response response = sg.api(request);
if (response != null) {
return true;
} else {
return false;
}
}
Define above static method and call with relevant arguments as your program wants.
Here is the code of a servlet that sends a mail with a PDF as attachment, through Sendgrid:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) {
....
ByteArrayOutputStream os = null;
try {
PDFGenerator pdfGenerator = new PDFGenerator(invoiceOut);
os = pdfGenerator.getPDFOutputStream();
} catch (Exception e) {
....
}
SendGrid sendgrid = new SendGrid(Constants.SENDGRID_API_KEY);
SendGrid.Email email = new SendGrid.Email();
email.addTo(....);
email.setFrom(....);
email.setFromName(....);
email.setSubject(....);
email.setHtml("......");
ByteBuffer buf = null;
if (os == null) {
//error...
} else {
buf = ByteBuffer.wrap(os.toByteArray());
}
InputStream attachmentDataStream = new ByteArrayInputStream(buf.array());
try {
email.addAttachment("xxxxx.pdf", attachmentDataStream);
SendGrid.Response response = sendgrid.send(email);
} catch (IOException e) {
....
throw new RuntimeException(e);
} catch (SendGridException e) {
....
throw new RuntimeException(e);
}
}
PDFGenerator is one of my classes in which getPDFOutputStream method returns the PDF as ByteArrayOutputStream.
I personally find it easier to directly construct the JSON request body as described in the API docs than to use Sendgrid's libraries. I only use the Sendgrid library for sending the request after I construct the JSON data myself.
When constructing the JSON data you need to specify at least a filename and the content (i.e., the PDF file). Make sure to Base64 encode the PDF file before adding it to the JASON data.
I'd include some code, but I do Python and not Java so not sure that would help.

Too many requests after uploading an image to google cloud storage

I watch a gcs bucket with watchbucket command. After that i am doing a upload. The watch command sends a notification to my servlet on app engine.
To be sure about that, i have a look at the logs. But it seems so, that some how the servlet keeps on getting request after request. Four times successfully and after that it gets an nullpointer exception. Why are there so many requests?
* EDIT *
On client side i use a meteorJS a Javascript framework. I added the extension slingshot to process the upload.
first i have to provide a the necessary info like acl, bucket, etc. like this:
Slingshot.createDirective('uploadSpotCover', Slingshot.GoogleCloud, {
bucket: 'upload_spot_cover',
GoogleAccessId: 'accessId',
acl: 'project-private',
maxSize: 0,
cacheControl: 'no-cache',
...
}
As you can see in line 153 slingshot uses a XMLHttpRequest to upload to Cloudstorage
https://github.com/CulturalMe/meteor-slingshot/blob/master/lib/upload.js#L144
On serverside my Servlet and it's logic look like this.
public class Upload extends HttpServlet {
private BucketNotification notification;
#Override
public void doPost(HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException {
this.notification = getBucketNotification(req);
UploadObject object = new UploadObject(notification);
CloudStorageHandler gcs = new CloudStorageHandler();
BlobstoreHandler bs = new BlobstoreHandler();
ImageTransformHandler is = new ImageTransformHandler();
/** GET DATA FROM GCS **/
byte[] data = gcs.getFileFromGoogleCloudStorage(object.getGcsBucket(), object.getGcsPath());
//BlobKey bk = bs.createBlobKey(object.getGcsBucket(), object.getGcsPath());
/******************/
/** TRANSFORMATION **/
byte[] newImageData = is.resizePicture(data, 1200, 1200, "JPEG", 65, 0.5, 0.5);
/******************/
/** STORE NEW RESIZED FILE INTO GCS BUCKET **/
UploadObject tmp = new UploadObject(object.getGcsPath(), "JPEG", "beispiel", 1200, 1200);
tmp.setData(newImageData);
gcs.saveFileToGoogleCloudStorage(newImageData, "beispiel", object.getGcsPath(), "JPEG", "public-read");
/******************/
/** CREATE SERVING URL via BLOBKEY **/
BlobKey bk_neu = bs.createBlobKey("beispiel", object.getGcsPath());
String servingUrl = is.createServingUrlWithBlobkey(bk_neu);
/******************/
log("Blobkey: "+ bk_neu.getKeyString());
log("url: "+ servingUrl);
res.setStatus(200);
}
private BucketNotification getBucketNotification(HttpServletRequest req) {
BucketNotification notification = null;
String jsonString ="";
try {
jsonString ="";
BufferedReader in = new BufferedReader(new InputStreamReader(req.getInputStream()));
for (String buffer;(buffer = in.readLine()) != null;jsonString+=buffer + "\n");
in.close();
notification = new Gson().fromJson(jsonString, BucketNotification.class);
} catch (IOException e) {
log("Failed to decode the notification: " + e.getMessage());
return null;
}
return notification;
}
}
I wrapped the specific service methods like save file to cloudstorage in its own handlers.
Because I do not know what you are using to upload, I'm guessing now. Most file uploads upload stuff in bulks, so not the whole file at once, but in smaller chunks until everything is up. This would explain the multiple calls you are getting.
Can you show us the server and the client code? Maybe that way we could see the problem.

How to upload file on GAE with GWT?

I've been trying this for hours, so maybe a fresh set of eyes will help.
I'm trying to upload a file to the server using GWT (using UiBinder) on Google App Engine. Following all the examples I can find, things look like they should work, but the server's 'post' method is showing 0 items uploaded.
Here's the client code:
FormUploader.ui.xml:
<g:FormPanel action="/formImageUploader" ui:field="formPanel">
<g:VerticalPanel>
<g:FileUpload ui:field="uploader"></g:FileUpload>
<g:Button ui:field="submit">Submit</g:Button>
</g:VerticalPanel>
</g:FormPanel>
FormUploader.java:
#UiField
FormPanel formPanel;
#UiField
FileUpload uploader;
#UiField
Button submit;
public FormUploader() {
initWidget(uiBinder.createAndBindUi(this));
formPanel.setEncoding(FormPanel.ENCODING_MULTIPART);
formPanel.setMethod(FormPanel.METHOD_POST);
addSubmitHandlers();
}
private void addSubmitHandlers() {
formPanel.addSubmitCompleteHandler(new SubmitCompleteHandler() {
#Override
public void onSubmitComplete(SubmitCompleteEvent event) {
Core.log("Inside submitComplete");
Window.alert(event.getResults());
}
});
}
web.xml:
<servlet>
<servlet-name>formImageUploader</servlet-name>
<servlet-class>com.company.server.FormImageUploader</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>formImageUploader</servlet-name>
<url-pattern>/formImageUploader</url-pattern>
</servlet-mapping>
FormImageUploader.java:
public class FormImageUploader extends HttpServlet {
protected void doPost(HttpServletRequest request, HttpServletResponse resp) throws ServletException, IOException {
log.info("Request.getContentLength(): " + request.getContentLength());
log.info("Content type: " + request.getContentType());
log.info("Parm names: ");
for (Object paramName : request.getParameterMap().keySet()) {
log.info("__Param name: " + paramName.toString());
}
log.info("Attribtue names: ");
Enumeration enu = request.getAttributeNames();
while (enu.hasMoreElements()) {
log.info("__Attribute name: " + enu.nextElement().toString());
}
log.info("Header names: ");
enu = request.getHeaderNames();
while (enu.hasMoreElements()) {
log.info("__Header name: " + enu.nextElement().toString());
}
resp.getWriter().println("Post Code has finished executing");
}
All this executes just fine, the client shows an alert box containing Post Code has finished executing, but I can't figure out how to get the content of the file I want to upload.
Here's the server side log:
Request.getContentLength(): 63
Content type: multipart/form-data; boundary=---------------------------194272772210834546661
Parm names:
Attribtue names:
__Attribute name: com.google.apphosting.runtime.jetty.APP_VERSION_REQUEST_ATTR
Can anyone help me see why I'm not seeing any files on the server after clicking "submit" on the client?
Thanks in advance
Edit:
Still can't figure out why I can't get the form data on the server. I'm following this example, and just trying to write the response out to the client, but the server is definitely not seeing any content from the client.
Server code:
try {
log.info("Inside try block");
ServletFileUpload upload = new ServletFileUpload();
log.info("created the servlet file upload");
res.setContentType("text/plain");
log.info("set the content type to text/plain");
FileItemIterator iterator = upload.getItemIterator(req);
log.info("Got the iterator for items");
while (iterator.hasNext()) {
log.info("__inside iterator.hasNext");
FileItemStream item = iterator.next();
log.info("__assigned the file item stream");
InputStream stream = item.openStream();
log.info("__opened said stream");
if (item.isFormField()) {
log.warning("Got a form field: " + item.getFieldName());
} else {
log.warning("Got an uploaded file: " + item.getFieldName() +
", name = " + item.getName());
int len;
byte[] buffer = new byte[8192];
while ((len = stream.read(buffer, 0, buffer.length)) != -1) {
res.getOutputStream().write(buffer, 0, len);
}
log.info("Done writing the input to the output stream");
}
}
} catch (Exception ex) {
log.severe("Exception; " + ex);
throw new ServletException(ex);
}
log.info("Done parseInput3");
Logs:
Inside parseInput3
Inside try block
created the servlet file upload
set the content type to text/plain
Got the iterator for items
Done parseInput3
Again, it's definitely not able to iterate over the file items on the server... any idea why?
Turns out, GAE has a completely different way of uploading files (Blobs):
http://code.google.com/appengine/docs/java/blobstore/overview.html#Uploading_a_Blob
You need to supply the form's on the client with a Blobstore Upload URL (which times out) from the server, retrieved using:
blobstoreService.createUploadUrl('urlToRedirectToAfterUploadComplete')
After setting that upload URL on the form's action, the form will be submitted and the blobstore service will redirect to the supplied URL, which can then access the BlobKey of the object stored by Google.
So with this, I added the following ChangeHandler to my GWT FileUpload:
private void addChangeListener() {
Core.log("Selected file: " + uploader.getFilename());
uploader.addChangeHandler(new ChangeHandler() {
#Override
public void onChange(ChangeEvent event) {
MyApp.SERVICES.getUploadUrl(new SuccessAsyncCallback<String>() {
#Override
public void onSuccess(String uploadUrl) {
form.setAction(uploadUrl);
form.submit();
Core.log("Submitted form with upload url: " + uploadUrl);
}
});
}
});
}
Then in the servlet I have the GAE redirect to after uploads:
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws Exception {
BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
Map<String, List<BlobKey>> blobs = blobstoreService.getUploads(req);
// where 'uploader' is the name of the FileUploader (<input type='file'>) control on the client
List<BlobKey> blobKeys = blobs.get("uploader");
resp.getOutputStream().print(blobKeys.get(0).getKeyString());
}
Note!
GAE will throw a OutOfMemory exception when trying to get the uploaded blob's value if the FileUpload control doesn't have a name="whateveryouwantthenametobe" attribute defined, even if you tell it (using UiBinder) that the id is ui:field="nameofControl"
You cannot write to the local file system - you should take a look at the files api.

Resources