I currently have a fully functional email notification system. It is all working correctly, but have noticed that one of the inline images, which is attached to the email via a rest service, is being requested three times.
It appears to be happening once transport.send is invoked. I'm assuming that when javamail goes to send email, it then calls for all inline images referenced and generates the base64 image and places it in the outgoing email.
My question is...
Is the above assumptions correct and why would it be calling the service multiple times when it has been verified in the raw email that it contains the image only once.
Below is a copy of the raw email with addresses changed, base64 images, text/plain and text/html versions removed.
Raw email
Date: Thu, 25 Feb 2016 20:40:04 -0800 (PST)
From: something.news.noreply#something.org
To: user.email#something.org
Message-ID: <56557433.31456461613977.JavaMail.something.news.noreply#something.orgh>
Subject: Business News for 02/26/2016
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_Part_6_670038123.1456461604268"
X-Priority: 3
------=_Part_6_670038123.1456461604268
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
TEXT VERSION OF EMAIL
------=_Part_6_670038123.1456461604268
Content-Type: multipart/related;
boundary="----=_Part_7_2073972040.1456461604268"
------=_Part_7_2073972040.1456461604268
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
HTML VERSION OF EMAIL
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="840da574-a395-4fb8-8f33-bcbb1837220e?t=1452039853427"
Content-Transfer-Encoding: base64
Content-ID: <#9dc060f2-84ec-4344-8871-68d4e4885b70_840da574-a395-4fb8-8f33-bcbb1837220e?t=1452039853427>
Content-Disposition: inline;
filename="840da574-a395-4fb8-8f33-bcbb1837220e?t=1452039853427"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="21ae1010-9675-4daa-8ac8-659052b943e8?t=1450724912940"
Content-Transfer-Encoding: base64
Content-ID: <#17ab1018-2bc5-44f5-9934-fb7b0f1a860b_21ae1010-9675-4daa-8ac8-659052b943e8?t=1450724912940>
Content-Disposition: inline;
filename="21ae1010-9675-4daa-8ac8-659052b943e8?t=1450724912940"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="a96af9b6-9093-4086-bf91-e5677bef533f?t=1450966405365"
Content-Transfer-Encoding: base64
Content-ID: <#15a682c9-95e8-42ee-880b-f52a47396341_a96af9b6-9093-4086-bf91-e5677bef533f?t=1450966405365>
Content-Disposition: inline;
filename="a96af9b6-9093-4086-bf91-e5677bef533f?t=1450966405365"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="25caf361-80a9-44c2-9ae6-97c19709becf?t=1450966427231"
Content-Transfer-Encoding: base64
Content-ID: <#2e9839ae-9c10-4c8f-9c4a-bc9e1a934fc3_25caf361-80a9-44c2-9ae6-97c19709becf?t=1450966427231>
Content-Disposition: inline;
filename="25caf361-80a9-44c2-9ae6-97c19709becf?t=1450966427231"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268
Content-Type: image/png; name=upChart
Content-Transfer-Encoding: base64
Content-ID: <#6ca90a68-1d24-4414-ac8a-8c84b50bb663_upChart>
Content-Disposition: inline; filename=upChart
BASE 64 IMAGE
THIS IS THE IMAGE THAT IS RETRIEVED VIA A WEB SERVICE CALL. IT CREATES A CHART
IMAGE THAT GET ATTACHED TO EMAIL.
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="10739ce0-c979-4e47-8cee-666304b9a92e?t=1451426510519"
Content-Transfer-Encoding: base64
Content-ID: <#14227419-3db7-444b-a704-7abaccd1e380_10739ce0-c979-4e47-8cee-666304b9a92e?t=1451426510519>
Content-Disposition: inline;
filename="10739ce0-c979-4e47-8cee-666304b9a92e?t=1451426510519"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268
Content-Type: image/png;
name="995344fa-765f-4a27-848c-d36d26c593f3?t=1451426553685"
Content-Transfer-Encoding: base64
Content-ID: <#f17f0535-a2a3-47a8-aca4-9ed883542de8_995344fa-765f-4a27-848c-d36d26c593f3?t=1451426553685>
Content-Disposition: inline;
filename="995344fa-765f-4a27-848c-d36d26c593f3?t=1451426553685"
BASE 64 IMAGE
------=_Part_7_2073972040.1456461604268--
------=_Part_6_670038123.1456461604268--
I have added I believe the relevant code. The code starts with what it started out like and then two changes I made based on the suggestion.
Started as:
try {
// Get a REST client so we can call the Highcharts service.
hcClient = new RestClient(hcServer, hcResource);
// Create a client response object.
ClientResponse cr = null;
// Initialize tries counter and loop a maximum of 3 tries
// if highcharts export server doesn't respond with a 200.
int tries = 0;
do {
// Increase tries count.
tries++;
// Call the service and place response data into our data object.
cr = hcClient.getWebResource().post(ClientResponse.class, params);
// Pull the response status from client.
upChartRespStatus = cr.getStatus();
// Determine if we received a 200 code or not.
if(upChartRespStatus == 200) {
// Obtain entity and build response.
response = Response
.ok(new ByteArrayInputStream(cr.getEntity(cbChart.getClass())))
.build();
} else {
// Log warning that chart was not received, what the response
// status was and what try we are on.
logger.warn("Unable to get requested chart, response code was "+upChartRespStatus+".");
logger.warn("Reattempting to get chart. The current count of retries is: "+tries);
}
} while (tries <= 3 && upChartRespStatus != 200);
} catch(Exception e) {
logger.error(e.getMessage());
} finally {
hcClient.destroy();
}
I thought maybe I would try sending a base64 encoded image. I didn't even get the image in the email doing this.
// Determine if we received a 200 code or not.
if(upChartRespStatus == 200) {
// Get entity from client request.
ByteArrayInputStream bais = new ByteArrayInputStream(cr.getEntity(cbChart.getClass()));
String b64Image = new String(Base64.encode(read(bais)));
// Obtain entity and build response.
response = Response
.ok(b64Image)
.header("Content-Type", "image/png")
.header("Content-Length", b64Image.length())
.header("Content-Transfer-Encoding", "BASE64")
.build();
} else {
// Log warning that chart was not received, what the response
// status was and what try we are on.
logger.warn("Unable to get requested chart, response code was "+upChartRespStatus+".");
logger.warn("Reattempting to get chart. The current count of retries is: "+tries);
}
My last attempt was going back to binary, but making sure that the headers were all in place. This last attempt did yield the chart image in the email, but still in all cases the transport.send is still calling the service three times before it actually sends.
// Determine if we received a 200 code or not.
if(upChartRespStatus == 200) {
// Get entity from client request.
ByteArrayInputStream bais = new ByteArrayInputStream(cr.getEntity(cbChart.getClass()));
// Obtain entity and build response.
response = Response
.ok(bais)
.header("Content-Type", "image/png")
.header("Content-Length", bais.available())
.header("Content-Transfer-Encoding", "BINARY")
.build();
} else {
// Log warning that chart was not received, what the response
// status was and what try we are on.
logger.warn("Unable to get requested chart, response code was "+upChartRespStatus+".");
logger.warn("Reattempting to get chart. The current count of retries is: "+tries);
}
Okay. So I got some success. Instead of setting headers at the service, I think that was to late. I took a look at the code that creates the email and specifically when the image gets added to its part. I've added what the code was and what I changed. By doing this, it is now only calling the service twice instead of three times.
private void addImagesInline(Multipart parent, List<URL> embeded, HashMap<String,String> cids) throws MessagingException {
if (embeded != null) {
for (URL img : embeded) {
final MimeBodyPart htmlPartImg = new MimeBodyPart();
DataSource htmlPartImgDs = new URLDataSource(img);
htmlPartImg.setDataHandler(new DataHandler(htmlPartImgDs));
String fileName = img.getFile();
fileName = getFileName(fileName);
String newFileName = cids.get(fileName);
boolean imageNotReferencedInHtml = newFileName == null;
if (imageNotReferencedInHtml) continue;
htmlPartImg.setHeader("Content-ID", "<"+newFileName+">");
htmlPartImg.setDisposition(BodyPart.INLINE);
htmlPartImg.setFileName(fileName);
parent.addBodyPart(htmlPartImg);
}
}
}
I added these to lines of code.
htmlPartImg.setHeader("Content-Type", "image/png");
htmlPartImg.setHeader("Content-Transfer-Encoding", "BASE64");
Thanks in advance for any replies!
I would expect the image to be requested twice. JavaMail will read the image once to determine what encoding is appropriate, based on the content of the image data. It will then read it again to encode it and include it in the message.
You can read the image yourself and save it locally to avoid multiple requests, or you can tell JavaMail what encoding to use by setting the Content-Transfer-Encoding header for the part; that way it won't have to guess what to use.
Related
We need to send a file of xbrl format (a type of xml) to a receiving system. However, the receiving system requires that we are sending some form data as well. Regarding the xbrl file we need to send with it a form data, name="data".
Regarding the second part, we need to send an email to the system via form data. And that is name="email".
(I'll be the first to admit that I really do not understand form data at all so I cant explain much better than this).
I have a custom pipeline and in the encode stage I have put my pipeline component (the entire Execute method is down below) and after that a MIME/SMIME component (standard Microsoft).
Take a look at how the request looks like when it is being sent from Postman (and fully accepted by the receiving system):
Content-Type: multipart/form-data; boundary=--------------------------035085236562027409735938
Content-Length: 299420
----------------------------035085236562027409735938
Content-Disposition: form-data; name="data"; filename="Dk_{5AA9547A-E103-40CD-A2F4-6B77F010E570}.xbrl"
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<!- ---- Lots of removed xbrl ---- ->
</xbrli:xbrl>
----------------------------035085236562027409735938
Content-Disposition: form-data; name="email"
Content-Type: text/plain
test#test.com
----------------------------035085236562027409735938--
Now what you see below is what we have managed to achieve this far:
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="_E9E8D52C-4722-488D-9FEC-713446D4A5C4_"
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_
MIME-Version: 1.0
Content-Type: application/xml; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Content-ID: {E8CA3F1A-F953-4FFB-A054-DEB8466A5D86}
Content-Description: body
<?xml version="1.0" encoding="utf-8"?>
<!- ---- Lots of removed xbrl ---- ->
</xbrli:xbrl>
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-ID: {3B35ECAC-990D-458C-8E7E-9C178CA99988}
Content-Description: email
Content-Disposition: attachment; filename="=?utf-8?B?QXR0YWPobWVudDF=?="
test#test.com
--_E9E8D52C-4722-488D-9FEC-713446D4A5C4_--
Now, a quick comparison reveals that we are closing in on the solution. The first important issue must be that the Content-Type (first part below MIME-version: 1.0) is becoming multipart/mixed when we need multipart/form-data.
Second issue is in the Content-Disposition which for both parts does not, at all, look like expected.
Please take a look at my pipeline component. What you see is the entire code from the Execute. Since formDataPart.Data only takes a stream I have a method that translates a string to a stream (MemoryStream).
// Variables
var outMsg = pContext.GetMessageFactory().CreateMessage();
outMsg.Context = PipelineUtil.CloneMessageContext(pInMsg.Context);
outMsg.Context.Promote("ContentType", "http://schemas.microsoft.com/BizTalk/2003/http-properties", "multipart/form-data;");
// Process xbrl file
var msgPart = pContext.GetMessageFactory().CreateMessagePart();
msgPart.PartProperties.Write("FileName", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "SomeID.xbrl");
msgPart.PartProperties.Write("ContentDescription", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "xml");
//msgPart.PartProperties.Write("ContentTransferEncoding", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "7bit");
msgPart.Data = pInMsg.BodyPart.Data;
msgPart.ContentType = "application/xml";
msgPart.Charset = "UTF-8";
outMsg.AddPart("data", msgPart, true);
// Form data (email)
var formDataPart = pContext.GetMessageFactory().CreateMessagePart();
formDataPart.ContentType = "text/plain";
formDataPart.Charset = "utf-8";
formDataPart.PartProperties.Write("ContentType", "http://schemas.microsoft.com/BizTalk/2003/http-properties", "form-data;");
formDataPart.Data = GenerateStreamFromString("test#test.com");
outMsg.AddPart("email", formDataPart, false);
return outMsg;
Does anybody know whow we can achieve this? I can also mention that we are using the MIME/SMIME component after this custom pipeline component.
I really think that the solution from Greg Forsythe would be the best solution since I would have total control over the message but I have no idea how to translate that into actual code.
Greg_suggestion
UPDATE: After so many trials and not coming anywhere we decided to ditch the above solution and go with Gregs solution instead. There is an issue there so I hope some C# guru can help us out. Please take a look at the new question if you are struggling with the form data request issue.
Second attempt here
Using Azure.AI.FormRecognizer 1.0.0-preview.2 and trying to upload a file and then run forms recognition on it. Code is
var fileName = Path.Combine(#"c:\temp\", sourceFile.FileName);
var fileUri = new Uri(fileName);
sourceFile.SaveAs(fileName);
var forms = await recogClient.StartRecognizeCustomFormsFromUri(modelId, fileUri).WaitForCompletionAsync();
The file URI becomes, for example, file:///c:/temp/DC002.pdf which I believe is a valid URI. However, when running StartRecognizeCustomFormsFromUri, I get the error:
Service request failed. Status: 400 (Bad Request) Content: {"error":{"code":"1003","message":"Parameter 'Source' is not a valid Uri."}} Headers: Transfer-Encoding: chunked x-envoy-upstream-service-time: REDACTED apim-request-id: REDACTED Strict-Transport-Security: REDACTED x-content-type-options: REDACTED Date: Wed, 27 May 2020 12:30:01 GMT Content-Type: application/json; charset=utf-8
FileUri needs to be a public accessible URL, you can not point to your local filesystem. If you would like to send a local file, you should send the file as a file stream.
The Uri is a great way to speed up the processing when your files are already on a blob storage, or any other public accessible cloud storage, saving time of not streaming the file to the Form Recognizer service.
Recognize forms from
File
Recognize forms from Uri
I really need your help!
I want to send a reply message with the Gmail REST API. Using the standard url:
https://www.googleapis.com/gmail/v1/users/userId/messages/send, everything works fine. By sending the raw data of the message and the thread id, the new message is attached to the same thread.
Now as I want our customers to be able to upload bigger attachments i use the upload url: https://www.googleapis.com/upload/gmail/v1/users/userId/messages/send?uploadType=media.
I send the request with Content-Type set to message/rfc822 as set in the documentation, but I can't find a way to send also the Thread id.
I tried adding it as a header (example: Thread-Id) to the MimeMessage, but it only works when i send the message to Outlook - there the message is sent to the same thread.
I have also populated the In-Reply-To header, the References header and the Subject is the same as in the original thread with additional 'Re:'. Gmail still creates a new thread for this message.
Update:
I have tried using the upload uri with parameter uploadType=multipart and here is my request:
Content-Type: multipart/related; boundary="=-EEoaSdATsa0it8EbymRVdg=="
--=-EEoaSdATsa0it8EbymRVdg==
Content-Type: application/json
Content-Transfer-Encoding: base64
eyJ0aHJlYWRJZCI6IjE1ZTdlZmJhYTc3MzViZjcifQ==
--=-EEoaSdATsa0it8EbymRVdg==
Content-Type: message/rfc822
From: deni.gencheva#gmail.com
Date: Fri, 27 Oct 2017 16:13:37 +0300
Subject: Re: RE:
Message-Id: <ESXEYD0PP2U4.RL2HHWMRGNXE#localhost.localdomain>
Sender: {sender email}
Reply-To: {sender email}
To: {one receiver email}
In-Reply-To:
=?us-ascii?q?
=3CHE1PR0402MB3323926A291565E98E66C1DCFC720=40HE1PR0402MB3323=2Eeur?=
=?us-ascii?q?prd04=2Eprod=2Eoutlook=2Ecom=3E?=
References:<HE1PR0402MB3323926A291565E98E66C1DCFC720#HE1PR0402MB3323.eurprd04.prod.outlook.com>
<CAJM2npmj3DWntngux8KovxPoNJ+KOS4tKm=LAo1YoG3vxWLQdg#mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-VCM8QFnp+o3VG1fmByraaw=="
--=-VCM8QFnp+o3VG1fmByraaw==
Content-Type: application/octet-stream; name="test document.rtf"
Content-Disposition: attachment; filename="test document.rtf"
Content-Transfer-Encoding: base64
Content-Id: <5e655580-14b6-0d1f-9e86-ad1c590691db>
{attachment base64 encoded content}
--=-VCM8QFnp+o3VG1fmByraaw==
Content-Type: multipart/alternative; boundary="=-pst5gHw2fNhIKAdrMRT2wg=="
--=-pst5gHw2fNhIKAdrMRT2wg==
Content-Type: text/plain; charset=utf-8
{message body as plain text}
--=-pst5gHw2fNhIKAdrMRT2wg==
Content-Type: text/html; charset=utf-8
{message body as html}
--=-pst5gHw2fNhIKAdrMRT2wg==--
--=-VCM8QFnp+o3VG1fmByraaw==--
--=-EEoaSdATsa0it8EbymRVdg==--
But API returns an error which states that multipart/related is not an allowed content type.
I am writing an app that decrypts encrypted emails for a client.
The encrypted data is in the form of an email attachment, which, when decrypted, looks like this :
Content-Type: multipart/mixed;
boundary="PGP_Universal_830ECF7A_AFB087B4_241DE401_9BE7FFD1"
--PGP_Universal_830ECF7A_AFB087B4_241DE401_9BE7FFD1
Content-Type: multipart/alternative;
boundary="PGP_Universal_904F5C3F_3A8C9E07_A3A24D11_F0FB260C"
--PGP_Universal_904F5C3F_3A8C9E07_A3A24D11_F0FB260C
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7BIT
[snip]
--PGP_Universal_904F5C3F_3A8C9E07_A3A24D11_F0FB260C
Content-Type: text/html;
charset=us-ascii
Content-Transfer-Encoding: QUOTED-PRINTABLE
[html content]
--PGP_Universal_904F5C3F_3A8C9E07_A3A24D11_F0FB260C--
--PGP_Universal_830ECF7A_AFB087B4_241DE401_9BE7FFD1
Content-Type: application/pdf;
name="BB_FW_60_Manual_Key.Enrollment_Final_v1.pdf"
Content-Transfer-Encoding: BASE64
Content-Disposition: attachment;
filename="BB_FW_60_Manual_Key.Enrollment_Final_v1.pdf"
[base64 data]
--PGP_Universal_830ECF7A_AFB087B4_241DE401_9BE7FFD1--
I am writing the app in Cascades. As I cannot find a native way of transforming this data into an email, to let the email client deal with the attached base64-encoded file and so on, I am hoping to find a C++ class (it can optionally depend upon Qt, obviously) that I can use on the BB10 which can parse these kind of multipart messages.
It is possible to get SPMIME to compile on BB10. It is LGPL so it might need to be compiled seperately and linked to if your app is not GPL.
I'm trying to upload a file using POST
here's my request :
POST /upload.php HTTP/1.1
Host: localhost
Content-Type: multipart/form-data; boundary=---------------------------552335022525
Content-Length: 192
-----------------------------552335022525
Content-Disposition: form-data; name="userfile"; filename="12.txt"
Content-Type: text/plain
blabla
-----------------------------552335022525--
Using HTTP live headers firefox plugin everything works
but when putting it a char *buffer and send it with winsocksapi I get 400 Bad Request error
You need a blank line between the headers and the payload.
Content-Length: 192
-----------------------------552335022525
This is part of the HTTP protocol. HTTP request headers end with the first empty line (CR-LF by itself.) What you are sending is resulting in the string
-----------------------------552335022525
being taken (along with the following two lines) as a request header which, of course, it isn't. The server can't make head or tail of that, so it responds with 400 Bad Request.
Also, sending the Content-length is not necessary with multipart/form-data, nor even a good idea, as the wrong value could create problems. The MIME multipart format is self describing.