Getting unexpected results when processing email via App Engine app - google-app-engine

I'm trying to set up the email receiver found here, to process incoming emails and send them out as POST data to a script on my server to be handled further from there. The issue I'm having is when I send one test email to foo#[myappname].appspotmail.com, the App Engine logs show that the email is continually "received" over and over again every couple of minutes, even though I only sent it once. Then after several minutes of this, when I go into settings and disable the app, I get at least one "Delivery to the following recipient failed permanently" message to the email account I was sending the emails from (makes sense, since the app is now disabled and not accepting any incoming mail).
What I'm having trouble understanding is why the application is behaving like it's getting multiple emails sent to it when it's only one. Do I need to modify the Python script to do something to delete or halt the email once it's been processed the first time? If so, does anyone have any suggestions as to how to do that? The Python script that I'm using is found here.

User voscausa answered my question-- the email requests kept retrying because the script was erroring. Thanks!

Related

Integromat custom webhook queue keeps filling with the same salesforce data

I have an Integromat scenario setup that starts with a custom webhook. The data to the webhook comes from Salesforce. Everything is working fine except the webhook queue seems to keep running the same data every 2 hours. For example, I do the thing in Salesforce that triggers the outbound message to the webhook, Integromat scenario triggers and does what I want.....Then two hours later it does it again....then two more hours it does it again. As if the server buffer is not clearing (wild guess, and I don't think Integromat would even allow access to clear this).
I had a very similar flow on Zapier, triggered by the exact same Salesforce event so I don't think it's Salesforce sending the data on a loop. I am trying to migrate the workflow from Zapier to Integromat for cost reasons but obviously can't do it if this bug isn't fixed.
Any ideas would be greatly appreciated. Thanks.
Thanks Diego and eyescream. I was able to see in Salesforce that the message was failing on the Salesforce side because an acknowledgment was not sent back. For anyone else who has this issue in Integromat, you need to add a second step Webhook Response and structure it like:
Status: 200
Body:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<notificationsResponse xmlns="http://soap.sforce.com/2005/09/outbound">
<Ack>true</Ack>
</notificationsResponse>
</soapenv:Body></soapenv:Envelope>
Then it worked and SF got the acknowledgment and stoped trying to send the message. I didn't need to add any custom headers.

Delay in receiving emails

Currently, we are using one Gmail API to receive MFA through Email. We are using this in automated tests to read to code.Until yesterday we are receiving emails in seconds and we are able to read the code. But starting from today there is some delay in receiving an Email and sometimes the email is not been received.
Is there any way to check these logs? Can anyone help with this?
Well, if you experienced some delay, based on this documentation there can be a delay of several minutes if the user exceeds their quota. You can also try to use the Users.messages: get to get the specific message and check if it has some delay in receiving.
I found here a related SO question about your problem. So just check it if it can help you. It uses IMAP for this issue.

Getting http 500 backend error when posting to Gmail API

I am using the Gmail API to put messages into a Google Apps email account. I use
the OAuth 2.0 authentication protocol with a service account. This is more or
less working fine. One of our customers has asked us to put messages
directly into a Google Vault. I don't see a Vault API, but I did find this
information related to the "insert" method (which is what we use to add
messages to a normal account):
parameter "deleted" (boolean): Mark the email as permanently deleted
(not TRASH) and only visible in Google Apps Vault to a Vault administrator.
Only used for Google Apps for Work accounts.
When I do this, some messages are accepted, but frequently I get http error
500 in response to the POST. The error text says "Backend Error". I thought
the pattern was that the first time the message was posted, it would work,
but the second time would generate the error. Therefore I was thinking it
was a duplicate check issue. However I now see some examples of messages
that fail immediately. The POST url looks like this:
https://www.googleapis.com/upload/gmail/v1/users/user#domain.com/messages?uploadType=multipart&internalDateSource=dateHeader&deleted=true&access_token=ABC...
As I mentioned, the same message to the same url (without deleted=true) will
always work. Any ideas what is causing the error?
Was just fighting this issue myself. Apparently the error has something to do if the message is compatible with the Google vault retention policies:
If I turn on a default policy of "Retain everything" then I've been able to get the messages to import correctly. HTH!
I'm using the import api method and the backendError seems to be related to filters/policies. For example we asked Google to reject messages with xls and macros and we get the error on mail with that kind of attachment

Mobile to Web Service Communication

I created a web service and a mobile application that communicate between each other. When everything is working, it works great. When the server doesn't respond, it starts to break down.
The mobile device sends a message to the server with a bunch of records. Getting the records on the server never seems to be a problem. It gets the records and then sends a response back to the mobile device that the update was received. The PROBLEM is that the mobile device doesn't always get the response, so it doesn't know it shouldn't send those records again for updating.
Next time it sends the records again and now I have duplicate records. How can I solve this?
Idea 1) Create a transaction number unique on the mobile device that I can compare against the server to see if the record was already uploaded. Then just don't write that record and attempt to send back the response that it was written.
Idea 2) Send the records to the server, but before writing them respond to the mobile device that they were received. This way the mobile device can tag them and then send another response to the server telling it to write them. At the point the mobile device almost doesn't care if it gets a response. Only thing, you don't know if the server ever got the message.
Looking for ideas on how to handle this that either confirm one of these ideas or has a completely different one.
I ended up creating logs that the device attempts to resolve when it gets back successful responses from the server.
I tag items as a batch of lines and send them up to the server. Once they are up there, I create a log about the success or failure of each line item in a batch of items and then save the log to the file system.
When the mobile device is unsuccessful in hearing back a response from the server, in rare cases, it asks the server about a batch number. If the server doesn't respond with a status of that batch, it assumes the server never received it and remarks those items for another upload attempt. If it hears back, it processes the success and failure line by line and then marks the items on the mobile device accordingly. If the mobile device doesn't ask about the log in the next upload, the server assumes the batch's lifecycle is complete and it no longer needs to maintain that log. It is then deleted.
The server doesn't delete a log until it has a successful request from the specific device no longer asking to hear about the log. So if I have log 1 on the server and the device doesn't ask in the next upload to hear back about that log, the server then removes that log assuming the device got the response it wanted or doesn't care about it anymore.

saving appengine mail from spam filters

One of my clients uses Trend Micro InterScan Messaging Security to protect their internal mail services.
Suddenly InterScan decided to filter out all messages coming from Google App Engine.
Unfortunately they haven't been able to whitelist the sender address as each e-mail gets a different one. For example, *3ckihSOVMMHlZHSL.JSMMHlZHSL.JS*#apphosting.bounces.google.com, with everything before the # being variable.
Update I'm including this screenshot of how Interscan sees the incoming e-mail. Notice that all senders are different:
If I look into the e-mail headers, the apphosting domain appears inside the Return-Path field:
Return-Path: <36kSiSwYIBh0883XL3E7.5EH883XL3E7.5E#apphosting.bounces.google.com>
The "From" field looks ok. It says what I set it to say, but the spam filter only looks at the Return-Path.
My client sysadmin doesn't want to whitelist the whole apphosting domain, as it wouldn't be only whitelisting my application.
How could I bypass this e-mail filters if I can't get an unique sender?
Thanks,
You can't change the return-path header of mail sent by App Engine. The way I see it, you have two options:
Whitelist everything from App Engine. Spam from App Engine is not a big problem, because it's expensive to send in the huge numbers spammers need, and we're constantly monitoring for spamming and shutting spammers down.
Whitelist based on the X-Google-Appengine-App-Id header, which will be set to the app ID of your app.
I am not well versed in spam filters, but it seems to me that if it can only whitelist based on one field, it is pretty lame. Unfortunately that does not help you. If this is an important client, and they absolutely refuse to budge, I see two possible paths forward:
Do some research into interscan to see if you can give the client some pointers (tactfully) on how to configure it to whitelist in such a way that your mail can get through, but only your mail.
Maintain a server outside app engine specifically for the purpose of sending emails. You can build a super simple web app that just sends out emails, and call it from within your app engine app.

Resources