For some reason I cannot figure out how to send emails using a gmail account, Appengine and Golang.
Here's what I've done:
I went to Google Cloud Platform > Appengine > Settings > Select Project and I added the gmail account on Email API authorized senders.
I' tried to make this work using the code from (https://golang.org/pkg/net/smtp/#pkg-examples) (func SendMail)
package main
import (
"log"
"net/smtp"
)
func main() {
// Set up authentication information.
auth := smtp.PlainAuth("", "user#gmail.com", "password", "smtp.gmail.com")
// Connect to the server, authenticate, set the sender and recipient,
// and send the email all in one step.
to := []string{"recipient#example.net"}
msg := []byte("To: recipient#example.net\r\n" +
"Subject: discount Gophers!\r\n" +
"\r\n" +
"This is the email body.\r\n")
err := smtp.SendMail(smtp.gmail.com:587", auth, "sender#example.org", to, msg)
if err != nil {
log.Fatal(err)
}
}
On the front-end (JavaScript) I get an unsuccessful response after trying to run this code.
I've been running this on the appengine staging server
I tried different smtp server, ports, users and it still not work (support.google.com/a/answer/176600?hl=en)
I found a few examples on github and some other blog and I tried them but it didn't make different.
github.com/golang/go/wiki/SendingMail
nathanleclaire.com/blog/2013/12/17/sending-email-from-gmail-using-golang/
On all the examples it everything looks straight forward but there's something that I'm definitely missing or misunderstanding.
There're some limitations with establishing raw tcp connections on GAE:
https://cloud.google.com/appengine/docs/go/sockets/#limitations_and_restrictions
I would recommend to use GAE mail api (which is slightly diffrent from standart smtp package) to send emails:
https://cloud.google.com/appengine/docs/go/mail/sending-receiving-with-mail-api
Related
I wanted to authenticate myself (Google Account) using a golang client against protected apps on Google App Engine where login: required or login: admin is specified in app.yaml.
First I wrote a simple OAuth2 offline access client but it didn't work at all - the server just redirects clients to Google Account's sign in page. I've tried with various Google API scopes and currently no luck.
package main
import (
"context"
"fmt"
"io"
"log"
"os"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
)
const (
AppURL = "https://login-requried-app.appspot.com"
AuthClientID = "....."
AuthClientSecret = "....."
AuthRedirectURL = "urn:ietf:wg:oauth:2.0:oob"
AuthScope = "https://www.googleapis.com/auth/cloud-platform"
)
func main() {
ctx := context.Background()
conf := &oauth2.Config{
ClientID: AuthClientID,
ClientSecret: AuthClientSecret,
Endpoint: google.Endpoint,
RedirectURL: AuthRedirectURL,
Scopes: []string{AuthScope},
}
url := conf.AuthCodeURL("state", oauth2.AccessTypeOffline)
fmt.Printf("Visit the URL for the auth dialog: %v\n", url)
fmt.Printf("Enter authentication code: ")
var code string
if _, err := fmt.Scan(&code); err != nil {
log.Fatal(err)
}
tok, err := conf.Exchange(ctx, code)
if err != nil {
log.Fatal(err)
}
client := conf.Client(ctx, tok)
res, err := client.Get(AppURL)
if err != nil {
log.Fatal(err)
}
defer res.Body.Close()
log.Println(res.Status)
io.Copy(os.Stdout, res.Body)
}
I'm looking for the details of GAE's user authentication modes used in such apps to write a non web browser client. I feel it's something different than standard OAuth2 authentication/authorization after reading App Engine Users API docs and code that is receiving user information via HTTP headers like X-AppEngine-User-Email.
Users Go API Overview
google.golang.org/appengine/user package document
user.Current() implementation (it's for flex env but almost the same in standard env)
UPDATE: After some research it looks like the GAE frontend uses SACSID cookie for tracking authenticated sessions, which isn't related to the OAuth2 standard. Indeed as stated in Users Go API document:
Note that using OAuth to identify your users is completely orthogonal to the standard user authentication modes. For example, pages marked with login: required or login: admin will refuse to load if the user is only authenticated via OAuth.
Is there any supported way for a CLI application to acquire SACSID properly authorized by user's consent?
Related questions:
Access an App Engine app from command line using OAuth2?
On Google App Engine, can I relate a Google OAuth 2 Token and a SACSID token I got using Android's AccountManager?
Given the situation you explain here, I suggest using a remote API. This way you can access App Engine services from your Go app.
First you have to configure your app.yaml file by adding the following:
- url: /_ah/remote_api
script: _go_app
You also have to add the following import to your .go source file:
import _ "google.golang.org/appengine/remote_api"
When this is done, deploy your updated app to App Engine:
gcloud app deploy app.yaml
The website I included here includes an example on how to use the remote API. You can try it and adapt your code if this works for you.
I'm trying to use Firestore in my AppEngine (standard environment) app written in Go. I've been following the "Getting Started with Cloud Firestore" guide and have been using the firestore package documentation to implement a simple example that works fine when running it on my local dev server.
However when I deploy the app and try the deployed version the call to DocumentRef.Set() fails with the error
rpc error: code = Unavailable desc = all SubConns are in TransientFailure
This is my code that reproduces the issue:
func init() {
http.HandleFunc("/test", testHandler)
}
type testData struct {
TestData string `firestore:"myKey,omitempty"`
}
func testHandler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
var firestoreClient *firestore.Client
var firebaseApp *firebase.App
var err error
conf := &firebase.Config{ProjectID: "my-project"}
firebaseApp, err = firebase.NewApp(ctx, conf)
if err != nil {
fmt.Fprintf(w, "Failed to create a new firestore app: %v", err)
return
}
firestoreClient, err = firebaseApp.Firestore(ctx)
if err != nil {
fmt.Fprintf(w, "Failed to create a new firestore client: %v", err)
return
}
data := testData{"my value"}
_, err = firestoreClient.Collection("testCollection").Doc("testDoc").Set(ctx, data)
if err != nil {
fmt.Fprintf(w, "Failed to create a firestore document: %v", err)
return
}
firestoreClient.Close()
fmt.Fprint(w, "Data stored in Firestore successfully")
}
As mentioned before, on the dev server this works fine. So there the returned page contains the text Data stored in Firestore successfully.
When running the deployed code I get Failed to create a firestore document: rpc error: code = Unavailable desc = all SubConns are in TransientFailure instead. Why do I get this error and how can I avoid it?
I've raised an issue about this in the Firestore client library issue tracker and it seems like the situation is a bit complex.
When using App Engine the Firestore client library's network connections goes trough the App Engine socket library. However sockets is only available for paid App Engine apps:
Sockets are only available for paid apps, and traffic from sockets is billed as outgoing bandwidth. Sockets are also limited by daily and per minute (burst) quotas.
So this is the reason why the Firestore client library fails. For small scale projects it's possible to enable billing of your App Engine app and still stay within the free range. If billing is enabled it should work when the app is deployed as well.
However if you are living within the European Union you are not allowed to have a paid App Engine app for non commercial purposes due to Google policies:
If you are located in the European Union and the sole purpose for which you want to use Google Cloud Platform services has no potential economic benefit you should not use the service. If you have already started using Google Cloud Platform, you should discontinue using the service. See Create, modify, or close your billing account to learn how to disable billing on your projects.
So if you are in Europe or for some other reason are unable to use have a paid App Engine app you will not be able to use the Firestore client library.
One alternative in this case is to use the Firestore REST API instead and manually make HTTP requests to Firestore. It's a bit more work, but for smaller scale projects it works.
On AppEngine you need to create a client that uses Http client provided by urlfetch service.
The firestore.NewClient() function accepts ClientOptions parameters that you can create using WithHTTPCLient() function.
Here is an article on issuing HTTP requests from AppEngine Go.
That should help.
I have been working on a Java app that connects to Yahoo mail to enable a user to search through their Yahoo email.
Recently, Yahoo suddenly (with only 5 weeks notice) discontinued Yahoo Mail API, which we were using and worked great. Then we re-engineered and switched to YQL. Unfortunately, for unknown reasons, this too has stopped working as of this week. The endpoint keeps returning an error. Even when YQL did work, it was occasional and sporadic. Even the Yahoo YQL console is returning errors. We have tried using JavaMAIL IMAP access to search for messages. We can connect to the IMAP server, but the JavaMAIL Search terms are not supported. I keep getting the error "SEARCH Server error - Please try again later". The same code works just fine for other IMAP services (like Aol mail).
So basically, with Yahoo Mail API gone, YQL not working, and IMAP not supporting searching, there is no programmatic way of searching Yahoo mail right now? Yahoo keeps telling us that the Yahoo API for IMAP access is the way ahead (see here https://developer.yahoo.com/mail/). But this is not live yet and there is no functioning documentation. Sending an email to imap-service#yahoo-inc.com was useless as no one responds to that anyway. They should learn a thing or two from Facebook on how to manage changes and maintain developer relations.
Does anyone have an alternative means to programatically search Yahoo Mail for emails with Java?
Thanks.
I managed to get IMAP access working with Yahoo through OAuth 2.0, but this code is in Python:
Follow the directions here to refresh the user's token:
https://developer.yahoo.com/oauth2/guide/index.html#refresh-token-label
(you can use the refresh_token that you received using OAuth 1.0)
Replace the email_address and access_token in the following code:
import logging
import imaplib
import datetime
import quopri
import hashlib
endpoint = 'imap.mail.yahoo.com'
email_address = 'REPLACE_THIS#yahoo.com'
access_token = 'REPLACE_THIS'
auth_string = 'user=%s\1auth=Bearer %s\1\1' % (email_address, access_token)
imap_conn = imaplib.IMAP4_SSL(endpoint) # the IMAP server we're connecting to
imap_conn.debug = 3 # for logging purposes
imap_conn.authenticate('XOAUTH2', lambda x: auth_string)
folders = imap_conn.list()
print folders
imap_conn.select('Inbox', readonly=True)
result, data = imap_conn.uid('search', None, '(FROM "Amazon.com")')
messages = data[0].split()
print 'messages:' + str(messages)
uids_to_fetch = ','.join(messages)
result, data = imap_conn.uid('fetch', uids_to_fetch, 'RFC822')
for counter, message in enumerate(data[::2]):# every other item in the list is not a message, but ")" so we skip it
# yield raw mail body, after decoding the quoted-printable encoding
print quopri.decodestring(message[1])
I've been trying to implement OAuth2 for Go with App engine server to server request.
Here is the code (following the example):
oauthConf, err := google.NewServiceAccountJSONConfig(
"./key.json",
"https://www.googleapis.com/auth/adsense.readonly",
)
if err != nil {
log.Fatal(err)
}
client := http.Client{Transport: oauthConf.NewTransport()}
resp, err := client.Get(urlStr)
...
And I get the error message:
http.DefaultTransport and http.DefaultClient are not available in App Engine.
I'm pretty sure the json.key file is valid as for the other stuff
Googling lead me to understand that it is preferred to use urlfetch, but I can't figure out how to make the setup with the oauth2 config.
The NewServiceAccountJSONConfig returns a oauth2.JWTConfig
https://github.com/golang/oauth2/blob/master/google/google.go#L69-L87
Its .NewTransport() you are using:
https://github.com/golang/oauth2/blob/master/jwt.go#L86-L88
Defaults to the http.DefaultTransport, that is not supported on appengine:
https://github.com/golang/oauth2/blob/master/jwt.go#L163-L168
I would go for using the AppEnigneConfig instead (if possible). See http://godoc.org/github.com/golang/oauth2/google
c := appengine.NewContext(nil)
config := google.NewAppEngineConfig(c, "https://www.googleapis.com/auth/bigquery")
// The following client will be authorized by the App Engine
// app's service account for the provided scopes.
client := http.Client{Transport: config.NewTransport()}
client.Get("...")
Otherwise if you need to use ServiceAccountJSONConfig, you probably can, but you'll need to use urlfetch Client and Transport. Have a look how AppEngineConfig is set up: https://github.com/golang/oauth2/blob/master/google/appengine.go
We have our mail setup with Google Apps. We want to be able to run some regular expressions on incoming mail and process this information.
Is this possible today with Google App Engine? Does Google provide some kind of infrastructure that can do this?
from google documentation here:
Receiving Mail
Your app can receive email at addresses of the following form:
string#appid.appspotmail.com
Note that even if your app is deployed on a custom domain, your app can't receive email sent to addresses on that domain.
Email messages are sent to your app as HTTP requests. These requests are generated by App Engine and posted to your app. In your app's configuration, you specify handlers that will be called to handle these HTTP requests. In your handlers, you receive the MIME data for email messages, which you then parse into its individual fields.
Email messages are sent to your app as HTTP POST requests using the following URL:
/_ah/mail/address
where address is a full email address, including domain name.
The ability to receive mail in your app is disabled by default. To enable your app to receive mail, you must specify that you want this service enabled in your app.yaml file by including this:
inbound_services:
- mail
The Python SDK defines InboundMailHandler, a webapp class for handling incoming email. To use InboundMailHandler, you subclass it and override the receive() method. The receive() method is called with an argument of class InboundEmailMessage, another class defined by the Python SDK.
InboundMailHandler is in the google.appengine.ext.webapp.mail_handlers package. You can create an instance of InboundEmailMessage like this:
import logging, email
from google.appengine.ext import webapp
from google.appengine.ext.webapp.mail_handlers import InboundMailHandler
from google.appengine.ext.webapp.util import run_wsgi_app
class LogSenderHandler(InboundMailHandler):
def receive(self, mail_message):
logging.info("Received a message from: " + mail_message.sender)
The InboundEmailMessage object includes attributes to access other message fields:
subject contains the message subject.
sender is the sender's email address.
to is a list of the message's primary recipients.
cc contains a list of the cc recipients.
date returns the message date.
attachments is a list of file attachments, possibly empty. Each value in the list is a tuple of two elements: the filename and the file contents.
original is the complete message, including data not exposed by the other fields such as email headers, as a Python email.message.Message.
handle_incoming_email.py
app.yaml
Update: It's now supported.
Processing incoming email is not yet supported. It is however on their roadmap: http://code.google.com/appengine/docs/roadmap.html
Google don't currently support handling email in App Engine, though it is on the roadmap. In the meantime, services like smtp2web will handle it for you (disclaimer: I wrote smtp2web).
You could setup an email account and have an external server (one you create and host outside of AE) access the gmail account via IMAP. Your "mail server" then reads the messages and accesses the /email API of your app on AE.
Python has an email module, so you could post the entire message there, or if that doesn't work (due to whatever restrictions), you could preprocess it on your mail server and post the simplified version to your app.
The downside is that you'll have to resort to polling for information, but that should be ok since email is accepted to have somewhat of a delay.
As another option to the answer from Gabriel I would recommend to use go environment of App Engine on Sending and Receiving Mail with the Mail API.
From The Documentation:
Receiving Mail
Your app can receive email at addresses of the following form:
anything#appid.appspotmail.com
Compare to a configuration on handling incoming mail in python as discussed here, enabling incoming mail in your app's app.yaml file is fairly simple:
inbound_services:
- mail
Name your app file as mail.go, then register a handler to the /_ah/mail/ path and read the email's data from the *http.Requestlike using net/mail like this:
func incomingMail(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
defer r.Body.Close()
var b bytes.Buffer
if _, err := b.ReadFrom(r.Body); err != nil {
log.Errorf(ctx, "Error reading body: %v", err)
return
}
log.Infof(ctx, "Received mail: %v", b)
}
Sending Mail
Follow this guideline to register your sender emails as authorized senders
Use the mail.Message type to set the sender, recipient, subject, and body of the message.
Send the email with the mail.Send function.
func confirm(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
addr := r.FormValue("email")
url := createConfirmationURL(r)
msg := &mail.Message{
Sender: "Example.com Support <support#example.com>",
To: []string{addr},
Subject: "Confirm your registration",
Body: fmt.Sprintf(confirmMessage, url),
}
if err := mail.Send(ctx, msg); err != nil {
log.Errorf(ctx, "Couldn't send email: %v", err)
}
}
Deploy
The complete sample code of both Receiving and Sending is available here on GitHub:
GoogleCloudPlatform/golang-samples/docs/appengine/mail/mail.go
To clone the sample code, go to your Console. Click the button to open Cloud Shell:
then similar with this quickstart enter the steps below:
$ SOURCEDIR=https://github.com/GoogleCloudPlatform/golang-samples.git
$ TUTORIALDIR=~/src/your-application-id/go_gae_samples
$ git clone $SOURCEDIR $TUTORIALDIR
$ cd $TUTORIALDIR
$ git checkout master
$ cat docs/appengine/mail/app.yaml
$ cat docs/appengine/mail/mail.go
$ goapp serve docs/appengine/mail/app.yaml
From here you can access the app on port 8080 by using Web preview.
To terminate press Ctrl+C in the Cloud Shell.
Finally you may deploy your app
goapp deploy -application your-application-id -version 0
Click the URL to visit it
http://your-application-id.appspot.com/
Then send an email to anything#your-application-id.appspotmail.com see if it works.