I have been working on a Java app that connects to Yahoo mail to enable a user to search through their Yahoo email.
Recently, Yahoo suddenly (with only 5 weeks notice) discontinued Yahoo Mail API, which we were using and worked great. Then we re-engineered and switched to YQL. Unfortunately, for unknown reasons, this too has stopped working as of this week. The endpoint keeps returning an error. Even when YQL did work, it was occasional and sporadic. Even the Yahoo YQL console is returning errors. We have tried using JavaMAIL IMAP access to search for messages. We can connect to the IMAP server, but the JavaMAIL Search terms are not supported. I keep getting the error "SEARCH Server error - Please try again later". The same code works just fine for other IMAP services (like Aol mail).
So basically, with Yahoo Mail API gone, YQL not working, and IMAP not supporting searching, there is no programmatic way of searching Yahoo mail right now? Yahoo keeps telling us that the Yahoo API for IMAP access is the way ahead (see here https://developer.yahoo.com/mail/). But this is not live yet and there is no functioning documentation. Sending an email to imap-service#yahoo-inc.com was useless as no one responds to that anyway. They should learn a thing or two from Facebook on how to manage changes and maintain developer relations.
Does anyone have an alternative means to programatically search Yahoo Mail for emails with Java?
Thanks.
I managed to get IMAP access working with Yahoo through OAuth 2.0, but this code is in Python:
Follow the directions here to refresh the user's token:
https://developer.yahoo.com/oauth2/guide/index.html#refresh-token-label
(you can use the refresh_token that you received using OAuth 1.0)
Replace the email_address and access_token in the following code:
import logging
import imaplib
import datetime
import quopri
import hashlib
endpoint = 'imap.mail.yahoo.com'
email_address = 'REPLACE_THIS#yahoo.com'
access_token = 'REPLACE_THIS'
auth_string = 'user=%s\1auth=Bearer %s\1\1' % (email_address, access_token)
imap_conn = imaplib.IMAP4_SSL(endpoint) # the IMAP server we're connecting to
imap_conn.debug = 3 # for logging purposes
imap_conn.authenticate('XOAUTH2', lambda x: auth_string)
folders = imap_conn.list()
print folders
imap_conn.select('Inbox', readonly=True)
result, data = imap_conn.uid('search', None, '(FROM "Amazon.com")')
messages = data[0].split()
print 'messages:' + str(messages)
uids_to_fetch = ','.join(messages)
result, data = imap_conn.uid('fetch', uids_to_fetch, 'RFC822')
for counter, message in enumerate(data[::2]):# every other item in the list is not a message, but ")" so we skip it
# yield raw mail body, after decoding the quoted-printable encoding
print quopri.decodestring(message[1])
Related
I am working on a web application with ReactJs frontend and Django backend. In this app, I will need to send calender notifications to the users and overall need a user authentication feature for which I planned to use google oauth. From react, I am able to log in the user and get the access tokens but since they expire in an hour, I planned to get the authorization code and use it to get the refresh/access tokens from the backend whenever a user logs in/needs to send API request. The issue is that I am not able to find any good resource on how to get refresh tokens from the backend given I have the authorization code. Most of the HTTP based methods I have found are very outdated and I have searched some of the google documentation but have not found anything worthwhile. Since I could not find any package that would handle this, I have been trying to send POST request to the URL mentioned here (https://developers.google.com/identity/protocols/oauth2/web-server#python_1) but only get a 400 response. Below are 2 methods I have tried.
to_send={'code':user_data['code'], 'client_id': cl_id , 'client_secret': cl_secret,
'redirect_uri':'http://localhost:3000/', 'grant_type':'authorization_code'}
test=requests.post('https://oauth2.googleapis.com/token', data=to_send)
print(test)
credentials = service_account.Credentials.from_service_account_file('path/key.json')
scoped_credentials = credentials.with_scopes(['https://www.googleapis.com/auth/drive.metadata.readonly'])
authed_session = AuthorizedSession(scoped_credentials)
response = authed_session.request('POST', 'https://oauth2.googleapis.com/token', data = to_send)
print(response)
Ive also tried other things like creating a flow object.
flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file( JSON_PATH, scopes=API_SCOPE)
flow.redirect_uri = REDIRECT_URL
flow.fetch_token(code=user_data['code'])
credentials = flow.credentials
where user_data['code'] is the authorization token and I get the error
oauthlib.oauth2.rfc6749.errors.InvalidGrantError: (invalid_grant) Bad Request
Consider this:
I am fetching rawbody messages by this call:
service.users().messages().get(userId='me', format='raw', id=msgid)
Then I am pushing rawbody messages by this Call:
service.users().messages().insert(userId='me', body=message)
Now when the mails contain attachments bigger 5MB, I encounter 413 "Request Entity Too Large.", and I can't push mails.
GMail API messages.insert Documentation advices to use
POST https://www.googleapis.com/upload/gmail/v1/users/userId/messages
instead of
POST https://www.googleapis.com/gmail/v1/users/userId/messages.
But Google API Client doesn't seem to have any documentation about how to call the above Url, and it keeps getting back to latter url.
How Can send post requests to first url(with /upload) with Google Api Client rather than its default?
How to use /upload url and set uploadType=multipart with Google APi Client?
Yeah, this was totally unclear from the documentation for the Google Python API client, but I found the solution in this other answer. It turns out that you use the same method (users().messages().insert()) but you pass media_body instead of body['raw']. Something like this should work:
from io import BytesIO
from base64 import urlsafe_b64decode
import googleapiclient.http
b = BytesIO()
message_bytes = urlsafe_b64decode(fetched_message['raw'])
b.write(message_bytes)
media_body = googleapiclient.http.MediaIoBaseUpload(b, mimetype='message/rfc822')
service.users().messages().insert(userId='me', media_body=media_body).execute()
I haven't tried using uploadType=multipart, but maybe you can figure it out from this documentation page and looking at the contents of the googleapiclient.http module.
I am trying to send an email from google app engine using the python 2.7 library but I keep getting Unauthorized sender in the logs. I have tried my gmail account I created the application with as the sender, I registered another gmail address as a developer and tried that but still get Unauthorized sender. I am not sure if it matters but I do have a domain name registered to this application.
Here is the code I am trying:
message = mail.EmailMessage()
message.sender = "ron.....#gmail.com"
message.subject = "Inquiry"
message.to = "ron.....#gmail.com"
message.body = "Please work"
message.send()
I have looked at other articles to no avail.
Google Appengine sending emails: [Error] unauthorized sender
InvalidSenderError: Unauthorized sender (Google App Engine)
from google.appengine.api import mail
mail.send_mail(sender="stackoverflow.com Hossam <Hossam#stackoverflow.com>",
to="rsnyder <rsnyder#stackoverflow.com>",
subject="How to send an e-mail using google app engine",
body="""
Dear rsnyder:
This example shows how to send an e-mail using google app engine
Please let me know if this is what you want.
Best regards,
""")
EDIT:
Note that sender must be an administrator of the application, so in case that you are not and administrator, follow these steps from the post google app engine: how to add adminstrator account
I found the issue. It was the wrong version of the code. I switched to a version 2 and didn't realize I had to activate it.
I have an appengine app that needs to access a single, hard-coded spreadsheet on Google Drive.
Up until now I have been achieving this as follows:
SpreadsheetService service = new SpreadsheetService("myapp");
service.setUserCredentials("myusername#gmail.com", "myhardcodedpassword");
When I tried this today with a new user, I got InvalidCredentialsException even though the username and password were definitely correct. I got an email in my inbox saying suspicions sign-ins had been prevented, and there seems to be no way to enable them again.
I am also aware that hardcoding passwords in source is bad practice.
However, I have read very widely online for how to enable OAuth/OAuth2 for this, and have ended up wasting hours and hours piecing fragments of information from blogs, stackoverflow answers etc, to no avail.
Ideally the solution would involve an initial process to generate a long-lived access token, which could then be hard-coded in to the app.
I want a definitive list of steps for how to achieve this?
EDIT: As Google have redesigned the API Console, the details of the steps below have changed - see comments
OK here goes, step by step
Go to Google Cloud Console and register your project (aka application)
You need to note the Client ID, and Client Secret
Go to the OAuth Playground, click the gear icon and choose and enter your own credentials
You will be reminded that you need to go back to the Cloud COnsole and add the Oauth Playground as a valid callback url. So do that.
Do Step 1, choosing the spreadsheet scope and click authorize
Choose your Google account if prompted and grant auth when prompted
Do Step 2, Click 'Exchange auth code for tokens'
You will see an input box containing a refresh token
The refresh token is the equivalent of your long lived username/password, so this is what you'll hard code (or store someplace secure your app can retrieve it).
When you need to access Google Spreadsheets, you will call
POST https://accounts.google.com/o/oauth2/token
content-type: application/x-www-form-urlencoded
client_secret=************&grant_type=refresh_token&refresh_token=1%2xxxxxxxxxx&client_id=999999999999.apps.googleusercontent.com
which will return you an access token
{
"access_token": "ya29.yyyyyyyyyyyyyyyyyy",
"token_type": "Bearer",
"expires_in": 3600
}
Put the access token into an http header for whenever you access the spreadsheet API
Authorization: Bearer ya29.yyyyyyyyyyyyyyyyy
And you're done
Pinoyyid has indeed provided wonderful help. I wanted to follow up with Python code that will allow access to a Personal (web-accessible) Google Drive.
This runs fine within the Google App Engine (as part of a web app) and also standalone on the desktop (assuming the Google App Engine SDK is installed on your machine [available from: https://developers.google.com/appengine/downloads]).
In my case I added https://www.googleapis.com/auth/drive scope during Pinyyid's process because I wanted access to all my Google Drive files.
After following Pinoyyid's instructions to get your refresh token etc., this little Python script will get a listing of all the files on your Google drive:
import httplib2
import datetime
from oauth2client.client import OAuth2Credentials
from apiclient.discovery import build
API_KEY = 'AIz...' # from "Key for Server Applications" from "Public API Access" section of Google Developers Console
access_token = "ya29..." # from Piinoyyid's instructions
refresh_token = "1/V..." # from Piinoyyid's instructions
client_id = '654....apps.googleusercontent.com' # from "Client ID for web application" from "OAuth" section of Google Developers Console
client_secret = '6Cl...' # from "Client ID for web application" from "OAuth" section of Google Developers Console
token_expiry = datetime.datetime.utcnow() - datetime.timedelta(days=1)
token_uri = 'https://accounts.google.com/o/oauth2/token'
user_agent = 'python urllib (I reckon)'
def main():
service = createDrive()
dirlist = service.files().list(maxResults=30)
print 'dirlist', dirlist
result = dirlist.execute()
print 'result', result
def createDrive():
credentials = OAuth2Credentials(access_token, client_id, client_secret, refresh_token, token_expiry, token_uri, user_agent)
http = httplib2.Http()
http = credentials.authorize(http)
return build('drive', 'v2', http=http, developerKey=API_KEY)
main()
I'm grateful to all who have provided the steps along the way to solving this.
I started using google API recently . I am using simpleauth https://github.com/crhym3/simpleauth for authentication to google app engine . Now I am using google blogger API for publishing my blog and fetching data .
This API requires access_token value to use the API https://developers.google.com/blogger/docs/3.0/using#RetrievingPostsForABlog for Authorization . I cant find a way to get value of access token .
Is there a way to get the value of acces token or am I doing something wrong ?
You need to register your webapp with Google to get a client ID and client secret. Then, you can configure your OAuth2 library with these details to allow you to send fully authenticated requests from your webapp to Blogger.
For the specific scenario you listed, retrieving a blog post, I think you can follow step 1 of this page and then follow these steps. You should be able to copy+paste the key from there into the query params of the GET request.
To issue fully authenticated requests, for publishing new posts, for example, you'll have to get your OAuth2 library with the client ID and client secret and have it issue the requests for you.