I have an App Engine site with a URL which is marked login:required in app.yaml. I need to access this URL from a command-line script. How do I do that now that OAuth2 has replaced ClientLogin?
Previous questions () all end up using ClientLogin, which has been removed:
How do you access an authenticated Google App Engine service from a (non-web) python client?
How to make an authenticated request from a script to appengine?
How do I use OAuth2 in a command line tool to access an application hosted on Google Appengine?
OAuth2 is the replacement, but when I try to use that, my script continually gets redirected to the login page.
Can anyone see what I'm doing wrong?
from google.appengine.tools import appengine_rpc_httplib2
authParams = appengine_rpc_httplib2.HttpRpcServerOAuth2.OAuth2Parameters(
access_token=None,
# id & secret come from http://console.developers.google.com
client_id="FIXME",
client_secret="FIXME",
# I'm not sure this is the right scope
scope="https://www.googleapis.com/auth/appengine.admin",
refresh_token=None,
credential_file=None
)
rpcServer = appengine_rpc_httplib2.HttpRpcServerOAuth2('example.appspot.com',
authParams,
None,
'ignored',
save_cookies=False,
auth_tries=3)
# Makes the actual GET request
result = rpcServer.Send('/some_url/', payload=None)
print result
(I'm using python but I'd be open to any way to do this from the command-line, even curl.)
Related
I got stuck on trying to access Azure blob storage from React app (thus from browser) with credentials.
And firstly I want to admit, that I am newbie in Azure, so maybe I misunderstood some basic concepts...
My current situation and goal:
I am developing React app (lets say MyReactApp). This app uses some files from Azure blob storage (lets say MyBlobStorage) -> it reads, creates and deletes blobs.
I started to develop it on my local and for this dev purpose I was connecting to MyBlobStorage with SAS - this worked perfectly.
MyReactApp is browser only, so it does not have any backend.
After finishing local development, I deployed it as Azure web app with SAS. What have I done:
created App service (lets say MyAppService)
register app in Azure Active Directory and use it as Identity Provider in MyAppService
After this the app works from Azure url perfectly too.
But my app on Azure should fulfill 2 conditions:
User has to log in with AAD account before access to MyReactApp
App itself must get rid of SAS (because it is not secure as it can be obtained from browser) and use some Azure credentials to connect to Azure blob storage
First condition - user log in:
I enabled "easy" logging in MyAppService -> Authentication and chose users, who can have access.
in Authentication section of app in AAD I set up Web type Redirect Uri as /.auth/login/aad/callback
Still everything works great - the user, who is assigned to the app, can log in and work with the app - so far so good, but now the problem comes
Second condition - I wanted to get rid of the SAS to access MyBlobStorage and use DefaultAzureCredentials:
I turned on managed identity for MyAppService and add it as Storage Blob Data Contributor for MyBlobStorage
I obtained user_impersonation and User.Read Api permissions for my app
I removed SAS and tried to add DefaultAzureCredentials to my code -> but it seems, that they can't be used in browser and only option is InteractiveBrowserCredentails
so I tried to use InteractiveBrowserCredentails like this:
this.interactiveBrowserCredential = new InteractiveBrowserCredential({
tenantId: "<appTenant>", // got from app registration on AAD
clientId: "<appClient>", // got from app registration on AAD
redirectUri: <MyAppServiceURi>/.auth/login/aad/callback // the same as in Azure AAD app,
});
this.blobSC = new BlobServiceClient(Constants.STORAGE_PATH, this.interactiveBrowserCredential);
My current result:
This shows login popup after getting to the page and after another signing in it results in error:
AADSTS9002326: Cross-origin token redemption is permitted only for the
'Single-Page Application' client-type.
I googled it of course and according to some answers I tried to change the Web type of redirect URI to SPA.
I tried it, but some other error showed up:
AADSTS9002325: Proof Key for Code Exchange is required for
cross-origin authorization code redemption.
Surprisingly this should be solved by changing from SPA to Web type...:) So I am trapped...
My expected result
In ideal world, I want to connect MyReactApp to MyBlobStorage without popup and with some secret credentials, just to say something like this.blobSC = new BlobServiceClient(Constants.STORAGE_PATH, credentials);
and be able to work with blobs.
Is it possible to access blob storage from browser without errors of course and ideally without popup, which needs another log in for user?
My complementary questions
Can I use somehow the logging info from the user (from his "easy" AAD logging)? I can get his info with GET call to /.auth/me, maybe it can be utilized, so I can use his info to access the blobs?
The solution should be working on localhost too (I tried to add http://localhost:3000/ to redirect uri, but without success), can it be done?
Thank you all, who read the whole story!
Below are my requirements.
Develop a flask app.
Use collections in the firebase in the app.
Deploy this app on Google App Engine using a standard service account
What I have done.
Created a service account
Downloaded the corresponding credentials json; I am calling it as key.json
written a main.py
cred = credentials.Certificate('key.json')
default_app = initialize_app(cred)
db = firestore.client()
user_ref = db.collection_group('Users')
#app.route('/', methods=['GET'])
def home():
return "<h1>Welcome to my first app</h1>"
#app.route('/users', methods=['GET'])
def getUsers():
try:
result = [user.to_dict() for user in user_ref .stream()]
return jsonify(result), 200
except Exception as e:
result = { "message:"failed"}
return jsonify(result), 500
I have tested this locally and also on deployed on Google App Engine.
In both the cases, key.json was in the same directory as the code.
I have verified that if this key.json is modified to store wrong data, then /users endpoint won't work and gives me a 500 error.
So far so good. I want to know if this is even the right approach.
I want the key.json authentication to applied even for the root / endpoint.
i.e., if the user supplies a valid key.json, only then the Welcome to my first app should be displayed.
Else, Unauthorized user message needs to be displayed.
As mentioned by #Gaefan and #DishantMakwana, as well as in this documentation:
An API key only identifies the application and doesn't require user authentication. It is sufficient for accessing public data.
So in order to authenticate/authorize your users you should reconsider your strategy. I would recommend you to follow the instructions in the Authenticating as an end user Documentation.
I have found that we can use Google Cloud Endpoints for API management. Works as a charm.
I am using the app.yaml's login: admin in handlers to restrict access to my app only to selected Google accounts (which I can edit in IAM). I'm using the python27 standard environment on GAE.
I would like to use the JSON API my app exposes from another server app (not hosted on GAE). Using a service account looks like a straightforward solution, but I am unable to get the scopes or the request itself right, so the endpoint would see an authenticated Google user.
The service-user currently has Project/Viewer role in the IAM. I tried a few more like AppEngine/Viewer, AppEngine/Admin. I also tried some more scopes.
My test code:
"""Try do do an API request to a deployed app
with the current service account.
https://google-auth.readthedocs.io/en/latest/user-guide.html
"""
import sys
from google.auth.transport.requests import AuthorizedSession
from google.oauth2 import service_account
def main():
if len(sys.argv) < 2:
sys.exit("use: %s url" % sys.argv[0])
credentials = service_account.Credentials.from_service_account_file(
'service-user.json')
scoped_credentials = credentials.with_scopes(
['https://www.googleapis.com/auth/cloud-platform.read-only'])
authed_http = AuthorizedSession(scoped_credentials)
response = authed_http.request('GET', sys.argv[1])
print response.status_code, response.reason
print response.text.encode('utf-8')
if __name__ == '__main__':
main()
There is no error, the request behaves like unauthenticated. I checked the headers on the server, and while requesting from the browser there are several session cookies, the AuthorizedSession request contains single Authorization: Bearer .. header.
Normally the roles you would need is App Engine Admin; it's designed for this purpose. It should also work with the viewer/editor/owner primitive roles. That being said, to make sure it's not a "role" issue, simply give it the project owner role and also the explicit App Engine Admin role and try again. This will eliminate any role-based issue.
Let me know if that works for you.
I have an appengine app that needs to access a single, hard-coded spreadsheet on Google Drive.
Up until now I have been achieving this as follows:
SpreadsheetService service = new SpreadsheetService("myapp");
service.setUserCredentials("myusername#gmail.com", "myhardcodedpassword");
When I tried this today with a new user, I got InvalidCredentialsException even though the username and password were definitely correct. I got an email in my inbox saying suspicions sign-ins had been prevented, and there seems to be no way to enable them again.
I am also aware that hardcoding passwords in source is bad practice.
However, I have read very widely online for how to enable OAuth/OAuth2 for this, and have ended up wasting hours and hours piecing fragments of information from blogs, stackoverflow answers etc, to no avail.
Ideally the solution would involve an initial process to generate a long-lived access token, which could then be hard-coded in to the app.
I want a definitive list of steps for how to achieve this?
EDIT: As Google have redesigned the API Console, the details of the steps below have changed - see comments
OK here goes, step by step
Go to Google Cloud Console and register your project (aka application)
You need to note the Client ID, and Client Secret
Go to the OAuth Playground, click the gear icon and choose and enter your own credentials
You will be reminded that you need to go back to the Cloud COnsole and add the Oauth Playground as a valid callback url. So do that.
Do Step 1, choosing the spreadsheet scope and click authorize
Choose your Google account if prompted and grant auth when prompted
Do Step 2, Click 'Exchange auth code for tokens'
You will see an input box containing a refresh token
The refresh token is the equivalent of your long lived username/password, so this is what you'll hard code (or store someplace secure your app can retrieve it).
When you need to access Google Spreadsheets, you will call
POST https://accounts.google.com/o/oauth2/token
content-type: application/x-www-form-urlencoded
client_secret=************&grant_type=refresh_token&refresh_token=1%2xxxxxxxxxx&client_id=999999999999.apps.googleusercontent.com
which will return you an access token
{
"access_token": "ya29.yyyyyyyyyyyyyyyyyy",
"token_type": "Bearer",
"expires_in": 3600
}
Put the access token into an http header for whenever you access the spreadsheet API
Authorization: Bearer ya29.yyyyyyyyyyyyyyyyy
And you're done
Pinoyyid has indeed provided wonderful help. I wanted to follow up with Python code that will allow access to a Personal (web-accessible) Google Drive.
This runs fine within the Google App Engine (as part of a web app) and also standalone on the desktop (assuming the Google App Engine SDK is installed on your machine [available from: https://developers.google.com/appengine/downloads]).
In my case I added https://www.googleapis.com/auth/drive scope during Pinyyid's process because I wanted access to all my Google Drive files.
After following Pinoyyid's instructions to get your refresh token etc., this little Python script will get a listing of all the files on your Google drive:
import httplib2
import datetime
from oauth2client.client import OAuth2Credentials
from apiclient.discovery import build
API_KEY = 'AIz...' # from "Key for Server Applications" from "Public API Access" section of Google Developers Console
access_token = "ya29..." # from Piinoyyid's instructions
refresh_token = "1/V..." # from Piinoyyid's instructions
client_id = '654....apps.googleusercontent.com' # from "Client ID for web application" from "OAuth" section of Google Developers Console
client_secret = '6Cl...' # from "Client ID for web application" from "OAuth" section of Google Developers Console
token_expiry = datetime.datetime.utcnow() - datetime.timedelta(days=1)
token_uri = 'https://accounts.google.com/o/oauth2/token'
user_agent = 'python urllib (I reckon)'
def main():
service = createDrive()
dirlist = service.files().list(maxResults=30)
print 'dirlist', dirlist
result = dirlist.execute()
print 'result', result
def createDrive():
credentials = OAuth2Credentials(access_token, client_id, client_secret, refresh_token, token_expiry, token_uri, user_agent)
http = httplib2.Http()
http = credentials.authorize(http)
return build('drive', 'v2', http=http, developerKey=API_KEY)
main()
I'm grateful to all who have provided the steps along the way to solving this.
I have run through the google example of using Authsub to retrieve Google feed data (http://code.google.com/appengine/articles/python/retrieving_gdata_feeds.html)
If I understand correctly, it is possible for a malicious hacker to call the 'next_url' (which google auth service calls with your token) and inject their own token?
Meaning that they could cause the Web apps to write to the Hackers google doc account instead of the authenticated user!
Does anyone know how to secure this url so that only google auth service can call it?
Below is the code I am referring to:
def get(self):
next_url = atom.url.Url('http', settings.HOST_NAME, path='/step1')
# Initialize a client to talk to Google Data API services.
client = gdata.service.GDataService()
gdata.alt.appengine.run_on_appengine(client)
# Generate the AuthSub URL and write a page that includes the link
self.response.out.write("""<html><body>
Request token for the Google Documents Scope
</body></html>""" % client.GenerateAuthSubURL(next_url,
('http://docs.google.com/feeds/',), secure=False, session=True))