I'm working on a project with Google App Engine. I am using continuous integration via Travis, and wish to be able to deploy directly from it. Due to a bug that will not be resolved directly, I can't rely on Travis' built-in GAE deployment, so I basically have to use mvn appengine:update manually. This requires me navigate to a generated URL and manually paste to the terminal an authentication code, which I can't do in automated builds.
It was suggested to me, however, that I do some Unix magic instead. While I can easily pick out the URL I need to navigate to from grep, I still need to log in to Google with my credentials in order to actually get the authentication code (which I can then grep out and pipe to the deployment program).
Given that, how do I log in to Google with my credentials, using only curl or similar command-line utilities?
I've accomplished similar things in the past using Service Accounts. These are likely a good fit for your problem.
Service Accounts will allow you to authenticate and upload your app without manual intervention.
Overview
A Service Account will allow you to do "passwordless" authentication like you may already do with ssh, and git, etc. by setting up your keys. This will remove the requirement that you log in manually, or follow the road to madness by trying to do a "manual" login automatically.
There are basically two steps:
Create your service account and key (with the right permissions)
Use that credential instead of what you're doing now
Resources
I think it's better to give a list of resources than concrete instructions since it's basically impossible to express concisely (even though it's a simple process, there's bound to be a lot of little things that annoy), everyone's requirements will be slightly different, and Google is likely to change the process at some point.
Using the Google Cloud Platform Console for App Engine | Permissions
Using OAuth 2.0 for Server to Server Applications
Setting up OAuth 2.0 | Service Accounts
gcloud auth activate-service-account
Hopefully that's enough to get you headed in the right direction.
Note
You'll likely have to spend some time looking at your .appcfg_oauth2_tokens_java and sorting out a variety of other annoyances, but I believe that this approach is the best way to solve your problem.
It sounds like you have a pretty straight-forward setup and that a Service Account alone will get you there, but if you need to get a little weird, the App Engine Admin API is always there.
Related
If a particular computer is making tons of accounts or flooding my server with other requests, could parse-server automatically check this behaviour and block the specified IP address?
Built-in rate limiting would also be a nice alternative, although it doesn't really solve the problem if the person continues to spam.
I am hosting on google app engine by the way.
I don't know about Parse itself, but from App Engine side you have DoS protection service controlled via dos.yaml file in your project that lets you blacklist IP blocks—sounds like that may help. It's not "automatic", though; you still need to manually update this file and issue appcfg.py update_dos <PROJECT_DIR> for changes to take effect.
I don't believe that this is a feature out of the box - see advanced options here: https://github.com/ParsePlatform/parse-server.
You'd need to look at controlling access to the Google App Engine (or another host - such as Microsoft Azure Web App) using a firewall (you can easily do this with Azure. I'm not familiar with Google App Engine, but imagine similar functionality is available.
However, I don't believe that a firewall is necessary - just better app security. Disable anonymous users - Parse Server Security
Firstly, just letting you know I have searched a fair bit here and I am aware of some of the other questions on this topic but none answer my question.
The authentication of the Local GAE differs from the appspot deploy and I need it not to with minimal work-around code.
I'm writing an HTML5 app and I can do the google authentication via a button and it updates all the correct tokens so I can access the profile in either GAE Launched apps or appspot deployed ones.
I need the google account details of the logged in user within the app
I am writing (for API calls to calendar and contacts for example)
, and I'd rather not have to write a login handler only for my local development platform - automated for simplicity or otherwise.
I've read that adding login:required forces a login, and on appspot this works perfectly. Locally it does nothing useful.
I've read that you can write a Python decorator to use #login_required - but I'm not writing in Python (It's php generating an HTML5 page). I could write a bit of a PHP wrapper to handle it, or automate a call in Javascript on page load - but this is the workaround I don't want to write because it's handled in the production environment for me.
I want the login:required option as everything is handled for me in
production
. I have googled the options for the login tag and nothing there suggests I can force a google login in the locally launched app. I have googled the launcher and settings, but nothing seems apparent.
I suppose I could live with the dev workaround, and the app could assume I'm authenticated and the JSON request handlers in my app would just use the login:required with the correct google tokens being passed once I am authenticated.
Do I have any other options?
This sounds like it could be a PHP runtime bug. login:required works fine on the python local dev server. Have you checked the issues page to see if it's been reported?
https://code.google.com/p/googleappengine/issues/list
Otherwise it's possible that it's bypassing the login on your dev server because you have some cookie in your browser indicating that you're already logged in. You might try clearing cookies
Alternatively (at least on the python devserver), you can go to your login page via http://localhost:8000/_ah/login to force a logout (obviously fix the hostname and port number)
There appears to be no way round this other than to write the whole OAuth handler yourself (or get one elsewhere) - significant overkill for a development environment only 'issue'.
I have written the app to handle the getting of the google profile details as it starts and force an authenticate if they are not present.
This means that the login:required will work as expected in the production world and force you to authenticate to google before you even get to the application... then the app just gets the profile details because the tokens are already present.
login:required in the dev environment just puts up a screen which you just 'ok', then the app attempts to gets the profile details but forces the authentication itself because there are no authentication tokens present.
It's unfortunate, but it's a single step in a development that users will not have to use, but it works.
I have a google app engine application which needs to be given a public-private key pair. I don't want to check this into source control because it will be accessible by too many people. Since this is GAE I can't use the build system to write the keys to the file system of the server.
Is there a known best practice for this?
My first thought was does Jenkins provide a way to manage keys securely? I know I can just copy the keys to a location on the jenkins server and copy them into the build but this project will be used by third party teams so I need to provide a UI based solution in jenkins. I did not find any relevant plugin but I would like to make sure there isn't a better way before writing my own.
There are of course several approaches to this. I believe certificates are a concern of admins, not developers.
What we do is have custom admin pages where we upload certificates to blobstore under separate namespace. Then we have an internal "service" (just a simple factory) so that other pieces of code can retrieve certs.
If you are happy to use a cloud based Jenkins, we (CloudBees) have an oauth based solution at appengine.cloudbees.com
You could roll your own. It is not excessively tricky. You will need to
Register with google's api console to get a client key and secret and define the endpoints that your app will show up as
Write some way if feeding those credentials to your Jenkins build. I would recommend using the credentials plugin
Either write a build wrapper that exposes the refresh token to your build (python sdk deployment) or exposes the access token (java sdk... Got to love that the two sdks do the same thing in different ways)
Or use our free service ;-)
I want to use Google Prediction from a Python Google App Engine Application. Google Prediction requires you to store your "prediction models" in Google Storage for Developers, in effect meaning that to use GP you must use GSD. Unfortunately, both GP and GSD seem to require OAuth 2.0 .
This Oauth stuff is really getting in the way though! All the examples I find seem to deal with the case of wanting for access a users data/credentials/identity/whatever using Oauth. I have no need for that. I simply want to access a resource (GP and GSD) from my server using http request. Repeat, I just want to use some of their services, I have no need at all to access any other users information!
I can see from my Google API console that I have created both a id and secret for my GAE applications domain. Is it not possible to just use these values to do OAuth authentication to other Google API's? Effectively saying "I am the application at domain xyz, here are my credentials, let me use your API"? It seems kind of ridiculous that Google is currently forcing people to use such a burdensome authentication system for things that they are trying to get people to try out?
I am hoping there is some magical awesome library that will take care of all these OAuth details for me. Short of that, a code example of how to do these things in Python App Engine would be useful. I just want to use the Google Prediction and Google Storage for Developers services from my python GAE app, but I am blocked by the burden of having to configure all of this OAuth stuff. Isn't there some easy way to do this?
Look at the Google API Python Client. You should just be able to put your tokens in and connect. There are some examples on the page that should give you enough information to connect in.
I'm the Product Manager at Google working on the Prediction API. The first thing I want to say is thank you for trying out the API and for reaching out to the world about your issues! We hear you! We are very aware of the difficulty of using the API in some cases and some of the pain OAuth2.0 can cause for the simple use cases. In particular I tried to do exactly what you were doing a few weeks ago and was myself rather frustrated! We're working on it!
OK, so, that's all nice and dandy, but do I have anything helpful for you? Hopefully I do! I managed to get my GAE application working with GP -- I shelved the GSD component for the moment as I ran out of time, so hopefully somebody else can lend you some sample code for that (it should involve using boto & OAuth to handle the tricky bits).
from apiclient.discovery import build
from oauth2client.client import OAuth2Credentials
# You can find an example oauth2client in the python prediction sample code
# Replace everything in <>'s
credentials = OAuth2Credentials(
"<access_token>", #probably empty string
"<client_id>",
"<client_secret>",
"<refresh_token>",
<Expiry>, # Probably None
"https://accounts.google.com/o/oauth2/token",
"<app_useragent>")
http = credentials.authorize(httplib2.Http())
service = build("prediction", "v1.3", http=http)
You should be able to get the client id an client secret from the API console. You can use any sample application or demo, e.g. the python sample code, to generate a refresh token.
Best of luck! Feel free to followup directly with me (zg#google.com) or post to our public discussion list (prediction-api-discuss#googlegroups.com) if you still have any trouble.
Looking at the Google App Engine API, it seems that despite all its great features, the User API is extremely limiting. It seems you can only authenticate people who have a Google account, or use an OpenID account, or via some OAuth kung fu (handshaking with a Facebook account etc).
This appears to be a major stumbling block for anyone who wants a proprietary user base by creating user accounts within the application. In short, I don't want my users to have to use or create a Google account to access my app.
Has anyone else come across this limitation and has it been a deal breaker for using the GAE? Am I missing something? It is possible to deploy my own Spring based security etc within the app and use my own User API? Comments on this issue greatly appreciated. Thanks.
You're free to completely ignore the Users API and implement your own authentication system, as you would in any other hosting environment. Nothing about App Engine prevents you from doing so.
The Users API is just there as a convenience, in case you'd like to spare yourself the effort of re-implementing everything, and spare your users the inconvenience of filling out another sign up form and remembering another set of credentials.
You can always implement your own user management system.
In my application I have used spring-security for this purpose. spring security 3.0.1 works perfectly fine with app engine 1.3.5. There may occur some issues integrating other versions of both. I found below links extremely useful :
http://www.google-app-engine.com/blog/post/Spring-security-fix-for-google-app-engine.aspx.
http://www.dotnetguru2.org/bmarchesson/index.php?p=1100
http://groups.google.com/group/google-appengine-java/browse_thread/thread/964e7f5e42840d9c