I'm just getting started with Google App Engine, and I'm having trouble figuring out how to securely provide credentials for external services to a deployed app. I saw that you can put environment variables in the app.yaml config file, but I don't want to store secret credentials in plain text on my local machine, and I would like to store the deploy agnostic parts of the app.yaml in version control.
What are the best practices for providing Google App Engine with sensitive deploy specific variables?
Related
I'm trying to build a a web app that does some image/pdf processing. the intent is to allow users to upload a file and get some response back.
My frontend stack is React and I'm using GCP Storage along with cloud functions that run based on the notification from storage.
All examples that I find for uploading files to Google Storage are for NodeJS implementations with Express and so is their SDK. only references I see for web is via Firebase but I'm not sure I want to go that route. is there any other way to build such functionality without an express implementation?
Most of JS libraries can work on both NodeJS and the Browser (where your React app runs).
Sometimes instead libraries are tied to NodeJs only, or browser only. It depends on the runtime APIs they use (eg. "fs" module is available to NodeJS only)
Moreover, before uploading files you want to make sure your users are authenticated and have permissions to upload. So normally you have:
Mobile app authenticating users
Mobile app invoking your backend/function for upload passing auth token
Backend/function validating the request authentication token
Backend/function pushing the file to the storage (the backend/function itself needs to run using a service account that's allowed to write to the storage bucket)
In the past when I've created a client-side web app using a framework like React, I can create a browser build and push it to a static hosting platform like AWS S3.
For Google Cloud Platform, it seems like the go-to approach is to deploy it to App Engine. If I'm understanding correctly, GAE is a combination of storage and compute resources.
Why can't I simply host the JavaScript app as prepared by webpack?
Google Cloud Storage (GCS) would be the way to go to host your static website on Google Cloud Platform (GCP). Notice that in order to use GCS you'll need to have a domain as well as a GCP project. The documentation is very straightforward on how to achieve this. Here you can find some examples and other useful information.
You could also host your static website on Google App Engine (GAE). But in this case you'll need to cleverly build your app.yaml file in order to manage your URL handlers, as well as structuring your project in a particular way. Simply uploading the JS app as prepared by webpack won't work. You can find all the useful information here in order to use GAE to host your static website.
I have a situation where an existing GAE App (let's call it app A) is running, but for non-technical reasons can't be modified. As users migrates to a new client version, we need to migrate their data from app A to a new GAE app (which I'll call app B).
Is there a way that I can grant app B access to app A's live datastore without modifying app A? My not modifying I mean not having to deploy new code. Changing setting or permissions in the Cloud Console is fine.
In case it matters, both apps that I'm referring to are written in Go.
It might not be possible to share the datastore across multiple GAE apps using the Google App Engine Standard Environment Client Libraries. At least for python it's not possible, donno about go.
But the Cloud Datastore Client Libraries can be used to share a datastore across many apps, even from outside Google Cloud.
Regardless of the particular way the old app accesses the datastore (language/library/etc.) it can be configured from the Cloud Console to allow access to a remote app. The exact procedure steps are captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?
The new app would be using the above-metioned client library with the old app's service account credentials (obtained in the above paragraph procedure) to access the old app's datastore.
I am looking to use Google Cloud Storage services outside of launching a google app engine application. My application is designed for running on a hosted server of our own, and I would like to still use the cloud storage service.
I've written a dev app that did this, and ran it with dev_appserver.py in the python GAE framework. It seems the dev_appserver.py is necessary as it fills in how to perform the RPC functions the cloud storage provides.
Is there a way to easily use google cloud storage service without the rest of it?
There is gsutil command line tool:
https://developers.google.com/storage/docs/gsutil
There is an xml/json restful service all at:
https://developers.google.com/storage/index
You'll need to enable the google cloud storage on the google console at:
https://code.google.com/apis/console/ and get the necessary api keys.
I am just starting to develop GAE web application. I am trying to get a list of bucket of google cloud storage. I found some code from web site.
there are 3 ways on the web what I found.
using Google storage API
https://code.google.com/p/google-api-java-client/source/browse/storage-serviceaccount-appengine-sample/src/main/java/com/google/api/client/sample/storage/appengine/serviceaccount/StorageSample.java?repo=samples&r=f0c6982b3cde8629511346641bfe4bb5eb28d73f
using Json liblary
but thsi sample needs to configure domain verification for bucket.
using python
our exisiting application is developped by gae for Java. I do not think thie python way can include in Java application.
then I would like to fix No1 way.
It show credential error. Buckets are authorized to everyone (All Authenticated Users )for now. I think I need to understand how the credential works. I have read the google developer page about 0Auth2. I could find a sample of bigquery. if you have a sample or good web sites about credential and google cloud storage, it will be great helpful.
the errors;
Error: com.google.api.client.googleapis.extensions.appengine.auth.oauth2.AppIdentityCredential.(Ljava/util/Collection;)V
what i did at API console:
creating client id (web application, service account, installed application each)
creating Simple API Access ( new server key , new browser key)
what I did at local machine;
Set ACL with gsutil
my developing environment
eclipse 3.6 helios ( Japanese language pack)
gae java for eclipse
gae 1.8.0
java 1.6.31
google storage API
google client library
others(tomcat,maven)
windows 7 English with Japanese language pack
Your first attempt (number 1) is the best way to focus on getting this to work. The code sample Marc provided uses the built-in App Engine App Identity module to securely identify the App Engine app to other Google APIs: https://developers.google.com/appengine/docs/java/appidentity/#Asserting_Identity_to_Google_APIs
However, there are 2 gotchas to check:
The App Identity module will not work using the local devserver. When running locally there is no way for the local server to securely identity itself to Google production APIs - so you should either look at using a service account with a downloadable key for local testing, or, easiest option (and what I'd recommend for now), is to deploy into a production App Engine environment to test.
Make sure the App Engine service account email address is added to the access control list for the Google Cloud Storage bucket that you are trying to access. Get this email address by looking under the Admin Console (for your App Engine app) -> Application Settings:
Service Account Name:
xxxxxxx#appspot.gserviceaccount.com
To add this to a Google Cloud Storage access control list, either add it to the project team for the project that owns the Google Cloud Storage bucket, or, if you're not using the default access control settings, add it directly to an ACL:
https://developers.google.com/storage/docs/accesscontrol
Service accounts with downloadable key - use this for local testing if you can't test in a production App Engine environment:
https://developers.google.com/accounts/docs/OAuth2ServiceAccount