Is there a way to port the functionality of
from google.appengine.api import app_identity
from google.appengine.api import urlfetch
in the flexible environment?
I have a service running on the flex environment which needs to make a request to the standard environment and would like to use App Engine's native app identity and X-Appengine-Inbound-Appid header to do so.
It seems that if the origin of the request is from the standard environment, the header is passed on to the flex environment but not the other way around?
Update: It is not possible to use the X-Appengine-Inbound-Appid header in this case because, as you observed, it is removed on the ingress side of the standard env service after being specifically set on the egress side of the flexible env service, following the suggested experiment:
Original post:
From App identity:
When running on the flexible environment, you can use a combination of
environment variables and the Google Compute Engine Metadata
service to obtain information about your application:
Application / project ID
Either the GCLOUD_PROJECT environment variable or
the /project/project-id resource in the metadata server
So you could try to get the app ID in the flex env service as mentioned above and manually set the X-Appengine-Inbound-Appid response header. It might not be removed on the ingress side of the standard env service (it's not explicitly mentioned to be removed in Request headers).
As for the urlfetch porting, from URL Fetch:
The flexible environment has no sandbox restrictions, so you can use
any HTTP library. We recommend using Requests.
So check for the specific method to set the response header for whichever HTTP library you decide to use.
Related
Thanks for your help in advance!
I'm trying to create a simple but secure way to inject secrets from a commandline call to gcloud app deploy app.yml.
I'm relying on secret information stored in environment variables in my app's runtime, which I can set using the following app.yml:
runtime: nodejs
env: flex
env_variables:
SECRET_KEY: "passed-in-value"
For example, I'd like to be able to do something like
SECRETKEY=${LOCAL_SECRET_VALUE} gcloud app deploy app.yml
or use a cli argument if there is one, but I don't see any here.
At the end of the day, I just need a simple way to inject secrets so I can deploy to my testing environment from my local machine, or to production from a github action. I could just write the app.yml dynamically from my ci script and inject local environment variables, but it seems like there must be a more canonical way.
I can set environment variables with app.yml using the following syntax.
I would like a solution which works with both standard and flex app engine if possible.
Google Cloud has silently released in Beta Secret Manager. The APIs are available but the client libraries aren't yet. The service will be announced soon and will help you to manage your secrets.
The principle is simple: In your yaml files and in your GIT, you simply save a reference to the secret, for example mySecret#2 where 2 is the version number, and mySecret the name of your secret.
Then, perform a simple call to the API to get access to the secret
https://secretmanager.googleapis.com/v1beta1/projects/<myProject>/secrets/<MySecret>/versions/<Version>:access
The call must be secured with a Bearer access token in the header. Be sure that the App Engine service account has the required roles (default service account: YOUR_PROJECT_ID#appspot.gserviceaccount.com), at least roles/
secretmanager.secretAccessor
Note: I don't know how the library will be implemented if you request the secret without giving an explicit version. Is the 1st version which will be taken? the lastest?
For now, there is a lot of manual overhead (get the token, build the URL, manage errors,...). If you can wait a couple of weeks, the life will be easier with client libraries!
E.g. create an app engine standard php72 app and perform a curl request to another app engine app. Note that the header X-Appengine-Inbound-Appid is not sent with the curl request.
The usual answer to this is to use URL fetch service - but I don't see this in php72 docs?
The header is added by the URLFetch service. As long as the service is not following redirects, it will add the X-Appengine-Inbound-Appid header automatically.
You can find more information in this documentation. [1]
[1] App Identity PHP API Overview:
https://cloud.google.com/appengine/docs/standard/php/appidentity/#asserting_identity_to_other_app_engine_apps
What is the correct way to map a custom domain to Google App Engine project?
I've added the custom domain, set up all the DNS records etc. and it is working! But how do I enforce HTTPS only? (Like the custom domain generated does)
From reading around many posts I have seen secure: always in a handler which I have tried, but doesn't seem to work (saw some comments that it doesn't work on flex environments?)
So how can I correctly map my domain to a flex php environment being HTTPS only?! Seems such a simple thing to want to be able to do!
While being somewhat related to a custom domain (for example the SSL certificate used by HTTPS being tied to the domain) the HTTPS-only behaviour is not really an attribute of the custom domain mapping operation, it's just a behaviour of the app itself (regardless of being mapped to a custom domain or not).
For standard environment GAE apps part of the HTTPS-only behaviour is an automatic http -> https redirection performed by the GAE infra on behalf of the app if secure: always is set in the app.yaml configuration file.
But that configuration has, indeed, been deprecated for the flexible environment GAE apps. The PHP-specific deprecation note can be found at app.yaml changes:
Secure handlers
The secure setting under handlers is now deprecated for the App
Engine flexible environment. If you need SSL redirection, you can
update your application code and use the X-Forwarded-Proto header to
redirect http traffic. You can also use the HTTP Strict Transport
Security response header.
You'll also note that there is no mentioning of secure (or handlers for that matter) in the app.yaml Configuration File doc.
So you need to take care of the http -> https redirection inside your app code (or the framework you use for it, if any and if it has such capability), possibly following the suggestions from the above quote. I can't really help with details on this as I'm not a PHP user.
I have an AppEngine webapp with a JavaScript tracker. I want it to make HTTP calls to a collector (which is a REST API that saves data into BigQuery) without making an entire app-to-app authentication process with OAuth/JWT or whatever.
According to this thread it's not possible for AppEngine to use a REST API located on a compute engine instance with an internal IP.
In addition, it seems that it's not possible to use GAE multibackends feature since it loses autoscaling and I really need this feature.
What is the proper Google Cloud way to achieve this ?
I presume that the collector is running on compute engine and the problem is how to verify that requests to the collector are genuine requests from the AppEngine service since the only way for the AppEngine app to reach the collector is via a public IP.
AppEngine has an Identity API for an AppEngine to assert to a third party service that a request is genuine. A more detailed (and less contrived) example of how to use this is discussed here. The code link no longer works but the code is available here.
This specific sequence from the last article seems to be more or less what you need:
Client App generates a signed blob by calling app_identity.signForApp(string_blob)
Client App exposes its public certificates on a public endpoint, for example clientapp.com/certs. In the demos below we use a trivial Json format to expose certificates, something like: {"cert1":"x509 cert pem", "cert2":"x509 cert pem 2"...}.
Client App sends a request to API App along with the signed blob and the URL that contains Client App's public certificates.
API App fetches Client App's public certificates from that URL
API App verifies the signature of the signed blob. The API App might perform other business logic like checking if the Client App (as identified by the URL of its public certificates) is on an access control list.
Both apps should agree on the same 'signed blob' format. In our demo we use JWT as the signed token format. The detailed spec can be found at http://self-issued.info/
I've read some info about authentication, but I would have thought that I could turn off my app's visibility and/or access to the public. This would be useful for alpha testing so surely a setting like this exists? Or do I need to build such things into the app itself?
Without some sort of authentication mechanism your app can't really distinguish between a request coming from you and one coming from someone else.
It might be a good idea to spend a bit of time to analyze your app's authentication requirements and maybe get it done now, while still in alpha.
Depending on the solution it may be fairly simple to integrate.
Google offers multiple authentication options, see What is the difference between Google identity toolkit, Google OAauth and Google+ sign in
I personally opted for the GIT kit for simplicity, flexibility and convenience.
It's possible secure your App's urls so only an authorised user or administrator can access them.
This can be done through the app.yaml file (Python, PHP and Go applications) or the web.xml deployment descriptor (Java applications).
Option A:
Just allow only admin access, in yourapp.yaml
- url: /*
login: admin
script: yourappname.app
Option B:
If you have an static IP (or with a few changes a week), you can detect the IP of the request and let run only from your IP:
class yourHandler(webapp2.RequestHandler):
def get(self):
userIP=self.request.remote_addr
if userIP=="220.123.211.120" # Change this with your static IP
...your code for authorized users.
Option C:
Check request domain (to ensure is called from your own authorized domains), and put some security client side.
class yourHandler(webapp2.RequestHandler):
def get(self):
origin=self.request.headers['Origin']
if origin=="www.yourdomain.com" # Change this with your domain/subdomain
...your code for authorized users.
# I recommend to put also the CORS headers for your own domain
self.response.headers['Access-Control-Allow-Origin'] = "www.yourdomain.com"
Personally, I have a mix of the three options plus a custom authentication to access private content.
By default, every service is born public. Change that, individually, by changing the --ingress setting for the service you want.
gcloud beta app services update <service-name> --ingress <value>
all (default): public to internet.
internal-only: only accessible for resources in the same Cloud Project.
internal-and-cloud-load-balancing: only accessible for resources in the same Cloud Project. And those requests came from configured Cloud Load Balancing.
1 Gateway + a bunch Microservices architecture example:
gcloud beta app services update ms-payment --ingress internal-only
gcloud beta app services update my-backend-gateway --ingress all << default!! Just for example purpose.
In this way, ms-payment is accessible only by resources within the same Cloud Project, even if they are in different VPCs.
Refer the documentation: https://cloud.google.com/appengine/docs/standard/java11/application-security#ingress_controls
I've found recently that you could also use IAP (Identity-Aware Proxy) IA-what? I found a tutorial that implements it on App Engine.
Tutorial for App Engine.
So I didn't want to rely on my own authentication implementation because I'm not an expert, and security it's something very hard to learn in a rush. In a nutshell
Deploy an IAP step 1
Add your app engine (or the whole scope) to your IAP
add your authorized emails on the left panel step 3. For access use:
IAP-Secured Web App User: Grants access to the app and other HTTPS
resources that use IAP.
My Personal opinion here: try to implement as many safety measures as possible (don't rely on one system only), usually they could fail.