I have many microservices in app engine only for internal use. But, by default, app engine opens service-project.appspot.com domain to public, and anyone can access them via http or https.
Is there a way to restrict access only for certain IP address?
The trivial way i can think of is checking source IP address in application code.
Or, I can create custom docker image with nginx configuration which checks source ip address. But, these are not quite clean solutions because access control is actually independent from application, and I don't want to hard code static IP address inside the container.
I assumed there is a way to setup firewall rule for app engine, but I could not find it. Identity-Aware Proxy seems like another option, but it is not available for app engine flex.
I know this is cold comfort, but we're working on re-enabling App Engine flex support for IAP. It's going to be more than just a few days, though.
https://cloud.google.com/appengine/docs/flexible/java/migrating#users has some options that might be more palatable than hardcoding IPs. You won't be able to use GCE firewall rules because the appspot.com traffic is coming through Cloud HTTP Load Balancer, so the GCE instance firewall only sees the IP of the load balancer. If you do want to verify IPs within your app, use X-Forwarded-For as described at https://cloud.google.com/compute/docs/load-balancing/http/#components .
Hope this helps! --Matthew, Cloud IAP engineer
Related
I have a python (flask) application running on Google App Engine (flex); the application is protected by the GAE firewall where:
Default rule is 'Deny' all ingress
There is a whitelist of IP addresses from which traffic is allowed.
I have some microservices deployed on Cloud Run (fully managed) which:
Receive requests from the GAE app (e.g. for heavy duty tasks)
Send the results of whatever they process as http requests back to handlers/endpoints in the GAE app
Thus the GAE app is the main point of interaction with clients and a dispatcher of heavy tasks, while the processing of those tasks is carried out by the microservices. I have set up a static outbound IP address of the Cloud Run hosted service which verfiedly works and traffic is routed through the NAT gateway as required in the documentation. The respective NAT IP address is on the firewall whitelist.
The problem is that the firewall still does not let in the Cloud Run >>> GAE app requests which bounce back with 403 statuses (of course, if I change the default firewall rule to 'Allow', traffic goes through). If I host the same microservice in a docker container on a GCE VM with a static IP address like this everything works flawlessly. This makes me hypothesize that albeit Cloud Run outbound traffic is indeed routed through the static IP address when traffic is towards addressees outside GCP, when I try to ping an internal (project-wise) asset it still goes though some dynamically selected IP (i.e. the static IP solution simply does not work). Unfortunately the logs don't show the 403-ed attempt so I can't see from what IP addresses those request seem to come (from a GAE standpoint).
I would be very grateful for ideas how this can be fixed as it greatly diminishes the value of the otherwise wonderful idea to have static outbound IP addresses for Cloud Run.
First, thank you both for your help and suggestions, they are very helpful. I found the solution with some kind help from Google:
When the Cloud Run microservice and the GAE app are hosted in the same project traffic is still routed through internal channels and appears to come from IP address 0.0.0.0 which can be whitelisted (so it would work) as long as one considers this address encompasses GCP assets which are parts of other projects too (to the best of my understanding)
A more robust solution seems to be setting up an externally facing load balancer as described here and putting it in front of the GAE app; in such a case, Cloud Run will indeed consistently use its static outbound IP address as described in the documentation
You are correct saying that the static IP is not honoured when packets are routed internally to GCP.
I think this is what you want. You have to allow in the firewall one of the IPs mentioned there (not sure which one right now).
Just as you and #Ema mentioned, this is an expected behavior having in mind that the traffic from Cloud Run to App Engine is intern.
When you use Cloud Nat to send all traffic there, it does happen. If you create a container and ping, let's say to www.github.com. You will find that the traffic goes through the IP you set. On the other hand, if you ping to www.google.com, given that the traffic is intern, and the site to reach out is in the same infrastructure, the request doesn't even goes through public internet.
Additionally, just to keep in mind Static outbound IP address is still in Beta and it is not recommended to use Beta features/products in production environments.
As you mentioned and as it is stated in Allowing requests from your services:
Creating a rule for IP 0.0.0.0 will apply to all Compute Engine instances with Private Google Access enabled, not only the ones you own. Similarly, allowing requests from 0.1.0.40 or 10.0.0.1 will allow any App Engine app to make URL Fetch requests to your app.
This questions might be of your interest:
What are the outbound IP ranges for GCP managed Cloud Run?
Possible to get static IP address for Google Cloud Functions?
Our AppEngine app is connecting to a remote service which requires a VPN and also required me to add entries to the hosts file on my local machine in order to connect to their endpoints.
e.g.
10.200.30.150 foo.bar.com
This is working fine when running the app locally, but I can't figure out how to set this up on Google Cloud to work once deployed.
I can't use the IP addresses directly because it errors that the IP is not on the cert's list.
How do I map the host names to the IPs in Google Cloud so that AppEngine can use them?
From the error mentioned in the comment I suspect connecting directly through the IP fails because the certificate doesn't recognize the IP to DNS mapping as valid and therefore the secure connection setup breaks. Based on the requirements of connecting to the API by VPN and tweaking the hosts mapping there are few things you may try.
The simplest approach that may work would be using a Google Compute Engine VM instance, since there you would able to manipulate the etc/hosts file and replicate the local machine setup. This VM could be used either as the main app service or as a proxy from App Engine to the 3rd party API endpoint. To go that route I would suggest taking a look at these two posts which explain how to change the etc/hosts file on GCE (Changing the file once wouldn't work as the VM periodically overrides it, see the posts for cronjob like workaround).
Separately, as your app runs in App Engine flexible environment there is the chance to provide a docker container with the app packaged. It may be possible to set the workaround above in the docker file and have it working in App Engine too.
We have 2 app engine app (flex and standard) running on separate projects and we want project A to request project B with https to xxx.appspot.com URL.
Our firewall on both projects Denies all IPs(*) and whitelisted App Engine internal addresss (10.1.0.41, 0.1.0.40, 10.0.0.1 and 0.1.0.30) as explained in the doc.
Yet we receive a "403 error forbidden access" (which disappears when disabling the firewall).
This post is similar to mine but the responses didn't help me.
Is there anything else I can do ?
Did anyone got this to work ?
Thank you in advance.
As you may already know, GCP Projects represent a trust boundary within an organization. Hence, inter-project communication between App Engine services would require Public IP communication or using Shared VPC, which allows connecting between networks from different projects. There should be no internal communication between App Engine Services over different projects. Hence, whitelisting App Engine internal IP addresses might not be useful in this situation.
About using Public App Engine IP addresses, as illustrated in this document. App Engine hosts services on a dynamic public IP address of a Google load balancer. Due to that, the IP address can be changed any time and any Static IP can not be provided. For outbound services, a large pool of IP addresses are used which you can obtain as outlined in this document
Is it possible to create a google app engine program that would route http requests to a server on a local network?
What would be the best way to build a program like this?
I am trying to get away from buying a server from a hosting provider and simply use a local network server instead, and use google apps as a sort of proxy. The firewall would be configured to allow access to the server from the google app engine servers only.
If this has been done before in an open source project that would be excellent, but I have not been able to find one.
If all you want is a domain name that points to your dynamic IP address, you could give Dynamic DNS a try. It's designed for your use case, and you won't need to write any code; you just need either a router that supports it or a server with cron. There are lots of providers, but I've had good experiences with Dyn DNS, specifically their Remote Access plan.
I know the Google App Engine docs say you can only have HTTPS on your foo.appspot.com domain, not with a custom domain.
But is it possible (and safe) to host a custom domain somewhere else, and set it up to proxy all HTTPS requests to https://foo.appspot.com?
If so, how would you recommend setting it up? And would it be much slower, compared to using https://foo.appspot.com directly?
Yes, it's possible to do this. It's secure if you trust your proxy and you use SSL from the proxy to the app. It will be noticeably slower, since there's a longer path between your user and your app. In addition, unless you get multiple proxies, all your traffic will have to go via a single global location, whereas the appspot domain is served from frontends all round the globe.