I am reading about dos.xml for configure Java Application against DOS attack.
I did not understand how this configuration works.
I know about DOS and DDOS, but in this case it is confused for me.
When I configure Blacks of IP or subnets, will all requests be blocked? I think that GAE should block only if it receives too many requests of the same IP
What is the best way to prevent too many requests (evil requests ) in my login page? I can not have IP whitelist, because the app is available for any places in the world.
What is the difference between a black IP in two.xml and block IP in App Engine firewall?
Thanks !
When I configure Blacks of IP or subnets, will all requests be blocked? I think that GAE should block only if it receives too many requests of the same IP
Yes, that's how it's suppose to work: the IP address will be blocked only when AppEngine thinks they're abusing your resources (whatever that means). See more here:
Do not use this service for security because some requests from blacklisted users can still get through to your application. This service is designed for quantitative abuse prevention, such as preventing DoS attacks, only.
Unfortunately there is no clear algorithm described how it's handling this so it's probably best not to rely on it.
What is the best way to prevent too many requests (evil requests ) in my login page? I can not have IP whitelist, because the app is available for any places in the world.
You would have to implement some kind of rate limiting (by IP addresses or by accounts being accessed) yourself. Shouldn't be very difficult by incrementing some memcache key's value on each authentication attempt and setting it to expire in N minutes (or whatever approach that works best for you).
What is the difference between a black IP in two.xml and block IP in App Engine firewall?
A couple of differences between dos.yaml and AppEngine Firewall that comes to mind:
AppEngine Firewall can completely block access from a given IP address while the addresses listed in dos.yaml may still reach your app
AppEngine Firewall allows up to 1,000 rules while dos.yaml can only handle up to a 100
With AppEngine Firewall you can set up a whitelist (i.e. give access to your app only to, say, your company's network and no one else)
Also, judging from the wording AppEngine docs are using - it seems dos.yaml isn't the recommended approach so you would likely need to implement something yourself.
Related
I have a python (flask) application running on Google App Engine (flex); the application is protected by the GAE firewall where:
Default rule is 'Deny' all ingress
There is a whitelist of IP addresses from which traffic is allowed.
I have some microservices deployed on Cloud Run (fully managed) which:
Receive requests from the GAE app (e.g. for heavy duty tasks)
Send the results of whatever they process as http requests back to handlers/endpoints in the GAE app
Thus the GAE app is the main point of interaction with clients and a dispatcher of heavy tasks, while the processing of those tasks is carried out by the microservices. I have set up a static outbound IP address of the Cloud Run hosted service which verfiedly works and traffic is routed through the NAT gateway as required in the documentation. The respective NAT IP address is on the firewall whitelist.
The problem is that the firewall still does not let in the Cloud Run >>> GAE app requests which bounce back with 403 statuses (of course, if I change the default firewall rule to 'Allow', traffic goes through). If I host the same microservice in a docker container on a GCE VM with a static IP address like this everything works flawlessly. This makes me hypothesize that albeit Cloud Run outbound traffic is indeed routed through the static IP address when traffic is towards addressees outside GCP, when I try to ping an internal (project-wise) asset it still goes though some dynamically selected IP (i.e. the static IP solution simply does not work). Unfortunately the logs don't show the 403-ed attempt so I can't see from what IP addresses those request seem to come (from a GAE standpoint).
I would be very grateful for ideas how this can be fixed as it greatly diminishes the value of the otherwise wonderful idea to have static outbound IP addresses for Cloud Run.
First, thank you both for your help and suggestions, they are very helpful. I found the solution with some kind help from Google:
When the Cloud Run microservice and the GAE app are hosted in the same project traffic is still routed through internal channels and appears to come from IP address 0.0.0.0 which can be whitelisted (so it would work) as long as one considers this address encompasses GCP assets which are parts of other projects too (to the best of my understanding)
A more robust solution seems to be setting up an externally facing load balancer as described here and putting it in front of the GAE app; in such a case, Cloud Run will indeed consistently use its static outbound IP address as described in the documentation
You are correct saying that the static IP is not honoured when packets are routed internally to GCP.
I think this is what you want. You have to allow in the firewall one of the IPs mentioned there (not sure which one right now).
Just as you and #Ema mentioned, this is an expected behavior having in mind that the traffic from Cloud Run to App Engine is intern.
When you use Cloud Nat to send all traffic there, it does happen. If you create a container and ping, let's say to www.github.com. You will find that the traffic goes through the IP you set. On the other hand, if you ping to www.google.com, given that the traffic is intern, and the site to reach out is in the same infrastructure, the request doesn't even goes through public internet.
Additionally, just to keep in mind Static outbound IP address is still in Beta and it is not recommended to use Beta features/products in production environments.
As you mentioned and as it is stated in Allowing requests from your services:
Creating a rule for IP 0.0.0.0 will apply to all Compute Engine instances with Private Google Access enabled, not only the ones you own. Similarly, allowing requests from 0.1.0.40 or 10.0.0.1 will allow any App Engine app to make URL Fetch requests to your app.
This questions might be of your interest:
What are the outbound IP ranges for GCP managed Cloud Run?
Possible to get static IP address for Google Cloud Functions?
I have very serious question that my App Engine instance’s IP changed automatically with specific time interval. So my live application stops working. Can you please help so that we can make it working again.
Right now to solve this issue we have to change IP every time & set in SQL database also manually which is not proper way to do that.
Thanks in advance.
From the Google App Engine FAQ and as mentioned in the comments
Static IP Addresses and App Engine apps
App Engine does not currently
provide a way to map static IP addresses to an application. In order
to optimize the network path between an end user and an App Engine
application, end users on different ISPs or geographic locations might
use different IP addresses to access the same App Engine application.
DNS might return different IP addresses to access App Engine over time
or from different network locations.
Outbound services, such as the URL Fetch, Sockets and Mail APIs, make
use of a large pool of IP addresses. The IP address ranges in this
pool are subject to routine changes. In fact, two sequential API calls
from the same application may appear to originate from two different
IP addresses.
Solution: Set SQl instance network IP as : 0.0.0.0/0
They above IP called as none IP means your instance accessible by any IP
As an update to Scott's answer addressing how to get a static IP address (inbound or outbound) for App Engine can be found in my recent (2021) answer from a related question. Yes, it's not available directly from the App Engine service itself, but it's possible with other "GCP infrastructure." (BTW [by the way], that FAQ knowledge base has been deprecated, so you'll be redirected elsewhere if you click that link.)
I have just created one google app engine application and one 2nd Generation MySQL instance in eu-west2 region. In GCP Console they both seems to be in eu-west2 region.
However when I try to gelocate my ip's they seem to be in somewhere in US.
What should I do to use GCP in eu-west2 region?
my GCP instances:
their locations:
Google has an extensive world wide network. What you are seeing is us routing you to Google's closest Point of Presence (POP), which from that point on you're on a software defined network (SDN). What this means is we get your traffic on to our fast network as quickly as possible and abstract away the details of getting you to the machine in question.
Check latency from you to these hosts, then spin up a VM in Europe and check latency from that VM to these hosts - you'll find the numbers will confirm they really are in eu-west2.
I faced the same issue, you can find more about it here: Outgoing HTTP Request Location on Google App Engine
Is about the Google Network usage, the outgoing traffic come from the "Point of Presence" instead of the location and it can be dynamic. In my case, I have no solution, since it's mandatory to my API to make the requests from Brazil =\
I am running my server on google app engine and i have all of my services (e.g MongoDB, Redis, Elasticsearch) are deployed on compute engine. Now i wanted to connect my compute engine instances from App engine only that's why i deleted all of my firewall rules of my compute engines which were connecting them from external ip's, now only the instances that are within the internal network of my google cloud project can connect to themselves, now i am just wondering about IP spoofing that as nobody from outside my internal network can connect to my instances now can they fake their ip by telling my firewall that their ip is the ip which any of my instance is having because if that can happen then my whole security will be breached.
Now one question does google cloud project's firewall implement any measures to secure our instances from IP Spoofing or we have to setup something in order to avoid that.
If any of you have any idea about this please enlighten me.
Thanks
It's not quite clear which spoofing scenario you are concerned about. These two come to mind:
External party spoofing packets for your internal network, ie. the 10.0.0.0/8 range. This is not possible as packets inside your network can only come from VMs and VPNs in that private network.
Spoofing packets from other Google / GCE IP ranges; eg. the ones used for external addresses: This should be caught by Google's network ACLs.
I would however not recommend to authenticate based on IP address. For example, if you are communicating over external IP addresses between GCE/GAE entities, it's easy to be too broad, also allowing other GCE/GAE customers. Even if you only whitelist single IP addresses there is a risk that over time, your setup becomes more complex. Imagine for example, if an employee deletes a GCE instance without also removing the IP from the whitelist. In that case, the IP would be released and available to other GCE customers who could then access your service.
Therefore, it's usually safer to use an application level authentication mechanism such as SSL client certificates.
I host a web service, and have recently getting many HTTP requests (up to several thousand per second) from IP addresses that start with 10 according to the attached log. From my limited networking knowledge, this prefix means the IP is a local one, not a WAN IP. Why would app engine report traffic from Google's own LAN IP? Furthermore, because of this it seems I'm unable to blacklist that IP range, which has been costing me quite a lot in quota fees! Any ideas of why I'd be seeing local IP addresses in the logs for these requests, and how i can block them before they reach my application?
Sigh, embarrassingly, there is an obvious reason a Google App Engine application would see an IP with prefix 10 - It's Google's Crawler. The issue was I was generating too many unique URLs and it was trying to crawl them all, leading to the obscene traffic volumes I was seeing. So I was, in a way, DOS attacking myself by letting the crawler know about too many unique URLs. A simple robots.txt fix seems to do the job for this traffic, although one bot with "User-agent: Feedfetcher" is still hitting the site. Obvious in hindsight but maybe it will help someone else.