I have two App Engine apps that need to communicate with each other. I am not using modules.
I use HTTP url fetch to communicate between them. Do I need to make the requests HTTPS to ensure secure communication.
I seem to remember an article stating that all communication within Google's infrastructure is encrypted/secure by default, therefore making HTTPS redundant.
Can anyone confirm this as it would be good to save some valuable CPU clock cycles with zero effort?
(I know how to ensure the identity of each server using the HTTP Headers so identity is not a problem.)
The purpose of SSL is to keep some hacker from sniffing your requests/responses to steal valuable info, like your online identity. A call from one Google server to another should not be exposed to any sniffing. Thus, in my opinion, SSL between Google instances is unnecessary.
SSL is not necessary if you set the follow_redirects parameter to False when invoking fetch. This will ensure the call will not leave Google’s infrastructure. Please see Making requests to another App Engine app or Google service section of this document.
On the other hand using SSL will not increase instance's CPU clock cycles, but the request-response will take a bit more.
Related
What is the best way to set up Google App Engine to always have at least one instance ready and available to handle requests when using automatic scaling? This is for a low traffic application.
There are settings here that allow you to control them but I am not sure what the best combination is and some of them sound confusing. For example, min-instances and min-idle-instances sound similar. I tried setting min-instances to 1 but I still experienced lag.
What is a configuration that from the end user's point of view is always on and handles requests without lag (for a low traffic application)? I
In the App Engine Standard environment, when your application load or handling a requests this may cause the users to experience more latency, however warmup requests might help you reduce this latency. Before any live requests get to that instance, warmup requests load the app's code onto a new one. If this is enabled, App Engine will detect if your application needs a new instance and initiate a warmup request to initialize a new instance. You can check this link for Configuring Warmup Requests to Improve Performance
Regarding min-instances and min-idle-instances these will only apply when warmup request is enabled. As you can see in this post the difference of these two elements: min-instances used to process the incoming request immediately while min-idle-instances used to process high load traffic.
However, you mentioned that you don't need a warmup so we suggest you to select App Engine Flexible and based on this documentation it must have at least one instance running and can scale up in response to traffic. Please take note that using this environment costs you a higher price. You can refer to this link for reference regarding the pricing of two environments in App Engine.
I have jobs and APIs hosted on cloud composer and App Engine that works fine. However for one of my job I would need to call an API that is IP restricted.
As far as I understand, I see that there's no way to have a fixed IP for app engine and cloud composer workers and I don't know what is the best solution then.
I thought about creating a GCE with a fixed IP that would be switched on/off by the cloud composer or app engine and then the API call would be executed by the startup-script. However, it restrains this to only asynchronous tasks and it seems to add a non desired step.
I have been told that it is possible to set up a proxy but I don't know how to do it and I did not find comprehensive docs about it.
Would you have advice for this use-case ?
Thanks a lot for your help
It's probably out of scope to you, but you could whitelist the whole range of app engine ip by performing a lookup on _cloud-netblocks.googleusercontent.com
In this case you are whitelisting any app engine applications, so be sure this api has another kind of authorization and good security. More info on the App Engine KB.
What I would do is install or implement some kind of API proxy on GCE. It's a bummer to have a VM on 24/7 for this kind of task so you could also use an autoscaler to scale to 0 (not sure about this one).
As you have mentioned: you can set up a TCP or UDP proxy in GCE as a relay, and then send requests to the relay (which then forwards those requests to the IP-restricted host).
However, that might be somewhat brittle in some cases (and introduces a single point of failure). Therefore, another option you could consider is creating a private IP Cloud Composer environment, and then using Cloud NAT for public IP connectivity. That way, all requests from Airflow within Composer will look like they are originating from the IP address of the NAT gateway.
I have an app engine service deployed in GAE (written in Node) to accept a series of click stream events from my website. The data is pushed as an CORS ajax call. I think, since the POST request can be seen in browser through the developer tools, somebody can use the app engine URL to post similar data from the browser console.( like in firefox, we can resend the URL. Chrome also has this features i guess)
A few options I see here is,
Use the firewall setting to allow only my domain to send data to the GAE.
This can still fail me since the requests can be made from the browser console repetitively)
Use a WAF ( Web Application Firewall) as a proxy to create some custom rule.
What should be my approach to secure my GAE service?
I don't think the #1 approach would actually work for you: if the post requests are visible in the browser's development tools it means they're actually made by the client, not by your website, so you can't actually use firewall rules to secure them.
Since requests are coming into the service after fundamentally originating on your own website/app (where I imagine the clicks in the mentioned sequences happen) I'd place the sanity check inside that website/app (where you already have a lot more context info available to make decisions) rather than in the service itself (where, at best, you'd have to go through loops to discover/restore the original context required to make intelligent decisions). Of course, this assumes that the website/app is not a static/dumb site and has some level of intelligence.
For example, for the case using browser development tools for replaying post requests you described - the website app could have some 'executed' flag attached to the respective request, set on the first invocation (when the external service is also triggered) and could thus simply reject any subsequent cloned/copy of the request.
With the above in place I'd then replace sending the service request from the client. Instead I'd use your website/app to create and make the service request instead, after passing through the above-mentioned sanity checks. This is almost trivial to secure with simple firewall rules - valid requests can only come in from your website/app, not from the clients. I suspect this is closer to what you had in mind when you listed the #1 approach.
I have a small question with possibly a complex answer. I have tried to research around, but I think I may not know the keywords.
I want to build a web service that will send a JSON response, which would be used for another application. My goal is having the App Engine server crawl a set of webpages and store the relevant values so the second application (client) would not need to query everything. It will only go to my server with the already condensed information.
I know, it's pretty common, but how can I defend from attackers who wish to exhaust my App Engine resources/quota?
I have been thinking on limiting the amount of requests by IP (say.. 200 requests / 5 minutes), but is that feasible? Or is there a better, and more clever way of doing it?
First, you need to cache the JSON. don't hit the datastore for every request. use memcache or possibly, depending on your requirements, you can cache the JSON in a static file in Cloud Storage. This simple is the best defender against DDOS, since every request adds minimal overhead.
Also, take a look in the DDOS protection service offered by app engine:
https://developers.google.com/appengine/docs/java/config/dos
You could require users to log-in then generate and send an auth key to the client app that must accompany any requests to the app engine service.
I have an 3rd Party API I'm wanting to use from my Silverlight 4 Application. It's interface is simply a web request (NOT A WEB SERVICE!) with form variables. According to everything I've read, this is not possible because of security issues? What security issues? Is there no way to do this? I can understand not allowing this with web services, but with simple web requests?
I cannot use my RIA Services gateway because the data is VERY sensitive and I cannot (and do not want to) transmit it under my own SSL cert. This is RIDICULOUS!
The same security issues apply in Adobe Flash. In JavaScript, it's not possible at all.
For Silverlight, the website root must contain a clientaccesspolicy.xml permitting the call.
http://msdn.microsoft.com/en-us/library/cc197955%28v=vs.95%29.aspx
To stop Cross Site Request Forgery (XSRF). Without this measure, what's to stop an attacker from issuing requests to online banking sites (or any site) on the users behalf? Exploiting the fact he may have cookies stored which will allow the request to go through.
It's too bad the internet has turned into an assumed guilty until proven innocent situation, as I could see many useful, non-malicious, uses for sending web requests.