Is there any software out there that assists in creating an intranet-style proxy for Google’s AppEngine? The problem I’m trying to solve is to enable corporate users to:
1. Enjoy a more responsive experience when plugged into their LAN,
and
2. Remain productive during Internet outages.
I'm not talking about a simple HTTP proxy server here... but instead something which can serve LAN users on behalf of GAE. Is this sort of thing at all possible?
If you're wanting to run an App Engine app on your own infrastructure, you probably want to check out TyphoonAE or AppScale. It sounds like you're looking for something that does even more than that, though - serving local users when disconnected, while still running on Google's infrastructure as well? I'm not sure how something like that would work, or if it's even possible. It would certainly need to be explicitly dealt with by the app, since there's no general way to handle partitioning in a database in a completely consistent fashion.
Related
I've been searching about if Google App Engine (GAE from now on) has some kind of DDOs protection. Apparently it had but it's been replaced with GAE Firewall which allows you to choose which IPs to allow to access your App.
This firewall raises 2 questions on me:
If someone tries to launch a DDOs attack on my App. Will they success or will they fail? I want to believe that Google will stop it automatically (or try at least) but I don't really know if it does.
Is there some kind of API for pogramming languages like Java to modify the allowed IPs in the Firewall so that I can programatically forbid IPs that I feel are dangerous? I haven't been able to find anything for this and it'd be really great.
Thanks!
You can manage your firewall rule programmatically using AppEngine Admin APIs which have a Java client available here (however I couldn't find the call for updating firewall rules).
Okay, I think I found it. Here's a list of libraries for various programming languages: https://cloud.google.com/apis/docs/cloud-client-libraries?hl=es
And what seems to be what I'm looking for is the FirewallClient which has method insertFirewall(InsertFirewallHttpRequest request): https://googleapis.dev/java/google-cloud-clients/latest/index.html
If you are concerned about the security of your app my recommendation is to use the Identity-aware-proxy. This should offer full protection to your app, not only for DDoS attacks.
Try to use Cloud Armor it’s the best solution :
https://cloud.google.com/armor
Problem. I'm looking for an agile way to shoot a docker container (stored on GCR.IO) to a managed service on GCP:
one docker container gcr.io/project/helloworld with private data (say, Cloud SQL backend) - can't face the real world.
a bunch of IPs I want to expose it to: say [ "1.2.3.4" , "2.3.4.0/24" ].
My ideal platform would be Cloud Run, but also GAE works.
I want to develop in agile way (say deploy with 2-3 lines of code), is it possible run my service secretly and yet super easily? We're not talking about a huge production project, we're talking about playing around and writing a POC you want to share securely over the internet to a few friends making sure the rest of the world gets a 403.
What I've tried so far.
The only think that works easily is a GCE vm with docker-friendly OS (like cos) where I can set up firewall rules. This works, but it's a lame docker app on a disposable VM. Machine runs forever and dies at reboot unless I stabilize it on cron/startup. Looks like I'm doing somebody else's job.
Everything else I've tried so far failed:
Cloud Run. Amazing but can't set up firewall rules on it, or Cloud Director, .. seems to work only with IAP which is painful to set up.
GAE. Works with multiple IPs and can't detach public IPs or firewall it. I managed to get the IP filtering within the app but seems a bit risky. I don't [want to] trust my coding skills :)
Cloud Armor. Only supports a HTTPS Load Balancer which I don't have. Nor I have MIGs to point to. I want simplicity.
Traffic Director and need a HTTP L7 balancer. But I have a docker container, on a single pod. Why do I need a LB?
GKE. Actually this seems to work: [1] but it's not fully managed (I need to create cluster, pods, ..)
Is this a product deficiency or am I looking at the wrong products? What's the simplest way to achieve what I want?
[1] how do I add a firewall rule to a gke service?
Please limit your question to one service. Not everyone is an expert on all Google Cloud services. You will have a better chance of a good answer for each service if they are separate questions.
In summary, if you want to use Google Cloud Security Groups to control IP based access you need to use a service that runs on Compute Engine as security groups are part of the VPC feature set. App Engine Standard and Cloud Run do not run within your project's VPC. This leaves you with App Engine Flex, Compute Engine, and Kubernetes.
I would change strategies and use Google Cloud Run managed by authentication. Access is controlled by Google Cloud IAM via OAuth tokens.
Cloud Run Authentication Overview
I have agreed with the John Hanley’s reply and I have up-voted his answer.
Also, I’ve learned that you are looking how to restrict access to your service through GCP.
By setting a firewall rules, You can limit access to your service by limiting the Source IP range as Allowed source, so that only this address will be allowed as source IP.
Please review another thread in Server Fault [1], stating how to “Restrict access to single IP only”.
https://serverfault.com/questions/901364/restrict-access-to-single-ip-only
You can do quite easily with a Serverless NEG for Cloud Run or GAE
If you're doing this in Terraform you can follow this article
If a particular computer is making tons of accounts or flooding my server with other requests, could parse-server automatically check this behaviour and block the specified IP address?
Built-in rate limiting would also be a nice alternative, although it doesn't really solve the problem if the person continues to spam.
I am hosting on google app engine by the way.
I don't know about Parse itself, but from App Engine side you have DoS protection service controlled via dos.yaml file in your project that lets you blacklist IP blocks—sounds like that may help. It's not "automatic", though; you still need to manually update this file and issue appcfg.py update_dos <PROJECT_DIR> for changes to take effect.
I don't believe that this is a feature out of the box - see advanced options here: https://github.com/ParsePlatform/parse-server.
You'd need to look at controlling access to the Google App Engine (or another host - such as Microsoft Azure Web App) using a firewall (you can easily do this with Azure. I'm not familiar with Google App Engine, but imagine similar functionality is available.
However, I don't believe that a firewall is necessary - just better app security. Disable anonymous users - Parse Server Security
I have a front-end angular app using firebase to store user data.
I currently do not have a backend set up, such as a node.js server.
I would like to use the Google Docs API to upload files from my app.
Since the Great Firewall of China does not (or makes unstable) the use of Google services, is it possible to place those services on the backend server and still use them reliably?
Perhaps after they have uploaded the document to firebase, a backend script retrieves it, uploads it to google docs, and then removes the record from firebase? Just trying to see if Google or similar services are even feasible for this use case.
I suppose the crux of my question is whether or not the calling of the Google API would be taking place on the user's computer, in which case would it become unstable?
** Updates for clarity:
I am deciding whether my firebase-backed app needs a more traditional backend like a node server to do things like: upload images and documents, send mail via Mandrill, etc... It would be helpful to me if I knew whether, after putting in the time to create a server, some of the services I am after (aka APIs) are any more resilient to the GFW than they would be if they ran on the client side. So if any one has had success in such a task, I would like to know.
** Technical update:
So, for example, if I run the Google Maps API on the client side, if the user is in China and is not running a VPN, accessing the API calls will either lag or time out or (rarely) success in returning the scripts. If I was somehow able to able to process the map query "off-site" aka on the server, could I then return with a static image of the map to a Chinese user without fail?
If I was somehow able to able to process the map query "off-site" aka
on the server, could I then return with a static image of the map to a
Chinese user without fail?
Yes, of course. What you are going to miss this way is all the front-end interactive functionality Google Maps offers. But if that's ok in your use case, sure.
I have never tried it with the GCF, but what I would do is this:
Google Maps <-> Your Reverse proxy <-> User
So, instead of the user visitng the real google maps site, it will be visiting your maps.mydomain.com site, that will be sitting in between, proxying everything.
Nginx is an excellent choice for a reverse proxy. If you need more control, there are good node.js reverse proxying packages that you an use to rewrite the content extensively before serving it (perhaps to obfuscate it in case the GCF blacklists content based on pattern matching, or to change the script names/links again to avoid pattern matching).
You are misunderstanding about the great firewall of China. I consulted for a couple of Chinese companies after the dot com crash so I can say this from personal experience, not hearsay.
It is mostly high-end Cisco hardware behind gateways behind their government telecom infrastructure. Nowadays they knock off what hardware they can, every chance they can, and spend money on specialized hardware to monitor cell phones systems.
There was a brief mention of the street-level surveillance hardware on 20/20 before the crash if you are interested in looking it up.
Not to discourage you, but I say set up whatever open servers you want with whatever frontends or backends you want, but the reality is the traffic is not going to be there.
That is why they call it an oppressive regime, they do not get to decide for themselves, remember?
This question is cross-posted on bitcoin.stackexchange, stackoverflow and bitcointalks.
I'm planning to build an application on Google App Engine that will heavily make use of Bitcoin trading. I've been Googling along a little but I couldn't find whether it is possible to run Bitcoin itself on App Engine (with Java). I have some experience with App Engine, but limited to a pure web-app centered usage.
I've read about a few people that have made applications using Bitcoin with App Engine as well that are hosting Bitcoin separately on an Amazon EC2 instance.
So, does anyone here either has experience with running Bitcoin in App Engine for Java or would anyone have an idea how this could possibly be done?
I know there are a lot of Bitcoin applications out there, I'd like to know how these manage their Bitcoin traffic.
I'm trying to avoid needing a separate Amazon service running all the time next to App Engine.
In fact, receiving Bitcoin can easily be done by using passive APIs like blockexplorer or blockchain, so I'm considering to find a reliable API to handle my outgoing payments. But this approach causes extreme dependency on this API service, which I actually want to avoid as much as possible.
I think you summed up the possiblities already.
depend on an external service providing notifications for
transactions and sending them i would advise against this.
have a second server running permanently and connect to it using json-rpc
running any type of p2p node on app engine will fail, because of the threading limitations on GAE.
a third possibility would be to use a stratum/electrum supernode, that way you are dependant on a 3rd party service, but at least it is well documented and you can set up one yourself easily. AFAIK, stratum is based on http.
source: i programmed a GAE app dealing with bitcoins about 6 monts ago. (using the second server approach)