I'd like to build an TCP receiver/listener/server to run on Google App-Engine. For example to receive messages over TCP on a specific port, similar to a syslog server, and process the request. Something in Go language would be ideal, but really anything appengine supports.
Does anyone know if and how I can setup appengine to proccess TCP requests?
AppEngine runs your applications in a sandboxed environment and you may not open sockets, as mentioned in their What Is Google App Engine? guide:
As with the Java and Python environments, not all the standard
library's functionality is available inside the sandbox. For example,
attempts to open a socket or write to a file will return an os.EINVAL
error.
App Engine's designed to handle HTTP requests, which are TCP requests. If you choose to use App Engine, you should probably design your server to handle HTTP requests.
If you need to run on a lower level, you're probably better off using something like Amazon which has tools like their elastic IPs that would make this much easier.
Related
I have jobs and APIs hosted on cloud composer and App Engine that works fine. However for one of my job I would need to call an API that is IP restricted.
As far as I understand, I see that there's no way to have a fixed IP for app engine and cloud composer workers and I don't know what is the best solution then.
I thought about creating a GCE with a fixed IP that would be switched on/off by the cloud composer or app engine and then the API call would be executed by the startup-script. However, it restrains this to only asynchronous tasks and it seems to add a non desired step.
I have been told that it is possible to set up a proxy but I don't know how to do it and I did not find comprehensive docs about it.
Would you have advice for this use-case ?
Thanks a lot for your help
It's probably out of scope to you, but you could whitelist the whole range of app engine ip by performing a lookup on _cloud-netblocks.googleusercontent.com
In this case you are whitelisting any app engine applications, so be sure this api has another kind of authorization and good security. More info on the App Engine KB.
What I would do is install or implement some kind of API proxy on GCE. It's a bummer to have a VM on 24/7 for this kind of task so you could also use an autoscaler to scale to 0 (not sure about this one).
As you have mentioned: you can set up a TCP or UDP proxy in GCE as a relay, and then send requests to the relay (which then forwards those requests to the IP-restricted host).
However, that might be somewhat brittle in some cases (and introduces a single point of failure). Therefore, another option you could consider is creating a private IP Cloud Composer environment, and then using Cloud NAT for public IP connectivity. That way, all requests from Airflow within Composer will look like they are originating from the IP address of the NAT gateway.
I'm using Google App Engine Standard to write a small application, let's call it AppX.
AppX is supposed to receive a POST message from another website, let's say B, and then do some processing and show on its mainpage.
The question is:
I don't know how to use dev_app server to debug. As if I use dev_app server, the server will run locally without https, then I don't know how to send a POST message from website B.
Google cloud shell is a very limited shell too, which does have limited ports enabled for outgoing connections only.
Although there might be a way to configure it, I think the easiest way to test calls from website B to a dev_app would be configuring a GCE virtual machine with a fixed IP. There you can configure the firewall freely, and also not worry any non-interactive session to finish abruptly.
I have read about Remote Desktop Client with AngularJS and Yeoman.It is using sockets internally.
http://blog.mgechev.com/2014/02/08/remote-desktop-vnc-client-with-angularjs-and-yeoman/
As my angular app is deployed on GAE and I have implemented channel API for notifications. So is there any way to use channel API for the same?
Or any other best way to do the same?
The simple answer is: No, the approach documented by the provided link will not work on "normal" app engine.
Here a couple of reasons:
It uses a continuous socket connection to connect to the VNC server
It uses server sockets* to provide a stream to Yeoman VNC
The Channel API is not an appropriate replacement for sockets
*) App Engine doesn't allow you to open listening sockets. Having the 60 second / 10 minute deadlines in place it wouldn't be practical anyway (unless manually scaled and thus taking all the good out of app engine)
I just wrote an extended answer on a slightly similar question here. Some of the points there could be of interest to you.
Consider using App Engine Managed VM. I believe you can also run node.js application on managed vms.
Previous answers have eluded to the fact that Google App Engine now supports FTP - but does it support sFTP as well for incoming data streams?
We are looking to simulate an sFTP endpoint on GAE to leverage its HIPAA compliance. If incoming sFTP is not possible on GAE - is there a simple layer of indirection one could create that effectively gives the same level of security yet still looks like an sFTP endpoint to outside systems?
It appears according to Google you can only create outbound sockets. "You cannot create a listen socket; you can only create outbound sockets."
Google App Engine currently limits you to 2,000 emails per day (for free) via their API.
I am trying to find a definitive answer if it is possible to use a third-party system if you need to send more. I know that they disallow raw sockets, so I would assume that there might be trouble with this approach... but surely I'm not the first to see it.
Worst case, I can build a simple offsite web service that my GAE can call... but I'd much rather just be able to send directly through an SMTP server.
Thanks!
Nope.
You're correct: you cannot make raw socket requests, nor any other direct outbound requests except through the urlfetch API. To talk to an external SMTP server, you would need to use a webservice as a proxy.
We use the Postmark mail outsourcing service via the hutools.postmark API. Since the communication is HTTP based, it works like a charm on Google AppEngine. This might be an option for you, although it is also a for-pay service. We use it to get arround GAEs sender restrictions.
I've successfully used third party providers for email services with Google App Engine. I've used both SendGrid and MailGun using their HTTP-API.