Is there any handy option for running MemSql on Google Cloud AppEngine and accessing it from dotnet application running on the same instance?
I'm not familiar with MemSql, but be aware that App Engine Flexible instances are restarted on a regular basis, and during that any locally saved data on disk will get lost, so hosting a database on Flexible is not a good choice. You may want to setup the DB on a GCE instance using persistent disk for storage.
Related
Use case: I am managing both GCP infrastructure and local infrastructure and am looking for a way for an app on Google App Engine to send data to/from a MySQL database on the local infrastructure, which is behind a VPN. I've set up and tested a Cloud VPN Gateway and VPC Connector that allows a Google Compute Engine instance to connect to the MySQL database and send and retrieve data.
Per this thread, and my own experimentation, the Google App Engine standard environment cannot currently connect to a local network via Google Cloud VPN directly. I've also been testing Cloud Function and Cloud Run to see if they can connect with the Cloud VPN, and it seems that they also have this limitation.
What I'd like to confirm is that only the Google App Engine flex environment OR a Google Compute Engine instance can connect through Cloud VPN. Google's documentation across all these resources doesn't ever outright say whether any of them can connect to Cloud VPN through a VPC Connector (just that they can connect to GCP networks via VPC Connector), so I'm hoping someone here can corroborate my testing. Additionally, is there any other GCP resource that can make use of this functionality that I've missed?
I have a App Engine Application written in Flask Python 3.7
My usecase is to get information from Composer Metadata DB (dag runs, dag success, dag failures etc) from Composer metadata DB and show as a dashboard inside App Engine Application (few charts).
Homework Done so far -
I was able to run sql queries on Composer metadata after logging in to one of the worker nodes (as worker nodes already have Cloud SQL proxy running which connects to Cloud SQL running in other container). This was done after creating a Compute engine under same VPC as of Composer and then doing ssh from Compute engine to one of the worker nodes.
Now the question is how to connect to Composer metadata DB that is under VPC from App Engine application using Cloud SQL Proxy ?
I would look at Serverless VPC connector, although it designed mainly for App Engine and Cloud Functions, you may consider implement this connector on a Flask app side that gives you opportunity to unify network between App engine and Compute engine nodes parties, thus you would be able to reach Cloud SQL proxy as well.
The setup is fairly much simple, though you just have to attach connector to the specific VPC network and region in the particular GCP project. The IP addresses pool must be in CIDR /28 range, reserved for a connector usage.
I am building a backend for an application with Google App Engine and Cloud SQL.
I do have:
A webserver as a proxy in front of my API server which handles sessions (using Cloud SQL and memcache) and calls the API
An API server which has access to the resource in the Cloud SQL instance
oAuth server which also needs Cloud SQL and memcache for tokens etc.
So my questions: Do I need three Cloud SQL project, which all have their own replica? Or is it ok to have one Cloud SQL project and all three App Engine projects access this Cloud SQL instance through the Cloud SQL proxy?
All projects will be located in the us-central region.
Would love to hear some thoughts.
Thanks!
I’m adding this information as a formal answer for the community. All credit goes to Dan Cornilescu.
You do not need to create 3 different projects. You can have 3 Google App Engine services running and a single Cloud SQL instance in the same project. That seems to be the best option for your situation. Using multiple services within a single project has its advantages one of them being increased performance.
Note that you could also have multiple Cloud SQL instances running in the same project. You can follow this document that talks more about creating a Cloud SQL instance:
Creating Instances
In case you need more information about Google App Engine services, this is a good resource:
Microservices Architecture on Google App Engine
We plan to use Appengine for our Node.JS REST-API which will serve content for our customers. The data will come from a MOngoDB Cluster (or Replicaset), which will also be hosted on Google Cloud Services. Some of the contents might be cached using the Memcache of AppEngine.
Now my question. Is it neccessary to setup a kind of VLAN or similar, to connect to the DB server? How can we assure that the DB server is connected with a good bandwidth / latency to the AppEngine instances? Or is this no matter of concern at all?
Another question, does Google Cloud have a kind of internal DNS, which can be used? So for example that we have a domainname like "ourmongodb.internal" which can be used in connection string of our REST API? Or how is the most recent approach to connect to our DB server instance?
If you use app engine flex you can setup your nodejs app engine server and the GCE VMs that run the mogoDB cluster in the same network and than connect them both in the internal IP space.
Somebody knows if is it possible connect an application from Appengine to a mysql database hosted in compute engine?
I'm trying to do this with python and i have this error:
Can't create TCP/IP socket (-1)
I'm using SqlAlchemy ORM which use the next configuration:
create_engine('mysql+mysqldb://root#ip/database')
and locally works but when i deploy the application to appengine doesn't work.
Thanks
Google App Engine, by default, runs code in a sandboxed environment, meaning that certain aspects of the Python runtime environment are restricted, or respond differently than they would otherwise. One of these aspects is outbound network connectivity—while GAE supports sockets, there are certain restrictions, and sockets are only available for paid apps.
The recommended options for storing information in a GAE app include the App Engine Datastore, Google Cloud SQL, and Google Cloud Storage. Google Cloud SQL is MySQL, and works with SQLAlchemy, so that's probably your best option.
If you absolutely need to run your own MySQL server (rather than using Google Cloud SQL) and connect to it from a GAE app, the other option is to use the managed VM environment, which permits unrestricted network access (since it's essentially a Google Compute Engine VM with the App Engine runtime on top).