I'm attempting to connect a Flask App Engine instance to a Cloud SQL instance and the connection works when I run things locally but doesn't work when I deploy it using gcloud app deploy. I'm using flask_sqlalchemy to connect to the database with the following URI string
app.config["SQLALCHEMY_DATABASE_URI"]= f"postgresql://postgres:{PASSWORD}#{PUBLIC_IP_ADDRESS}:5432/{DBNAME}"
However this results in a timeout error that looks like this:
OperationalError: (psycopg2.OperationalError) could not connect to server: Connection timed out
Try taking a look at these pages:
Connecting from App Engine standard environment to Cloud SQL
Connecting from App Engine flexible environment to Cloud SQL
There are instructions for both Private and Public IP.
You may be also interesting this demo app:
https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/cloud-sql/postgres/sqlalchemy
I have set up 2 projects in my Google Cloud console. The following are the two projects that I have set up in my console.
1. Cloud SQL + App Engine
2. App Engine (New)
So the idea is App Engine (1) is running the same database as App Engine (2). I have already set up the IAM Permission Page and IAM Admin and Project Page.
I have given both projects as Project Editor role too, but still my (2) still can't connect to my (1) database.
Any help please?
Granting access to an application does not automatically enable a database user account to connect to the instance.
You may connect through a proxy, in which case you should follow these steps:
Enable the API
Install the proxy client on your local machine
Determine how you will authenticate the proxy
If required by your authentication method, create a service account
Determine how you will specify your instances for the proxy
Start the proxy
Update your application to connect to Cloud SQL using the proxy
You can find related details on the Connecting to Cloud SQL from External Applications.
This documents provides steps that cover configuring access for IP connections as well.
If you connect from within the app engine environment, you may have a look at Connecting from App Engine.
I have a dotnet core 2.0 application running in Google App Engine Flexible Environment. Within the same Google project I have a Cloud SQL - MySQL database. On the Cloud SQL Instance details page, under the Authorizations tab, it states
Apps in this project: All authorized.
However, I cannot access the database from my application unless I add the 0.0.0.0/0 route to the Authorized networks section.
What can I do to give my application db access without opening my database to the world?
Update 2018-05-21 from Jeffery Rennie (accepted answer)
App Engine now supports connecting to a Cloud SQL instance using a port number instead of a unix domain socket. So now, you can add something like this to your app.yaml:
beta_settings:
cloud_sql_instances: "your-project-id:us-central1:instance-name=tcp:5432"
And specify Host=cloudsql in your connection string in your appsettings.json:
"ConnectionString": "Uid=aspnetuser;Pwd=;Host=cloudsql;Database=visitors"
In the sample above, the port is 5432, which is the default port for a PostgreSQL database. For a MySQL database, use port 3306.
A full example with instructions for deploying to App Engine can be found here:
https://github.com/GoogleCloudPlatform/dotnet-docs-samples/tree/master/appengine/flexible/CloudSql
The ideal solution is to use a unix domain socket to connect from your app engine instance to Cloud SQL. That's how other programming languages like Python and PHP do it. Unfortunately, the MySQL connector does not work with domain sockets. I see no reason why it can't, but it doesn't. I hope they fix that issue soon.
As described in https://cloud.google.com/appengine/kb/#static-ip,
Note that using static IP address filtering is not considered a safe
and effective means of protection. For example, an attacker could set
up a malicious App Engine app which could share the same IP address
range as your application. Instead, we suggest that you take a defense
in depth approach using OAuth and Certs.
If certificates are not sufficient to protect your application, then the only remaining option I see today is to build a custom runtime that runs the Cloud SQL Proxy. The proxy can forward a local ip port number to a unix domain socket. If you have built a docker image or two, then it's not too bad.
I will update this answer as the situation improves.
Update 2018-05-21
App Engine now supports connecting to a Cloud SQL instance using a port number instead of a unix domain socket. So now, you can add something like this to your app.yaml:
beta_settings:
cloud_sql_instances: "your-project-id:us-central1:instance-name=tcp:5432"
And specify Host=cloudsql in your connection string in your appsettings.json:
"ConnectionString": "Uid=aspnetuser;Pwd=;Host=cloudsql;Database=visitors"
In the sample above, the port is 5432, which is the default port for a PostgreSQL database. For a MySQL database, use port 3306.
A full example with instructions for deploying to App Engine can be found here:
https://github.com/GoogleCloudPlatform/dotnet-docs-samples/tree/master/appengine/flexible/CloudSql
While you are not wrong that "apps in this this project: All authorized" seems to suggest you can out-of-the-box just use your App Engine app with Cloud SQL, but there are limitations.
First of all, your Cloud SQL needs to be a 2nd generation instance, and secondly, there are specific instructions that's dependent on the language you use and the App Engine type (standard or flex).
If your situation fit all the requirements, it should work.
For your specific use case, you need the .Net instructions, it does say you need to add a network with 0.0.0.0/0 access and an user account. The user authentication + SSL should provide the security you need.
My Google App Engine app fails to connect to my Google Cloud SQL instance with:
java.lang.RuntimeException: The Google Cloud SQL API is not enabled for project [{project-id}]. Please use the Google Developers Console to enable it: https://console.cloud.google.com/apis/api/sqladmin/overview?project={project-id}
at com.google.cloud.sql.core.SslSocketFactory.obtainInstanceMetadata(SslSocketFactory.java:389) ~[jdbc-socket-factory-core-1.0.4.jar!/:na]
at com.google.cloud.sql.core.SslSocketFactory.fetchInstanceSslInfo(SslSocketFactory.java:284) ~[jdbc-socket-factory-core-1.0.4.jar!/:na]
at com.google.cloud.sql.core.SslSocketFactory.getInstanceSslInfo(SslSocketFactory.java:264) ~[jdbc-socket-factory-core-1.0.4.jar!/:na]
at com.google.cloud.sql.core.SslSocketFactory.createAndConfigureSocket(SslSocketFactory.java:183) ~[jdbc-socket-factory-core-1.0.4.jar!/:na]
at com.google.cloud.sql.core.SslSocketFactory.create(SslSocketFactory.java:152) ~[jdbc-socket-factory-core-1.0.4.jar!/:na]
at com.google.cloud.sql.mysql.SocketFactory.connect(SocketFactory.java:48) ~[mysql-socket-factory-1.0.4.jar!/:na]
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2222) ~[mysql-connector-java-5.1.44.jar!/:5.1.44]
That's strange, because I thought the API was enabled. The APIs dashboard has an entry for Google Cloud SQL with no activity and a "disable" button.
The app uses the Cloud SQL Socket Factory:
jdbc:mysql://google/{database}?socketFactory=com.google.cloud.sql.mysql.SocketFactory&cloudSqlInstance={instance connection name copied from console}&useSSL=false
When the app runs locally it connects to and updates the remote Cloud SQL instance.
The Cloud SQL instance is 2nd generation, HA, and requires SSL. All of the "gcloud sql" commands I've tried succeeded. I can interact with the database using command line tools and a local Cloud SQL Proxy.
Before I click the "Enable" button on the API I'd like to understand why the API already appears in the API dashboard (with an option to disable), and why only Cloud SQL Socket Factory connections from a Google App Engine instance fail.
Configuring Google Cloud SQL to work with an app built using GWT Eclipse EE Plugin and am having authentication issues.
Google Command Line Tool - try running ./google_sql.sh instance
Need guidance setting up GoogleCL for CloudSQL. I installed regular GoogleCL using MacPorts. What next? I tried:
echo 'export PATH=Applications/eclipse/plugins/com.google.appengine.eclipse.sdkbundle_1.7.0/appengine-java-sdk-1.7.0/bin/:$PATH' >> ~/.profile
ECLIPSE
Eclipse asks for Instance Name and Database name in two separate fields. I've tried every permutation of Project ID:database name (e.g. petproject:dog).
Here is the Eclipse error report (its the same every time):
Could not connect to Profile (name-of-the-project-in-Eclipse.GoogleCloudSQL.DevInstance).
Error creating SQL Model Connection connection to Profile (name-of-the-project-in-Eclipse.GoogleCloudSQL.DevInstance). (Error: Access denied for user 'my-login-id'#'localhost' (using password: YES))
Access denied for user 'my-login-id'#'localhost' (using password: YES)
Error creating Google Cloud SQL Connection factory connection to Profile (pse:
Could not connect to Profile (name-of-the-project-in-Eclipse.GoogleCloudSQL.DevInstance). (Error: Access denied for user 'my-login-id'#'localhost' (using password: YES))
Other
I can get a test app to work just fine. I have the test app running through a domain that I own. I've created a Google Cloud SQL project and instance. I've given permission for my app to use the instance on the Google API console. I have local mysql running and created the database/schema on my machine. I can't link up my local database to the GWT because the entire Google Cloud SQL needs to function.
Thanks -- very much appreciated.