I am trying to use Flask SQLAlchemy to establish a database connection between Google App Engine and Google Cloud SQL.
I am able to establish the connection in a Flex environment but not Standard whenever I deploy. I am not sure why this is the case. I am receiving this error response from the Error Logs:
OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'localhost' ([Errno 97] Address family not supported by protocol)")
This is the connection string I am using to connect to the Cloud SQL database
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://<username>:<password>#/<database>?unix_socket=/cloudsql/<instanceName>'
When testing and developing locally, I am able to use this connection string to connect to the database through the cloud_sql_proxy but this connection string fails when I deploy and attempt to access it externally.
I've looked through similar questions but haven't seen any that reference this specific error response yet.
I am using the Second Gen Cloud SQL version
Related
I'm trying to use a PostgreSQL database with Prisma for the first time in my project. However, I'm failing to connect to the Cloud SQL instance I created on Google Cloud. When I run npx prisma db push, I'm getting the error:
Please make sure your database server is running at `34.86.23.139`:`5432`.
ERROR Command failed with exit code 1: prisma db push
I made sure the database server was running, and was able to successfully connect to it using Cloud Shell. The public IP address of the server is 34.86.23.139, the name of the database is test-db, and the connection name is test:us-east4:test-db. Given that the connection string should be in the formal of postgresql://USER:PASSWORD#HOST:PORT/DATABASE, I made this within .env:
//.env
DATABASE_URL="postgresql://myemail#gmail.com:mypassword#34.86.23.139/test-db"
// I also tried DATABASE_URL="postgresql://myemail#gmail.com:mypassword#34.86.23.139/test:us-east4:test-db" but same error.
And this is my schema.prisma:
// prisma/schema.prisma
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
model Thing {
id Int #id #default(autoincrement())
name String
}
Please let me know what I'm doing wrong. I'm a newbie so go easy on me :) Thank you in advance.
Cloud SQL instances are configured to block all traffic to the public IP unless you specifically allow an IP using Authorized Networks.
Alternatively, you can run the Cloud SQL Auth Proxy next to your application and it will create a listener on localhost that proxies traffic to your instance without having to configure Authorized Networks.
What I'm trying to do
I am trying to manage my SQL Server instance on Cloud SQL (GCP) with SQL Server Management Studio (SSMS).
What I've done
I followed the steps in the Google Cloud documentation (here).
I followed steps 1-5 (condensed version below):
Install Cloud SQL
Run gcloud init
Run gcloud auth login (P.S. I'm Owner on Project level)
Download and install Google Cloud SQL Proxy (as per the instructions here)
Enable Cloud SQL Admin API
Install the proxy client (Windows PC for me, x64 - renamed file to cloud_sql_proxy.exe)
Determine proxy authentication method (as per documentation here). I chose Cloud SDK since I need to download it in step 1 anyway
Skip (not using service account authentication)
Determine proxy instance specification (as per documentation here). Using Instances specified on proxy invocation method.
I only have private IP, so by default it will select the correct IP (as per documentation here)
Same as step 5 (below)
5 . Start the proxy. I ran (in cmd) ./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:1433 (obviously replacing the <INSTANCE_CONNECTION_NAME> portion with my instance name.
At this point, everything seems to be working:
Listening on 127.0.0.1:1433 for <INSTANCE_CONNECTION_NAME>
Ready for new connections
New connection for <INSTANCE_CONNECTION_NAME>
New connection for <INSTANCE_CONNECTION_NAME>
Now following the instructions here, I attempt to connect to the database instance using SSMS (127.0.0.1, SQL Server Authentication, sqlserver, my super awesome password), Connect.
My issue
I now get the following error messages in the Cloud SDK window:
couldn't connect to <INSTANCE_CONNECTION_NAME>: dial tcp <PRIVATE_IP>:<PORT>: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Throttline refreshCfg(<INSTANCE_CONNECTION_NAME>: it was only called XXXs ago
(and it repeats a few times)
In SSMS I get the following prompt:
My question
How do I fix this issue and properly connect to my database instance?
When setting the Connectivity settings for the SQL Server instance, I changed it to Private IP and unchecked the Public IP setting. Re-enabling the Public IP setting allowed me to establish a connection:
External applications can still connect to the instance through the Cloud SQL Proxy
This indirectly indicates to me that Cloud SQL Proxy needs the Public IP option to be enabled in order to establish a connection.
I'm very new to Google Cloud and running applications in general. I currently have a Django app running in a Docker container on Google Flexible App Engine that connects to a Google Cloud SQL (PostgreSQL) instance in the same project. The latest version has been running for about 3 days now without issue.
The Problem:
Today I started receiving OperationalError: server closed the connection unexpectedly errors repeatedly from the application.
I can run the Cloud SQL Proxy and it starts up normally (Ready for new connections), but if I try to connect with psql, I receive the error:
psql: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
And the proxy reports:
couldn't connect to "<instance_name>:us-central1:<instance_name>":
dial tcp <ip address>:3307: connect: connection refused
On SSHing into my running flex app instance and running sudo docker logs <cloud proxy container>, the last lines are, similarly:
couldn't connect to "<instance_name>:us-central1:<instance_name>":
dial tcp <ip address>:3307: getsockopt: connection refused
Things I've Tried/Checked
Restarted cloud sql instance. The instance itself is running fine and I can access it using cloud shell from the console.
Checked db instance name and ip address - they match.
Restarted the flex app engine instance. No change as far as I can tell.
Upgraded my local copy of cloud_sql_proxy to 1.09.
Checked quotas - I don't seem to have hit any API or simultaneous connection limits.
I'm able to connect to the sql instance by authorizing my local IP address.
I'm able to connect to a different (but very similar) Google Cloud SQL instance using the proxy locally so I'm not sure if the proxy is at fault.
Any help at all would be appreciated, at this point I'm out of ideas. Thank you!
This could also be an issue if the CloudSQL instance is configured with only a private IP address. Per a small paragraph hidden in the documentation:
The proxy does not provide a new connectivity path; it relies on existing IP connectivity. For example, you cannot use the proxy to connect with an instance using Private IP unless the proxy is using a VPC network that has been configured for private services access.
In this case, the only solution seems to be adding a public IP to the server.
I first restarted the Cloud SQL instance. That did not help. Then, I simply clicked "Stop" for the SQL instance and once it had, clicked "Start" and now it works. This is pretty random and annoying.
In my case, I had upgraded the machine type of the SQL instance earlier in the day and it seems like on doing so, Google Cloud simply "restarts" the instance where as what is needed is "stop" and then "start". This is only a guess.
tl;dr Stop and then Start the Cloud SQL Instance. Don't Restart as "Restart" != "Stop + Start"
Hope it helps others who face this random issue.
We ended up "fixing" the problem by rolling back to an earlier backup. Google Support noted "the issue started right around the maintenance window for your Cloud SQL instance, so it's possible that a change was made that caused the connection to break".
The Scenario:
A webApp (Not WebSite nor WebService) on Azure using WebAPI fails to connect to its Azure Database. There is no WebApp IP to open in the database firewall although I have opened (temporarily) 0.0.0.0 - 255.255.255.255 which did not help.
We receive the following generic error message:
"A network-related or instance-specific error occurred while
establishing a connection to SQL Server. The server was not found or
was not accessible. Verify that the instance name is correct and that
SQL Server is configured to allow remote connections. (provider: Named
Pipes Provider, error: 40 - Could not open a connection to SQL
Server)"
Running the Web App locally it connects to the Azure database fine.
The WebApp connects to the database through EF6.
So we know the connection string works, we know the database is accessible and the WebAPI works ... it only fails when the WebApp is placed on Azure.
How can we resolve this?
Ok, the solution is a little more complex than it needs to be. You can't get a webApp to communicate with a WebSql database without serious fudging. However, the correct although more costly solution is this:
Create a Ms SQL Database VM.
Create a Ms Windows VM for the API
Create a vNetwork and place both VMs on the network
Configure the API to use the internal IP of the network that belongs to the
Database VM
Voila.
I've set up a new Cloud Service in MS Azure. When I try to connect to the database, it appears an "SqlException: Network-related or instance-specific error ... Error 40: Could not open a connection to SQL Server".
This account has already set up some Cloud Services that are able to connect to the same database, but the new one I just uploaded cannot. I checked all the configurations and I can't find the solution.
In addition, I connected to the Cloud Service - Remote Desktop and I typed
telnet 1433
in order to check the connection but it fails.
Any ideas will be helpful! Thanks