I have just started with google appengine. I am developing an app in php. If I wanted to see the result, I used
dev_appserver.py ./ --php_executable_path /usr/bin/php5-cgi
(It doesn't work without the --php_executable_path parameter for me)
It works fine, except I cannot connect to the cloud sql instance. I read this https://cloud.google.com/appengine/docs/php/cloud-sql and tried to connect to the cloud sql by:
$sql = new mysqli(null,
'root', // username
'', // password
,
null,
'/cloudsql/:'
);
If I deploy something to appengine, this works, but what should I do, if I need to debug the app localy and it depends on the database?
#DTing's answer is correct that Google's docs encourage you to use a local MySQL from the local development server and recommend that pattern.
However, if you disagree and want to run the development server against your "production" SQL in the cloud, that's supported, too (just not encouraged because a bug during development could destroy your production data!).
Specifically, you follow the general instructions at, and pointed to by, https://cloud.google.com/sql/docs/getting-started#work (ignoring the appengine-specific part): make sure your Cloud SQL instance has an IP address, enable the outside-visible IP address of your workstation, make sure the SQL instance has a root password -- then check everything is working with a command line MySQL client, e.g
[[Note: to verify your workstation's outside-visible IP address, use e.g a browser to visit a site such as http://checkmyip.com/ ]]
$ mysql --host=INSTANCE_IP --user=root --password
and once everything is set up properly you just follow the instructions at https://cloud.google.com/appengine/docs/php/cloud-sql/#PHP_Using_a_local_MySQL_instance_during_development :
To connect to a Cloud SQL instance from your development environment,
substitute "127.0.0.1" with the instance IP address. You do not use
the "/cloudsql/"-based connection string to connect to a Cloud SQL
instance if your App Engine app is running locally in the Development
Server.
If you want to use the same code locally and deployed, you can use a
Special $_SERVER keys variable (SERVER_SOFTWARE) to determine where
your code is running. This approach is shown below.
So for example if your Cloud SQL's IP address is 12.34.56.78, you'd use
$sql = new mysqli('12.34.56.78:3306',
'<username>',
'<password>',
<database-name>
);
when $_SERVER['SERVER_SOFTWARE'] is not set or does not contain Google App Engine (which means you're running on the local development server).
https://cloud.google.com/appengine/docs/php/cloud-sql/#PHP_Using_a_local_MySQL_instance_during_development
The Guestbook example above shows how your application can connect to a Cloud SQL instance when the code runs in App Engine and connect to a local MySQL server when the code runs in the Development Server. We encourage this pattern to minimize confusion and maximize flexibility.
Related
We got one GCP project contains 3 Cloud SQL instance and each one in a different GCP region for a different group of users.
What we would like is when the user login, I need to connect them to differnt instance based on a master table (another master SQL instance)?
Is that the best way to do, or we can do it differently?
Our application resides on Google app engine with Python flex env.
Thanks in advance!
Thinking about using the app engine custom domain mapping to point connect the user to different SQL instance by using different URL.
For App Engine Flexible, you can configure the Cloud SQL Proxy to support more than one Cloud SQL instance. Just use different port numbers for each SQL instance when setting up the proxy. If you are using unix sockets, just specify the instance names.
For example:
unix sockets:
./cloud_sql_proxy -dir=/cloudsql \
-instances=myProject:us-central1:myInstance,myProject:us-central1:myInstance2
Your connection string includes:
/cloudsql/myProject:us-central1:myInstance2
tcp:
./cloud_sql_proxy \
-instances=myProject:us-central1:myInstance=tcp:3306,myProject:us-central1:myInstance2=tcp:3307
The tcp method, specify the host as 127.0.0.1 and port (3306 or 3307).
I was moving all website to google cloud and encounter a performance problem.
I set up a VM instance on Compute Engine and a Cloud SQL server.
And connect the Joomla website from VM to Cloud SQL server using provided IP address. (Seems public IP)
The performance is really slow compared to the website using local database inside the VM itself.
So, my question is, is there a way to find local IP to connect to Cloud SQL since our web server is also on the Google Cloud infra itself.
Or, the only way is to stick with the database inside VM?
Update
I set up the Cloud proxy using this guide.
Can connect to mysql prompt with the proxy now.
But still cannot find a way to let joomla use this cloud proxy to connect to the database.
The fastest, easiest, and most secure way to connect to your Cloud SQL instance from your Compute instance is by using the Cloud SQL Proxy. There are multiple reasons for this, but here are the main ones:
Secure connections: The proxy automatically encrypts traffic to and from the database using TLS 1.2 with a 128-bit AES cipher; SSL certificates are used to verify client and server identities.
Easier connection management: The proxy handles authentication with Google Cloud SQL, removing the need to provide static IP addresses.
There's also the fact that you only need a static and small number of instances (1 in your case) connecting to the database, so you don't really need to overcomplicate your setup, you can just drop this binary into your instance, run it as a daemon, and instantly have a fast lane to your Cloud SQL instance (I use "fast lane" here because the traffic will go through Google Cloud's internal network).
Setting up the Cloud SQL Proxy comes down to enabling the Cloud SQL API, giving the service account of your intance access to the Cloud SQL API, making sure the binary has execution permissions (chmod +x), and giving it the connection string to the Cloud SQL instance. You seem to be having issues using the Proxy, so if you need more troubleshooting ideas, you can find them in the documentation. The tutorial you've followed should have detailed instructions on how to do these steps.
After all of that and after making sure the Proxy is running, connecting Joomla to the database should be similar to how you do it via the MySQL client. You should point your Joomla installation to localhost (or 127.0.0.1), give it a set of credentials to access the database itself (you can create database users via the Console), give your Joomla database's name, and that should be it!
Don't forget that the Proxy needs to be running in TCP mode! That should be as simple as adding =tcp:LOCAL_PORT_TO_LISTEN_ON to the connection string parameter you're passing to the Proxy. Here's an example of how to run the Proxy:
./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:3306
Virtual Private Cloud (VPC) helps to increase the performance.
Private Google access enables virtual machine (VM) instances on a subnetwork to reach Google APIs and Services using an internal IP address rather than an external IP address. You can use Private Google access to allow VMs without Internet access to reach Google services.
Here you get more details: https://cloud.google.com/vpc/docs/private-google-access
I have a dotnet core 2.0 application running in Google App Engine Flexible Environment. Within the same Google project I have a Cloud SQL - MySQL database. On the Cloud SQL Instance details page, under the Authorizations tab, it states
Apps in this project: All authorized.
However, I cannot access the database from my application unless I add the 0.0.0.0/0 route to the Authorized networks section.
What can I do to give my application db access without opening my database to the world?
Update 2018-05-21 from Jeffery Rennie (accepted answer)
App Engine now supports connecting to a Cloud SQL instance using a port number instead of a unix domain socket. So now, you can add something like this to your app.yaml:
beta_settings:
cloud_sql_instances: "your-project-id:us-central1:instance-name=tcp:5432"
And specify Host=cloudsql in your connection string in your appsettings.json:
"ConnectionString": "Uid=aspnetuser;Pwd=;Host=cloudsql;Database=visitors"
In the sample above, the port is 5432, which is the default port for a PostgreSQL database. For a MySQL database, use port 3306.
A full example with instructions for deploying to App Engine can be found here:
https://github.com/GoogleCloudPlatform/dotnet-docs-samples/tree/master/appengine/flexible/CloudSql
The ideal solution is to use a unix domain socket to connect from your app engine instance to Cloud SQL. That's how other programming languages like Python and PHP do it. Unfortunately, the MySQL connector does not work with domain sockets. I see no reason why it can't, but it doesn't. I hope they fix that issue soon.
As described in https://cloud.google.com/appengine/kb/#static-ip,
Note that using static IP address filtering is not considered a safe
and effective means of protection. For example, an attacker could set
up a malicious App Engine app which could share the same IP address
range as your application. Instead, we suggest that you take a defense
in depth approach using OAuth and Certs.
If certificates are not sufficient to protect your application, then the only remaining option I see today is to build a custom runtime that runs the Cloud SQL Proxy. The proxy can forward a local ip port number to a unix domain socket. If you have built a docker image or two, then it's not too bad.
I will update this answer as the situation improves.
Update 2018-05-21
App Engine now supports connecting to a Cloud SQL instance using a port number instead of a unix domain socket. So now, you can add something like this to your app.yaml:
beta_settings:
cloud_sql_instances: "your-project-id:us-central1:instance-name=tcp:5432"
And specify Host=cloudsql in your connection string in your appsettings.json:
"ConnectionString": "Uid=aspnetuser;Pwd=;Host=cloudsql;Database=visitors"
In the sample above, the port is 5432, which is the default port for a PostgreSQL database. For a MySQL database, use port 3306.
A full example with instructions for deploying to App Engine can be found here:
https://github.com/GoogleCloudPlatform/dotnet-docs-samples/tree/master/appengine/flexible/CloudSql
While you are not wrong that "apps in this this project: All authorized" seems to suggest you can out-of-the-box just use your App Engine app with Cloud SQL, but there are limitations.
First of all, your Cloud SQL needs to be a 2nd generation instance, and secondly, there are specific instructions that's dependent on the language you use and the App Engine type (standard or flex).
If your situation fit all the requirements, it should work.
For your specific use case, you need the .Net instructions, it does say you need to add a network with 0.0.0.0/0 access and an user account. The user authentication + SSL should provide the security you need.
I backed up my Cloud SQL V1 DB to a bucket and restored to a new 2nd-gen instance. Connecting via the console works fine. I can select, look at procs etc. Connecting to it from my Java GAE app doesn't. The GAE app and the Cloud SQL DB are in the same project.
I get these values in the logs:
GAE log:
java.sql.SQLInvalidAuthorizationSpecException: Not authorized to access instance: xxxx
SQL log:
[Warning] User 'root'#'%' was assigned access 0x1fffffff but was allowed to have only 0x1fff7dff.
Any tips?
Make sure your JDBC URL uses the jdbc:google:mysql:// prefix and not the old/deprecated jdbc:google:rdbms:// prefix.
While jdbc:google:rdbms:// continues to work for First Generation instances, it won't work when connecting to Second Generation instances. jdbc:google:mysql:// will work for both.
So Im following the google ruby guide to create and setup a cloud sql instance. Under 'Create and configure a Cloud SQL instance' step 4 it tells you to allow all network fields so the instance is open to all traffic, then underneath that it gives the warning:
This configuration leaves your Cloud SQL instance open to traffic from everyone, everywhere. It is used only for demonstration purposes. In production environments, restrict access to only those IP addresses that need access.
I haven't setup VM servers on a cloud environment before so I have no idea on what IP addresses I should be giving access to the SQL instance or what ones 'need access' do I just change it to the IP of my VMs?
In the context of the guide that you linked, the IP whitelist is necessary so you can access your Cloud SQL instance from your development server on your local computer. For that specific purpuse, you can just whitelist your computer's IP (see http://www.whatsmyip.org) instead off all the world.
When your application is going to be running on App Engine, you don't need to whitelist the IP. There is a separate access control list for that in the Cloud Console where you can list the App Engine applications authorized to connect.