Querying Amazon Web Services RDS via Google App engine Standard Environment seems to be very slow and is proportional to number of records that are being fetched.
For example, if the query returns one record, it takes 100ms and if it returns 10 records, it takes 1 sec. This is slowing our apis down a lot.
Has anyone else faced this? If yes, what have you to done to sort the issue?
PS: We were on google-cloud-sql prior to migrating to aws rds. Migrating due to financial constraints.
Related
Problem: Cloud SQL instances run indefinitely and are monetarily expensive to host.
Goal: Save money while not compromising on database availability.
It has been almost four years and Google Cloud has not fulfilled this feature request that has already been implemented on AWS with their Aurora RDS.
Since it does not seem that on demand Cloud SQL that auto-scales to zero is coming any time soon, will the following strategy work?
Have instances of Cloud SQL, a Baby and a Papa. They follow the master/slave replica principle, with twist. The Baby
instance is small with few vCPU's and low memory, it always runs, but
does so cheaply. However, the Papa instance is expensive with high vCPU and high
memory but runs only when needed.
To begin, only the Baby Cloud SQL instance is running so it is the master that accepts reads/writes. The Papa Cloud SQL instance is not running.
Since I am using standard app engine that
will auto-scale to zero with no traffic, schedule a cron job that
checks every 10 min if no app engine instances exists. In this case,
the application has no traffic. If this is not the case, the Papa Cloud SQL instance is started. Once started, the Papa instance
becomes the master that accepts reads/writes while the Baby instance
becomes a slave replica capable of only reads.
If the cron job detects the app engine has zero instances running, this means there is no traffic. Thus, the Papa Cloud SQL instance is
stopped and the Baby Cloud SQL replica is promoted to master and can accept reads/writes.
In this way, the expensive Papa instance runs on demand. If there is a traffic
spike when the Papa instance is stopped or rebooting, the Baby
instance will still be able to respond to requests.
This strategy ensures that the expensive Papa Cloud SQL instance only runs with traffic. Is this Baby-Papa dynamic possible on Google Cloud?
Cloud SQL has an Admin API that can be used to manipulate your Cloud SQL instances in such a way. You could build pieces of what you are describing using Cloud Scheduler to trigger a Cloud Function which uses the API to start and stop instances, or even promote/demote them to master.
However, it's probably a bad idea. These operations can take several minutes to complete and would give you dramatic increases to cold start times for requests. Additionally, SQL servers prefer to be long running for a reason - they use resources to cache and optimize queries to improve performance. Start, stoping, and resizing instances can cause you to lose these benefits.
It's better to consider - do you actually need a relational database? If not, it's probably better to use something like Firestore, which is a serverless product.
If you determine that you do indeed need a relational database, can you optimize your use for a smaller Cloud SQL instance? Can you cache queries using Memorystore or Firestore as listed above, or instead use the services I described above to export the results on a timed basis, which would be easier for your app to consume?
Would it be better to start and stop your Cloud SQL instance when there is no traffic? If you traffic is based around certain predictable times, you could schedule your instance to resize at the start and stop of these time periods.
Finally, if cost is really an option, you could run your own SQL server on a GCE instance. This means you have to do pretty much all of the management yourself (install, updates, maintenance, etc), but it would be cheaper.
All of these are probably much more functional solutions than trying to shoehorn non-serverless infrastructure to match a serverless workload.
Im using the Google Cloud Run service to host my Spring application in a docker container. The database is running in the Google SQL service. My problem is that the requests from the application to the database can take up to 2 minutes. See the Google Cloud Run log (the long requests are painted yellow). And here's the Dockerfile and Docker Compose File
The database is quite empty, it contains about 20 tables but each contains only few rows, so no request is bigger than few kB. And to make it more strange, after re-deploying the application the requests are fast again. But after few minutes, hours or even after a whole day the requests slow down again. When I start the application on my local machine the requests are always fast (to my local SQL and Google SQL instance), never had any slow connection. All actions within my application that doesn't require any DB request are still fast and takes only few ms.
Both services are running in the same region (europe-west) and CPU usage of the run service is never higher than 15%, of the Google SQL never above 3%. The Google SQL uses 1 CPU and 3.75GB, the Google run service has 4GB RAM and 2CPUs. But increasing the power of the Google Run Service and Google SQL doesn't improve the request latency. Google Cloud SQL is using MySQL 5.7 (like my local DB).
And after seeing the logs only warnings are shown in the filtered Google SQL log (I really dont know why this happens). Additionally here are my DB connection settings in the Spring config. But I dont think this has any impact, the config works perfect when connecting my local application to my local SQL instance or to the Google SQL instance.
But maybe one of you has an idea?
While not a real answer, there is a bug filed at Google that is tracking the issue:
https://issuetracker.google.com/issues/186313545
This is really hurting our customers experience and makes us loose trust in the service quality of cloud run. Even more so if there is no feedback from Google to know if they are even addressing the issue.
Edit:
The issue now seems to be resolved, according to the interactions in https://issuetracker.google.com/issues/186313545
I have recently completed two migrations from Hadoop to Snowflake on AWS. I am now moving on to work with another customer where snowflake is on Azure. I wanted to know if there are any feature difference between Snowflake on AWS and snowflake on Azure.
Thanks
not in terms of Snowflake - most customers pick depending on the other systems they use and the deployment (to avoid the extra costs related to data transfer - Snowflake doesn't charge for data load, but AWS and Azure do charge to get the data into the correct deployment if it's not there already)
Metadaten Operations in Azure are slightly slower.
And the maximum filesize ist limited to 256 MB while its 5 GB on AWS and Google.
I know this question was asked here before but why should I choose cloud sql?
I'm developing mobile game that will be relaying mostly on backend/database. I'm targeting 100.000 dau. I was looking at some options for hosting and cloud vps + cloud sql database are much more expensive that buying vps/dedicated machine.
I get that cloud take from me the administration/replication/backup part but the cost of it is insane.
For example, 4 cores 24gb ram and 100gb ssd https://www.ovh.pl/vps/vps-cloud-ram.xml cost ~45$.
For the same specs on google cloud I would need to pay more than 100$. We talking vps here which is cheaper than google cloud sql.
For example, I can have one backend server, one database server, one replication server, and one weak backup server for the price of 1 vps in google cloud.
Is this really worth it?
Am I not seeing something?
I know that my time/team time cost a lot of money but we talking a small server here if we would want to scale this up to something bigger, buying a dedicated server is even a 10% of the google cloud vps price.
Let me explain the problem. I have a magento project with 3 million products and more than 6 million urls. The problem ist only the database because of this much products. I would like to only autoscale the google cloud sql database. Then it would always respond adequate. I know its possible for google compute engine and that also includes the database. So is it possible with google or another cloud sql provider?
You cannot scale databases the same way Autoscaler is doing for Compute Engine managed instances. Autoscaling capabilities of Compute Engine works for stateless VMs. Databases are stateful. You can use read replications to scale Cloud SQL. Read Replica instances allow data from the master instance to be replicated to one or more slaves. This setup can provide increased read throughput. Visit this artcile for different read replica scenarios.