I have a small Java GAE hobby project using Google's CloudSQL MySQL and Hibernate. I have already used up my $300 promotional credit, and have been surprised just how expensive the CloudSQL is. Looking closer at the billing, it seems clear that I am being billed for almost continual running. A typical month invoice might specify:
Cloud SQL DB standard Intel N1 1 VCPU running in Americas (with 30% promotional discount): 719 Hours
I am certain that the db is actually be utilized a tiny fraction of that time. What are the factors in a GAE app that might cause the CloudSQL to be "running"? Is it primarily driven by the DB connection/pooling, as managed by Hibernate? Or is it dependent upon whether or not a GAE instance is running (which I believe is configurable?)?
Here are the connection pooling settings in my persistence.xml:
<property name="hibernate.c3p0.max_size" value="100" />
<property name="hibernate.c3p0.min_size" value="1" />
<property name="hibernate.c3p0.acquire_increment" value="1" />
<property name="hibernate.c3p0.idle_test_period" value="300" />
<property name="hibernate.c3p0.max_statements" value="50" />
<property name="hibernate.c3p0.timeout" value="500" />
Cloud SQL 2nd generation instances are billed for every minute the instance is running, regardless of if the instance is being used or not.
Cloud SQL 1st generation instances had a "per use" billing option, where you were only billed for hours while the instance was being used. Support for 1st gen instances is ending on Jan 30th, 2020.
Related
My JBPMM/PAM application under JBoss has been getting intermittent optimistic lock exceptions so I wanted to make sure that my application's
Multiple last resources have been added to the current transaction
warnings are harmless. I get these warning despite using an XA driver and
<property name="com.arjuna.ats.arjuna.allowMultipleLastResources" value="true"/>
My application uses multiple datsources pointing to different databases via Hibernate and JPA. However, the XA driver used for querying our SQL Server DB seems like it should allow distributed transactions - so why does Arjuna still issue warnings? Is there a simple configuration change I can make to avoid these warnings?
<drivers>
<driver name="sqlserver" module="com.microsoft.jdbc">
<xa-datasource-class>com.microsoft.sqlserver.jdbc.SQLServerXADataSource</xa-datasource-class>
</driver>
</drivers>
The reason here is that you are working with two datasources which are part of one global transactions - i.e., those will be the two different databases.
The datasources are (most probably) configured as standard datasources and not XA datasources. Only XA datasource is capable to correctly participate within the XA transaction (transaction with multiple participants - https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.4/html-single/managing_transactions_on_jboss_eap/index#xa_versus_non_xa_transactions).
The MSSQL implements the XA tranactions thus you could just need to change the datasource configuration to be configured as an XA datasource.
https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.4/html-single/configuration_guide/index#example_microsoft_sqlserver_xa_datasource
Problem: Cloud SQL instances run indefinitely and are monetarily expensive to host.
Goal: Save money while not compromising on database availability.
It has been almost four years and Google Cloud has not fulfilled this feature request that has already been implemented on AWS with their Aurora RDS.
Since it does not seem that on demand Cloud SQL that auto-scales to zero is coming any time soon, will the following strategy work?
Have instances of Cloud SQL, a Baby and a Papa. They follow the master/slave replica principle, with twist. The Baby
instance is small with few vCPU's and low memory, it always runs, but
does so cheaply. However, the Papa instance is expensive with high vCPU and high
memory but runs only when needed.
To begin, only the Baby Cloud SQL instance is running so it is the master that accepts reads/writes. The Papa Cloud SQL instance is not running.
Since I am using standard app engine that
will auto-scale to zero with no traffic, schedule a cron job that
checks every 10 min if no app engine instances exists. In this case,
the application has no traffic. If this is not the case, the Papa Cloud SQL instance is started. Once started, the Papa instance
becomes the master that accepts reads/writes while the Baby instance
becomes a slave replica capable of only reads.
If the cron job detects the app engine has zero instances running, this means there is no traffic. Thus, the Papa Cloud SQL instance is
stopped and the Baby Cloud SQL replica is promoted to master and can accept reads/writes.
In this way, the expensive Papa instance runs on demand. If there is a traffic
spike when the Papa instance is stopped or rebooting, the Baby
instance will still be able to respond to requests.
This strategy ensures that the expensive Papa Cloud SQL instance only runs with traffic. Is this Baby-Papa dynamic possible on Google Cloud?
Cloud SQL has an Admin API that can be used to manipulate your Cloud SQL instances in such a way. You could build pieces of what you are describing using Cloud Scheduler to trigger a Cloud Function which uses the API to start and stop instances, or even promote/demote them to master.
However, it's probably a bad idea. These operations can take several minutes to complete and would give you dramatic increases to cold start times for requests. Additionally, SQL servers prefer to be long running for a reason - they use resources to cache and optimize queries to improve performance. Start, stoping, and resizing instances can cause you to lose these benefits.
It's better to consider - do you actually need a relational database? If not, it's probably better to use something like Firestore, which is a serverless product.
If you determine that you do indeed need a relational database, can you optimize your use for a smaller Cloud SQL instance? Can you cache queries using Memorystore or Firestore as listed above, or instead use the services I described above to export the results on a timed basis, which would be easier for your app to consume?
Would it be better to start and stop your Cloud SQL instance when there is no traffic? If you traffic is based around certain predictable times, you could schedule your instance to resize at the start and stop of these time periods.
Finally, if cost is really an option, you could run your own SQL server on a GCE instance. This means you have to do pretty much all of the management yourself (install, updates, maintenance, etc), but it would be cheaper.
All of these are probably much more functional solutions than trying to shoehorn non-serverless infrastructure to match a serverless workload.
There are numerous questions about using a db connection pool with Google App Engine, but a lot has changed recently. Up to this point, I could never get a connection pool to work with GAE. However, I think some recent develops may allow connection pooling to work, which may be why it is mentioned in the Google documentation (which seems to have recently been updated).
https://cloud.google.com/sql/docs/mysql/connect-app-engine
Can someone confirm that connection pools can be used?
1) We used Google Cloud SQL 1st gen and the database could deactivate (go to sleep). This would make any existing connections stale.
With a 2nd gen database, there is no deactivtion of databases. So this may address the problem.
2) Many connection pool implementations used threads.
With Java 8 being supported on GAE, it looks like threads are permitted.
3) Some people suggest that GAE's limited number of database connections (12) are a reason to use connection pools. The connection pool size could be set to GAE's limit and thus an app would never exceed the limit.
a) First, documentation indicates a much larger number of connections, based on the size of the database.
https://cloud.google.com/sql/docs/quotas
b) Second, if there is a limit for a GAE app, is the limit per individual server instance or for an entire GAE app?
Any confirmation that the above thinking makes sense would be appreciated.
Regarding 1) Yes, the Cloud SQL instances of 2nd generation, your instances don't deactivate unless it's for maintenance etc.
2) I don't see why you can't use threads to connect to a 2nd generation Cloud SQL database. With Java 8, you can absolutely do that. To check how many threads you have open, you can run mysql> SHOW STATUS WHERE Variable_name = 'Threads_connected';
For 3a), I would go with the official documentation link that you provided already but remember that database connections consume resources on the server and the connecting application. Always use good connection management practices to minimize your application's footprint and reduce the likelihood of exceeding Cloud SQL connection limits. The limit of 12 connections was indeed in place in the past but it doesn't exist anymore.
3b) When a limit or quota refers to a Google App Engine app, then it's for the whole app unless it's specified that it's per instance. More specifically for Cloud SQL connections, you can find the limits here and there is actually a limit that is specific to instances. You can't have more than 100 concurrent connections for each App Engine instance running in a standard environment.
I hope that helps!
I know this question was asked here before but why should I choose cloud sql?
I'm developing mobile game that will be relaying mostly on backend/database. I'm targeting 100.000 dau. I was looking at some options for hosting and cloud vps + cloud sql database are much more expensive that buying vps/dedicated machine.
I get that cloud take from me the administration/replication/backup part but the cost of it is insane.
For example, 4 cores 24gb ram and 100gb ssd https://www.ovh.pl/vps/vps-cloud-ram.xml cost ~45$.
For the same specs on google cloud I would need to pay more than 100$. We talking vps here which is cheaper than google cloud sql.
For example, I can have one backend server, one database server, one replication server, and one weak backup server for the price of 1 vps in google cloud.
Is this really worth it?
Am I not seeing something?
I know that my time/team time cost a lot of money but we talking a small server here if we would want to scale this up to something bigger, buying a dedicated server is even a 10% of the google cloud vps price.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I have a number of VMs on Windows Azure (Iaas) hosting a website. There are a number of load-balanced front-end VMs, all connecting to a single VM with SQL Express. It works well.
However!
I'm getting random restarts across all the VMs. As for the front-end VMs (with IIS), since they are load balanced, the site is not affected and the load balancer adjusts accordingly. But when the VM hosting the database is restarted, the site is down until the DB is up again. It takes < 3min to boot up, but that's still unacceptable if it happens frequently enough. Although the restarts are relatively rare (2 a month per VM), sometimes we get a week with 4 restarts per VM, which gets frustratingly annoying. Not all VMs restart as frequently and I cannot figure out a pattern. Restarts are also unexpected (pull-the-power-cable type of restarts, and not shutdowns). Datacenter is West Europe.
Microsoft emphasises that SLA only covers 2VMs in an availability set, which I can't have for the database VM (and the enterprise SQL edition costs an arm and three legs). Also, SQL Azure isn't an option as the application is very chatty, and the SQL Azure database was being throttled during peak times (though it works super smooth with SQL Express on a Medium VM!).
My question(s):
Is it normal to have so many restarts? Are there other people having the same problem? What is your experience with such an environment on Azure? What can I do to minimise this downtime?
Thanks all!
Is it normal to have so many restarts?
Yes this can happen in a given month, you need to stand up SQL Server in high availability mode to really get this to work.
Yes it does cost an arm and leg. ;(
What is your experience with such an environment on Azure?
Some months are really good some months are bad, depends on your cluster and which datacenter you are in. MS have mixed range our hardware out in there datacenters. That does not mean they are running on old laptops in some datacenters but it does mean in my experience the new datacenters tend to have better kit in them and thus less restarts. I.e we use USA East.
What can I do to minimise this downtime?
High availability with a witness is the only way to give you availability in VM and yes it cost and arm and leg.
Other serious options. Cache Cache ..You should use computer cache, azure cache and try to minmize your calls to the database. This might reduce your chatty app and allow you to step back in SQL Azure, but might give you enough to for the failover to recover back.
Queues Queues would help you application recover and give you user a message of we are working on it.
Use SQL Azure as failover. Data sync using SQL Azure Sync from Premise (Not sure this works with Express) to SQL Azure and write into you app code to pick up the connection error and failover.
Look at using other parts of Azure for parts of your app to reduce your amount of calls coming into SQL , i.e Can you move stuff to table storage ?
HTHS give you some ideas.
Windows Azure Infrastructure Services (IaaS) has only been in General Availability (GA, or production) about 3 weeks, since April 16 (see announcement here). Prior to GA, there was no SLA and you would have seen more frequent OS restarts as various patches were still being applied to the Host OS. Are you saying that this pattern has continued at the same velocity since April 16?
Now that IaaS is GA, I wouldn't expect 4 restarts in a week. That said: there are several reasons you'd see a restart:
Host hardware failure (this takes down all Guest OSs running on that host)
Host software update (and only if requiring a restart of the Host os). Host OS reboots shouldn't be happening at the frequency you're seeing.
Guest OS issues. Here's where things depart from PaaS (web/worker role Cloud Services). In IaaS, there's no Guest OS maintenance done by Azure; this is all in your hands. It's possible to get reboots if installing Windows Updates automatically. Possibly you could be running into an application-level issue causing the box to become unresponsive for a long period of time, resulting in the Azure fabric controller rebooting your box as it thinks it's unhealthy. And... your app could be somehow crashing the box.
If you've ruled out application error and are sure the VMs are in good health at the time they're rebooting, you may need to open a support ticket with Microsoft to help diagnose the issue further.