I'm hosting my back end project on Google Cloud (App Engine Flex Instance), for now I have only 10 users but they charge me 250$ per month now, because I use several core, and so I used 2400 hours of accumulated instance time. Insane for only 10 users and not so much traffic!
Can I reduce or limit the number of core used by my back end?
As you can see here, the price for App Engine Flexible is computed as vCPU per core hour of usage. Basically, it does not matter if users reach your back-end project. It matters only if many users reach your App Engine Flexible deployment, increasing the number of resources required to serve them, thus increasing the price.
Yes, you can reduce the number of cores used in the back end, through the resource settings of your app.yaml configuration file. You might also want to check service scaling settings, to control the way App Engine Flexible assigns more resources based on your service's demands.
Related
I want to host an application in the google engine, the purpose of my application is to get data from a different server once in a day. So I don't want my application to work full time in day(As it is costlier). I just want to run the application for an hour in a day and pay only on that basis. Is it possible to do it in that way?
All depend of your needs. App Engine standard has 28H free per day of F1 instance type, and 9H of B1 instance type (depend of your scaling type).
App Engine flexible doesn't have free quota -> This is expensive for some hours per days.
If App Engine standard matches your need, take care of the timeout. Each request can last more than 60 secondes. You can use Cloud Task to defer background task. By the way, a task can last up to 24h.
If App Engine standard doesn't match, because of language limitation, third party library/binary limitation and you need to have a container and that's why you need to run on App Engine Flexible, you can consider Cloud Run. However, the request are limited to 15 minutes and you have a generous free tier.
So, provide more inputs if you want more advice.
I am aware that Google cloud has an IaaS offering which allows point click installation of drupal sites using Bitnami. But I am wondering how this compares with using App Engine or container engine (instead of IaaS).
Per OP--
Large is defined as: "upto 1 million daily users visits to the website and growing"
App Engine, Kubernetes Engine (formerly Container Engine), and Compute Engine are tools that solve different problems.
From what you have shared about your use case, you have the following requirements:
Migrate a large (single) Drupal website
Ensure that it scales beyond 1 million user visits per day
All 3 products will support these requirements.
The important factors that you will need to consider when evaluating these products are:
Cost
Maintenance Effort
When evaluating the products against these two factors in the context of your requirements, you might reach a clearer winner.
For the cost ratings below, it is a best guess using you high level requirements.
Google App Engine - allows you to easily deploy, scale, and update your application. It takes care of all of the "system administration" tasks for you. Rolling out updates to deployed services is also done quite easily via the command line.
Cost is "medium" when compared to running a small single Compute Engine VM but App Engine only provisions what it believes you need to serve the traffic you are receiving, as Drupal requires a DB (can be managed by Google Cloud SQL), this will need to be provisioned separately and will add to the total cost.
Maintenance Effort is "low" because App Engine handles all of the operations tasks.
Google Kubernetes Engine - is a managed Kubernetes providing "Container Infrastructure as a Service". Great for when you have various sized services and you want to bin pack and utilize a cluster of machines for different services. Given you are running a single sized application - a Drupal website, you won't really be taking advantage of this. Scaling and administration is also a little bit more complicated as you will need to scale your Pod as well as the Kubernetes Engine cluster.
Cost is "low" as Kubernetes Engine zonal masters are free and you are only charged for the compute resources of the nodes.
Maintenance Effort is "high" as you will have to configure scaling of your cluster and pods. Operate the Kubernetes Engine cluster and manage your application through the Kubernetes API. A lot more moving parts, when compared to App Engine.
Google Compute Engine - as mentioned, there are Cloud Launcher options to run Drupal. However, once the instance is deployed, you will need to take care of updating. Also, that particular Cloud Launcher version deploys to a single VM. You will probably want to setup Load Balancing, a Managed Instance Group, and Cloud SQL to ensure it is highly available and will scale.
Cost is "low" as you could run your Drupal application and DB on a single small VM, however it may not scale.
Maintenance Effort is "high" as at a minimum, you will have to apply OS patches. To have High Availability and scaling you will have to configure a managed instance group, load balancer, and a startup script/VM image.
In conclusion, if you are an experienced System Administrator, you might prefer to deploy to Compute Engine and manage the system yourself for a potentially lower cost.
If you don't want any operational overhead, App Engine is the way to go.
Kubernetes Engine is too sophisticated a tool for the problem you are solving.
I'm redesigning a monolith application into a MicroServices architecture and am hoping to use Google Cloud Platform (GCP) to host the entire solution. I'm having a very hard time understanding their costing breakdown, and am concerned that my costs will be uncontrollable after I build it. This is for a personal project but I'm hoping will have many users after I launch so I want to get the underlying architecture right and at the same time have reasonable costs initially when I launch.
Here is my architecture:
MicroServices 1 - 4 (Total 4 API Services):
Runs on App Engine
Exposes a REST API and saves data to DataStore
Initially each API should get hit around 200 times a day
MicroService 5 (Events triggered API Service):
Runs on App Engine
Listens for PubSub events and saves to DataStore (basically I have a sensor that pushes data to this Service for storage)
Initially the PubSub should receive events around 200 times a day
MicroService 6-7 (Total 2 UI Services):
Runs on App Engine
These are UIs so people can login and use the systems. The UIs are lightweight frond end apps that use the REST Services above to populate user data in a nice way.
Each UI Service should be used around 3 hours a day
So in Total I have 7 MicroServices with each running as AppEngine "Services" in a single GCP "Project". A DataStore is shared between these APIs within this Project.
As I have 7 App Engine instances running, and they only need to be operational for a short period of time per day, how does the pricing work?
I want to use App Engine because it's completely Managed, which is one of my design requirements. But I'm hoping AppEngine has some kind of Sleep Mode, so that when there is no usage it does not bill?
Any help in understanding what my monthly costs would be would be appreciated.
Thanks very much.
Update 8/2/2017
I've decided to stay out of GCP for now. As I hope to have 7 App Engines Services running in Flex (as they are node.js) I don't seem to get access to a free tier or the ability to scale idle services to 0 instances.
This means I'll be paying full price for these services. (i.e. 7 X Full App Engine VM Cost per Monthly :O )
This is an expense I cant have just for a POC of a proper MicroService design. Instead I'm going to continue with my MicroService design but use a 10$ DigitalOcean box and Dokku to containerise my Services. If this works well and I have a need I will migrate this design to GCP (or AWS)
The full outline of App Engine instance handling is available at https://cloud.google.com/appengine/docs/python/how-instances-are-managed .
In short, your best bet is to enable automatic scaling and set
max_idle_instances = 0
in your app.yaml.
That means that your app will autoscale to handle traffic as needed and shut down the instances afterwards. Also
When settling back to normal levels after a load spike, the number of idle instances can temporarily exceed your specified maximum. However, you will not be charged for more instances than the maximum number you've specified.
Later - when load time becomes more important you can set min_idle_instances to a more suitable number - this allows for responsive apps.
am concerned that my costs will be uncontrollable after I build it
You should be aware that automatically scalable GAE apps always have cost components dependent on the external user request patterns which are not controllable.
For example, in the standard GAE env, the way those 200 requests/day are distributed matters significantly:
if they are evenly distributed they will come in less than 15 min apart - the minimum billed time per instance lifetime, so the respective service will be billed for minimum 24 instance hours per day (very close to the daily 28 free instance-hours/day for billed apps, only a single-service app using the smallest instance class can fit in it).
if they are all received within a 15 minutes interval the service will be billed for 0.5 instance hours daily (which can easily fit in the free daily quota even with multiple services and/or with more powerfull instance classes).
The actual scalability configuration of each service can matter as well. See, for example,
The only way to keep costs under strict control is via the daily budget configuration (but hitting that limit means your app's functionality will be temporarily crippled).
All other usage-based costs being equal due to the functionality being performed you have some (potentially significant) control over costs via:
the GAE environment type selected for each service:
the standard env is billed by instance hours and includes a free daily quota
the flex env has no free daily quota.
the number of services: you could start with fewer services by combining their functionalities (you can still keep them modularized for later split). The expected initial load you describe can easily fit within the free daily budget with just a single standard env service.
Once the app usage picks up and the free daily quotas percentage in the total costs become neglijible you can gradually split the app into multiple services as needed. In general this can be a relatively simple task if the app is properly modularized.
I'm very new to using Google cloud services. As I can see Google App Engine has two way of deployment. The first one is using App Engine sandbox and the second is managed VM.
So I'm interested in pricing. Is there any difference in price? For example if I choose managed VM instead sandbox.
Pricing for the Sandbox can be found on the App Engine Pricing page.
Pricing for the Managed VM's can be found on the Compute Engine Pricing page:
While in beta, pricing is based on Compute Engine Pricing for each VM. Pricing will change in the future.
Based on the linked Price pages:
The Minimum cost for a Sandboxed instance: $0.05 / hour (F1 class, 128MB RAM, 600MHz CPU)
The Minimum cost for a Managed VM: $0.063 (n1-standard-1, 3.75GB RAM, 2.75GCEU CPU)
For other classes of sandbox instances see this page: Adjusting Application Performance
Before jumping to the conclusion that Managed VMs are cheaper: each app gets 28 free instance-hours per day for Sandbox, so chances are good you won't even have to pay for any. Also with further configuration you can achieve to only pay for further "used" instance hours (e.g. you can play with min_idle_instances and max_idle_instances in your module config so additional instances only count toward instance hour billing when they are active (serving requests)).
Notes:
Price shouldn't be the only (or most important) reason to chose one over the other. They are for different things with quite different characteristics.
Sandbox instances are primarily for your application front-end: they can automatically scale as your traffic changes/grows. Many restrictions are enforced.
Managed VMs are good for background operations which can be long or CPU consuming, most of the restrictions are not applied.
In a nutshell: MVMs are for the same price about 10x more powerful.
Default GAE instances are F1: 600Mhz, 128MB, $0.05/h
Comparable MVM: 2500Mhz, 3.75GB, $0.05/h (typical use)
It's not about price, it's about environment. You would not have SQL or root FS on GAE, you can't open ports. Your code base would be hard to migrate to another VPS. As a trade-off you get zero cost maintenance and effortless instant scaling.
I'm doing a prototype backend and in the near future I expect little traffic but while testing I consumed all my 300$ free trail.
How can I configure my app to consume the least possible resources? I need things like limiting the number of instances to 1, using a cheap machine, sleep whenever possible, I've read something about Client vs Backend intances.
With time I'll learn the config that best suits me, but now I need the CHEAPEST config to get going.
BTW: I am using managed-vms with Dart.
EDIT
I've been recommended to configure my app.yaml file, what options would you recommend to confront this issue?
There are two train of thought for your issue.
1) Optimization of code: This is very difficult for us as we are not privy to your App's usage and client-base and architecture. In general, it depends on what Google App Engine product you use the most, for example: Datastore API call (fetch, write, delete... etc...), BigQuery and Cloud SQL. Even after optimization, you can still incur a lot of cost depending on traffic.
2) Enforcing cheap operation: This is easier and I think this is what you want. You can manually enforce a daily budget (in your billing setup page) so the App never cost more than a certain amount per day. You can also artificially lower the maximum amount of idling instances to 0 and use the smallest instance possible (F1 for frontend).
For pricing details see this article - https://cloud.google.com/appengine/pricing#Billable_Resource_Unit_Costs
If you use managed VM -- you'll be billed for Compute Engine Instance prices, not for App Engine Instances, and, as I know, the minimum possible instance to use as Managed VM is "g1-small" which costs you $0.023 per hour full sustained usage (if it will be turned on all month), so you minimum bill will be 0.023 * 24 * 30 = $16.56 only for instance hours. Excluding disk and traffic. With minimum amount of datastore operations you may stay on free quota.
Every application consumes resources differently. To minimize your cost, you need to know what resources used the majority of your expenses and go from there.
If it is spent on extra instances that were just sitting there - then trim the number of instances to the minimum required and use a lower class instance. If you are seeing a lot of expense on datastore calls - then look at optimizing your entities and take advantage of memcache.
Lowest Cost for a simple app:
Use App Engine Standard. It scales to zero instances, so will not cost anything if there is no traffic. With App Engine Flex you will pay for the instance hours and the Flex (GCE) instances are bigger.
Use autoscaling with max instances, F1 instance class:
With autoscaling you do not need to guess how many instances you need. F1 are the smallest instances. Set the max instances in case you get DoS'd or more traffic than you can afford.
Stop Instances:
You can stop the App Engine versions when you do not expect the app to be used. The will be no charge for instance hours for either Standard or Flex. For Flex there will be disk charges. The app will be ready to go when you need it again.
App Engine Version Cleanup:
Versions are easy to create and harder to remove. Here is a post on project cleanup. See this post on App Engine cleanup
https://medium.com/google-cloud/app-engine-project-cleanup-9647296e796a