The app engine dashboard has an Instances view, but it shows the instances in hourly interval. Is there a way to set this interval to day, so I can see the daily sum of instance usage for the last week, for example?
I tried clicking the gray pills, but nothing happens. The arrow on the right reveals the 3 metrics shown, but they cannot be clicked either.
To monitor the instance count of my app engine services up to 30 days, I go to [1].
There you can filter by service name and time.
[1] https://console.cloud.google.com/appengine/instances
Here you may find an overview of all the Monitoring tools GCP offers for your Application Performance Management.
Additionally, you can use the Metrics Explorer to display App Engine Metrics, while all versions of your GAE will be concurrently displayed in one chart.
There you may select the metric you would like to display, as for example: "Memory Usage". Furthermore, you can reduce the amount of data returned for a metric by combining data from similar time series, using the "Group By" and "Aggregator" filtering methods.
I hope this information helps.
Related
I'm redesigning a monolith application into a MicroServices architecture and am hoping to use Google Cloud Platform (GCP) to host the entire solution. I'm having a very hard time understanding their costing breakdown, and am concerned that my costs will be uncontrollable after I build it. This is for a personal project but I'm hoping will have many users after I launch so I want to get the underlying architecture right and at the same time have reasonable costs initially when I launch.
Here is my architecture:
MicroServices 1 - 4 (Total 4 API Services):
Runs on App Engine
Exposes a REST API and saves data to DataStore
Initially each API should get hit around 200 times a day
MicroService 5 (Events triggered API Service):
Runs on App Engine
Listens for PubSub events and saves to DataStore (basically I have a sensor that pushes data to this Service for storage)
Initially the PubSub should receive events around 200 times a day
MicroService 6-7 (Total 2 UI Services):
Runs on App Engine
These are UIs so people can login and use the systems. The UIs are lightweight frond end apps that use the REST Services above to populate user data in a nice way.
Each UI Service should be used around 3 hours a day
So in Total I have 7 MicroServices with each running as AppEngine "Services" in a single GCP "Project". A DataStore is shared between these APIs within this Project.
As I have 7 App Engine instances running, and they only need to be operational for a short period of time per day, how does the pricing work?
I want to use App Engine because it's completely Managed, which is one of my design requirements. But I'm hoping AppEngine has some kind of Sleep Mode, so that when there is no usage it does not bill?
Any help in understanding what my monthly costs would be would be appreciated.
Thanks very much.
Update 8/2/2017
I've decided to stay out of GCP for now. As I hope to have 7 App Engines Services running in Flex (as they are node.js) I don't seem to get access to a free tier or the ability to scale idle services to 0 instances.
This means I'll be paying full price for these services. (i.e. 7 X Full App Engine VM Cost per Monthly :O )
This is an expense I cant have just for a POC of a proper MicroService design. Instead I'm going to continue with my MicroService design but use a 10$ DigitalOcean box and Dokku to containerise my Services. If this works well and I have a need I will migrate this design to GCP (or AWS)
The full outline of App Engine instance handling is available at https://cloud.google.com/appengine/docs/python/how-instances-are-managed .
In short, your best bet is to enable automatic scaling and set
max_idle_instances = 0
in your app.yaml.
That means that your app will autoscale to handle traffic as needed and shut down the instances afterwards. Also
When settling back to normal levels after a load spike, the number of idle instances can temporarily exceed your specified maximum. However, you will not be charged for more instances than the maximum number you've specified.
Later - when load time becomes more important you can set min_idle_instances to a more suitable number - this allows for responsive apps.
am concerned that my costs will be uncontrollable after I build it
You should be aware that automatically scalable GAE apps always have cost components dependent on the external user request patterns which are not controllable.
For example, in the standard GAE env, the way those 200 requests/day are distributed matters significantly:
if they are evenly distributed they will come in less than 15 min apart - the minimum billed time per instance lifetime, so the respective service will be billed for minimum 24 instance hours per day (very close to the daily 28 free instance-hours/day for billed apps, only a single-service app using the smallest instance class can fit in it).
if they are all received within a 15 minutes interval the service will be billed for 0.5 instance hours daily (which can easily fit in the free daily quota even with multiple services and/or with more powerfull instance classes).
The actual scalability configuration of each service can matter as well. See, for example,
The only way to keep costs under strict control is via the daily budget configuration (but hitting that limit means your app's functionality will be temporarily crippled).
All other usage-based costs being equal due to the functionality being performed you have some (potentially significant) control over costs via:
the GAE environment type selected for each service:
the standard env is billed by instance hours and includes a free daily quota
the flex env has no free daily quota.
the number of services: you could start with fewer services by combining their functionalities (you can still keep them modularized for later split). The expected initial load you describe can easily fit within the free daily budget with just a single standard env service.
Once the app usage picks up and the free daily quotas percentage in the total costs become neglijible you can gradually split the app into multiple services as needed. In general this can be a relatively simple task if the app is properly modularized.
Where can I find the total front-end instance hours that I have used in my previous days? It seems I can only see today's total.
AFAIK the actual instance-hours numbers are not directly available, at least not in the developer console.
What you might find helpful would be the historical graph of the instance usage, for the last (up to) 30 days, which you'll find in the Dashboard after selecting the Instances display mode, the desired timescale and the billed instance estimate graph:
Hover the mouse cursor over the graph and the actual billed instance estimate value will be displayed below the graph and the corresponding date and time is displayed in the top right corner. Note that there is some noticeable delay in updating the values, tho.
The graph detail level depends on the selected timescale, the coarsest being ~2h (on the 30 days timescale), so if you need more than just a general idea about usage trending you'd need to take care of:
averaging and/or integrating on a daily basis
accounting for the free daily quotas
accounting for instance class (I'm unsure if that's already taken into account or not)
You could also try the billing date export feature. From
Export billing data to a file:
You can export your daily usage and cost estimates automatically to a
CSV or JSON file stored in a Google Cloud Storage bucket you
specify. You can then access the data via the Cloud Storage API, CLI
tool, or Google Cloud Platform console
...
Alternatively, you can export detailed data to a Google BigQuery
dataset. For more information, see Export billing data to
BigQuery.
Note: I haven't actually tried this export feature. so I can't tell if the instance hours values are in there. There's also the still open GAE issue 10716 which suggests GAE stats might not be included.
I aim to create an application that will be deployed thanks to Google App Engine.
Before that I would like to calculate the cost of Google App Engine.
For this I have to provide the Number of instances, per hour. How can caluculate this number of Instances ?
To reformulate, imagine I have 1 thousand users connected, how many users can 1 instance afford ?
Thank you for your answer and help
Regards
Benoit
It's really hard to answer without more info. App engine consumption will depend on the nature of your app, the average session span and usage hours, how well you optimize your implementation and so much more variables...
Don't think about how many instances, think about how many instance hours you'll need to serve your users if you can figure how much you'll be consuming you could get an estimate using the App engine pricing calculator .
Completely depends on the code that runs on each request. You need to make it more specific to get a better answer. Ive had carefully implemented apps that receives thousands of requests per minute with a single instance. How long each request takes and the memory it consumes is key as well as the frontend instance type you select to use.
Appengine also lets you calibrate some parameters like min/max latency to control when more instances are launched.
I understand that Flurry gives analytics including: user retention, daily users, average active session time etc. But can I use it or some other service to tell me how many users have clicked a certain button, how much time users spend on a specific view etc?
Yes, you can. Localytics (I work there), Flurry and other app analytics services support the recording of "events" that do exactly what you described. The events also have "attributes" that you can use to track additional details, like time (which we generally recommend bucketing -- ie, 0-5 seconds, 6-10 sections, etc.). Here's a link to our integration docs: http://www.localytics.com/docs/iphone-integration/
You might also want to track screens, and then combine events and screens into into conversion funnels.
I am building a demo for a banking application in App Engine.
I have a Users table and Stocks table.
In order for me to be able to list the "Top Earners" in the application, I save a "Total Amount" field in each User's entry so I will later be able to SELECT it with ORDER BY.
I am running a cron job that runs over the Stocks table and update each user's "Total Amount" in the User's table. The problem is that I often get TIMEOUTS since the Stocks table is pretty big.
Is there anyway to overcome the time limit restriction in App Engine, or is there any workaround for these kind of updates (where you MUST select many entries from a table that result a timeout)?
Joel
The usual way is to split the job into smaller tasks using the task queue.
You have several options, all will involve some form of background processing.
One choice would be to use your cron job to kick off a task which starts as many tasks as needed to summarize your data. Another choice would be to use one of Brett Slatkin's patterns and keep the data updated in (nearly) realtime. Check out his high performance data-pipelines talk for details.
http://code.google.com/events/io/2010/sessions/high-throughput-data-pipelines-appengine.html
You could also check out the mapper api (app engine map reduce) and see if it can do what you need.
http://code.google.com/p/appengine-mapreduce/