What exactly is a Google App Engine Instance as per their plans to change the billing model as they exit preview?
Is an instance a VM with a set of memory and a fixed amount of dedicated CPU power, memory etc or something else?
Is an instance a VM with a set of memory and a fixed amount of dedicated CPU power, memory etc or something else?
Yes, exactly that it is. A VM with a set of memory and a fixed amount of dedicated CPU power.
I can't remember how many instances a google server has but they said that in a I/O session. They also said what kind of servers google is using for App Engine ( quad core with some big amount of ram). Here it's the recording if I'm not wrong https://www.youtube.com/watch?v=7B7FyU9wW8Y
http://code.google.com/appengine/docs/adminconsole/instances.html#Introduction_to_Instances for a general description.
http://code.google.com/appengine/docs/python/config/backends.html for specifics on backends, where you have more control over the resource that you're either reserving or using on demand.
Related
In Google Cloud Platform Monitoring, is there any way to monitor memory usage by App Engine instance? I see that there is the Memory usage metric for the GAE Application resource type, but I don't see a Memory usage metric for the GAE Instance resource type.
My particular use case is that I'd like to see (and make alerts based on) memory usage per instance.
Currently there's no built-in metric to monitor per instance memory. The only way to monitor the Memory Usage within a GAE Instance is by creating a custom metric, in this link you can find more information about custom metrics.
Just remember that the amount of instances will increase according to the load, so a per instance alert shouldn't reflect the actual load of your application, that's why the memory usage is normally measured by group on the built in metric rather than per instance.
I have a few requests that needs to use extensive amount of memory i.e. 40 MB more than other requests.
At the default of 10 max concurrent requests using a F1 auto-scaling instance, it can potentially use 400+ MB, which is way more than the 130MB-ish system memory it has available. There is no memory utilization setting in the yaml file, so I wonder what can be done to prevent such situations.
Google App Engine don't have any memory utilization beside Python Garbage Collection.
My Advise is,
Try to release memory as soon as response
Try to optimize memory usage on that part, may be you need to use other service to help solving memory usage problem, eg. file serving via Google Storage, etc...
Scale up instance to F2 which more suitable for production, but you still need to optimize your memory usage for higher usage.
I am developping an application using App Engine to collect, store and deliver data to users.
During my tests, I have 4 data sources which send HTTP POST requests to the server every 5s (all requests are exactly uniform).
The server stores received data to the datastore using Objectify.
At the beginning, all requests are manage by 1 instance (class F1) with 0.8 QPS, a latency of 80ms and 80MB of memory.
But during the following hours, the used memory increases and goes over the limit of F1 Instance.
However, the scheduler doesn't start another instance. When I stop all traffic, average memory never decreases.
Now I have 150MB memory instead of 128MB (limit of F1 class) and I stopped all the traffic.
I Tried to set performance settings manually or automatic, disable Appstats without any improvement.
I use Memcache and datastore, don't have any cron or task queues and the traffic is always the same.
What are the possible reasons the average memory increase?
Is it a bug of the admin console?
Which points define the quantity of memory used per request?
Another question:
Does Google have special discount for datastore read/write ( >30 million ops / day ) ?
Thank you,
Joel
Regarding the special price, I don't think there is. If your app needs this amount of read/write quota you should look into optimizing to minimize write and perhaps implement some sort of bulk writing if possible.
On the memory issue. You should post your code in order to get a straight answer since there are too many things to look into when discussing memory usage. Knowing more about your case will help in producing a straight answer.
Cheers,
Kjartan
I have a 2 GB database and a front end that will likely handle 10-15 hits during the day. Is the AWS (with MySQL RDS) free-tier a good place to start?
Will CakePHP apps encounter time-outs or other resource issues, due to sizing of the Micro Instance?
Micro Instance (from Amazon): Micro instances (t1.micro) provide a small amount of
consistent CPU resources and allow you to increase CPU capacity in
short bursts when additional cycles are available. They are well
suited for lower throughput applications and web sites that require
additional compute cycles periodically. You can learn more about how
you can use Micro instances and appropriate applications in the Amazon
EC2 documentation.
Micro Instance 613 MiB of memory, up to 2 ECUs (for short periodic
bursts), EBS storage only, 32-bit or 64-bit platform
If you're only getting a very small amount of hits, you can probably run your application and mysql database on a micro instance.
The micro will be free, but you will have to pay for the RDS.
You should not notice any issues - we do most of our testing on micros, and our database is larger than yours.
It will work perfectly for your scenario.
I have deployed myself for other clients applications with at least twice requirements as yours and they worked fine.
If your application does operations saving and retrieving files from the disk I would like to suggest you giving a try to Amazon S3.
I am learning about the Apache Cassandra database [sic].
Does anyone have any good/bad experiences with deploying Cassandra to less than dedicated hardware like the offerings of Linode or Slicehost?
I think Cassandra would be a great way to scale a web service easily to meet read/write/request load... just add another Linode running a Cassandra node to the existing cluster. Yes, this implies running the public web service and a Cassandra node on the same VPS (which many can take exception with).
Pros of Linode-like deployment for Cassandra:
Private VLAN; the Cassandra nodes could communicate privately
An API to provision a new Linode (and perhaps configure it with a "StackScript" that installs Cassandra and its dependencies, etc.)
The price is right
Cons:
Each host is a VPS and is not dedicated of course
The RAM/cost ratio is not that great once you decide you want 4GB RAM (cf. dedicated at say SoftLayer)
Only 1 disk where one would prefer 2 disks I suppose (1 for the commit log and another disk for the data files themselves). Probably moot since this is shared hardware anyway.
EDIT: found this which helps a bit: http://wiki.apache.org/cassandra/CassandraHardware
I see that 1GB is the minimum but is this a recommendation? Could I deploy with a Linode 720 for instance (say 500 MB usable to Cassandra)? See http://www.linode.com/
How much ram you needs really depends on your workload: if you are write-mostly you can get away with less, otherwise you will want ram for the read cache.
You do get more ram for you money at my employer, rackspace cloud: http://www.rackspacecloud.com/cloud_hosting_products/servers/pricing. (our machines also have raided disks so people typically see better i/o performance vs EC2. Dunno about linode.)
Since with most VPSes you pay roughly 2x for the next-size instance, i.e., about the same as adding a second small instance, I would recommend going with fewer, larger instances than more, smaller ones, since in small numbers network overhead is not negligible.
I do know someone using Cassandra on 256MB VMs but you're definitely in the minority if you go that small.