I have an script in App engine that gets called every 10min. I am the only user.
The script pulls data from a web source does light processing and returns an image. It takes several minutes to run the first time. The source gets updated every 10min, so the next time my script runs, (10min later), it returns in a few seconds.
I'm using over 30 instance hours a day which is over the 28 free hours.
I read somewhere that every time an instance starts, it uses a minimum of 15min. (so 144x15=36hrs)
Therefore, am I better off trying to keep the instance running 24hrs (using up 24hrs) and limiting to one instance max? Perhaps setting idle_timeout to 10min. Another potential way to save would be to somehow pause my script during late night/early morning hours.
This is where you likely found the 15 minutes minimum statement and it refers to when the accrual of instance hours ends. If you look at that documentation you will see that it depends on what type of scaling you are using.
Here are my tought in all options you mentioned on your question for staying in the free tier:
Use only one instance. The problem with this approach is that giving a higher load to a single instance may increase other cost such as vCPU and memory since all the processing will be done there, also, it's likely that it's going to take longer to run the script.
Pause the script on low activity hours, this will be the ideal if the acumulated load when turning it back on it not too big.
Setting idle_timeout to 10min. This won't work if it runs every 10 min on all instances, since your app runs every 10min and app engine will only stop charging you every 15min idle, which will never happen, unless it does not run every 10min for every instance, if that is the case, it could be worth trying.
So, summing it up, all 3 options have their pros and cons, I would suggest that you test all of them and see what options suits your needs better.
Hope this helped.
Related
How can one run multiple instances of a Script Using Google App Engine's Cron system?
By default, it will run, then wait the specified interval before running again, which means that only one instance runs. What i am looking for is how one can get a script that takes 2+ minutes to run start a new instance every 30-60 seconds regardless of if it is running already or not, which does assume the script does not interfere with itself if multiple instances are running. this would effectively allow the script to deal with several times more information in the same period of time.
Edit, Completely reworded the question.
You only get resolution to the minute. To get finer-grained, you'll need instances that know whether they should handle the request from chron immediately, of if they'll have to sleep 30 seconds first. A 30 second sleep uses up half of the 60 second request deadline. Depending on the workload you expect to handle, this might require that you use Modules.
By the way, I'm not aware of any guarantee that a job scheduled for 01:00 will fire at exactly 01:00:00 (and not at, say, 01:00:03).
Since the cron service doesn't allow intervals below 1 min you'd need to achieve staggering script launching in a different manner.
One possibility would be to have a cron entry handler running every 2 mins which internally sleeps for 30 seconds (or as low as your "few seconds of each-other" requirements are) between triggering the respective script instance launches.
Note: the sleeps would probably burn into your Instance Hours usage. You might be able to incorporate the staggered triggering logic into some other long-living task you may have instead of simply sleeping.
To decouple the actual script execution from the cron handler (or the other long-living task) execution you could use dedicated task queues for each script instance, with queue handlers sharing the actual script code if needed. The actual triggering would be done by enqueueing tasks in the respective script instance queue. As a bonus you may further control each script instance executions by customizing the respective queue configuration.
Note: if your script execution time exceeds the 2 minutes cron period you may need to take extra precautions in the queue configurations as there can be extra delays (due to queueing) which could push lauching of the respective script instance closer to the next instance launch.
Working off Dave W. Smith's answer, The Line would be
every 1 minute from 00:00 to 23:59
Which means that it would create a new instance every minute, even if the script takes longer than a minute to run. It does seem that specifying seconds is not possible.
I am terribly worried why my Google App Engine Application consumes super fast to its Front End Instance Hours. It's like 1 hour a day and then my Instance hour is reach its quota. Why I am experiencing this? I already read some articles regarding on this but it seems not solved. What is the right value of Idle Instance and Pending Latency? Thanks for helping guys.
In your Application Dashboard, go to Application Settings
Under performance, check the Frontend Instance Class - An F1 will cost you one instance hour and hour, F2 will be 2, etc. You probably want it set to F1.
Set pending and idle instances to automatic-automatic - this means appengine will scale down your instances to the minimum required.
Assuming you have low volume and no particular memory or CPU requirements, these settings will allow you to run all day for free.
If you are running any backends (check under the Main -> Backends ), these will consume instance hours as well based on the type (B1, B2 etc). You can make these more cost effective by making them dynamic.
My guess is that your instances are staying active for the default 12 hours after the last activity, which, for a Cloud SQL instance in a test environment, causes a lot of extra charges. I haven't yet determined how to programmatically shutdown instances, but you can change the default idle time before shutdown in the appengine-web.xml file (for Java), or the app.yaml file (for Python). I changed my ".xml" file so that my instances shut down after five minutes of inactivity by adding the following lines immediately before the final </appengine-web-app> line:
<basic-scaling>
<idle-timeout>5m</idle-timeout>
</basic-scaling>
I found this information on the following page: https://developers.google.com/appengine/docs/java/modules/
The Python information can be found here:
https://developers.google.com/appengine/docs/python/modules/
How do I go about optimizing my Google App Engine app to reduce instance hours I am currently using/paying for?
I have been using app engine for a while and the cost has been creeping upwards. I now spend enough on GAE to invest time into reducing the expense. More than half of my GAE bill is due to frontend instance hours, so it's the obvious place to start. But before I can start optimizing, I have to figure out what's using the instance hours.
However, I am having difficulty trying to determine what is currently using so many of my frontend instance hours. My app serves many ajax requests, dynamic HTML pages, cron jobs, and deferred tasks. For all I know there could be some runaway process that is causing my instance usage to be so high.
What methods or techniques are available to allow me to gain visibility into my app to see where I am using instance hours?
Besides code changes (all suggestions in the other answer are good) you need to look into the instances over time graph.
If you have spikes and constant use, the instances created during the spikes wont go to sleep because appengine will keep using them. In appspot application settings, change the "idle instances" max to a low number like 1 (or your actual daily average).
Also, change min latency to a higher number so less instances will be created on spikes.
All these suggestions can make an immediate effect on lowering your bill, but its just a complement to the code optimizations suggested in the other answer.
This is a very broad question, but I will offer a few pointers.
First, examine App Engine's console Dashboard and logs. See if there are any errors. Errors are expensive both in terms of lost business and in extra instance hours. For example, tasks are retried several times, and these reties may easily prolong the life of an instance beyond what is necessary.
Second, the Dashboard shows you the summary of your requests over 24 hours period. Look for requests with high latency. See if you can improve them. This will both improve the user experience and may reduce the number of instance hours as more requests can be handled by each instance.
Also look for data points that surprise you as a developer of your app. If you see a request that is called many more times that you think is normal, zero in on it and see what it is happening.
Third, look at queues execution rates. When you add multiple tasks to a queue, do you really need all of them to be executed within seconds? If not, reduce the execution rate so that the queue never needs more than one instance.
Fourth, examine your cron jobs. If you can reduce their frequency, you can save a bunch of instance hours. If your cron jobs must run frequently and do a lot of computing, consider moving them to a Compute Engine instance. Compute Engine instances are many times cheaper, so having such an instance run for 24 hours may be a better option than hitting an App Engine instance every 15 minutes (or even every hour).
Fifth, make sure your app is thread-safe, and your App Engine configuration states so.
Finally, do the things that all web developers do (or should do) to improve their apps/websites. Cache what can be cached. Minify what needs to be minified. Put images in sprites. Split you code if it can be split. Use Memcache. Etc. All of these steps reduce latency and/or client-server roundtrips, which helps to reduce the number of instances for the same number of users.
Ok, my other answer was about optimizing at the settings level.
To trace the performance at a granular level use the new cloud trace relased today at google i/o 2014.
http://googledevelopers.blogspot.com/2014/06/cloud-platform-at-google-io-enabling.html
1-2 days ago startup time increase from 1-3 second to 10-30 seconds with same code.
I use python 2.7 multithreading.
Code in this request read one value from memcache and return it to user. If memcache empty it read and render simple html-template from template in local filesystem. Both use equal cpu_ms.
Same code work fine in test application. Startup time in test application about 1-2 second.
I send production issue yesterday in night, but don't receive answer.
I try change instance type from F1 to F4, startup time for F4 8-10 seconds.
AppID of my app: f1f2ru
Log record before problem:
Log record at start problem:
Log record now:
Log record in test app:
Without logs etc it's hard to know. But who knows, perhaps Google is running low on resources and your free apps are paying the price.
https://developers.google.com/appengine/docs/adminconsole/instances#Loading_Requests
Currently I get about a 5 second startup time from cold on one of my very simple apps. 30 seconds does seem a long time however.
https://developers.google.com/appengine/docs/python/config/appconfig#Warmup_Requests
App Engine frequently needs to load application code into a fresh instance. This happens when you redeploy the application, when the load pattern has increased beyond the capacity of the current instances, or simply due to maintenance or repairs of the underlying infrastructure or physical hardware.
Or just pay:
https://developers.google.com/appengine/docs/adminconsole/performancesettings?hl=en
The Idle Instances sliders control the minimum and maximum number of idle instances available to your application at any given time.
The upper slider sets the minimum number of idle instances:
Note: In order to specify the minimum number of idle instances, you must have a paid app.
As usual, the more you pay the better the service.
Are the libraries the same from before or after? Adding libraries to the source code can and often does increase start up time (AppEngine needs to load them). It is something easily overlooked, and some of them are really really bloated.
I don't know reason of first increase startup time.
Problem of increase operations time is my mistake:
my application do query:
object.date < datetime.datetime.now()
and my datastore have no objects with date < datetime.now(), but it have objects with date = None (null). I forget about null it is normal, indexed value and null < any value.
I am using Task Queue in GAE for performing some background work for my application. I have come to know that there is a 10 minute time limit for a particular task. My concern is how do I test this thing in my local environment. I tried thread sleep but it didn't throw any exception as mentioned in google app engine docs. Also is this time limit is measured by CPU time or the actual time.
Thanks.
The time is measured in wall clock time. The development server doesn't enforce time limits, although it's unclear why you'd want to test it because it's unlikely your tests will perform the same as they will in production, so trying to guess how much you'll be able to accomplish in 10 minutes on the production servers by seeing how much you can accomplish in 10 minutes on the development server will fail horribly.
For your development server, start a timer when a task is initiated. keep checking in your code if you reached 10 mins wall clock time. When you reach, throw a DeadlineExceededError. It would be better to have the try and except statements in the class handlers which call a particular function of your code.