I have a requirement where my application job must trigger every 10min which I configured using camel timer. Now the issue is this bundle is running in two different servers and both trigger same time as well.. is there a way to manage the timer in both servers so that it will not run at same time and do it periodically?
A very minimalist but often sufficient solution would be to use the delay option (for setting an initial delay before the first trigger) of Camel Timer.
Calculate a random initial delay on startup according to your period. In your case a random delay between 0 and 10 minutes and set it as delay.
However, with "bad luck" they can still start very close to each other.
It will be more effective to use Camel Quartz instead of Camel Timer on burki solution, the cron mask will be more effective.
For a robust solution, you should look at idempotency but it takes time to set up.
Related
I am trying to figure out how to run a service(1) when it does not receive any calls.
I want to use Microservices Architecture.
Basically i want to run this service (1) when the other service(2) is receiving calls and all data.
As the service(1) i mentioned is not receiving it would not have to spawn new instances and i would want only the service(2) to scale.
I have noticed scheduling jobs with cron yaml but the number of calls is limited.
I need to get this service(1) to be active every 1 min when service(2) is active.
It's hard to give a good answer without knowing more about what service (1) has to do when it is 'active'. It sounds you want cron to launch a task every minute.
You can use cron in conjunction with push queues: https://cloud.google.com/appengine/docs/standard/go/taskqueue/push/
When creating a push queue task, you can set the property delay before adding it to the queue: https://cloud.google.com/appengine/docs/standard/go/taskqueue/reference#Task
(For me in Python they called it countdown https://cloud.google.com/appengine/docs/standard/python/refdocs/google.appengine.api.taskqueue.taskqueue#google.appengine.api.taskqueue.taskqueue.add)
You could have a cron job that fires every 24 hrs. That cron job would load up your push queue with tasks who's delays are staggered. The delay of the first one is 1 min, the delay of the second one is 2 min, etc.
We are implementing multi-user timer with Google App Engine. When the timer ends we need to do some calculations and send results to users. Several users should be able to start and pause the timer from different browsers. We will use Channels API for communication. How could we trigger the calculations at specific time?
One idea that we have is, when the timer starts, to create a push task with eta set to timer finish time. When that task runs, check for timer state which is stored in memcache or datastore, and create another task if the state was modified due to pausing the timer. If the timer is finished the task launches the calculations. Are there better approaches, since there is no guarantee that the task will run exactly at eta?
You could have a constantly repeating task (re-enqueing itself, eventually with a delay specified in seconds) which would check the timer state and perform the calculations if appropriate or just return if not.
How can one run multiple instances of a Script Using Google App Engine's Cron system?
By default, it will run, then wait the specified interval before running again, which means that only one instance runs. What i am looking for is how one can get a script that takes 2+ minutes to run start a new instance every 30-60 seconds regardless of if it is running already or not, which does assume the script does not interfere with itself if multiple instances are running. this would effectively allow the script to deal with several times more information in the same period of time.
Edit, Completely reworded the question.
You only get resolution to the minute. To get finer-grained, you'll need instances that know whether they should handle the request from chron immediately, of if they'll have to sleep 30 seconds first. A 30 second sleep uses up half of the 60 second request deadline. Depending on the workload you expect to handle, this might require that you use Modules.
By the way, I'm not aware of any guarantee that a job scheduled for 01:00 will fire at exactly 01:00:00 (and not at, say, 01:00:03).
Since the cron service doesn't allow intervals below 1 min you'd need to achieve staggering script launching in a different manner.
One possibility would be to have a cron entry handler running every 2 mins which internally sleeps for 30 seconds (or as low as your "few seconds of each-other" requirements are) between triggering the respective script instance launches.
Note: the sleeps would probably burn into your Instance Hours usage. You might be able to incorporate the staggered triggering logic into some other long-living task you may have instead of simply sleeping.
To decouple the actual script execution from the cron handler (or the other long-living task) execution you could use dedicated task queues for each script instance, with queue handlers sharing the actual script code if needed. The actual triggering would be done by enqueueing tasks in the respective script instance queue. As a bonus you may further control each script instance executions by customizing the respective queue configuration.
Note: if your script execution time exceeds the 2 minutes cron period you may need to take extra precautions in the queue configurations as there can be extra delays (due to queueing) which could push lauching of the respective script instance closer to the next instance launch.
Working off Dave W. Smith's answer, The Line would be
every 1 minute from 00:00 to 23:59
Which means that it would create a new instance every minute, even if the script takes longer than a minute to run. It does seem that specifying seconds is not possible.
I have a jBPM process setup with a boundary timer on a human task set for 30s (for testing purposes) - this is to escalate to another task if the time expires.
This normally functions correctly - when the task is reached and 30s are up, the flow is moved to the next task.
However, if I bounce the server, it seems that none of the timers are recreated and the flow sits on that task indefinitely.
The chances of the server being bounced in the real world are fairly high, as the timeouts will be more likely to last a couple of days.
Does anyone know if this is a known issue?
How are you executing your process, using the execution server as part of jbpm-console or embedding the engine yourself?
If you are embedding the engine yourself, note that you need to reinitialize your RuntimeManager upon restart (don't wait on the first request to do this, as this won't reactivate timers).
I am using Task Queue in GAE for performing some background work for my application. I have come to know that there is a 10 minute time limit for a particular task. My concern is how do I test this thing in my local environment. I tried thread sleep but it didn't throw any exception as mentioned in google app engine docs. Also is this time limit is measured by CPU time or the actual time.
Thanks.
The time is measured in wall clock time. The development server doesn't enforce time limits, although it's unclear why you'd want to test it because it's unlikely your tests will perform the same as they will in production, so trying to guess how much you'll be able to accomplish in 10 minutes on the production servers by seeing how much you can accomplish in 10 minutes on the development server will fail horribly.
For your development server, start a timer when a task is initiated. keep checking in your code if you reached 10 mins wall clock time. When you reach, throw a DeadlineExceededError. It would be better to have the try and except statements in the class handlers which call a particular function of your code.