In Task Queues code is executed to connect to the server side
through URL Fetch.
My file queue.yaml.
queue:
- Name: default
rate: 10 / m
bucket_size: 1
In such settings, Tusk performed all at once, simultaneously.
Specificity is that between the requests should be delayed at least 5
sec. Task must perform on stage with a difference> 5 sec. (but
does not parallel).
What are the values set in queue.yaml?
You can't specify minimum delays between tasks in queue.yaml, currently; you should do it (partly) in your own code. For example, if you specify a bucket size of 1 (so that more than one task should never be executing at once) and make sure the tasks runs for at least 5 seconds (get a start=time.time() at the start, time.sleep(time.time()-(5+start)) at the end) this should work. If it doesn't, have each task record in the store the timestamp it finished, and when it start check if the last task ended less than 5 seconds ago, and in that case terminate immediately.
The other way could be store the task data in table. In your task-queue add a id parameter. Fetch 1st task from table and pass its id to task queue processing servlet. In servlet at the end delay for 5 second and feth next task, pass its id and.... so on.
Related
how can I check my requirement of 100 requests are processed in less than 1 second in my gatling3 report. I ran this using jenkins.
my simulation looks like as below
rampConcurrentUsers(1) to (100) during (161 second),
constantConcurrentUsers(100) during (1 minute)
Below is my response time percentile graph of two executions for an interval of one second.
enter image []1 here
What does the min,max here will tell us, i am assuming the percentages 25%-99% are the completion of the request.
Those graph sections are not what you're after - they show the distribution of response times and the number of active users.
So min is the fastest response time
max is the longest
95% is the response time for which 95% of your requests were under
and so on...
So what you could do is look at the section of your graph corresponding to the 100 constant concurrent users injection stage. In this part you would require that the max response time always be under 1 second
(Note: there's something odd with your 2nd report - I assume it didn't come from running the stated injection profile as it has more than 100 concurrent users active)
I have a scenario where I want to use rules purely as a scheduled job for invoking other services. I am using a solution similar to Answer 2 on this. So I have rule 1 which looks like:
rule "ServiceCheck"
timer ( int: 3m 5m )
no-loop true
when
then
boolean isServiceEnabled = DummyServices.getServiceEnabledProperty();
if(isServiceEnabled){
ServicesCheck servicesCheck = new ServicesCheck();
servicesCheck.setServiceEnabled(true);
insert(servicesCheck);
}
end
This inserts a servicesCheck object every 5 minutes if services are enabled. Once this object is inserted my other rules fire and retract the servicesCheck fact from there.
The problem I am facing is when I switch off the app and start it next day. At that time, the ServiceCheck rule gets fired a load of times before coming to a stop. My assumption is that the last fired time is saved in the session and when I restart, it finds a difference between current time and saved time and fires the rules for number of times till the 2 times match in the session. So effectively, to catch up for 1 hr gap from shutdown to restart, it will fire the rule 12 times in this case as the interval set is 5 mins. Is there a way using which I can update the last fired time on the rules session so that it starts working like a fresh new start without catching up for lost time.
I suppose you are persisting the entire session? I suppose you have a shutdown procedure. You can use a single Fact, let's call it Trigger. Modify your rule to
rule "ServiceCheck"
timer ( int: 3m 5m )
when
Trigger()
then
// ... same
end
You'll have to insert one Trigger fact after startup and retract it during shutdown.
Later
I've set up an experiment (using 5.5.0) where a session is running, being called with fireUntilHalt in one thread, with a rule like "ServiceCheck". Another thread will sleep some time, halt the session after retracting the Trigger fact. After more than double the interval of the timer firing, the second thread inserts the Trigger again, signals the first thread to re-enter fireUntilHalt(), and the second thread will repeat its cycle. I can observe silence during the period where the Trigger is retracted.
If, however, the Trigger is not retracted/re-inserted, there'll be a burst of firings after the session has been restarted.
This indicates that retracting and re-inserting a Trigger does indeed stop and restart a timer rule.
I have route which when sent a message invokes a refresh service
I only want the service to be invoked at most every 1 minute
If the refresh service takes longer than 1 minute (e.g. 11 minutes) I don't want requests for it to queue up
The first part: every 1 minutes is easy, I just create an aggregator with a completionTimeout of 1 mins
The part about stopping requests queueing up is not so easy and I can't figure out how to construct it
e.g.
from( seda_in )
.aggregate( constant(A), blank aggregator )
.completionTimeout( 1000 )
.process( whatever )...
if the process takes 15 seconds then potentially 15 new inoke messages could be waiting for the process when it finishes. I want at most just 1 to be waiting for however long the process takes. (its hard to predict)
how can I avoid this or structure it better to achieve my objectives?
I believe you would be interested in seeing the Throttler pattern, which is documented here http://camel.apache.org/throttler.html
Hope this helps :)
EDIT - If you are wanting to eliminate excess requests, you can also investigate setting a TTL (Time to live) header within JMS and add a concurrent consumer of 1 to your route, which means any excess messages will also be discarded.
Google app engine task queues have configuration as (example)
<queue>
<name>mail-queue</name>
<rate>5/m</rate>
<bucket-size>10</bucket-size>
</queue>
Here, what does the 'bucket-size' mean? I could not find a comprehensive documentation about this in google app engine documentation.
Does specifying this as 10 means that if 100 tasks are queued at an instant only 10 of those will be put in the queue and rest will be ignored?
bucket-size is perfectly described here:
Limits the burstiness of the queue's processing, i.e. a higher bucket size allows bigger spikes in the queue's execution rate. For example, consider a queue with a rate of 5/s and a bucket size of 10. If that queue has been inactive for some time (allowing its "token bucket" to fill up), and 20 tasks are suddenly enqueued, it will be allowed to execute 10 tasks immediately. But in the following second, only 5 more tasks will be able to be executed because the token bucket has been depleted and is refilling at the specified rate of 5/s.
If no bucket_size is specified for a queue, the default value is 5.
For your case it means that if 100 messages are queued, only ten are directly being executed and another 5 every next minute. You won't loose any messages, but they will queue up if your bucket-size and rate is too low.
We recently got some data back on a benchmarking test from a software vendor, and I think I'm missing something obvious.
If there were 17 transactions (I assume they mean successfully completed requests) per second, and 1500 of these requests could be served in 5 minutes, then how do I get the response time for a single user? Is this sort of thing even possible with benchmarking? I have a lot of other data from them, including apache config settings, but I'm not sure how to do all the math.
Given the server setup they sent, I want to know how I can deduce the user response time. I have looked at other similar benchmarking tests, but I'm having trouble measuring requests to response time. What other data do I need to provide here to get that?
If only 1500 of these can be served per 5 minutes then:
1500 / 5 = 300 transactions per min can be served
300 / 60 = 5 transactions per second can be served
so how are they getting 17 completed transactions per second? Last time I checked 5 < 17 !
This doesn't seem to fit. Or am I looking at it wrongly?
I presume be user response time, you mean the time it takes to serve a single transaction:
If they can serve 5 per second than it takes 200ms (1/5) per transaction
If they can serve 17 per second than it takes 59ms (1/17) per transaction
That is all we can tell from the given data. Perhaps clarify how many transactions are being done per second.