JMeter Think Time - timer

Apologies if this request is similar to others - I am new to JMeter and have searched for other relevants posts but couldn't find anything - or maybe I just didn't understand them!
I'm performance testing a system with a web based application. The front end system will be processing records submitted into the system via MQ - the front end allows the user to pick up a record from the queue, validate some detail, make changes and submit the changes.
There will be 20 users using the front end to do this message validation, update and submission.
Each user is expected to need 30 seconds to pick a message from queue, make changes and resubmit - so we are expecting 1 user to process 120 records/hour, so 20 users will be expected to process 2400 records/hour
The picking up the record off the queue, changing it and submitting the changes will be done via 3 individual web pages.
SO - think time across the 3 pages has been defined as 24 seconds (leaving 6 of the 30 second limit for rendering, server responses, db calls etc.)
However I don't know how to specify this within JMeter. From my reading I can see that I can add a Timer in as a parent to a sampler and I assume I can add a Timer in as a parent of the Recording Controller? - but I need to be able to specify that the 24 second think time is spread across those 3 different pages.
I read a post elsewhere suggesting that if I record using the proxy after adding the Gaussian Random Timer in as a child of the Test Plan (parent to everything else) then the http proxy will record the think time as a ${T} variable in the Gaussian Random Timer - I tried this and this didn't work (also I don't want to rely on this - I'd like to be able to understand and make changes to think time properly rather than relying on JMETER to do it for me.)
To reiterate - 20 users, 30 seconds for 1 user to complete a transaction, TT defined as 24 seconds - I am struggling what Timer to use, where to put it so that the think-time is spread across the samplers that equate to the GETS associated with the 3 pages the user will navigate through.
Apologies for the lengthy post - I just wanted to be clear and concise.
Many thanks in advance,

As per JMeter Timers documentation
Note that timers are processed before each sampler in the scope in which they are found; if there are several timers in the same scope, all the timers will be processed before each sampler.
Timers are only processed in conjunction with a sampler. A timer which is not in the same scope as a sampler will not be processed at all.
To apply a timer to a single sampler, add the timer as a child element of the sampler. The timer will be applied before the sampler is executed. To apply a timer after a sampler, either add it to the next sampler, or add it as the child of a Test Action Sampler.
Now regarding "what timer to use"
There are 2 scenarios:
Virtual-User-oriented scenario - when you try to simulate N users working together
Goal-Oriented-scenario - when you try to produce N hits per second load.
In case of scenario 1 even Constant Timer can be quite enough, besides it will provide repeatability of results. See above quote for information on where to put your timer(s)
In case of scenario 2 you'll need Constant Throughput Timer. If 20 users process 2400 records per hour and each record assumes 3 web page calls, it means that 7200 requests will be made in one hour which in its turn stands for 120 requests per minute (this is what you should enter into the timer's "throughput" area) or 2 requests per second.

Related

How to have a constant in feature scenarios in Behave

Is there a way to set a global variable/constant in .feature files in Behave?
For an analytical service, I have many scenarios like this one
Scenario: Some scenario
Given do some action
And wait for 90 seconds while the action results are ready
Then verifying some result
And recently the requirements has updated and the service can wait for a longer time. This requirement may be changed in future. Is there a way not to find and replace all the "wait for 90 seconds" but have some constant in a feature file that I can update in one place?
My current approach is to refactor the step into wait for a reasonable time while the action results are ready and set the constant of reasonable time in Python. But in this approach, it's not clear from the tests logs what is the reasonable time for a specific run.
Waiting a constant amount of time is bad practice
Correct scenario defininition should be:
Scenario: Some scenario
Given do some action
And wait for the action results are ready
Then verifying some result
In the step implementation of "wait for the action results are ready" an active wait must be made that will end when results are ready

TumblingProcessingTimeWindows processing with event time characteristic is not triggered

My use-case is quite simple I receive events that contain "event timestamp", and want them to be aggregated based on event time. and the output is a periodical processing time tumbling window every 10min.
More specific, the stream of data that is keyed and need to compute counts for 7 seconds.
a tumbling window of 1 second
a sliding window for counting 7 seconds with an advance of 1 second
a windowall to output all counts every 1s
I am not able to integration test it (i.e., similar to unit test but an end-to-end testing) as the input has fake event time, which won't trigger
Here is my snippet
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
val oneDayCounts = data
.map(t => (t.key1, t.key2, 1L, t.timestampMs))
.keyBy(0, 1)
.timeWindow(Time.seconds(1))
.sum(2)
val sevenDayCounts = oneDayCounts
.keyBy(0,1)
.timeWindow(Time.seconds(3), Time.seconds(1))
.sum(2)
// single reducer
sevenDayCounts
.windowAll(TumblingProcessingTimeWindows.of(Time.seconds(1)))
.process(...)
I use EventTime as timestamp and set up an integration test code with MiniClusterWithClientResource. also created some fake data with some event timestamp like 1234l, 4567l, etc.
EventTimeTrigger is able to be fired for sum computation but the following TumblingProcessingTimeWindow is not able to trigger. I had a Thread.sleep of 30s in the IT test code but still not triggered after the 30s
In general it's a challenge to write meaningful tests for processing time windows, since they are inherently non-deterministic. This is one reason why event time windows are generally prefered.
It's also going to be difficult to put a sleep in the right place so that is has the desired effect. But one way to keep the job running long enough for the processing time window to fire would be to use a custom source that includes a sleep. Flink streaming jobs with finite sources shut themselves down once the input has been exhausted. One final watermark with the value MAX_WATERMARK gets sent through the pipeline, which triggers all event time windows, but processing time windows are only fired if they are still running when the appointed time arrives.
See this answer for an example of a hack that works around this.
Alternatively, you might take a look at https://github.com/apache/flink/blob/master/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TumblingProcessingTimeWindowsTest.java to see how processing time windows can be tested by mocking getCurrentProcessingTime.

Building thinking time delay with time based tasks?

I have this meter job for which I need to build some think time between certain HTTP Requests. But during those thinking time I still need to send a keep alive request on specific interval.
For example:
User login
get some profile information.
Then he start to do some work.
each unit of work is delayed by some random delay varying from 1 to 30 minutes.
During that time we still need to send to there server a ImAlive request at fix interval (like 5 minutes).
Once the thinking time is expired which could be at 17m12s for example, then the loop exit.
For simulate the delay you can use Runtime Controller which will be executed with given seconds you define the keep alive requests, inside Runtime Controller add Timer as Gaussian Random Timer to add delay between keep alive requests.
You can use While Controller with condition like:
${__groovy(${__time(,)} - ${TESTSTART.MS} < 1032000,)}
Where:
__time() function - returns current time in milliseconds since start of Unix epoch
${TESTSTART.MS} - pre-defined JMeter property where test start time lives
__groovy() function - allows execution of arbitrary Groovy code
1032000 - milliseconds representation of 17m12s - (17 * 60 + 12) * 1000
So children of the While Controller will be executed for 17 minutes and 12 seconds after test start. If needed you can add another condition just in case you want to exit the loop earlier. See Using the While Controller in JMeter guide for more details.

How to stream delayed data from a real-time data source in Scala

I have to design a way to stream data in Scala/Java, with a certain delay compared to the moment I receive the data. I have an API for my original data source (the real time one) which lets me query it as it was a database, using a query-like format, and also to get notification when something happens, much as with EHCACHE.
Now I want for example to stream data with different delays, according to the user privileges. Some of the users will see data streamed with 0 delay, other with 15 minutes, other with 60 minutes.
I will therefore need a top-level cache composed of multiple lower level caches (0 delay, 15 min delay, 60 min delay).
In a first moment I would like the cache to contain only elements of the same type. This would make the things simpler, because I can decide for a unique ID for the elements and I would have only to route the request to the right cache according to the required delay
In a second moment, I would like my delayed cache to be queryable. Is there a part of the code from EHCache which I can recycle, for example?
An idea: Convert the stream data into some kind of event objects that implement Delayed. As soon as such a real-time event occurs, put it into DelayedQueue. Another thread can then call take() in a cycle and retrieve and process the delayed events.

Is there an elegant way to post messages to AWS SQS with visibility delay of longer than 15 minutes?

In Amazon Web Services, their queues allow you to post messages with a visibility delay up to 15 minutes. What if I don't want messages visible for 6 months?
I'm trying to come up with an elegant solution to the poll/push problem. I can write code to poll the SQS (or a database) every few seconds, check for messages that are ready to be visible, then move them to a "visible queue", or something like that. I wish there was a simpler, more reliable method to have messages become visible in queues far into the future without me having to worry about my polling application working perfectly all the time.
I'm not married to AWS, SQS or any of that, but I'd prefer to find a cloud-friendly solution that is stable, reliable and will trigger an event far into the future without me having to worry about checking on its status every day.
Any thoughts or alternate trees for me to explore barking up are welcome.
Thanks!
It sounds like you might be misunderstanding the visibility delay. Its purpose is to make sure that the polling application doesn't pull the same item off the queue more than once.
In other words, when the item is pulled off the queue it becomes invisible for a predetermined period of time (default is 30 seconds, max is 15 minutes) in case the polling system has a cluster of machines reading from the queue all at once.
Here's the relevant documentation:
http://docs.amazonwebservices.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/IntroductionArticle.html#AboutVT
...and the sentence in particular that relates to my comment is:
"Immediately after the component receives the message, the message is still in the queue. However, you don't want other components in the system receiving and processing the message again. Therefore, Amazon SQS blocks them with a visibility timeout, which is a period of time during which Amazon SQS prevents other consuming components from receiving and processing that message."
You should be able to use SQS for your purpose since you can leave an item in the queue for as long as you want.
7 years later, and Amazon still doesn't support the feature you need!
The two ways you can sort of get it to work are:
have messages contain a delivery target datetime in their message_attributes, and have the workers that consume the queue's messages just delete and recreate any message that is consumed before its target, with delay = max(0, min(secs_until_target_datetime, 900)) ; that would allow you to effectively schedule a message for any arbitrary future time;
or,
(slightly less frequent and costly:) similarly, if a message isn't due to be handled yet, recreate it and change its visibility timeout to be timeout = max(0, min(secs_until_target_datetime, 43200))
The disadvantage of using visibility timeout is that any read will re-trigger it.
There has been a direct AWS solution possible since 2016-12-01: AWS Step Functions
Each execution can last/idle up to one year, persists the state between transitions, and doesn't cost you any money while it waits.

Resources