On my Gatling reports, I noticed that "Response Time Percentiles" and "Latency Percentiles over time" charts are quite identical. In which way are they different?
I saw this post, which makes me even more unsure:
Latency Percentiles over Time (OK) – same as Response Time Percentiles
over Time (OK), but showing the time needed for the server to process
the request, although it is incorrectly called latency. By definition
Latency + Process Time = Response time. So this graphic is supposed to
give the time needed for a request to reach the server. Checking
real-life graphics I think this graphic shows not the Latency, but the
real Process Time. You can get an idea of the real Latency by taking
one and the same second from Response Time Percentiles over Time (OK)
and subtract values from current graphs for the same second.
Thanks in advance for your help.
Latency basically tells how long it takes to receive the first packet for each page request throughout the duration of your load test. If you look at this chart in the Gatling documentation, the first spike is just before 21:30:20 on the x axis and tells you that 100% of the pages requested took longer than 1000 milliseconds to get the first packet from source to destination, but that number fell significantly after 21:30:20.
Related
I have two operators, a source and a map. The incoming throughput of of the map is stuck at just above 6K messages/s whereas the message count reaches the size of the whole stream (~ 350K) in under 20s (see duration). 350000/20 means that I have a throughput of at least 17500 and not 6000 as flink suggests! What's going on here?
as shown in the picture:
start time = 13:10:29
all messages are already read by = 13:10:46 (less than 20s)
I checked the flink library code and it seems that the numRecordsOutPerSecond statistic (as well as the rest similar ones) operate on a window. This means that they display average throughput but of the last X seconds. It's not the average throughput of the whole execution
I'm trying to increase Requests Per Second in Gatling with fewer Users (Each User-created will send API requests in a loop). I achieved 300 RPS with 35 Users. However, even if I increase the users to 70 or 150, I cannot get a higher rps than 300. With increased user count, the RPS is more than 300 for the initial few seconds but later just hovers around 300.
I tried both atOnceUsers and rampUsers, but still couldn't achieve higher RPS.
Is there any way to increase RPS with fewer users?
You need to find out where your constraint is - look at the results coming back from the simulation and examine how the response time changes with the number of requests. You may be being throttled by your application under test
I want to measure the amount of time taken to do any kind of operation inside apps such as Creo Parametric 5.0, Adobe Premiere Pro, Maya, Adobe Creative, Lightroom CC or any other design app
The idea is to measure the performance (time taken per operation) to catch performance issues.
When you create your library of action, you can create a decorator that log and time any actions so you can monitor whats going on
A method which is hacky and time consuming but generic (even for softwares not providing (good) scripting), could be to record your screen at high speed (such as 60 fps). Then you look at the frames to count the frames between giving the order (click, enter key) and the result (updated display).
The precision will be in the order of 1 / recording frequency (16 ms if recording at 60 fps). A drawback is that you are likely to measure the time of more than just the operation you are interested in, for instance if you want to bench the loading of a file, you will also measure the time it took to render it after (which may/should be negligible).
I was able to apply this method using https://github.com/SerpentAI/D3DShot (increase the framebuffer size which by default last 1 second). Note that the frame numbers when exporting to files are going backward in time.
It may be possible to make this method less hacky by using computer vision algorithms to not have to count frames manually.
I want to test how many subscribers I can connect to a publisher, which is sending out messages fast, but not with a maximum speed, e.g. every microsecond.
The reason is, if I send out messages with maximum speed, I miss messages at the receiver ( High-water-mark ).
I thought, I can use nanosleep(), and it works nice with 20 messages a second ( sleep: 50000000 [ns] ). But with a shorter sleeping time, it gets worse: 195 (5000000), 1700(500000), 16000 (50000) messages. And with even shorter sleeping times, I don't really get more messages. It seems that the sleep-function itself needs some time, I can see this, if I print out timestamps.
So, I think, it is the wrong way to run a function with a specific rate. But I didn't find a way to do that in another way.
Is there a possibility to send out roughly 1000000 messages a second?
Q: How to send out messages with a defined rate?
Given API is v.4.2.3+, one can use a { pgm:// | epgm:// }-transport class for this very purpose and setup the adequately tuned .setsockopt( ZMQ_RATE, <kbps> ) plus exhibit some additional performance related tweaking of buffer-sizing ( ZMQ_SNDBUF, ZMQ_IMMEDIATE, ZMQ_AFFINITY, ZMQ_TOS and ZMQ_MULTICAST_MAXTPDU ) with some priority-mapping e.t.c., so as to safely get as close to the hardware limits as needed.
Q: Is there a possibility to send out roughly 1,000,000 messages a second?
Well, given not more than about a 1000 [ns] per a message-to-wire dispatch latency, a carefull engineering is due to take place.
The best candidate for such rate would be to use the inproc:// transport class, as this does not rely on ZeroMQ's Context-instance IO-thread(s) performance / bottlenecks inside an external O/S scheduler ( and will definitely work faster than any other kind of available transport-classes ). Still, it depends, if it can meet less than the required 1000 [ns] latency, based on your application design and message sizes ( Zero-Copy being our friend here to better meet the latency deadline ).
I have a setup with a Beaglebone Black which communicates over I²C with his slaves every second and reads data from them. Sometimes the I²C readout fails though, and I want to get statistics about these fails.
I would like to implement an algorithm which displays the percentage of successful communications of the last 5 minutes (up to 24 hours) and updates that value constantly. If I would implement that 'normally' with an array where I store success/no success of every second, that would mean a lot of wasted RAM/CPU load for a minor feature (especially if I would like to see the statistics of the last 24 hours).
Does someone know a good way to do that, or can anyone point me in the right direction?
Why don't you just implement a low-pass filter? For every successfull transfer, you push in a 1, for every failed one a 0; the result is a number between 0 and 1. Assuming that your transfers happen periodically, this works well -- and you just have to adjust the cutoff frequency of that filter to your desired "averaging duration".
However, I can't follow your RAM argument: assuming you store one byte representing success or failure per transfer, which you say happens every second, you end up with 86400B per day -- 85KB/day is really negligible.
EDIT Cutoff frequency is something from signal theory and describes the highest or lowest frequency that passes a low or high pass filter.
Implementing a low-pass filter is trivial; something like (pseudocode):
new_val = 1 //init with no failed transfers
alpha = 0.001
while(true):
old_val=new_val
success=do_transfer_and_return_1_on_success_or_0_on_failure()
new_val = alpha * success + (1-alpha) * old_val
That's a single-tap IIR (infinite impulse response) filter; single tap because there's only one alpha and thus, only one number that is stored as state.
EDIT2: the value of alpha defines the behaviour of this filter.
EDIT3: you can use a filter design tool to give you the right alpha; just set your low pass filter's cutoff frequency to something like 0.5/integrationLengthInSamples, select an order of 0 for the IIR and use an elliptic design method (most tools default to butterworth, but 0 order butterworths don't do a thing).
I'd use scipy and convert the resulting (b,a) tuple (a will be 1, here) to the correct form for this feedback form.
UPDATE In light of the comment by the OP 'determine a trend of which devices are failing' I would recommend the geometric average that Marcus Müller ꕺꕺ put forward.
ACCURATE METHOD
The method below is aimed at obtaining 'well defined' statistics for performance over time that are also useful for 'after the fact' analysis.
Notice that geometric average has a 'look back' over recent messages rather than fixed time period.
Maintain a rolling array of 24*60/5 = 288 'prior success rates' (SR[i] with i=-1, -2,...,-288) each representing a 5 minute interval in the preceding 24 hours.
That will consume about 2.5K if the elements are 64-bit doubles.
To 'effect' constant updating use an Estimated 'Current' Success Rate as follows:
ECSR = (t*S/M+(300-t)*SR[-1])/300
Where S and M are the count of errors and messages in the current (partially complete period. SR[-1] is the previous (now complete) bucket.
t is the number of seconds expired of the current bucket.
NB: When you start up you need to use 300*S/M/t.
In essence the approximation assumes the error rate was steady over the preceding 5 - 10 minutes.
To 'effect' a 24 hour look back you can either 'shuffle' the data down (by copy or memcpy()) at the end of each 5 minute interval or implement a 'circular array by keeping track of the current bucket index'.
NB: For many management/diagnostic purposes intervals of 15 minutes are often entirely adequate. You might want to make the 'grain' configurable.