I am using constantUsersPerSec to inject users in my simulation scrip, according to the docs I was expecting to see that number of users/requests being performed, so that rate would act as limit (and a baseline too).
However, in some situations (seems random), the number of requests go over this limit.
This is the metrics from my load balancer:
In there, we can see a drop of the requests and then a peak that goes over the rate.
To compare, this is a normal run:
The gatling injection script:
val scenario: ScenarioBuilder = scenario("Test")
.exec(doHttpRequest())
setUp(
scenario.inject(
constantUsersPerSec(30).during(5.minutes),
constantUsersPerSec(60).during(10.minutes)
).protocols(httpProtocol)
)
Gatling version: 3.7.3.
The injector host is a m5.large EC2 instance.
The CPU utilization for both executions is not that high:
Network out (bytes):
You should probably look at your infrastructure: the injector host, the network between the injector and your LB, etc. Are you able to reproduce with Gatling Cloud?
Related
I asked about this in the appengine user group here and wasn't able to resolve the issue.
The issue I'm having is that, for seemingly a very light endpoint and others like it, latency seems to be an issue. Here's an example request as shown in GCP's Trace tool:
I'm aware that if it's a request that spawned a new instance or if memory usage is high, that would explain high latency. Neither of which are the case here.
It seems that intermittently some endpoints simply take a good second or two to respond on top of however long the endpoint itself takes to do its job, and the bulk of that is "untraced time" under GCP Stackdriver's Trace tool. I put a log entry as early as I seemingly possibly could, in webapp2's RequestHandler object on initialization. You can see that in the screenshot as "webapp2 request init".
I'm not familiar enough with webapp2's inner working to know where I could put a log elsewhere that could help explain this, if anywhere.
These are the scaling settings for that appengine service defined in my yaml file:
instance_class: F4
automatic_scaling:
max_idle_instances: 1
max_pending_latency: 1s
max_concurrent_requests: 50
Not sure what other information would be useful here.
I have configured a version of my default service on Google App Engine Standard (Java, though that shouldn't make any difference) to use basic scaling and run a single B2 instance:
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>${app.id}</application>
<version>tasks</version>
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
<module>default</module>
<instance-class>B2</instance-class>
<basic-scaling>
<idle-timeout>60s</idle-timeout>
<max-instances>1</max-instances>
</basic-scaling>
<!-- Other stuff -->
</appengine-web-app>
Despite not getting any request for almost 28 minutes, the instance did not shutdown by itself (I manually shut it down with appcfg.cmd stop_module_version...):
There are no background threads.
Why doesn't this instance shutdown? Two days prior the instance ran almost the whole day, idle, with this configuration... so what's the issue?
The definition of idle is that no new requests are received in x amount of time. What if the last request is taking 20 minutes to execute? Shouldn't idle be defined as the time since the last request finished?
I posted this question on SeverFault (since this is not a programming question) but was told that StackOverflow would be the better site...
GCP Support here:
I tried to reproduce the same behaviour but either using 1m or 60s, instances would be shutdown after serving its last request.
Nonetheless, when I had any long lasting requests, threads, and/or Task queues running for minutes, the instance wouldn't be shutdown until this request was finished. You can also find this information here for both Manual/basic scaling :
Requests can run for up to 24 hours. A manually-scaled instance can
choose to handle /_ah/start and execute a program or script for many
hours without returning an HTTP response code. Task queue tasks can
run up to 24 hours.
In your case, it seems there was a request lasting for minutes prior being finished and thus, the instance was active (rather than idle) throughout until you manually stopped it. You may find this Instance life cycle documentation useful as well.
If you believe this behaviour isn't what you experienced back then, I'd suggest you to create a private issue tracker so we can investigate further. Make sure to provide the project number and all required details in there (fresh samples). Once created, share the issue tracker number so we can look into this.
I am creating a performance test suite in jmeter with webdriver sampler. I am able to do it but when I see the results(load time).It keeps changing every time I run the test for same URL (homepage). How do I get accurate results?
Performance testing using WebDriver Sampler is not what it is designed for.
As per WebDriver Sampler Tutorial
Note: It is NOT the intention of this project to replace the HTTP Samplers included in JMeter. Rather it is meant to compliment them by measuring the end user load time.
and
The Web Driver Sampler is meant to be run in conjunction with a HTTP Sampler so that from the server's perspective, the load is production like. At the same time, the web sampler will simulate the user experience of interacting with the website/webapp whilst the server is under load, hence measuring the real user's experience at the same time.
So I would recommend the following approach:
Develop your test using HTTP Request Samplers
Follow recommendations from How to make JMeter behave more like a real browser guide
Add a separate Thread Group with 1 virtual user and put WebDriver Sampler in there to measure real-life user experience during load test.
My workplace is connected via cable internet with limited up/downstream, so for load-testing a web application I am using a distributed approach:
An instance of jmeter-server (v2.4) is running on a remote linux box which has proper internet connectivity.
The controlling jmeter GUI (also 2.4) is running on my desktop.
The two are connected via a VPN.
The SUT is a web-application hosted in another datacenter.
This works fine - I am able to start the remote thread execution from the jmeter GUI and receive live results in the Summary Report.
The weird part is that my ability to put real load on the SUT is limited by my local internet connection (inbound, downstream). Tracing the link between the local jmeter GUI and the remote jmeter-server, I can see that all HTTP-traffic from SUT towards jmeter-server is sent to the local jmeter GUI. So if the jmeter-server is retrieving a 1MB file from the SUT, it's then sending it to my local jmeter GUI (over my slow internet link).
Is that how it should be? I was expecting that only the key measurements would be transported (success, latency, etc).
[PS: I know I can scp my testplan across to the remote box and run it in a headless jmeter. But then I don't see live results...]
This is probably because you are having a "View Tree Results" in your jmeter script. This tells the server to send all responses back to the master so you can view them. That will quickly fill your bandwith and is memory intensive. Try to disable all listeners in your script and only have the "View Aggregate results" one in there.
With high load, you're not interested in individual results. There is also no need to open up your firewall in your datacenter if you use ssl tunneling. I've documented that in a blogpost because I couldn't find a simple step-by-step online, you might want to check that out too at http://rolfje.com/2012/02/16/distributed-jmeter-through-vpn-and-ssl/
I would recommend you to write your results to a file. Before doing so, you can optionally perform a select / deselect of result save configuration by clicking on configure button and/or by checking write success/error to log. By unselecting result sampler data, save response messages etc you can reduce the amount of data being sent to your console.
I THINK the way you have it set up, both the Local instance of Jmeter and the remote are running the script at the same time, hence traffic is being directed to your local box.
What I've found to be more effective is to remote INTO the machine running JMeter. Everything is then local to the proper machine, and I don't have to worry about bottle necks.
I would recommend having a simple data writer element that saves to a file on the REMOTE linux box.
In the GUI, you could still have just the aggregate report to keep an eye on what's happening.
Hope this helps. If not, I would recommend posting to the JMeter user group:
http://jmeter.512774.n5.nabble.com/
how can i record the web server log files and replay the the recorded server logs so that i can latter run the logs and check the load on sql server.
ex. number of request on and the number of resposes on sql server
Build a tool that can parse the log files from the web server and replay the requests again. Then you generate test cases by accessing your site using a normal web browser and record the log files.
You can then replay the same sequence over and over again and your tool could also read multiple logs and play them back simultaneously to simulate concurrency.
Make sure that the log files include all GET and POST request parameters as well as timestamps.
Here are a couple tools that you might find useful:
Selenium is a suite of tools specifically for testing web applications.
http://seleniumhq.org/
ab (Apache HTTP server benchmarking tool) is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving.
http://httpd.apache.org/docs/2.0/programs/ab.html
What's wrong with using a tool? Tools like selenium will exercise the front-end, you can probably setup a server and a collection of clients running on separate machines to simulate user interaction.
If you want to exercise the backend more directly you could write a unit test I suppose, but unit testing isn't really designed for doing that sort of thing.