I'm developing a trace analyzer tool for UnetStack. My tool can determine TxCount, RxCount, and end-to-end delay for each successfully received packet by using a simulation's trace.json file. Next, I want to calculate the throughput. How do I do that?
Is there enough data in the trace.json file to calculate throughput?
There are many different definitions for throughput, depending on which layer traffic you're looking at. The tracer included with UnetStack computes throughput at the PHY layer, by measuring the total amount of time that a node is busy receiving successful frames normalized by the total amount of simulation time x number of nodes.
You could do the same by looking at the RxFrameNtf entries in trace.json, but will need additional information such as CONTROL and DATA frame duration from your simulation settings in order to complete the computation, as this information is not published in the trace.json in the current version (3.x) of UnetStack.
The relevant information is currently more easily available from trace.nam.
P.S. We may expose the duration information in a later version of UnetStack.
Related
I made an experiment with my Apache Flink streaming application trying out different sizes to the tumbling time window. I fed the application the same data each time and measured the time the application took to emit data after doing some calculations. As expected, a large time window took a longer time to emit the output than a smaller window, but only up to a point. When the window became smaller then say, 14ms, the overhead costs (I guess) of the computation made it so that the time to emit the output took longer than when using say, a 16ms window.
How would you explain this, specifically in an Apache Flink streaming application? What are the specifics of these overhead costs? The application is integrated with Kinesis Data Analytics.
The network memory buffer's default value is 100ms. It stores your records for at most 100ms before sending them, or if the buffer is full.
To me, measuring the performance of any window size under this value will have little meaning. I guess the overhead you mention might become dominant, since your system is also waiting for the 100 ms to be elapsed (I'm assuming you're not filling the buffers, which default to 32KiB; which would be 2MiB/s if they're filled every 15ms).
Try setting execution.buffer-timeout to 5 (ms) for optimizing latency, or to -1 for optimizing throughput, and reexecute your workload.
Since this is Kinesis Data Analytics, you might have to do it programatically:
env.setBufferTimeout(5);
env.generateSequence(1,10).map(new MyMapper()).setBufferTimeout(5);
I am working on IBM cloudant performance testing. ( No SQL DB hosted in IBM cloud).
I am trying to identify the breaking point ( max input/sec).
I am triggering this request (POST) with JSON data.
I am unable to determine to design this test plan and thread group.
I need to determine the breaking point ( maximum allowed request/second).
Please find my Jmeter configuration above
The test type, you're trying to achieve is the Stress Test, you should design the workload as follows:
Start with 1 virtual user
Gradually increase the load
Observe the correlation between increasing number of virtual users and the throughput (number of requests per second) using i.e. Transaction Throughput vs Threads chart (can be installed using JMeter Plugins Manager)
Ideally throughput should increase proportionally to the increasing number of threads (virtual users). However applications have their limit therefore at certain stage you will run into the situation when the number of virtual users increases and throughput decreases. The moment just before throughput degradation is called saturation point and this is what you're looking for.
P.S. 20 000 virtual users might be a little bit high number for a single JMeter engine, you might need to consider switching to Distributed Testing
My problem is exactly similar to this except that Backpressure in my application is coming as "OK".
I thought the problem was with my local machine not having enough configuration, so I created a 72 core Windows machine, where I am reading data from Kafka, processing it in Flink and then writing the output back in Kafka. I have checked, writing into Kafka Sink is not causing any issues.
All I am looking for are the areas that may be causing a split in Throughput among task slots by increasing parallelism?
Flink Version: 1.7.2
Scala version: 2.12.8
Kafka version: 2.11-2.2.1
Java version: 1.8.231
Working of application: Data is coming from Kafka (1 partition) which is deserialized by Flink (throughput here is 5k/sec). Then the deserialized message is passed through basic schema validation (Throughput here is 2k/sec).
Even after increasing the parallelism to 2, throughput at Level 1 (deserializing stage) remains same and doesn't increase two fold as per expectation.
I understand, without the code, it is difficult to debug so I am asking for the points which you can suggest for this problem, so that I can go back to my code and try that.
We are using 1 Kafka partition for our input topic.
If you want to process data in parallel, you actually need to read data in parallel.
There are certain requirements to read data in parallel. The most important once are that the source is able to actually split the data into smaller work chunks. For example, if you read from a file system, you have multiple files, or the system subdivides the files into splits. For Kafka, this necessarily means that you have to have more partitions. Ideally, you have at least as many partitions than you have max consumer parallelism.
The 5k/s seems to be the maximum throughput that you can achieve on one partition. You can also calculate the number of partitions by the maximum throughput you want to achieve. If you need to achieve 50k/s, you need at least 10 partitions. You should use more to also catch up in case of reprocessing or failure recovery.
Another way to distribute the work is to add a manual shuffle step. That means, if you keep the single input partition, you would still only reach 5k/s, but after that the work is actually redistributed and processed in parallel, such that you will not see a huge decline in your throughput afterwards. After a shuffle operation, work is somewhat evenly distributed among the parallel downstream tasks.
For benchmarking purposes, I have to simulate an online data stream (Time Series data), to be ingested by various databases.
In this particular use case, Time Series data should be recorded at very high frequencies.
I would process this stream with Apache Kafka.
Stream is essentially composed by many real valued numbers, with their timestamp and a text label.
I have seen many Kafka producer examples, but noone fitted my use case really good: how can I use Kafka in order to GENERATE and PROCESS such a data stream??
EDIT:
Using python APIs, I am currently able to directly send data to various databases. The remaining issue regards now speed: my target frequency is near to 15kHz.
I can barely reach 2kHz using the same (multi-threaded) Python script.
Any suggestion?
We're being asked to spec out production database hardware for an ASP.NET web application that hasn't been built yet.
The specs we need to determine are:
Database CPU
Database I/O
Database RAM
Here are the metrics I'm currently looking at:
Estimated number of future hits to
website - based on current IIS logs.
Estimated worst-case peak loads to
website.
Estimated number of DB queries per
page, on average.
Number of servers in web farm that
will be hitting database.
Cache polling traffic from database
(using SqlCacheDependency).
Estimated data cache misses.
Estimated number of daily database transactions.
Maximum acceptable page render time.
Any other metrics we should be taking into account?
Also, once we have all those metrics in place, how do they translate into hardware requirements?
What I have been doing lately for server planning is using some free tools that HP provides, which are collectively referred to as the "server sizers". These are great tools because they figure out the optimal type of RAID to use, and the correct number of disk spindles to handle the load (very important when planning for a good DB server) and memory processor etc. I've provided the link below I hope this helps.
http://h71019.www7.hp.com/ActiveAnswers/cache/70729-0-0-225-121.html?jumpid=reg_R1002_USEN
What I am missing is a measure for the needed / required / defined level of reliability.
While you could probably spec out a big honking machine to handle all the load, depending on your reliabiltiy requirements, you might rather want to invest in smaller, but multiple machines, and into safer disk subsystems (RAID 5).
Marc
In my opinion, estimating hardware for an application that hasn't been built and designed yet is more of a political issue than a scientific issue. By the time you finish the project, current hardware capability and their price, functional requirements, expected number of concurrent users, external systems and all other things will change and this change is beyond your control.
However this question comes up very often since you need to put numbers in a proposal or provide a report to your manager. If it is a proposal, what you are trying to accomplish is to come up with a spec that can support the proposed sofware system. The only trick is to propose a system that will not increase your cost for competiteveness while not puting yourself at the risk of a low performance system.
If you can characterize your current workload in terms of hits to pages, then you can then:
1) calculate the typical type of query that will be done for each page
2) using the above 2 pieces of information, estimate the workload on the database server
You also need to determine your performance requirements - what is the max and average response time you want for your website?
Given the workload, and performance requirements, you can then calculate capacity. The best way to make this estimate is to use some existing hardware, run a simulated database workload on a database on that hardware, and then extrapolate your hardware requirements based on your data from the first steps.