I'm trying to write a web server in C under Linux using protocol HTTP1.1 .
I've used select for multiple requests and I'd like to implement persistent connections but it didn't work so far 'cause I can't set a timeout properly. How can I do it? I think about setsockopt function:
setsockopt(connsd, SOL_SOCKET, SO_RCVTIMEO, (char *)&tv, sizeof(tv))
where tv is a struct timeval. This isn't working either.
Any suggestions?
SO_RCVTIMEO will only work when you are actually reading data. select() won't honor it. select() takes a timeout parameter in its last argument. If you have a timer data structure to organize which connections should timeout in what order, then you can pass the soonest to timeout time to select(). If the return value is 0, then a timeout has occurred, and you should expire all timed out connections. After processing live connections (and re-setting their idle timeout in your timer data structure), you should again check to see if any connections should be timed out before calling select() again.
There are various data structures you can use, but popular ones include the timing wheel and timer heap.
A timing wheel is basically an array organized as a circular buffer, where each buffer position represents a time unit. If the wheel units is in seconds, you could construct a 300 element array to represent 5 minutes of time. There is a sticky index which represents the last time any timers were expired, and the current position would be the current time modulo the size of the array. To add a timeout, calculate the absolute time it needs to be timed out, modulo that by the size of the array, and add it to the list at that array position. All buckets between the last index and the current position whose time out has been reached need to be expired. After expiring the entries, the last index is updated to the current position. To calculate the time until the next expiration, the buckets are scanned starting from the current position to find a bucket with an entry that will expire.
A timer heap is basically a priority queue, where entries that expire sooner have higher priority than entries that expire later. The top of a non-empty heap determines the time to next expiration.
If your application is inserting a lots and lots of timers all the time, and then cancelling them all the time, then a wheel may be more appropriate, as inserting into the wheel and removing from the wheel is more efficient than inserting and removing from a priority queue.
The simplest solution is probably to keep a last-time-request-received for each connection, then regularly check that time and if it's too long ago then close the connection.
Related
I have developed an energy aware routing protocol, now for performance evaluation I want to calculate end-to-end packet transmission delay when packets travel through a multi-hop link. I am unable to decide which timing information to consider whether to consider the simulation time available in log file (log-0.txt) or the modem's transmission time (txtime and rxtime). Please let me know the method to calculate end-to-end delay in UnetStack.
The simulation time (first column in the log files below, in milliseconds) is synchronized across all simulated nodes, and so you can use it to compute end-to-end delays if you log a START time at your source node, and END time at your destination node.
Example log file:
5673|INFO|org.arl.unet.sim.SimulationAgent/4#570:call|TxFrameNtf:INFORM[type:DATA txTime:2066947222]
6511|INFO|org.arl.unet.sim.SimulationAgent/3#567:call|TxFrameNtf:INFORM[type:DATA txTime:1157370743]
10919|INFO|org.arl.unet.sim.SimulationAgent/4#570:call|TxFrameNtf:INFORM[type:DATA txTime:2072193222
In this example, node 4 (SimulationAgent/4) transmits at time 5673. Node 3 (SimulationAgent/3) then transmits at time 6511. And so on...
The txTime and rxTime are in microseconds, but are local to each node. So they can be used to get time differences for events in the same node, but cannot directly be compared across nodes.
I am trying to shape a JMeter test involving a Concurrency Thread Group and a Throughput Shaping Timer as documented here and here. the timer is configured to run ten ramps and stages with RPS from 1 to 333.
I want to set up the Concurrency Thread Group to use the schedule feedback function and added the formula in the Target concurrency field (I have updated the example from tst-name to the actual timer name). ramp-up time and steps I have set to 1 as I assume the properties are not that important if the throughput is managed by the timer; the Hold Target Rate time is 8000, which is longer than the steps added in the timer (6200).
When I run the test, it ends without any exceptions within 3 seconds or so. The log file shows a few rows about starting and ending threads but nothing alarming.
The only thing I find suspicious is the Log entry "VirtualUserController: Test limit reached, thread is done plus thread name.
I am not getting enough clues from the documentation linked here to figure this out myself, do you have any hints?
According to the documentation rampup time and steps should be blank:
When using this approach, leave Concurrency Thread Group Ramp Up Time and Ramp-Up Steps Count fields blank"
So your assumption that setting them to 1 is OK, seems false...
If I have a very long akka-stream pipeline, is there a way to handle timeouts such that the timeout doesn't start until the first element gets to a given spot in a pipeline?
For example, let's say I have a pipeline in which it takes the first element 2+ minutes to reach the final sink, but after that, elements should come in every second or so. Is this something akka has taken into account? Or do I have to set timeouts on my graph shapes individually in this case?
I'm not an expert in Network Programming. I basically have two kinds of clients who have different time-outs. I am supposed to use UDP with connected sockets for client-server communication.
The problem is twofold:
a) I need to mark as died whichever client (alternatively, socket) does not respond for t1 seconds. Using select would time out if none of the sockets in read_fd_set have anything to read within the timeout value. So, how do I time-out any one socket which is not having data to read for quite some time?
Currently, whenever select returns, I myself keep track of which sockets are responding and which not. And I add t1.tu_sec to the individual time elapsed of each client (socket). Then, I manually close and exclude from FD_SET the socket which does not respond for (n) * (t1.tu_sec) time. Is this a good enough approach?
b) The main problem is that there are two kinds of clients which have different time-outs, t1 and t2. How do I handle this?
Can I have two select()s for the two kinds of clients in the same loop? Would it cause starvation without threads? Is using threads advisable (or even required) in this case?
I've been roaming around the web for ages!
Any help is much appreciated.
This is just a special case of a very common pattern, where a select/poll loop is associated with a collection of timers.
You can use a priority queue of tasks, ordered on next (absolute) firing time; the select timeout is always then just the absolute time at the front of the queue.
when select times out (and just before the next iteration, if your tasks may take a long time to complete), get the current time, pull every task that should already have executed off the queue, and execute it
(some) tasks will need to be re-scheduled, so make sure they can mutate the priority queue while you do this
Then your logic is trivial:
on read, mark the socket busy
on timer execution, mark the socket idle
if it was already idle, that means nothing was received since the last timer expiry: it's dead
A quick solution that comes to my mind, is to keep the sockets in a collection sorted by the time remaining until the nearest timeout.
Use select with the timeout set to the smallest time remaining, remove/close/delete the timed-out socket from the collection, and repeat.
So, in pseudo-code it might look like this:
C = collection of structs ( socket, timeout, time_remaining := timeout )
while (true) {
sort_the_collection_by_time_remaining
next_timeout = min(time_remaining in C)
select ( sockets in C, next_timeout )
update_all_time_remaining_values
remove_from_C_if_required //if timeout occured
}
It can easily be solved with a single select call. For each socket have two values related to the timeout: The actual timeout; And the amount of time until timeout. Then count down the "time until timeout" every 0.1 second (or similar), and when it reaches zero close the socket. If the socket receives traffic before the timeout simply reset the "time until timeout" to the timeout value and start the down-count again.
I have an application that keeps emitting data to a second application (consumer application) using TCP socket. How can I calculate the total time needed from when the data is sent by the first application until the data is received by the second application? Both the applications are coded using C/C++.
My current approach is as follow (in pseudocode):
struct packet{
long sent_time;
char* data;
}
FIRST APP (EMITTER) :
packet p = new packet();
p.data = initialize data (either from file or hard coded)
p.sent_time = get current time (using gettimeofday function)
//send the packet struct (containing sent time and packet data)
send (sockfd, p, ...);
SECOND APP (CONSUMER)
packet p = new packet();
nbytes = recv (sockfd, p, .....); // get the packet struct (which contains the sent time and data)
receive_time = get current time
data transfer time = receive time - p.senttime (assume I have converted this to second)
data transfer rate = nbytes / data transfer time; // in bytes per second
However the problem with this is that the local clock time between the 2 applications (emitter and consumer) are not the same because they are both running on different computers, leading this result to a completely useless result.
Is there any other better way to do this in a proper way (programmatically), and to get as accurate data transfer rate as possible?
If your protocol allows it, you could send back an acknowledgementn from the server for the received packet. This is also a must if you want to be sure that the server received/processed the data.
If you have that, you can simply calculate the rate on the client. Just substract the RTT from the length of the send+ACK intervall and you'll have a quite accurate measurement.
Alternatively you can use a time syncronization tool like NTP to synchronize the clocks on the two servers.
First of all: Even if your times were in sync, you would be calculating latency, not throughput. On every network connection chances are, that there is more than one packet en route at a given point in time, rendering your single-packet approach useless for throughput measurement.
E.g. Compare the ping time from your mobile to a HTTP server with the max download speed - ping time will be tens of ms, packet size will be ca. 1.5KByte, which would result in a much lower max throughput than observerd when downloading.
If you want to measure real throughput, use a blocking socket on the sender side and send e.g. 1 million packets as fast as the system will allow you, on the receiving side measure time between arrival of first packet and arrival of last packet.
If OTOH you want to accurately measure latency, use
struct packet{
long sent_time;
long reflect_time;
char* data;
}
and have the server reflect the packet. On the client side check all three timestamps, then reverse roles to get a grip on asymetric latencies.
Edit: I meant: The reflect time will be the "other" clock, so when running the test back and forth you will be able to filter out the offset.