I have to manage multiple timers for a UDP file transfer application,
after a timeout the server had to resend packets to the client, but there are more than one packet a time that could cause the timeout.
So I have to manage a timer for each packet. How can I do this?
I can't use alarm because it cancelled the previous timers and also works only with seconds.
You need to keep an array of structs containing timeouts for each packet you want to keep track of.
Each array element should contain the starting time and expected ending time for each timeout. When it's time to set the timer, check all entries in the array to see which one is expected to time out first. Then subtract that time from the current time to get your timeout value for select.
When the socket read times out, go through the list again and for each packet whose timeout time is prior to the current time, handle the timeout for that packet.
Take a look at the source of a multicast file transfer application I wrote called UFTP for an example of how this can be implemented. Specifically, look at the getrecenttimeout function in client_loop.c.
Related
I am receiving UDP packets at the rate of 10Mbps. Each packet is formed of around 1109 bytes.
So, it makes more than 1pkt/ms that I am receving on eth0. The recvfrom() in C receives the packet and passes on the packet to Java. Java does the filtering of the packets and the necessary processing.
The bottlenecks are:
recvfrom() is too slow:fetching takes more than 10ms possibly because it does not get the CPU.
Passing of the packet from C to Java through the interface(JNI) takes 1-2 ms.
The processing of packet in Java itself takes 0.5 to 1 second depending on if database insertion or image processing needs to be done.
So, the problem is many delays add up and more than half of the packets are lost.
The possible solutions could be:
Exclude the need for C's recvfrom() completely and implement UDP fetching directly in Java (Note: On the first place, C's recvfrom() was implemented to receive raw packets and not UDP). This solution may reduce the JNI transfer delay.
Implement multi-threading on the UDP receive function in Java. But then an ID shall be required in the UDP packets for the sequence because in multi-threading the order of incoming packets is not guaranteed. (However, in this particular program, there is a need for packets to be ordered). This solution might be helpful in receiving all packets but the protocol which sends the data needs to be modified to add a sequence identifier. Due to multi-threading, the receiver might have higher chances to get the CPU and packets can be quickly fetched.
In Java, a blocking queue can be implemented as a huge buffer which stores the incoming packets. The Java parser can then use the packets from this queue to process it. However, it is not sure if the receiver function will be fast enough and put all the received packets in the queue without dropping any packet.
I would like to know which of the solutions could be optimal or a combination of the above solutions will work. Any help or suggestions would be greatly appreciated.
How long is this burst going on? Is it continuous and will go on forever? Then you need beefier hardware that can handle the load. Possibly some load-balancing where multiple servers handle the incoming data.
Does the burst only last a short wile, like in at most a second or two? Then have the lower levels read all packets as fast as it can, and put in a queue, and let the upper levels get the messages from the queue in its own time.
It sounds like you may be calling recvfrom() with your socket in blocking mode, in which case it will not return until the next packet arrives. If the packets are being sent at 10ms intervals, perhaps due to some delay on the sending side, then a loop of blocking recvfrom() calls would appear to take 10ms each. Set the socket to non-blocking and use something like select() to decide when to call it. Then profile everything to see where the real bottleneck lies. My bet would be on one or more of the JNI passthroughs.
Note that recvfrom() is not a "C" function, it is a system call. Java's functions just add layers on top of it.
I am working on redesign an existing L2TP(Layer 2 tunneling protocol) code.
For L2TP , the number of tunnels we support is 96K. L2TP protocol has a keep-alive mechanism where it needs to send HELLO msges.
Say if we have 96,000 tunnels for which L2TPd needs to send HELLO msg after configured timeout value , what is the best way to implement it ?
Right now , we have a timer thread , where for every 1sec , we iterate and send HELLO msges. This design is a old design which is not scaling now.
Please suggest me a design to handle large number of timers.
There are a couple of ways to implement timers:
1) select: this system call allows you to wait on a file descriptor, and then wake up. You can wait on a file descriptor that does nothing as a timeout
2) Posix Condition Variables: similar to select, they have a time out mechanism built in.
3) If you are using UNIX, you can set a UNIX signal to wake up.
Those are basic ideas. You can see how well they scale to multiple timers; I would guess you'd have to have multiple condvars/selects for some handful of the threads.
Dependingo on the behaviour you want, you would probably want a thread for every 100 timers or so, and use one of the mechanisms above to wake up
one of the timers. You'd have a thread sitting in a loop, and keeping
track on each of the 100 timeouts, then waking up.
Once you exceed 100 timers, you would simply create a new thread and have it manage the next 100 timers and so on.
I don't know if 100 is the right granularity, but it's something you'd play with.
Hopefully that's what you are looking for.
Typically, such requirements are met with a delta-queue. When a timeout is required, get the system tick count and add the timeout interval to it. This gives the Timeout Expiry Tick Count, (TETC). Insert the socket object into a queue that is sorted by decreasing TETC and have the thread wait for the TETC of the item at the head of the queue.
Typically, with asocket timeouts, queue insertion is cheap because there are many timeouts with the same interval and so new timeout insertion will normally take place at the queue tail.
Management of the queue, (actually, since insertion into the sorted queue could take place anywhere, it's more like a list than a queue, but whatever:), is best kept to one timeout thread that is normally performing a timed wait on a condvar or semaphore for the lowest TETC. New timeout-objects can then be queued to the thread on a thread-safe concurrent queue and signaled to the timeout-handler thread by the sema/condvar.
When the timeout thread becomes ready on TETC timeout, it could call some 'OnTimeout' method of the object itself, or it might put the timed-out object onto a threadpool input queue.
Such a delta-queue is much more efficient for handling large numbers of timeouts than any polling scheme, especially for requirements with longish intervals. No polling is required, no CPU/memory bandwidth wasted on continual iterations and the typical latency is going to be a system clock-tick or two.
It is dependent on the processor/OS, kernel version, architecture.
In linux, one of the option is to use its timer functionality for multiple timers. Addition of timer can be done using add_timer in linux. You can define it using timer_list and initilialize internal values of timer using init_timer.
Followed by it register it using add_timer after filling timer_list(timeout(expire), function to execute after timeout(function), parameter to the function(data)) appropriately for respective timer. If jiffies is more than or equal to timeout(expire), then the respective timer handler(function) shall be triggered.
Some processors have provisioning for timer wheels(that consists of a number of queues that are placed equally in time in slots) which can be configured for a wide range of timers,timeouts as per the requirement.
I'm building a network sniffer that will run on a PFSense for monitoring an IPsec VPN. I'm compiling under FreeBSD 8.4.
I've chosen to use libpcap in C for the packet capture engine and Redis as the storage system.
There will be hundreds of packets per second to handle, the system will run all day long.
The goal is to have a webpage showing graphs about network activity, with updates every minutes or couple of seconds if that's possible.
When my sniffer will capture a packet, it'll determine its size, who (a geographical site, in our VPN context) sent it, to whom and when. Then those informations needs to be stored in the database.
I've done a lot of research, but I'm a little lost with libpcap, specifically with the way I should capture packets.
1) What function should I use to retrieve packets ? pcap_loop ? pcap_dispatch ? Or pcap_next_ex ? According to what I read online, loop and dispatch are blocking execution, so pcap_next_ex seems to be the solution.
2) When are we supposed to use pcap_setnonblock ? I mean with which capture function ? pcap_loop ? So if I use pcap_loop the execution won't be blocked ?
3) Is multi-threading the way to achieve this ? One thread running all the time capturing packets, analyzing them and storing some data in an array, and a second thread firing every minutes emptying this array ?
The more I think about it, the more I get lost, so please excuse me if I'm unclear and don't hesitate to ask me for precisions.
Any help is welcome.
Edit :
I'm currently trying to implement a worker pool, with the callback function only putting a new job in the job queue. Any help still welcome. I will post more details later.
1) What function should I use to retrieve packets ? pcap_loop ? pcap_dispatch ? Or pcap_next_ex ? According to what I read online, loop and dispatch are blocking execution, so pcap_next_ex seems to be the solution.
All of those functions block waiting for packets if there's no packet ready and you haven't put the pcap_t into non-blocking mode.
pcap_loop() contains a loop that runs indefinitely (or until pcap_breakloop() is called, and error occurs, or, if a count was specified, the count runs out). Thus, it may wait more than one time.
pcap_dispatch() blocks waiting for a batch of packets to arrive, if no packets are available, loops processing the batch, and then returns, so it only waits at most one time.
pcap_next() and pcap_next_ex() wait for a packet to be available and then returns it.
2) When are we supposed to use pcap_setnonblock ? I mean with which capture function ? pcap_loop ? So if I use pcap_loop the execution won't be blocked ?
No, it won't be, but that means that a call to pcap_loop() might return without processing any packets; you'll have to call it again to process any packets that arrive later. Non-blocking and pcap_loop() isn't really useful; you might as well use pcap_dispatch() or pcap_next_ex().
3) Is multi-threading the way to achieve this ? One thread running all the time capturing packets, analyzing them and storing some data in an array, and a second thread firing every minutes emptying this array ?
(The array would presumably be a queue, with the first thread appending packet data to the end of the queue and the second thread removing packet data from the head of the queue and putting it into the database.)
That would be one possibility, although I'm not sure whether it should be timer-based. Given that many packet capture mechanisms - including BPF, as used by *BSD and OS X - deliver packets in batches, perhaps you should have one loop that does something like
for (;;) {
pcap_dispatch(p, -1, callback);
wake up dumper thread;
}
(that's simplified - you might want to check for errors in the return value from pcap_dispatch()).
The callback would add the packet handed to it to the end of the queue.
The dumper thread would pull packets from the head of the queue and add them to the database.
This would not require that the pcap_t be non-blocking. On a single-CPU-core machine, it would rely on the threads being time-sliced by the scheduler.
I'm currently trying to implement a worker pool, with the callback function only putting a new job in the job queue.
Be aware that, once the callback function returns, neither the const struct pcap_pkthdr structure pointed to by its second argument nor the raw packet data pointed to by its third argument will be valid, so if you want to process them in the job, you will have to make copies of them - including a copy of all the packet data you will be processing in that job. You might want to consider doing some processing in the callback routine, even if it's only figuring out what packet data needs to be saved (e.g., IP source and destination address) and copying it, as that might be cheaper than copying the entire packet.
I have a UDP client that send a number of packets to a server, I need to set a period of time between every packet, in other words I want to control the sending time of each packet.
How can I dot it?
Help!
You cannot ask the socket to send the data at a certain point in time. All control you have about the sending time is by not calling send/sendto() until you want the sending to happen - even then, the TCP/IP stack is free to delay the actual packet sending, so you can only hope for the best. Basically, you get the current time from the OS, put the packet into the socket to be sent, sleep until the next packet is due, put the next packet into the socket, and so on.
i am trying to implement a tcp ping function. And I hope to make the rate and pattern of sending messages configurable. For example, send 5000 msg in 5 seconds, burst 2000 at first then 3 msg/ms for 1000ms.
Any idea how to make it happen? Thanks in advance.
ps, I am using c socket programming, write and read to send and receive msg.
I may be missing something but isn't all you have to do is have a loop where you send 2000 messages and then put the thread on Sleep() for 1ms and send 3 packets each time until you have sent the rest 3000 packets.
One thing you should know is that measuring how long time it takes for code to execute is hard. Since you are using TCP, which uses buffers, send will block if there is not enough buffer space until it can send the next message depending on amount of data, size and network status.