Currently working with sockets, I am wondering whether it makes sense to reduce the amount of calls to send for performance.
As far as I understood, there is a send buffer and the data is not dispatched immediately (?), but then I am wondering how long the kernel waits before actually sending the data and how much overhead would be caused by it if I call send multiple times instead of once?
For TCP, there is a send buffer controlled by the Nagle algorithm (and its interaction with delayed acks from the receiver).
There isn't equivalent delay/buffering mechanism for UDP.
You haven't said which protocol you're using, but if it is TCP you probably don't need to do anything. For latency-sensitive code it can still be worth buffering writes just to avoid the syscall overhead, but I suppose you'd already know if that was your situation.
Related
Should I use non-blocking or blocking TCP sockets when using an I/O multiplexing API like poll(2) or epoll(2)?
Some people suggest using non-blocking sockets here but the I/O multiplexing APIs inform you anyway if there is data to read so what is wrong with a blocking socket here?
If your TCP server is single-threaded and uses blocking I/O, then it's likely that any client that connects to it will be able to deny service to all of the other clients simply by sending only a partial-message, or alternatively by refusing to read any data from its TCP socket after the server sends data. In the former case, the server may block for a long time (perhaps forever) waiting for the entire message to be received from the client; during that time, the server will not be able to respond to other clients. In the latter case, the server will block for a long time (perhaps forever) waiting the client to read some TCP data so that the server-socket's send-buffer can be drained enough to fit some more outgoing data to that client.
One way to avoid that problem is to set all of the server's sockets to non-blocking I/O mode; that way the server knows it can never get "stuck" inside a recv() or a send() call, and thus can remain responsive to all clients regardless of whether any particular client is behaving nicely, or not. In the non-blocking design, the only place the server ever blocks is inside select() or poll() or similar, because those calls are designed to return whenever any client needs service, rather than blocking on only a single client. (the tradeoff is that with non-blocking I/O your server's buffering/queueing logic will need to be a bit more elaborate, since you can no longer assume that any particular fixed number of bytes will be sent or received during any given send or receive operation)
The other way to avoid the problem is to make a multi-threaded server; that has the advantage that each client gets its own thread, and therefore a badly-behaved client will block only its own thread and not the threads servicing other clients. The disadvantage is that now your server is multi-threaded, with all of the additional pitfalls that multithreading introduces.
(and, for completeness, the third approach is simply to ignore the possibility of badly-behaved/poorly-connected clients, and use a single-threaded/blocking model. That works fine for toy examples where clients are expected to be non-hostile, and where the network they are connecting over is reliable, but doesn't work so well in real life)
Non-blocking IO is used when you prefer an error response (EWOULDBLOCK / EAGAIN) over your thread waiting (blocking) until an IO operation becomes possible.
This leads to the question of how is the IO multiplexing achieved?
If you're using a thread-per-connection model (or a process-per-connection), using blocking IO might be more comfortable.
However, if the same thread is serving multiple IO objects, blocking IO would be hazardous and could bring the whole application to a halt.
It is better to use non-blocking IO when a single thread serves multiple IO objects.
Note that the issue might not be noticeable at first when polling (using select / poll or epoll/kqueue).
Since the IO operations are only performed by a code path that already "knows" that the IO operation will not block (it was polled and known to be an available operation).
This masks the issue that somewhere in the code an IO operation might be called directly without polling first, resulting in a blocking IO call that will grind the application to a halt.
I am developing a proxy server using WinSock 2.0 in Windows. If I wanted to develop it in blocking model, select() was the way to wait for client or remote server to receive data from. Is there any applicable way to do this so using I/O Completion Ports?
I used to have two Contexts for two directions of data using I/O Completion Ports. But having a WSARecv pending couldn't receive any data from remote server! I coudn't find the problem.
Thanks in advance.
EDIT. Here's the WorkerThread Code on currently developed I/O Completion Ports. But I am asking about how to implement select() equivalence.
I/O Completion Ports provide an indication of when an I/O operation completes, they do not indicate when it is possible to initiate an operation. In many situations this doesn't actually matter. Most of the time the overlapped I/O model will work perfectly well if you assume it is always possible to initiate an operation. The underlying operating system will, in most cases, simply do the right thing and queue the data for you until it is possible to complete the operation.
However, there are some situations when this is less than ideal. For example you can always send to a socket using overlapped I/O. You can do this even when the remote peer is not reading and the TCP stack has started to use flow control and has filled the TCP window... This simply uses resources on your local machine in a completely uncontrolled manner (not entirely uncontrolled, but controlled by the peer, which is not ideal). I write about this here and in many situations you DO need to actively manage this kind of thing by tracking how many outstanding I/O write requests you have and using that as an indication of 'readiness to send'.
Likewise if you want a 'readiness to recv' indication you could issue a 'zero byte' read on the socket. This is a read which is issued with a zero length buffer. The read returns when there is data to read but no data is returned. This would give you the indication that there is data to be read on the connection but is, IMHO, pointless unless you are suffering from the very unlikely situation of hitting the I/O page lock limit, as you may as well read the data when it becomes available rather than forcing multiple kernel to user mode transitions.
In summary, you don't really need an answer to your question. You need to look at how the API works and write your code to work with it rather than trying to force the API to work in a way that other APIs that you are familiar with work.
Would the following single threaded UDP client application see a performance benefit from using epoll over simply calling recvfrom/sendto on non-blocking sockets?
Let me explain the client.
I am writing single threaded UDP based client (custom protocol) that both sends and receives data using non-blocking I/O and my colleague suggested I use epoll for this.
The client sends and receives multiple packets of information that are all associated with a unique session id and multiple sessions can be run simultaneously.
If I use epoll, there will be a limited number of maybe 10-20 file descriptors which epoll_wait could wait on. Each file descriptor would be associated with one session. So that's maximum 10 - 20 sessions and this number will be enforced.
Each session has it's own state machine. From a single thread I need to run each state machine reasonably frequently and poll the associated socket as well.
In my case, I'd have to use epoll_wait with a timeout of zero or some very small value so that I can give CPU time to run the state machines for each session.
If there is data for a session then it needs to be directed to the associated state machine.
However, I can't really see much benefit of this design with such a small number of file descriptors.
The way I see it is I have two design options:
1. In my main loop using epoll I can poll the descriptors using epoll_wait with either a small timeout or no timeout.
How it handles data at this point is where I'm getting a bit stuck... either I read it right away and then throw it into a queue for each state machine to pick up when it's run, or I set a flag on the state machine to tell it that data is waiting and when the state machine runs it'll pick it up with a call to recvfrom. Or, I read the data and handle it right away and run the state machine for it.
Or...
2, Just run each state machine from the main loop and call recvfrom. If I get some data, handle it. If I don't then do whatever else the state machine requires. Is there huge overhead calling recvfrom when there is no data?
With going the epoll route I'm coding in some extra complexity. If there is a strong likelyhood for it be faster in my case then I will start doing it. However, if the second way which really simple works just as well then I would not use epoll.
Any thoughts?
No, and in fact performance will be much worse using epoll if adding and removing file descriptors from the set to poll is anything but an extremely rare event. With poll, a single syscall performs the entire operation. With epoll, you need multiple syscalls to modify the set and then wait on it.
Unless you're writing a server that's intended to scale to tens, hundreds, or thousands of thousands of long-term persistent connections, epoll is not only premature optimization, but actually a pessimization. It's also completely nonstandard and non-portable.
Say there are two programs running on a computer (for the sake of simplification, the only user programs running on linux) one of which calls recv(), and one of which is using pcap to detect incoming packets. A packet arrives, and it is detected by both the program using pcap, and by the program using recv. But, is there any case (for instance recv() returning between calls to pcap_next()) in which one of these two will not get the packet?
I really don't understand how the buffering system works here, so the more detailed explanation the better - is there any conceivable case in which one of these programs would see a packet that the other does not? And if so, what is it and how can I prevent it?
AFAIK, there do exist cases where one would receive the data and the other wouldn't (both ways). It's possible that I've gotten some of the details wrong here, but I'm sure someone will correct me.
Pcap uses different mechanisms to sniff on interfaces, but here's how the general case works:
A network card receives a packet (the driver is notified via an interrupt)
The kernel places that packet into appropriate listening queues: e.g.,
The TCP stack.
A bridge driver, if the interface is bridged.
The interface that PCAP uses (a raw socket connection).
Those buffers are flushed independently of each other:
As TCP streams are assembled and data delivered to processes.
As the bridge sends the packet to the appropriate connected interfaces.
As PCAP reads received packets.
I would guess that there is no hard way to guarantee that both programs receive both packets. That would require blocking on a buffer when it's full (and that could lead to starvation, deadlock, all kinds of problems). It may be possible with interconnects other than Ethernet, but the general philosophy there is best-effort.
Unless the system is under heavy-load however, I would say that the loss rates would be quite low and that most packets would be received by all. You can decrease the risk of loss by increasing the buffer sizes. A quick google search tuned this up, but I'm sure there's a million more ways to do it.
If you need hard guarantees, I think a more powerful model of the network is needed. I've heard great things about Netgraph for these kinds of tasks. You could also just install a physical box that inspects packets (the hardest guarantee you can get).
I have to implement a game server in C which handles multiple clients and continuously exchange information with them. The client may not be sending information at all times.Should I assign a thread with non-blocking socket to them or use select() call.
Which one is better?
Both will work just as well in most cases. Note that the thread version will use blocking sockets, and the select-based uses non-blocking sockets. In the case of a server, you may feel that events for data received is a good model.
The threaded version will have the memory-overhead of allocating a stack for each thread (often the size of a page), but you can program as if you have only one client.
The evented version needs to maintain state between callbacks, which can be more work, but again, in servers it feels quite natural.
select() is the way to go, that's what it's made for. If you go for the threaded non-blocking approach, you will have to implement a sleep after each tick or the thread will use all available cpu time. So, the worst case response time, if one client is sending data, is your sleep value. You could also implement one thread per socket and make it blocking, but depending on how many sockets you have, that will be much overhead.
With select() you can watch all sockets at once (no matter if they are blocking or not, btw) and only have to process those which are active.
If you are on linux an have many sockets to watch, you can take a look at epoll()