What happens if I have one socket, s, there is no data currently available on it, it is a blocking socket, and I call recv on it from two threads at once? Will one of the threads get the data? Will both get it? Will the 2nd call to recv return with an error?
One thread will get it, and there's no way to tell which.
This doesn't seem like a reasonable design. Is there a reason why you need two threads calling recv() on the same socket?
Socket implementations should be thread-safe, so exactly one thread should get the data when it becomes available. The other call should just block.
I can't find a reference for this, but here's my understanding:
A vendor's guarantee of thread-safety may mean only that multiple threads can each safely use their own sockets; it does not guarantee atomicity across a single call, and it doesn't promise any particular allocation of the socket's data among multiple threads.
Suppose thread A calls recv() on a socket that's receiving TCP data streaming in at a high rate. If recv() needs to be an atomic call, then thread A could block all other threads from executing, because it needs to be running continuously to pull in all the data (until its buffer is full, anyway.) That wouldn't be good. Hence, I would not assume that recv() is immune to context switching.
Conversely, suppose thread A makes a blocking call to recv() on a TCP socket, and the data is coming in slowly. Hence the call to recv() returns with errno set to EAGAIN.
In either of these cases, suppose thread B calls recv() on the same socket while thread A is still receiving data. When does thread A stop getting data handed to it so that thread B can start receiving data? I don't know of a Unix implementation that will try to remember that thread A was in the middle of an operation on the socket; instead, it's up to the application (threads A and B) to negotiate their use of it.
Generally, it's best to design the app so that only one of the threads will call recv() on a single socket.
From the man page on recv
A recv() on a SOCK_STREAM socket
returns as much available information
as the size of the buffer supplied can
hold.
Lets assume you are using TCP, since it was not specified in the question. So suppose you have thread A and thread B both blocking on recv() for socket s. Once s has some data to be received it will unblock one of the threads, lets say A, and return the data. The data returned will be of some random size as far as we are concerned. Thread A inspects the data received and decides if it has a complete "message", where a message is an application level concept.
Thread A decides it does not have a complete message, so it calls recv() again. BUT in the meantime B was already blocking on the same socket, and has received the rest of the "message" that was intended for thread A. I am using intended loosely here.
Now both thread A and thread B have an incomplete message, and will, depending on how the code is written, throw the data away as invalid, or cause weird and subtle errors.
I wish I could say I didn't know this from experience.
So while recv() itself is technically thread safe, it is a bad idea to have two threads calling it simultaneously if you are using it for TCP.
As far as I know it is completely safe when you are using UDP.
I hope this helps.
Related
I've seen a few write-ups comparing select() with poll() or epoll(), and I've seen many guides discussing the actual usage of select() with multiple sockets.
However, what I can't seem to find is a comparison to a non-blocking recv() call without select(). In the event of only having 1 socket to read from and 1 socket to write to, is there any justification for using the select() call? The recv() method can be setup to not block and return an error (WSAEWOULDBLOCK) when there is no data available, so why bother to call select() when you have no other sockets to examine? Is the non-blocking recv() call much slower?
You wouldn't want a non-blocking call to recv without some other means for waiting for data on the socket as you poll infinitely eating up cpu time.
If you have no other sockets to examine and nothing else to do in the same thread, a blocking call to read is likely to be the most efficient solution. Although in such a situation, considering the efficiency of this is like to be premature optimisation.
These kinds of considerations only tend to come into play as the socket count increases.
Nonblocking calls are only faster in the context of handling multiple sockets on a single thread.
If there is no data available, and you use non-blocking IO, recv() will return immediately.
Then what should the program do ? You would need to call recv() in a loop until data becomes available - this just uses CPU for pretty much no reason.
Spinning on recv() and burning CPU in that manner is very undesirable; you'd rather want the process to wait until data becomes available and get woken up; that's what select()/poll() and similar does.
And, sleep() in the loop in order to not burn CPU is not a good solution either. You'd introduce high latency in the processing as the program will not be able to process data as soon as the data is available.
select() and friends let you design the workflow in such a way that slowness of one socket does not impede the speed at which you can serve another. Imagine that data arrives fast from the receiving socket and you want to accept it as fast as possible and store in memory buffers. But the sending socket is slow. When you've filled up the sending buffers of the OS and send() gave you EWOULDBLOCK, you can issue select() to wait on both receiving and sending sockets. select() will fall through if either new data on the receiving socket arrived, or some buffers are freed and you can write more data to the sending socket, whichever happens first.
Of course a more realistic use case for select() is when you have multiple sockets to read from and/or to write to, or when you must pass the data between your two sockets in both directions.
In fact, select() tells you when the next read or write operation on a socket is known to succeed, so if you only try to read and write when select allows you, your program will almost work even if you didn't make the sockets non-blocking! It is still unwise to do, because there exist edge cases when the next operation still may block despite select() reported that the socket as "ready".
On the other hand, making the sockets non-blocking and not using select() is almost never advisable because of the reason explained by #Troy.
I have 20 threads all sending data on single tcp socket at a time and receiving data as well. When I run my application I don’t see any synchronization issues but according to my understanding, there may be some issues when two threads simultaneously try to write to tcp socket or when one thread is writing while other is reading.
If my understanding is correct, why don’t I face any errors?
Sometimes when you don't look both ways before crossing the street, you still get to the other side of the street safely. That doesn't mean it will work successfully every time you do it.
Here's the thing, you say "you don't see any synchronization issues", but that's only because it happens to do what you want it to do. Flip this around -- the reason you don't see any synchronization issues is because you happen to want it to do what it happens to do. Someone who expected it to do something else would see synchronization issues with the very same code.
In other words, you flipped a coin that could come up heads or tails. You expected it to come up heads, knowing that it was not guaranteed. And it came up heads. There's no mystery -- the explanation is that you expected what it happened to do. Had you expected something else, even had it done the very same thing, it would not have done what you expected.
First, the send and receive streams of each socket are independent. There should be no problem with one thread sending while another is receiving.
If multiple threads attempt to write to one socket, the behaviour is, in general, undefined. In practice, a write call from one of the threads will get into a lock in the TCP stack state-machine first, preventing any other threads from entering, write its data, release the lock and exit the stack, so allowing write calls from other threads to proceed. The would allow single write calls to be serialized. If your protocol implementation can send all PDU's with one write call, then fine. If a PDU requires more than one write call, then your outgoing PDU's can get sliced as the write calls from the multiple threads get interleaved.
Making receive calls from multiple threads to one socket is just... something. Even if the stack internal synchro allows only one receive call per socket at a time, the streaming nature of TCP would surely split up the received data in a pseudo-arbitrary manner across the threads. Just don't do it, it's crazy.
TCP already has a mechanism for multiplexing data streams - multiple sockets. You should use them correctly.
If you need to multiplex data streams across one socket, you should add a data routing protocol on top of TCP and implement this protocol in just one receive thread. This thread can keep a list of virtual connections and so service stream/message requests from other threads.
When a socket is signalled as being OK to write by a call to select(), how can I know how much data I can send without blocking? (In the case of full send buffers etc.)
Does inclusion in the set returned by select() signify that the socket is ready for at least one byte of data, and will send() then return a short count of written bytes?
Or will it block when I call send() with a len parameter that is bigger than the available buffer space? If so, how do I know the maximum amount?
I'm using regular C sockets on Linux.
The send call should not block on the first call, and should send at least one byte on the first call -- assuming you are using a stream protocol and assuming it's not interrupted by a signal, etc. However, there are really only two ways to figure out how much data you can send:
Call select after every call to send to see if more data can be sent.
Put the socket in non-blocking mode, and call send until it gives an EAGAIN or EWOULDBLOCK error.
The second option is preferred. (The third option is to do it in a different thread and simply let the thread block, which is also a good solution. In the past, threading implementations weren't as mature so non-blocking mode was seen as necessary for high-performance servers.)
You cannot know. You have to sent the socket to be non-blocking, and then pay attention to the return value that tells you how much it has written.
Can we call send from one thread and recv from another on the same socket?
Can we call multiple sends parallely from different threads on the same socket?
I know that a good design should avoid this, but I am not clear how these system APIs will behave. I am unable to find a good documentation also for the same.
Any pointers in the direction will be helpful.
POSIX defines send/recv as atomic operations, so assuming you're talking about POSIX send/recv then yes, you can call them simultaneously from multiple threads and things will work.
This doesn't necessarily mean that they'll be executed in parallel -- in the case of multiple sends, the second will likely block until the first completes. You probably won't notice this much, as a send completes once its put its data into the socket buffer.
If you're using SOCK_STREAM sockets, trying to do things a parallel is less likely to be useful as send/recv might send or receive only part of a message, which means things could get split up.
Blocking send/recv on SOCK_STREAM sockets only block until they send or recv at least 1 byte, so the difference between blocking and non-blocking is not useful.
The socket descriptor belongs to the process, not to a particular thread. Hence, it is possible to send/receive to/from the same socket in different threads, the OS will handle the synchronization.
However, if the order of sending/receiving is semantically significant, you yourself (respectively your code) have to ensure proper sequencing between the operations in the different threads - as is always the case with threads.
I don't see how receiving in parallel could possibly accomplish anything. If you have a 3 bytes message, 1 thread could get the 1st 2 bytes and another the last byte, but you'd have no way of telling which was which. Unless your messages are only a byte long, there is no way you could reliably make anything work with multiple threads receiving.
Multiple sends might work, if you sent the entire message in a single call, but I'm not sure. It's possible that one could overwrite another. There certainly wouldn't be any performance benefit to doing so.
If multiple threads need to send, you should implement a synchronized message queue. Have one thread that does the actual sending that reads messages from the queue and have the other threads enqueue whole messages. The same thing would work for receiving, but the receive thread would have to know the format of the messages so it could deserialize them properly.
I have a worker thread that is listening to a TCP socket for incoming traffic, and buffering the received data for the main thread to access (let's call this socket A). However, the worker thread also has to do some regular operations (say, once per second), even if there is no data coming in. Therefore, I use select() with a timeout, so that I don't need to keep polling. (Note that calling receive() on a non-blocking socket and then sleeping for a second is not good: the incoming data should be immediately available for the main thread, even though the main thread might not always be able to process it right away, hence the need for buffering.)
Now, I also need to be able to signal the worker thread to do some other stuff immediately; from the main thread, I need to make the worker thread's select() return right away. For now, I have solved this as follows (approach basically adopted from here and here):
At program startup, the worker thread creates for this purpose an additional socket of the datagram (UDP) type, and binds it to some random port (let's call this socket B). Likewise, the main thread creates a datagram socket for sending. In its call to select(), the worker thread now lists both A and B in the fd_set. When the main thread needs to signal, it sendto()'s a couple of bytes to the corresponding port on localhost. Back in the worker thread, if B remains in the fd_set after select() returns, then recvfrom() is called and the bytes received are simply ignored.
This seems to work very well, but I can't say I like the solution, mainly as it requires binding an extra port for B, and also because it adds several additional socket API calls which may fail I guess – and I don't really feel like figuring out the appropriate action for each of the cases.
I think ideally, I would like to call some function which takes A as input, and does nothing except makes select() return right away. However, I don't know such a function. (I guess I could for example shutdown() the socket, but the side effects are not really acceptable :)
If this is not possible, the second best option would be creating a B which is much dummier than a real UDP socket, and doesn't really require allocating any limited resources (beyond a reasonable amount of memory). I guess Unix domain sockets would do exactly this, but: the solution should not be much less cross-platform than what I currently have, though some moderate amount of #ifdef stuff is fine. (I am targeting mainly for Windows and Linux – and writing C++ by the way.)
Please don't suggest refactoring to get rid of the two separate threads. This design is necessary because the main thread may be blocked for extended periods (e.g., doing some intensive computation – and I can't start periodically calling receive() from the innermost loop of calculation), and in the meanwhile, someone needs to buffer the incoming data (and due to reasons beyond what I can control, it cannot be the sender).
Now that I was writing this, I realized that someone is definitely going to reply simply "Boost.Asio", so I just had my first look at it... Couldn't find an obvious solution, though. Do note that I also cannot (easily) affect how socket A is created, but I should be able to let other objects wrap it, if necessary.
You are almost there. Use a "self-pipe" trick. Open a pipe, add it to your select() read and write fd_set, write to it from main thread to unblock a worker thread. It is portable across POSIX systems.
I have seen a variant of similar technique for Windows in one system (in fact used together with the method above, separated by #ifdef WIN32). Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. The downside is that, of course, you have to re-open it every time.
However, in the aforementioned system, both of these methods are used rather sparingly, and for unexpected events (e.g., signals, termination requests). Preferred method is still a variable timeout to select(), depending on how soon something is scheduled for a worker thread.
Using a pipe rather than socket is a bit cleaner, as there is no possibility for another process to get hold of it and mess things up.
Using a UDP socket definitely creates the potential for stray packets to come in and interfere.
An anonymous pipe will never be available to any other process (unless you give it to it).
You could also use signals, but in a multithreaded program you'll want to make sure that all threads except for the one you want have that signal masked.
On unix it will be straightforward with using a pipe. If you are on windows and want to keep using the select statement to keep your code compatible with unix, the trick to create an unbound UDP socket and close it, works well and easy. But you have to make it multi-threadsafe.
The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.
To be simple:
a global var saves the socket handle, then close the global socket, the select() will return immediately: closesocket(g_socket);