I'm coding for a linux platform using C. Let's say I have 2 threads. A and B.
A is an infinite loop and constantly trying to find out if there is data on the socket localhost:8080, where as B is a thread that spends most of its time in a blocked state until A calls mutex unlock function on a mutex that B uses to block itself. A will unlock B when it received appropriate data on the socket.
So you see here is a problem. B is "event driven" largely whereas A is in a constant running state. My target platform isn't resource rich so I wish A could be "activated" and enter running state only when it received data on socket, instead of constantly looping.
So how can I do that? If it matters - I wish to do this for both UDP and TCP sockets.
There are Multiple was of doing what you want in a clean was. One approach, you are kind of using already, is a event system. A real event system would be overkill for the kind of problem you are dealing with, but can be found here. This is a (random) better implementation, capable of listening for multiple file descriptors and time based events, all in a single thread.
If you want to build one yourself, you should take a look at the select or poll function.
But I agree with #Jeremy Friesner, you should definitely use the functions made for socket programming, they are perfect for your kind of problem. Only use the event system approach if you really need it (with multiple sockets/timed events).
You simply call recv (or recvfrom, recvmsg, etc) and it doesn't return until some data has been received. There's no need to "constantly try to find out if there is data" - that's silly.
If you set the socket to non-blocking mode then recv will return even if there's no data. If that's what you're doing, then the solution is simple: don't set the socket to non-blocking mode.
Related
I have 20 threads all sending data on single tcp socket at a time and receiving data as well. When I run my application I don’t see any synchronization issues but according to my understanding, there may be some issues when two threads simultaneously try to write to tcp socket or when one thread is writing while other is reading.
If my understanding is correct, why don’t I face any errors?
Sometimes when you don't look both ways before crossing the street, you still get to the other side of the street safely. That doesn't mean it will work successfully every time you do it.
Here's the thing, you say "you don't see any synchronization issues", but that's only because it happens to do what you want it to do. Flip this around -- the reason you don't see any synchronization issues is because you happen to want it to do what it happens to do. Someone who expected it to do something else would see synchronization issues with the very same code.
In other words, you flipped a coin that could come up heads or tails. You expected it to come up heads, knowing that it was not guaranteed. And it came up heads. There's no mystery -- the explanation is that you expected what it happened to do. Had you expected something else, even had it done the very same thing, it would not have done what you expected.
First, the send and receive streams of each socket are independent. There should be no problem with one thread sending while another is receiving.
If multiple threads attempt to write to one socket, the behaviour is, in general, undefined. In practice, a write call from one of the threads will get into a lock in the TCP stack state-machine first, preventing any other threads from entering, write its data, release the lock and exit the stack, so allowing write calls from other threads to proceed. The would allow single write calls to be serialized. If your protocol implementation can send all PDU's with one write call, then fine. If a PDU requires more than one write call, then your outgoing PDU's can get sliced as the write calls from the multiple threads get interleaved.
Making receive calls from multiple threads to one socket is just... something. Even if the stack internal synchro allows only one receive call per socket at a time, the streaming nature of TCP would surely split up the received data in a pseudo-arbitrary manner across the threads. Just don't do it, it's crazy.
TCP already has a mechanism for multiplexing data streams - multiple sockets. You should use them correctly.
If you need to multiplex data streams across one socket, you should add a data routing protocol on top of TCP and implement this protocol in just one receive thread. This thread can keep a list of virtual connections and so service stream/message requests from other threads.
I'm trying to implement Bittorrent client. in order to receive pieces from different peers, The client should manage multiple socket.
Well-known solution that I know are
1. Each thread has one socket.
2. Using select() call, non-blocking I/O.
3. a mix of 1 and 2.
The first solution requires too many threads. The second solution wastes CPU time since it continue to checks maximum 50 socket. Also, when deciding to use the third solution, I don't know how many threads a single process use.
Which solution is the best one, to receive a fairly large file?
Is there any web page that give me a good solution?
Any advice would be awesome.
Some High Level Ideas from my side. : )
Have a main thread in which you will be doing the "select" / "poll" call for all the connections.
Have a thread pool of worker threads
If for a particular connection, select indicates that there is data to read, then pass the socket + additional information to one of the free worker threads for receiving / sending data on that connection.
Upon completion of the work, the worker thread returns to the free worker thread queue, which can be used again for another connection.
Hope this helps
You're right, the first solution is the worst.
The second one, with select() can do the job, but there's a problem: select() has a complexity of log(n). You should use /dev/poll, epoll(), kqueue() or whatever, but don't use select().
Don't use one thread per socket !! You will loose a lot of time due to the context switch.
You should have:
A Listener thread : just do all the accept and put the new socket
in a Worker thread.
Multiple Worker thread: do all the other stuff. It will check if there's data available and will handle it. A Worker thread manage many sockets.
Take a look at the Kegel's c10k page if you want more informations.
Check some Open Source BitTorrent client and check the code to get some ideas, it is the best thing you could do.
I recommend you to check BitTorrent in C or Hadouken in C# for example:
https://github.com/bittorrent
https://github.com/hadouken/hdkn
A couple of days ago I had to investigate a problem where my application was showing abnormally high CPU usage when it was (apparently) in idle state. I tracked the problem down to a loop which was meant to block on a recvfrom call while the socket had been set to O_NONBLOCK-ing resulting in a spin lock. There were two ways of solving the problem: set the socket to blocking or poll for available data on the socket using poll or select. I chose the former as it was simpler. But I am wondering why any one would create a non-blocking socket and then poll on it separately. Doesn't a blocking socket do the same? What are the uses cases where one would use a non-blocking socket and poll combination? Are there any advantages to it in general cases?
Using poll() or select() with a non-blocking file descriptor gives you two advantages:
You can set a timeout to block for;
You can wait for any of a set of file descriptors to become useable.
If you only have a single file descriptor (socket) to wait for, and you don't mind waiting indefinitely on it, then yes; you can just use a blocking call.
The second advantage is really the killer use case for select() and friends. It means that you can handle multiple socket connections, as well as standard input and standard output and possibly file I/O, all with a single thread of control.
I´m posting here, because although the question is old. It came up in my google search somehow and has definitely not been answered properly.
The accepted answer merely highlights two advantages of using non-blocking sockets but does not really go into detail or answer the actual question.
NOTE : Unfortunately most online "tutorials" or code snippets only feature blocking socket code, so knowledge on non-blocking sockets is less spread.
As to when you would you use one compared to the other ... in general blocking sockets are only used in online code snippets. In all (good) production applications non-blocking sockets are used. I´m not ignorant, if you know of an implementation that uses blocking sockets (and sure that´s very well possible in combination with threads) - or let´s be more specific that uses blocking sockets in a single thread - please do let me know.
Now I can give you a very easy to understand example, and there are many others out there. Let´s take the example of a gaming server. Games advances at ticks, regular intervals where the game state progresses whether or not the player provides input (mouse / keyboard) to change the state of the game. Now when sockets come into play in Multiplayer games - if you were to use blocking sockets the game state would not advance unless the players were sending updates - so if they have internet problems, the game state would never consistently update and propagate changes to all players. You would have a rather choppy experience.
Now using non-blocking sockets, you can run the gameserver on a single-thread, updating the gamestate as well as the sockets, with a ... let´s say 50ms timeout interval - and socket data is only read from connected users when they actually send something, and then fed into the server simulation, processed and fed into the game state calculation for the next tick.
resulting in a spin lock.
That condition normally is called a tight loop.
There were two ways of solving the problem: set the socket to blocking or poll for available data on the socket using poll or select. I chose the former as it was simpler.
Are you sure that other code parts do not already use poll() (or select()) and expect the socket to be in non-blocking mode?
Otherwise, then yes, the switch to the blocking mode is the simplest solution.
Best backward-compatible solution would have been before calling recvfrom() to use poll() to wait for the socket to become readable. That way ensures that other parts of the code would work precisely as before.
But I am wondering why any one would create a non-blocking socket and then poll on it separately. Doesn't a blocking socket do the same?
For the case of recvfrom() no major difference is known to me.
What are the uses cases where one would use a non-blocking socket and poll combination? Are there any advantages to it in general cases?
Could be a simple coding mistake. Or somebody might have thought that recv'ing in tight loop would somehow increase the performance.
It is always better to make sockets as nonblocking because even a blocking socket becomes ready state sometimes (when data arrived but has checksum error and that is discarded) - even when there is no data to read. So make it nonblocking, wait for the data availability through poll then read. I think this is the main advantage.
I do not understand what the difference is between calling recv() on a non-blocking socket vs a blocking socket after waiting to call recv() after select returns that it is ready for reading. It would seem to me like a blocking socket will never block in this situation anyway.
Also, I have heard that one model for using non blocking sockets is try to make calls (recv/send/etc) on them after some amount of time has passed instead of using something like select. This technique seems slow and wasteful to be compared to using something like select (but then I don't get the purpose of non-blocking at all as described above). Is this common in networking programming today?
There's a great overview of all of the different options for doing high-volume I/O called The C10K Problem. It has a fairly complete survey of a lot of the different options, at least as of 2006.
Quoting from it, on the topic of using select on non-blocking sockets:
Note: it's particularly important to remember that readiness notification from the kernel is only a hint; the file descriptor might not be ready anymore when you try to read from it. That's why it's important to use nonblocking mode when using readiness notification.
And yes, you could use non-blocking sockets and then have a loop that waits if nothing is ready, but that is fairly wasteful compared to using something like select or one of the more modern replacements (epoll, kqueue, etc). I can't think of a reason why anyone would actually want to do this; all of the select like options have the ability to set a timeout, so you can be woken up after a certain amount of time to perform some regular action. I suppose if you were doing something fairly CPU intensive, like running a video game, you may want to never sleep but instead keep computing, while periodically checking for I/O using non-blocking sockets.
The select, poll, epoll, kqueue, etc. facilities target multiple socket/file descriptor handling scenarios. Imagine a heavy loaded web-server with hundreds of simultaneously connected sockets. How would you know when to read and from what socket without blocking everything?
If you call read on a non-blocking socket, it will return immediately if no data has been received since the last call to read. If you only had read, and you wanted to wait until there was data available, you would have to busy wait. This wastes CPU.
poll and select (and friends) allow you to sleep until there's data to read (or write, or a signal has been received, etc.).
If the only thing you're doing is sending and receiving on that socket, you might as well just use a non-blocking socket. Being asynchronous is important when you have other things to do in the meantime, such as update a GUI or handle other sockets.
For your first question, there's no difference in that scenario. The only difference is what they do when there is nothing to be read. Since you're checking that before calling recv() you'll see no difference.
For the second question, the way I see it done in all the libraries is to use select, poll, epoll, kqueue for testing if data is available. The select method is the oldest, and least desirable from a performance standpoint (particularly for managing large numbers of connections).
I have a worker thread that is listening to a TCP socket for incoming traffic, and buffering the received data for the main thread to access (let's call this socket A). However, the worker thread also has to do some regular operations (say, once per second), even if there is no data coming in. Therefore, I use select() with a timeout, so that I don't need to keep polling. (Note that calling receive() on a non-blocking socket and then sleeping for a second is not good: the incoming data should be immediately available for the main thread, even though the main thread might not always be able to process it right away, hence the need for buffering.)
Now, I also need to be able to signal the worker thread to do some other stuff immediately; from the main thread, I need to make the worker thread's select() return right away. For now, I have solved this as follows (approach basically adopted from here and here):
At program startup, the worker thread creates for this purpose an additional socket of the datagram (UDP) type, and binds it to some random port (let's call this socket B). Likewise, the main thread creates a datagram socket for sending. In its call to select(), the worker thread now lists both A and B in the fd_set. When the main thread needs to signal, it sendto()'s a couple of bytes to the corresponding port on localhost. Back in the worker thread, if B remains in the fd_set after select() returns, then recvfrom() is called and the bytes received are simply ignored.
This seems to work very well, but I can't say I like the solution, mainly as it requires binding an extra port for B, and also because it adds several additional socket API calls which may fail I guess – and I don't really feel like figuring out the appropriate action for each of the cases.
I think ideally, I would like to call some function which takes A as input, and does nothing except makes select() return right away. However, I don't know such a function. (I guess I could for example shutdown() the socket, but the side effects are not really acceptable :)
If this is not possible, the second best option would be creating a B which is much dummier than a real UDP socket, and doesn't really require allocating any limited resources (beyond a reasonable amount of memory). I guess Unix domain sockets would do exactly this, but: the solution should not be much less cross-platform than what I currently have, though some moderate amount of #ifdef stuff is fine. (I am targeting mainly for Windows and Linux – and writing C++ by the way.)
Please don't suggest refactoring to get rid of the two separate threads. This design is necessary because the main thread may be blocked for extended periods (e.g., doing some intensive computation – and I can't start periodically calling receive() from the innermost loop of calculation), and in the meanwhile, someone needs to buffer the incoming data (and due to reasons beyond what I can control, it cannot be the sender).
Now that I was writing this, I realized that someone is definitely going to reply simply "Boost.Asio", so I just had my first look at it... Couldn't find an obvious solution, though. Do note that I also cannot (easily) affect how socket A is created, but I should be able to let other objects wrap it, if necessary.
You are almost there. Use a "self-pipe" trick. Open a pipe, add it to your select() read and write fd_set, write to it from main thread to unblock a worker thread. It is portable across POSIX systems.
I have seen a variant of similar technique for Windows in one system (in fact used together with the method above, separated by #ifdef WIN32). Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. The downside is that, of course, you have to re-open it every time.
However, in the aforementioned system, both of these methods are used rather sparingly, and for unexpected events (e.g., signals, termination requests). Preferred method is still a variable timeout to select(), depending on how soon something is scheduled for a worker thread.
Using a pipe rather than socket is a bit cleaner, as there is no possibility for another process to get hold of it and mess things up.
Using a UDP socket definitely creates the potential for stray packets to come in and interfere.
An anonymous pipe will never be available to any other process (unless you give it to it).
You could also use signals, but in a multithreaded program you'll want to make sure that all threads except for the one you want have that signal masked.
On unix it will be straightforward with using a pipe. If you are on windows and want to keep using the select statement to keep your code compatible with unix, the trick to create an unbound UDP socket and close it, works well and easy. But you have to make it multi-threadsafe.
The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.
To be simple:
a global var saves the socket handle, then close the global socket, the select() will return immediately: closesocket(g_socket);