I'm working on a software that will be running on HP blades/Linux, each blade will have multiple programs and I'm considering UDP for the IPC communication. The size of messages between the blades/programs won't be bigger than a 400Bytes.
I used to use TCP before and I'm not experienced in using UDP so the question here is this, is using UDP for cluster communication wise based on your experience?
It depends on your requirements for reliability. As you know UDP provides no delivery guarantees or even ordering guarantees (packets may arrive out-of-order). If your application is tolerant of this, or if you can make it tolerant with relatively simple code, UDP is definitely a better choice - it is lower latency, lower overhead, and programmatically simpler to deal with.
If reliability is an absolute requirement, then unless you're really hard core and trying to squeeze every last ounce of performance out of your cluster, just use TCP. Otherwise you'll simply find yourself trying to reinvent the mechanisms TCP uses to guarantee reliability, and you probably won't do as good a job as TCP does (it's had decades of tweaking and tuning).
Also note that on small LANs, despite the lack of any guarantee, UDP is quite reliable, but even in a perfect setup you still have to expect the occasional dropped packet. The more complex your cluster's network gets and the higher the bandwidth utilization of the system, the less reliable it will be.
Related
I'm a C programmer and may write programs in Linux area to connect machines over internet. After I examined that speedtest.net can't upload or has a very poor upload speed, I decided to examine a simple TCP socket connection and see whether it's really slow, and found that yes, it's really slow. I've rented a VPS outside of my country. I don't know what's happening by the government in infrastructure and how packets are routed and how they're restricted. Examining what I saw in speedtest.net, again in a simple socket connection proves me that I can't have a chance. When the traffic in shaped so, there's no way. It proves that it's not a restriction on HTTPS or any application layer protocol, when just a simple TCP socket connection also can't succeed to gain reasonable speed. The speed is below 10 kilobytes per second! Damn!
In contrast, after I got disappointed, I examined some barrier breakers like CyberGhost extension for Chrome. I wondered when I saw that it may overcome the barrier by increasing the upload speed to about 200 kilobytes per second! How?! They can't use any method closer to hardware than sockets.
Now I come here to consult with you and see what ideas you may have about it, so that I may write a program or change the written program based on it.
Thank you
Consider the prototypical multiplayer game server.
Clients connecting to the server are allowed to download maps and scripts. It is straightforward to create a TCP connection to accomplish this.
However, the server must continue to be responsive to the rest of the clients via UDP. If TCP download connections are allowed to saturate available bandwidth, UDP traffic will suffer severely from packet loss.
What might be the best way to deal with this issue? It definitely seems like a good idea to "throttle" the TCP upload connection somehow by keeping track of time, and send() on a regular time interval. This way, if UDP packet loss starts to occur more frequently the TCP connections may be throttled further. Will the OS tend to still bunch the data together rather than sending it off in a steady stream? How often would I want to be calling send()? I imagine doing it too often would cause the data to be buffered together first rendering the method ineffective, and doing it too infrequently would provide insufficient (and inefficient use of) bandwidth. Similar considerations exist with regard to how much data to send each time.
It sounds a lot like you're solving a problem the wrong way:
If you're worried about losing UDP packets, you should consider not using UDP.
If you're worried about sharing bandwidth between two functions, you should consider having separate pipes (bandwidth) for them.
Traffic shaping (which is what this sounds like) is typically addressed in the OS. You should look in that direction before making strange changes to your application.
If you haven't already gotten the application working and experienced this problem, you are probably prematurely optimizing.
To avoid saturating the bandwidth, you need to apply some sort of rate limiting. TCP actually already does this, but it might not be effective in some cases. For example, it has no idea weather you consider the TCP or UDP traffic to be the more important.
To implement any form of rate limiting involving UDP, you will first need to calculate UDP loss rate. UDP packets will need to have sequence numbers, and then the client has to count how many unique packets it actually got, and send this information back to the server. This gives you the packet loss rate. The server should monitor this, and if packet loss jumps after a file transfer is started, start lowering the transfer rate until the packet loss becomes acceptable. (You will probably need to do this for UDP anyway, since UDP has no congestion control.)
Note that while I mention "server" above, it could really be done either direction, or both. Depending on who needs to send what. Imagine a game with player created maps that transfer these maps with peer-to-peer connections.
While lowering the transfer rate can be as simple as calling your send function less frequently, attempting to control TCP this way will no doubt conflict with the existing rate control TCP has. As suggested in another answer, you might consider looking into more comprehensive ways to control TCP.
In this particular case, I doubt it would be an issue, unless you really need to send lots of UDP information while the clients are transferring files.
I wold expect most games to just show a loading screen or a lobby while this is happening. Neither should require much UDP traffic unless your game has it's own VOIP.
Here is an excellent article series that explains some of the possible uses of both TCP and UDP, specifically in the context of network games. TCP vs. UDP
In a later article from the series, he even explains a way to make UDP 'almost' as reliable as TCP (with code examples).
And as always... and measure your results. You have no way of knowing if your code is making the connections faster or slower unless you measure.
"# If you're worried about losing UDP packets, you should consider not using UDP."
Right on. UDP means no guarentee of packet delivery, especially over the internet. Check the TCP speed which is quite acceptable in modern day internet connections for most users playing games.
I am implementing bit torrent for P2p file sharing. Let's say, Maximum of among 100 peers sharing simultaneously. TCP Connections are setup between each peer to every other peer. Initially, One peer has whole file and it starts sharing pieces and subsequently, all peers share their pieces.
Typically, piece size is 50kB - 1MB. I am wondering, What is the best approach to write such application in C. Using threads with epoll or libevent??
Can anybody please give positives/negatives of different possible approaches??
If we're only talking about 100 peer connections at any given moment, the traditional approach of using select or poll on a group of TCP sockets will work out just fine.
EPoll helps solve the problem of when you need to scale to thousands of long running connections. Read the doc on the C10K problem for more details.
I've heard good things about libevent. I believe it's an abstraction on top of epoll and other socket functions that provides a few nice things. If it makes your programming easier, then by all means use it. But you probably don't need it for performance.
Libevent is essentially a wrapper around epoll, mostly considered for writing good portable code. Since its a wrapper, drawbacks of epoll will be retained and does not add much from performance perspective. If portability is not concern, epoll should just work fine. Even better, if the volume is considerably less than one still use poll.
I am developing a UDP client module in Solaris using C, and there are 2 design modules:
(1) Create a socket, and send all messages through this socket. The receive thread only call recvfrom on this socket.
(2) Create a group of sockets. When sending message, select a socket randomly from the socket pool. The receive thread needs to call poll or select on a group of sockets.
When the throughput is low, I think the first design module is OK.
If the throughput is high, I am wondering whether the second design module can be better?
Because it will dispatch messages to a group of sockets, and this maybe improve UDP datagram delivery successful rate and more efficient.
There's still only one network. You can have as many sockets, threads, whatever, as you like. The rate-determining step is the network. There is no point to this.
The question here primarily depends on how parallel the computer is (number of cores) and how parallel the algorithm is. Most likely your CPU cores are vastly faster than the network connection anyway and even one of them could easily overwhelm the connection. Thus on a typical system option (1) will give significantly better performance and lower drop rates.
This is because there is a significant overhead to using a UDP port on several threads or processes due to the internal locking the OS has to do to ensure the packets' contents are not multiplexed and corrupted, this causes a significant performance loss and significantly increased chance of packet loss where the kernel gives up waiting for other threads and just throws your pending packets away.
In the extreme case where your cores are very slow and your connection extremely fast (say a 500 core super computer with a 10 - 100Gbit fibre connection) option two could become more feasible, the locking would be less likely as the connection would be fast enough to keep many cores busy without them tripping over each other and locking often, this will -not- increase reliability (and may slightly decrease it) but might increase throughput depending on your architecture.
Overall in nearly every case I would suggest option 1, but if you really do have an extreme throughput situation you should look into other methods, however if you are writing software for this kind of system you would probably benefit from some more general training in massively parallel systems.
I hope this helps, if you have any queries please leave a comment.
I have a CentOS 5.9 machine set up with 5000+ IP addresses (secondary) for eth2.
My program only uses 2 for 2 UDP sockets (1 RX, 1 TX).
When I run the application, the CPU usage is almost 100% all the time.
When I drop down the number of the IP addresses (10), everything go to the normal - hardly 1% CPU usage.
Program is basically a client - server application. It uses non blocking r/w and epoll_wait()
for event waiting.
Can someone please explain to me why so high CPU usage for binary that only use small portion
of configured addresses.
I don't think the question posted talks about number of sockets but rather number of addresses on the interface. Although it seems a little strange as to why your program goes too high in CPU with this number, but in general number of addresses will affect the performance of the IP stack to deal with incoming packets and outgoing packets. Like when you call a send, and your socket is not bound, kernel needs to determine an IP address to put in the packet based on the destination address, and if that takes time it will show up in your process context.
But these still does not explain much, I guess putting a gprof will be a good idea.
Handling thousands of sockets takes specialized software. Most network programmers naively use "select" and expect that to scale up to thousands of sockets well... which it definitely does not. A more event-driven model scales much better ... the event being a new socket or data on the socket, etc.
For Linux and Windows I use Libevent. It's a socket wrapper and not very hard to use and it scales nicely to ten-of-thousands of sockets.
http://libevent.org/
Look at the website here and you can see the logarithmic graph that shows tens of thousands of sockets performing as though they were 100. Of course, if the sockets are super busy, then you are right back to low-performance, but most sockets in the world are mostly quiet and this is where libevent shines. There are other libraries as well like ZeroMq (C# mono), libev, Boost.ASIO.
http://zeromq.org/
http://libev.schmorp.de/bench.html
http://www.boost.org/doc/libs/1_36_0/doc/html/boost_asio.html
Here is my working, super-simple sample. You'll need to add threading protections but with less than an hour's work, you could easily support a few thousand simultaneous connections.
http://pastebin.com/g02S2RTi