I'm working on a basic UDP socket file transfer server/client setup, using go-back-n windowing, and unfortunately am stuck doing it using Winsock due to assignment constraints.
Normally in order to manage timeouts on outstanding packets I would just use signal() but am unsure as to how/if this actually really works on Windows, and if this is actually the best solution. Is there some best way to handle these sorts of socket timeouts? Or am I best just managing timeouts with select()?
If your application has a "main()" function then using select() to manage timeouts is the most convenient as it has the advantage that it uses only socket api calls, so the code should work on any platform supporting a bsd style socket api and doesn't require a windows message loop.
If you are writing a window GUI style application - usually with a WinMain() entry point and a message loop, then WSAAsyncSelect() on a socket handle will get read (and write) ready notification messages posted to a HWND - SetTimer likewise posts periodic WM_TIMER notifications, and GetTickCount can be used to detect which socket has been idle too long.
Related
I've made a test C program that creates an AF_PACKET socket, creates x amount of threads via pthreads, and within each thread performs epoll on the socket's file descriptor. This program was made for Linux and I've compiled it using GCC on Ubuntu 18.04. I've submitted a GitHub Gist of the program here since it's 200+ lines of code. I am still fairly new to C and network programming. Therefore, I'm sure there are many improvements I can make to the code. I am open to suggestions!
I have two main questions:
Is there a better way to receive and process high amounts of packets/traffic in a user space program than the above? I've read using pthreads along with epoll would be the best option, but I've also looked into select and standard poll.
When the program above is executed without any debug output via fprintf(), each thread consumes 100% CPU on the epoll_wait() function within the while loop. Is this normal behavior or am I using epoll incorrectly? I've looked at some other examples and I use epoll the same way as the examples do. I've taken a look at the manual page for epoll and I believe I'm using it correctly in my case. I've also tried setting a timeout for the epoll_wait() function, but it was still consuming 100% CPU per thread (which I'd expect due to the while loop).
I plan to make a program that will redirect traffic after inspecting the traffic and I expect a lot of incoming packets which is why I wanted to see if there is a better way to receive and process high amounts of packets. I also understand I could just use standard SOCK_DGRAM or SOCK_STREAM sockets and bind them to an IP and port. However, I do want to process and inspect all incoming traffic to an interface and forward traffic if necessary (e.g. if the destination address matches a forwarding rule). I also wasn't sure if I should make multiple sockets in this case (perhaps a socket per thread). I did do this initially, but it resulted in unexpected behavior and it was only ever reading from one socket descriptor anyways. Perhaps I wasn't creating the new sockets properly.
Any help is highly appreciated and if you need any more information, please let me know.
Thank you for your time.
I would like to use the recvmmsg call to read multiple UDP messages from ONE single socket at once. I'm reading data from a single multicast group.
When I read TCP data, I usually use poll/select with a non-blocking socket (and timeout) to be notified when that is ready to be read. I follow this approach as I am aware of the issue of spurious wakeup and potential troubles of having a blocking socket.
As my application must be very quick, if I follow the same approach with recvmmsg I will introduce an extra system call (poll/select) that might slow down the execution.
So my two questions are the following:
With UDP, can I safely read from BLOCKING sockets using recvmmsg without poll/select or do I have to apply the same principle I've used for TCP (non-blocking+poll)?
Suppose I have a huge amount of multicast traffic, would you go for non-blocking socket + recvmmsg only (no poll) and burn a lot of CPU?
I am using Linux: CentOS 7 and Oracle Linux.
You can always use blocking mode, with both TCP and UDP sockets.
If you want to impose a read timeout there is setsockopt() with the SO_RCVTIMEO option.
I follow this approach as I am aware of the issue of spurious wakeup
What spurious wakeup? Never seen it in 25 years of network programming.
and potential troubles of having a blocking socket.
Never heard of those either.
Using select() and non-blocking mode with a single socket is pointless unless your platform doesn't support SO_RCVTIMEO. It's an extra system call, for a start.
The option of using blocking or non-blocking depends on what is the final purpose of the application.
- Say it's just a sample chat application showing the usage of UDP combined with TCP then you can use either.
- But if you are planning to make this module a part of highly used application with lots of data flowing then probably creating multiple threads/processes to handle different tasks will come in handy. The parent thread will to wait for the message but for processing it will spawn a different child thread and hence make the parent available for the next message.
But in a nutshell I don't see any issue with your first option of using a blocking socket without poll/select for a UDP application considering it's just for homework purposes.
I'm looking into making a tcp client in C.
But before I start I would like some advise from you.
The client is for controlling lights and other devices.
With this client I will need to send commands, but I can also receive events when a light is turned off by someone.
Just sending commands with sockets is no problem.
But at the same time I have to read for incoming packages.
How is this best solved?
Do I need to use threads or can I solve this with something like libevent?
How do most people implement something like this?
This library should be as portable as possible. (linux, unix, windows, ...)
(I'm not looking for code, but just a best approach)
BSD sockets are bidirectional, and you can check if your socket has an incoming data or not by select(2)
http://rhoden.id.au/doc/sockets2.html
or search for "socket select" in SO: https://stackoverflow.com/search?q=socket+select
IMHO using libevent is too much for your purpose. It's nice but you have long way to go to realize its power. Actually libevent is a wrapper for select(2) or its improved successor epoll(2) etc.
I think you have to have two connections, one for sending command, one for getting respond from server when someone turns off light, most of time two conns are sleep. if you only create one connection, you have to query light status every xx ms, most of time it wastes bandwidth. if your app is not serious, server can use UDP to notify client when the light is turned off by someone, which solution only needs one live TCP connection.
I'm writing a TCP server/client application on Windows, to become familiar with the Winsock API. I come from an UNIX background and would like to know which of these could be the best approach to implement the application:
First the specification
Must scale well on multiprocessor and single-processor systems.
No hardset limit of connections.
Application can both listen for connections, acting as server, and act as client.
Multi threaded.
First approach:
Non-blocking select-like socket for listening, in the 'server' thread.
for each client connecting we spawn a separate thread.
Second approach:
Blocking socket for listening, in the 'server' thread.
for each client connecting we spawn a separate thread.
Third approach:
Non-blocking select-like socket for listening, in the 'server' thread.
No separate thread for each incoming connection, the protocol would need state information kept across sessions I suppose.
I wonder what is the most efficient and scalable approach, and especially if it can work with a UDP socket too.
Note: I'm writing the application in plain and old C. No .NET nor C++ involved, C++ exceptions disabled too.
As Gary says, I/O Completion Ports are the most efficient way to manage multiple network connections in a non-blocking/async manner on Windows platforms.
With IOCP you get notified when your networking operations complete and you can process these completions with a small number of threads. You get to decide how many threads you allocate to process the completions and the kernel decides when to use the threads that you're providing. It uses them in a LIFO order, to reduce context switching, so that if you are only using the minimal number of threads required at any point and you're reusing the same threads rather than cycling through all of the threads that you have available for use.
The asynchronous nature of IOCP programming can be a little confusing to start with, but once you get the hang of it it's fairly straight forward.
I have some free IOCP server code which demonstrates the basics and provides some example servers that are pretty easy to build on. You can find the code here: http://www.serverframework.com/products---the-free-framework.html. That page also links to some articles that I wrote to explain the code.
Relating this to the detail of your question. You should be looking at a variation on your third approach. Use AcceptEx() to accept new connections, this can be used in an asynchronous manner and so you don't need a separate thread for connection acceptance and can use the threads that are also processing your overlapped/async read and write operations.
I've written an asynchronous client which does not use blocking sockets, so if you're interested in that approach, then take a look at my client: http://codesprout.blogspot.com/2011/04/asynchronous-http-client.html
It's an HTTP client, but I've shown very little HTTP protocol processing in there, it's all just .NET sockets. The server would work in a similar way: you can take advantage of the *Async methods such as AsseptAsync.
Under Windows, the best performances are achieved by using I/O completion calls.
This is because the lists and queuing mechanism is done in the kernel, far from the heavy user-mode overhead (which drags your code down if you dare to do the hard work yourself).
Unfortunately, Windows I/O completion calls need to allocate many threads to scale and this is quickly killing the performances (as compared to Linux epoll which can scale independently of the number of worker threads you decide to involve in the task).
Recently, I discovered http://gwan.com/ a Web server which came from Windows and was then ported under Linux. And their authors describe the problem in details on their forum.
I have to implement a game server in C which handles multiple clients and continuously exchange information with them. The client may not be sending information at all times.Should I assign a thread with non-blocking socket to them or use select() call.
Which one is better?
Both will work just as well in most cases. Note that the thread version will use blocking sockets, and the select-based uses non-blocking sockets. In the case of a server, you may feel that events for data received is a good model.
The threaded version will have the memory-overhead of allocating a stack for each thread (often the size of a page), but you can program as if you have only one client.
The evented version needs to maintain state between callbacks, which can be more work, but again, in servers it feels quite natural.
select() is the way to go, that's what it's made for. If you go for the threaded non-blocking approach, you will have to implement a sleep after each tick or the thread will use all available cpu time. So, the worst case response time, if one client is sending data, is your sleep value. You could also implement one thread per socket and make it blocking, but depending on how many sockets you have, that will be much overhead.
With select() you can watch all sockets at once (no matter if they are blocking or not, btw) and only have to process those which are active.
If you are on linux an have many sockets to watch, you can take a look at epoll()