I want to make a TCP client that works on Windows, Linux and osx (most important) in C. The code that I've found on SO might work on linux but not on osx and vice versa. So what do I need to make sure that it works on all three?
Thanks!
Assuming you are planning to write your code to use the BSD sockets API, you'll find that most TCP client code that works on Linux will work on MacOS/X with little or no modification, and vice versa.
Getting the code to work under Windows as well is a little bit trickier, as required #includes are different, and there are a number of cases where Windows' TCP stack (aka WinSock) behaves a little bit differently than that the TCP stacks of the other two OS's. That said, Windows does support most of the BSD sockets API, and with a bit of #ifdef-ing you can come up with a program that will compile and run correctly on all three OS's. You'll need to test and debug on all three OS's, of course; never assume that just because something works on one OS that it will work everywhere.
Depending on your program's particular needs (and your interests), it may well make sense to follow Duck's advice and find a networking library that has already done the above-described work for you; but if you prefer to "roll your own", that is doable too. A good approach when writing to the BSD sockets API is: whenever you find a piece of code that you have to write differently for different OS's, hide the implementations of that code snippet inside a function together (with an #ifdef so that the right code gets compiled under each OS), so that the rest of your program doesn't have to remember how to deal with that unpleasant detail anymore. Do that enough times and you'll end up the proud maintainer of your own cross-platform networking library ;)
I'd recommend getting your program working under Linux and/or OS/X first, and once you're happy with it, then porting it over to Windows. Some "gotchas" to watch out for when porting your network code to Windows:
Under Windows, you #include windows.h or winsock2.h to get the network definitions you need. (If you want the newer WinSock2 API, you have to include winsock2.h, and always do it before any #include of windows.h, or you'll get the wrong API version... it's a real circus)
Under Windows you have to call WSAStartup() before doing any networking stuff (if you forget, all your networking calls will error out)
Under MacOS/X and Linux, file descriptors and sockets are largely interchangeable (i.e. you can select() on STDIN_FILENO, etc). Under Windows, they are not.
Under MacOS/X and Linux, you can find out why a call failed by checking errno. Under Windows, you call WSAGetLastError() instead.
Under MacOS/X and Linux, you destroy a socket with close(). Under Windows you do it with closesocket().
Under MacOS/X and Linux, you can (if you choose) call read() and write() on your TCP socket to receive/send data (respectively). Under Windows, that won't work, you must call send() and recv() only. (send() and recv() will work under MacOS/X and Linux too)
To set a socket to non-blocking mode under Windows, you have to call ioctlsocket(fd, FIONBIO, &mode). Under MacOS/X and Linux, you call instead fcntl(fd, F_SETFL, flags).
More fun Windows-networking gotchas can be read at the WinSock Lame List.
Related
I would like to use the PJSIP library to implement a small SIP softphone on an embedded system. Since this embedded system does not offer Linux or support POSIX, I would like to port the PJLIB library only partially, as described here (https://www.pjsip.org/porting.htm#mozTocId30930). The threading function can be deactivated via a macro, but I'm not quite sure yet how I have to set up this new transport function or where exactly it has to be included so that I can also bypass the IOQUEUE implementation and the PJLIB socket abstraction.
On my embedded system (Keil RTX) I can allocate a UDP socket and register a callback which is called on a network event. I also have a send function which I can use to send data packets. Although I have already looked into the stack, I can't find a way to get started.
Has anyone already dared to the partial porting and can give me a brief assistance. Thank you !
See how Symbian port worked (I think it might be removed from recent versions, but it should be still downloadable) - it was also based on non-POSIX sockets. Create your own platform-specific socket file and ioqueue file.
The man page for open() states for O_ASYNC:
This feature is available only for terminals, pseudoterminals,
sockets, and (since Linux 2.6) pipes and FIFOs. See fcntl(2) for
further details.
But I've used linux timers with epoll() successfully and setting O_ASYNC with fcntl() on a timer fd does not return an error. Obviously, no signals are being sent either. My question is, is it possible to get O_ASYNC working with linux timers? Are there any examples online? I know about the POSIX alternative, but was hoping to avoid it.
We can see that F_SETFL calls setfl which calls a fasync function specific to the type of file.
By searching for fasync we can see how async support is implemented in many devices. Seems that it's not too complicated as mostly it only needs the device to store the async registration and send the signal (here is the fasync function implementation for this kind of file).
Going back to setfl we can notice that if the file type's fasync function is null, it just silently succeeds. This could be a bug, or it could be intentional. Let's assume it's a bug.
Now that the bug is in the kernel, there are probably programs relying on it, which would stop working if the bug was fixed. If a program did break and someone complained about it, the fix would get undone so the program would keep working, because Linus doesn't like to break programs. If it doesn't break any programs that actually exist (which is unlikely, in my opinion), it can be fixed.
Another option is to update the documentation.
Another option is to make it actually work.
My question is, is it possible to get O_ASYNC working with linux timers?
It's unlikely (but still possible) that any program is setting O_ASYNC on a timerfd since it doesn't work - so it's unlikely that it will break compatibility. And it looks like it's not terribly complicated to implement, based on the other examples. So, go ahead and write this patch and send it to the mailing list.
If you meant if it's possible to implement on today's kernels, without a patch, the answer is no, it is not. Here is the timerfd ops structure and there is no entry for fasync
Are there any examples online?
Yes, the examples are the source code for all the other kinds of files that support fasync.
I'm trying to implement an echo TCP server as a loadable kernel module.
Should I use sock_create, or sock_create_kern?
Should I use accept, or kernel_accept?
I mean it does make sense that I should use kernel_accept for example; but I don't know why. Can't I use normal sockets in the kernel?
The problem is, you are trying to shoehorn an user space application into the kernel.
Sockets (and files and so on) are things the kernel provides to userspace applications via the kernel-userspace API/ABI. Some, but not all, also have an in-kernel callable, for cases when another kernel thingy wishes to use something provided to userspace.
Let's look at the Linux kernel implementation of the socket() or accept() syscalls, in net/socket.c in the kernel sources; look for SYSCALL_DEFINE3(socket, and SYSCALL_DEFINE3(accept,, SYSCALL_DEFINE4(recv,, and so on.
(I recommend you use e.g. Elixir Cross Referencer to find specific identifiers in the Linux kernel sources, then look up the actual code in one of the official kernel Git trees online; that's what I do, anyway.)
Note how pointer arguments have a __user qualifier: this means the data pointed to must reside in user space, and that the functions will eventually use copy_from_user()/copy_to_user() to retrieve or set the data. Furthermore, the operations access the file descriptor table, which is part of the process context: something that normally only exist for userspace processes.
Essentially, this means your kernel module must create an userspace "process" (enough of one to satisfy the requirements of crossing the userspace-kernel boundary when using kernel interfaces) to "hold" the memory and file descriptors, at minimum. It is a lot of work, and in the end, it won't be any more performant than an userspace application would be. (Linux kernel developers have worked on this for literally decades. There are some proprietary operating systems where doing stuff in "kernel space" may be faster, but that is not so in Linux. The cost to do things in userspace is some context switches, and possibly some memory copies (for the transferred data).)
In particular, the TCP/IP and UDP/IP interfaces (see e.g. net/ipv4/udp.c for UDP/IPv4) do not seem to have any interface for kernel-side buffers (other than directly accessing the rx/tx socket buffers, which are in kernel memory).
You have probably heard of TUX web server, a subsystem patch to the Linux kernel by Ingo Molnár. Even that is not a "kernel module server", but more like a subsystem that an userspace process can use to implement a server that runs mostly in kernel space.
The idea of a kernel module that provides a TCP/IP and/or UDP/IP server, is simply like trying to use a hammer to drive in screws. It will work, after a fashion, but the results won't be pretty.
However, for the particular case of an echo server, it just might be possible to bolt it on top of IPv4 (see net/ipv4/) and/or IPv6 (see net/ipv6/) similar to ICMP packets (net/ipv4/icmp.c, net/ipv6/icmp.c). I would consider this route if and only if you intend to specialize in kernel-side networking stuff, as otherwise everything you'd learn doing this is very specialized and not that useful in practice.
If you need to implement something kernel-side for an exercise or something, I'd recommend steering away from "application"-type ideas (services or similar).
Instead, I would warmly recommend developing a character device driver, possibly implementing some kind of inter-process communications layer, preferably bus-style (i.e., one sender, any number of recipients). Something like that has a number of actual real-world use cases (both hardware drivers, as well as stranger things like kdbus-type stuff), so anything you'd learn doing that would be real-world applicable.
(In fact, an echo character device -- which simply outputs whatever is written to it -- is an excellent first target. Although LDD3 is for Linux kernel 2.6.10, it should be an excellent read for anyone diving into Linux kernel development. If you use a more recent kernel, just remember that the example code might not compile as-is, and you might have to do some research wrt. Linux kernel Git repos and/or a kernel source cross referencer like Elixir above.)
In short sockets are just a mechanism that enable two processes to talk, localy or remotely.
If you want to send some data from kernel to userspace you have to use kernel sockets sock_create_kern() with it's family of functions.
What would be the benefit of TCP echo server as kernel module?
It makes sense only if your TCP server provides data which is otherwise not accessible from userspace, e.g. read some post-mortem NVRAM which you can't read normally and to send it to rsyslog via socket.
I just finished a small project written in C, where I read a data stream from a serial port and parse the incoming data.
The software is written for POSIX systems (using termios) and follows the standard steps to working with serial i/o
Opening the serial device using open()
Configuring communication parameters (termios)
Set blocking mode on file handle (fcntl)
Perform read() on serial interface.
Perform close() on serial interface when done.
Other than the socket parts, the code is straight ANSI C.
My question is, how involved would it be to make the code work on a windows platform.
The port would not be written by me, I'd only like to give an indication to others who might be interested in porting it (i.e. trivial, not so trivial, rip your eyes out insanity inducing).
Also if someone has Windows with "Windows Services for UNIX", would they be able to use the code without modifying it?
So, if anyone has experience with this could you please share.
It should be pretty easy to do. The names are very different, but the sequence of calls and concepts are very similar.
What you are looking for is the DCB structure which should be used with the SetComState() function to set baudrate, stopbits etc. Then use SetCommTimeouts() and set the timeout values in the COMMTIMEOUTS structure to make subsequent read calls blocking.
Here is a short introduction as a pretty PDF. (Backup.)
I've been toying around with libevent2, and I've got reading files working, but it blocks. Is there any way to make file reading not block just within libevent. Or, do I need to use another IO library for files and make it pump events that I need.
fd = open("/tmp/hello_world",O_RDONLY);
evbuffer_read(buf,fd,4096);
The O_NONBLOCK flag doesn't work either.
In POSIX disks are considered "fast devices" meaning that they always block (which is why O_NONBLOCK didn't work for you). Only network sockets can be non-blocking.
There is POSIX AIO, but e.g. on Linux that comes with a bunch of restrictions making it unsuitable for general-purpose usage (only for O_DIRECT, I/O must be sector-aligned).
If you want to integrate normal POSIX IO into an asynchronous event loop it seems people resort to thread pools, where the blocking syscalls are executed in the background by one of the worker threads. One example of such a library is libeio
No.
I've yet to see a *nix where you can do non-blocking i/o on regular files without resorting to the more special AIO library (Though for some, e.g. solaris, O_NONBLOCK has an effect if e.g. someone else holds a lock on the file)
Please take a look at libuv, which is used by node.js / io.js: https://github.com/libuv/libuv
It's a good alternative to libeio, because it does perform well on all major operating systems, from Windows to the BSDs, Mac OS X and of course Linux.
It supports I/O completion ports, which makes it a better choice than libeio if you are targeting Windows.
The C code is also very readable and I highly recommend this tutorial: https://nikhilm.github.io/uvbook/