Does anyone have experience with Linux aio functions (io_*, not posix aio)? It would be great if someone could provide a link to some examples (or provide some examples here). Also, what are your general observations/comments about their use?
I am working on an I/O library and someone suggested I have a look at them. They are known to perform better than POSIX aio in certain cases and I would like to have a look.
Thanks.
Update: this shows an example for the native linux io interface
(This is an example on the posix aio interface).
As to some of the commentters on the question: the aio library allows a program to issue multiple parallel request in a way where the kernel can execute them in the order which is most efficient for seeks and disk rotation -- i.e. the io request may not be executed in the order which they were issues which is different from making synchronous request in a thread. In very IO intensive applications this can dramatically increase IO performance, but for most applications it will just add complexity.
Related
I am completely new to threading in C so wanted to check my idea was valid and relatively straightforward to program before starting on it. I want to write a program that can read from 2 COM ports simultaneously on two different threads (One thread per COM port) using Modbus RTU. I am relatively proficient using libmodbus functions etc it is just the threading part I require help with.
So, a few questions:
1) Is this possible to implement relatively easily (any examples no matter how simple would be much appreciated), and if so what considerations with regards to memory will need to be made?
2) Which relevant header files are required to implement multithreading in C?
So, a few questions: 1) Is this possible to implement relatively easily (any examples no matter how simple would be much appreciated)
Threading is a rather advanced topic in itself, and then you also have the system-specific ways of using a COM port. You'll have to study multi-threading and the OS-specific COM port library routines. It is not trivial.
what considerations with regards to memory will need to be made?
Shouldn't be a concern, except that you might want to implement some buffer system, which in turn must be protected by mutexes if used by multiple threads.
Which relevant header files are required to implement multithreading in C?
Depends on the system. For Linux and other POSIX systems, use pthreads (POSIX threads) from pthread.h. They may or may not be supported by a Windows compiler. Otherwise, Windows uses its own thread routines, CreateThread etc. RAD tools like Visual Studio or C++ Builder have wrapper classes around CreateThread. Other systems might have different libraries.
There was an attempt to standardize threading with the C11 standard, but that one is still in the experimental phase and we have yet to tell if it will be used or turn out a fiasco - few if any compilers support C11 threads. For now, pthreads is the most used industry standard.
I'm going to use aio for async read. When aio completes and signal handler is triggered, I may need to do another aio_read call and proceed.
aio_read isn't mentioned among safe functions (in man signal). Ordinary read is, though.
What are the dangers of doing subsequent aio_read calls inside aio signal handler?
As the author of proposed Boost.AFIO which can make use of POSIX AIO, I strongly recommend against using POSIX AIO at all. I am hardly alone in this opinion, #arvid is similarly against: http://blog.libtorrent.org/2012/10/asynchronous-disk-io/. The API itself is poorly designed and as a result scales poorly with load unless you use OS-specific alternatives or extensions to AIO like BSD kqueues. POSIX AIO is essentially useless as-is.
Additionally, AIO calls are not signal safe on Linux, which you are probably using. This is because on Linux they are implemented in userspace using an emulation based on a threadpool. On BSD, AIO calls have a proper kernel syscall interface, but in the kernel turn into - yes you guessed it - a threadpool based emulation unless O_DIRECT is turned on.
You are therefore much better off on POSIX of simply always using a threadpool unless all your i/o is with O_DIRECT on. If O_DIRECT is indeed always on, Linux provides a custom kernel API detailed at http://man7.org/linux/man-pages/man2/io_submit.2.html which is fairly effective, and on BSD if you replace signal driven handling with BSD kqueues (https://www.freebsd.org/cgi/man.cgi?kqueue, see EVFILT_AIO) then with O_DIRECT things can also scale well, better than a threadpool anyway.
Use of signal based completion handling on ANY POSIX platform has dreadful performance. AFIO v2 provides a generic POSIX AIO backend, and it is dreadful, dreadful, dreadful. Avoid like the plague.
Note that a threadpooled synchronous API design is portable, scales well for most use cases, and is what I (and indeed arvid) would recommend to anybody without highly specialised needs like writing a database backend where you need very tight control over the physical storage layer, and anything but O_DIRECT|O_SYNC isn't an option.
Ok, all that said, if you really really want to use signal driven aio, I assume this is because you want to multiplex your file i/o with non-file i/o stuff and you therefore can't use aio_suspend() which is the proper API for doing this. The way AFIO v2 handles this is to use a realtime signal to interrupt aio_suspend() when something not aio related needs to be processed, it can then be handled and aio_suspend() restarted. You need to be very careful in handling races and deadlocks, and you'll need to carefully mask and unmask the signal for the thread calling aio_suspend() lest the realtime signal gets lost and you get a lost wakeup. All in all, it's not worth it for the typically much lower i/o performance you get over a threadpool + synchronous APIs.
I am reading a book about network progamming in C. It is from 2004.
In the example code, author is using select C function to accept multiple connections from the client. Is that function deprecated today?
I see that there are different ways to accept multiplexed I/O like poll and epoll. What are the advantages?
It's not deprecated, and lots of programs rely on it.
It's just not the best tool as it has some limitations:
The number of file descriptors is limited (OS specific, usually possible to increase it with kernel recompiling).
Doesn't scale well (with lots of fds): the whole FD set must be maintained, and re-initialized as select manipulates it.
Feel free to use it if these aren't relevant for you. Otherwise use poll/libevent if you're looking for a cross-platform solution, or in some rare-cases epoll/kqueue for platform specific optimized solutions.
It's not deprecated in its behavior, but its design may have performance issues. For example, linux epoll() documentation states:
API can be used either as an edge-triggered or a level-triggered inter‐
face and scales well to large numbers of watched file descriptors.
Since the efficient alternatives are specific to each operating system, an option better than directly using select() is to use a cross platform multiplexing library (which uses the best implementation available), examples being:
libevent
libev
libuv
If you're developing for a specific operating system, use the recommended implementation for high performance applications.
However, since some people don't like current libraries for I/O multiplexing (due to "being ugly"), select is still a viable alternative.
I am dealing with an existing project (in C) that is currently running on a single thread, and we would like to run on multiple platforms AND have multiple threads. Hopefully, there is a library for this, because, IMHO, the Win32 API is like poking yourself in the eye repeatedly. I know about Boost.Thread for C++, but, this must be C (and compilable on MinGW and gcc). Cygwin is not an option, sorry.
Try OpenMP API, it's multi-platform and you can compile it with GCC.
Brief description from the wikipedia:
OpenMP (Open Multi-Processing) is an application programming interface
(API) that supports multi-platform shared memory multiprocessing
programming in C, C++, and Fortran,[3] on most platforms, processor
architectures and operating systems, including Solaris, AIX, HP-UX,
Linux, macOS, and Windows. It consists of a set of compiler
directives, library routines, and environment variables that influence
run-time behavior.
I would use the POSIX thread API - pthread. This article has some hints for implementing it on Windows, and a header-file-only download (BSD license):
http://locklessinc.com/articles/pthreads_on_windows/
Edit: I used the sourceforge pthreads-win32 project in the past for multi-platform threading and it worked really nicely. Things have moved on since then and the above link seems more up-to-date, though I haven't tried it. This answer assumes of course that pthreads are available on your non-Windows targets (for Mac / Linux I should think they are, probably even embedded)
Windows threading has sufficiently different functionality when compared to that of Linux such that perhaps you should consider two different implementations, at least if application performance could be an issue. On the other hand, simply implementing multi-threading may well make your app slower than it was before. Lets assume that performance is an issue and that multi-threading is the best option.
With Windows threads I'm specifically thinking of I/O Completion Ports (IOCPs) which allow implementing I/O-event driven threads that make the most efficient use of the hardware.
Many "classic" applications are constructed along one thread/one socket (/one user or similar) concept where the number of simultaneous sessions will be limited by the scheduler's ability to handle large numbers of threads (>1000). The IOCP concept allows limiting the number of threads to the number of cores in your system which means that the scheduler will have very little to do. The threads will only execute when the IOCP releases them after an I/O event has occurred. The thread services the IOC, (typically) initiates a new I/O and returns to wait at the IOCP for the next completion. Before releasing a thread the IOCP will also provide the context of the completion such that the thread will "know" what processing context the IOC belongs to.
The IOCP concept completely does away with polling which is a great resource waster although "wait on multiple object" polling is somewhat of an improvement. The last time I looked Linux had nothing remotely like IOCPs so a Linux multi-threaded application would be constructed quite differently compared to a Windows app with IOCPs.
In really efficient IOCP apps there is a risk that so many IOs (or rather Outputs) are queued to the IO resource involved that the system runs out of non-paged memory to store them. Conversely, in really inefficient IOCP apps there is a risk that so many Inputs are queued (waiting to be serviced) that the non-paged memory is exhausted when trying to temporarily buffer them.
If someone needs a portable and lightweight solution for threading in C, take a look at the plibsys library. It provides you thread management and synchronization, as well as other useful features like portable socket implementation. All major operating systems (Windows, Linux, OS X) are supported, various other less popular operating systems are also supported (i.e. AIX, HP-UX, Solaris, QNX, IRIX, etc). On every platform only the native calls are used to minimize the overheads. The library is fully covered with Unit tests which are run on a regular basis.
glib threads can be compiled cross-platforms.
The "best"/"simplest"/... answer here is definitely pthreads. It's the native threading architecture on Unix/POSIX systems and works almost as good on Windows. No need to look any further.
Given that you are constrained with C. I have two suggestions:
1) I have a seen a project (similar to yours) that had to run on Windows and Linux with threads. The way it was written was that it (the same codebase) used pthreads on Linux and win32 threads on Windows. This was achieved by a conditional #ifdef statement wherever threads needed to be created such as
#ifdef WIN32
//use win32 threads
#else
//use pthreads
#endif
2) The second suggestion might be to use OpenMP. Have you considered OpenMP at all?
Please let me know if I missed something or if you want more details. I am happy to help.
Best,
Krishna
From my experience, multi threading in C for windows is heavily tied to Win32 APIs. Other languages like C# and JAVA supported by a framework also tie into these core libraries while offering their thread classes.
However, I did find an openthreads API platform on sourceforge which might help you:
http://openthreads.sourceforge.net/
The API is modeled with respect to the Java and POSIX thread standard,
I have not tried this myself as I currently do not have a need to support multiple platforms on my C/C++ projects.
I've been toying around with libevent2, and I've got reading files working, but it blocks. Is there any way to make file reading not block just within libevent. Or, do I need to use another IO library for files and make it pump events that I need.
fd = open("/tmp/hello_world",O_RDONLY);
evbuffer_read(buf,fd,4096);
The O_NONBLOCK flag doesn't work either.
In POSIX disks are considered "fast devices" meaning that they always block (which is why O_NONBLOCK didn't work for you). Only network sockets can be non-blocking.
There is POSIX AIO, but e.g. on Linux that comes with a bunch of restrictions making it unsuitable for general-purpose usage (only for O_DIRECT, I/O must be sector-aligned).
If you want to integrate normal POSIX IO into an asynchronous event loop it seems people resort to thread pools, where the blocking syscalls are executed in the background by one of the worker threads. One example of such a library is libeio
No.
I've yet to see a *nix where you can do non-blocking i/o on regular files without resorting to the more special AIO library (Though for some, e.g. solaris, O_NONBLOCK has an effect if e.g. someone else holds a lock on the file)
Please take a look at libuv, which is used by node.js / io.js: https://github.com/libuv/libuv
It's a good alternative to libeio, because it does perform well on all major operating systems, from Windows to the BSDs, Mac OS X and of course Linux.
It supports I/O completion ports, which makes it a better choice than libeio if you are targeting Windows.
The C code is also very readable and I highly recommend this tutorial: https://nikhilm.github.io/uvbook/