According to my research, there is a automatic cleanup mechanism on /dev/shm directory, made by systemd, with removeipc configuration.
Will systemd remove my posix semaphores (under /dev/shm) at runtime of my application even if I don't unlink them according to its removeipc feature ?
I don't want to change the systemd settings. So the systemV semaphore is a good alternative for me? I don't see but is there a automatic cleanup stuff on sysV ipc objects? Or should I continue with Posix semaphores?
Related
I need to create named lock that work correctly with multi-thread application for Linux. Each instance of application could use more than one named-lock with different names.
I know about fcntl/flock, but it doesn't work if try to lock twice from different thread of one application or from one thread.
I know about open(..., O_CREATE | O_EXCL), but this file-lock will not be removed if application was killed by signal KILL or was crashed with segmentation fault and there is needed manual removing of lock-files after restart application.
Any another ways?
If you just need to run under modern Linux, you could use file-private locks. If that's not an option, you'll have to build your own thread-safe locking abstraction on top of fcntl locks. SQLite is public domain and has implemented that, so you could look at that for inspiration. If GPLed code is okay: OpenJDK has another, incompatible implementation of the same thing.
O_EXCL does not perform locking (beyond the file creation step), so that's usually not helpful.
Other options are System V and POSIX semaphores, but these usually do not work as well as fcntl locks when processes day. A robust, process-shared mutex in a file mapping could be an option as well, but you need to be careful to stay within the POSIX semantics as far as serialization to disk is concerned (basically, you need to reinitialize the mutex every time the application starts from scratch, after a reboot or libc update).
I am writing a joystick library in C and I would like to make it thread safe. Is it okay to use pthread mutexes? Will they work even when the application that uses my library uses, for example, ACE as a thread framework (not sure if ACE does not just extends pthread, but let's assume it does not)?
Same goes for Windows: Can I use Windows' CriticalSection in combination with pthread in mingw+gcc? Or is threading something the OS must do so that the native implementation is always used (pthread on Linux and CriticalSection on Windows)?
Threads can operate at the user level and therefore there is no guarantee that the OS will schedule all concurrency. With that said, you should always seek to make your library thread-safe without requiring a specific locking mechanism on your users.
For example, you can make all functions "pure" or reentrant. You can also offer a version that is explicitly not thread-safe and an alternative version that requires a specific library, like pthread. In the most extreme case you can offer lock-free synchronization.
What are the differences between the functions included in <semaphore.h> and <sys/sem.h>?
Does exist a situation where is better to use a header or the other?
<sys/sem.h> provides the interface for XSI (originally Unix System V) semaphores. These are not part of the base POSIX standard (they're in the XSI option which is largely for traditional Unix compatibility) and while they are not considered obsolescent/deprecated yet, many programmers consider them deprecated, and POSIX advises:
APPLICATION USAGE
The POSIX Realtime Extension defines alternative interfaces for interprocess communication. Application developers who need to use IPC should design their applications so that modules using the IPC routines described in XSI Interprocess Communication can be easily modified to use the alternative interfaces.
The advantages and disadvantages of XSI semaphores is that they are, and must be due to the way their interface works, kernel-space objects. The main benefit this gives you is the ability to set them up so that the kernel can back-out operations if the process exits or is killed unexpectedly. The main cost is that every operation is a round-trip to kernel-space, i.e. they're very slow. The interfaces for using them are also very obtuse and hard to learn, and they're necessarily a process-shared resource, meaning you have to deal with a shared namespace and resource cleanup issues.
<semaphore.h> defines POSIX semaphores, which are designed in such a way that they can be implemented entirely in userspace, except in the contended case where the process will call into the kernel to go to sleep. Their performance should be near-optimal (i.e. almost impossible to beat rolling your own) but they're not quite as featureful as XSI semaphores. POSIX semaphores also offer you the choice of whether you want a process-local semaphore (for use in a multi-threaded environment, or even, in some conditions, a signal handler in a single-threaded program) or a process-shared one, and in the latter case, you also have the choice whether to let the system handle allocating it in a shared namespace by name, or to obtain shared memory yourself and initialize it in shared memory.
They are not the same/replacement for one another but two different implementations and provide different set of functions. semaphore.h is posix implementation and sys/sem.h is sysV's implementation. POSIX is considered to be lighter and used widely.
I've been toying around with libevent2, and I've got reading files working, but it blocks. Is there any way to make file reading not block just within libevent. Or, do I need to use another IO library for files and make it pump events that I need.
fd = open("/tmp/hello_world",O_RDONLY);
evbuffer_read(buf,fd,4096);
The O_NONBLOCK flag doesn't work either.
In POSIX disks are considered "fast devices" meaning that they always block (which is why O_NONBLOCK didn't work for you). Only network sockets can be non-blocking.
There is POSIX AIO, but e.g. on Linux that comes with a bunch of restrictions making it unsuitable for general-purpose usage (only for O_DIRECT, I/O must be sector-aligned).
If you want to integrate normal POSIX IO into an asynchronous event loop it seems people resort to thread pools, where the blocking syscalls are executed in the background by one of the worker threads. One example of such a library is libeio
No.
I've yet to see a *nix where you can do non-blocking i/o on regular files without resorting to the more special AIO library (Though for some, e.g. solaris, O_NONBLOCK has an effect if e.g. someone else holds a lock on the file)
Please take a look at libuv, which is used by node.js / io.js: https://github.com/libuv/libuv
It's a good alternative to libeio, because it does perform well on all major operating systems, from Windows to the BSDs, Mac OS X and of course Linux.
It supports I/O completion ports, which makes it a better choice than libeio if you are targeting Windows.
The C code is also very readable and I highly recommend this tutorial: https://nikhilm.github.io/uvbook/
Is there any way to perform POSIX shared synchronization objects cleanup especially on process crash? Locked POSIX semaphores unblock is most desired thing but automatically 'collected' queues / shared memory region would be nice too. Another thing to keep eye on is we can't in general use signal handlers because of SIGKILL which cannot be caught.
I see only one alternative: some external daemon which accepts subscriptions and 'keep-alive' requests working as watchdog so not having notifications about some object it could close / unlock object in accordance to registered policy.
Has anyone better alternative / proposition? I never worked seriously with POSIX shared objects before (sockets were enough for all my needs and are much more useful by my opinion) and I did not found any applicable article. I'd gladly use sockets here but can't because of historical reasons.
Rather than using semaphores you could use file locking to co-oridinate your processes. The big advanatge of file locks being that they are released if the process terminates. You can map each semaphore onto a lock for a byte in a shared file and know that locks will get released on exit; in mosts version of unix the bytes you lock don't even have to exist. There is code for this in Marc Rochkind's book Advanced Unix Programming 1st edition, don't know if it's in the latest 2nd edition though.
I know this question is old, but another great solution is POSIX robust mutexes. They automatically unlock and enter an "inconsistent flag" state when the owner dies, and the next thread to attempt locking the mutex gets an EOWNERDEAD error but succeeds in becoming the new owner of the mutex. It's then able to clean up whatever state the mutex was protecting (which could be in a very bad inconsistent state due to asynchronous termination of the previous owner!) and mark the mutex as consistent again before unlocking it.
See the documentation on robust mutexes here:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_lock.html
The usual way is to work with signal handlers. Just catch the signals and call the cleanup functions.
But your watchdog daemon has some merits, too. It would surely make the system more simple to understand and manage. To make it more simple to administrate, your application should start the daemon when it's not running and the daemon should be able to clean up any residue from the last crash.