I am using POSIX mandatory file locks through fcntl. I'm wondering if those locks are reentrant, ie. can a process acquire a lock it already owns ?
Advisory locks through fcntl are on a per process base and just accumulate locked intervals on the file for the given process. That is, it is up to the application to keep track of the intervals and any unlock call for an interval will unlock it, regardless on how many lock calls had been made for that interval.
Even worse, the closing of any file descriptor for the file cancels all locks on the file:
As well as being removed by an explicit F_UNLCK, record locks are
automatically released when the process terminates or if it closes any
file descriptor referring to a file on which locks are held. This is bad: it means that a process can lose the locks on a
file like
/etc/passwd or /etc/mtab when for some reason a library function decides to open, read and close it.
Related
If a process uses open(2) (or similar) to obtain more than one
descriptor for the same file, these descriptors are treated
independently by flock(). An attempt to lock the file using one of
these file descriptors may be denied by a lock that the calling
process has already placed via another descriptor.
If flock() treats the descriptors independently, why locking the file using one of the file descriptors would be denied by a lock placed via another descriptor? What does independent here mean?
Also if I unlock one of the descriptor, would other descriptors unlock as well?
treated independently by flock() means that flock() will not "ask" one descriptor, when attempting to modify the other. However, it doesn't mean they are truly independent. If flock() tries to lock one, while the other is already locked, this attempt may block.
Think of it as 2-levels mechanism. flock() looks at only one descriptor at a time, but eventually, upon the lock attempt, the system tries to move to the dipper level and actually lock, and there the problem occurs.
Also if I unlock one of the descriptor, would other descriptors unlock as well?
I'm not sure. This quote (below) states that this indeed is the case if a file has multiple descriptors from fork(2), dup(2). However there is nothing that says so in the 2nd paragraph that treats multiple open(2) which leads me to believe that it is just not a good thing to do :)
From here:
Locks created by flock() are associated with an open file description
(see open(2)). This means that duplicate file descriptors (created
by, for example, fork(2) or dup(2)) refer to the same lock, and this
lock may be modified or released using any of these file descriptors.
Furthermore, the lock is released either by an explicit LOCK_UN
operation on any of these duplicate file descriptors, or when all
such file descriptors have been closed.
If a process uses open(2) (or similar) to obtain more than one file
descriptor for the same file, these file descriptors are treated
independently by flock(). An attempt to lock the file using one of
these file descriptors may be denied by a lock that the calling
process has already placed via another file descriptor.
Suppose your process has two file descriptors, fd1 and fd2, that operate on the same file. If you lock a segment of the file on fd1, and then lock another overlapping segment also on fd1, the two locks won't interfere with each other because they're on the same file descriptor.
However, if the second lock was applied on fd2 instead of fd1, then the locks would be overlapping and the second lock would be deemed to interfere with first and would fail, despite the fact that it is the same process doing the locking.
This is the sense in which the locks on the file descriptors are independent of each other — the locking system doesn't check which process owns the interfering locks on a different file descriptor; it is sufficient that it is not the current file descriptor.
When you unlock one descriptor, you don't change the locks on any other file descriptor.
I'm reading for hours but can't understand what is the difference between the two locks. The only thing I understand is that fcntl() lock is offering a granular lock that can lock specific bytes and that only fcntl() supports NFS locking.
It's said that the difference is in their semantics, how do they behave when being duplicated by dup() or while fork(), but I can't understand what is the difference in practice.
My scenario is that I'm writing to a log file in a fork() based server, where every forked process is writing to the same file when something happens. Why would I want to use flock() and why would I want to use fcntl() locks?
I have tried to figure out the differences based on available documentation and took the following conclusions (please correct me if I am wrong):
With fcntl() (POSIX):
you create a lock record on the file at filesystem level including process id.
If the process dies or closes any filedescriptor to this file, the lock record gets removed by the system.
A request for an exclusive lock shall fail if the file descriptor was not opened with write access.
simply: fnctl locks work as a Process <--> File relationship, ignoring filedescriptors
flock() (BSD) is different (Linux: since kernel 2.0, flock() is implemented as a system call in its own right rather than being emulated in the GNU C library as a call to fcntl):
flock() creates locks on systems's "Open file descriptions". "Open file descriptions" are generated by open() calls.
a filedescriptor (FD) is a reference to a "Open file description". FDs generated by dup() or fork() refer to the same "Open file description".
a process may generate multiple "Open file descriptions" for one file by opening() the file multiple times
flock() places it's locks via a FD on a "Open file description"
therefore flock() may be used to synchronize file access among processes as well as threads (in one ore more processes).
see flock(2) and especially open(2) man pages for details on "Open file descriptions".
In Your scenario you probably want to use fcntl() based locks, because your forked processes will open() the logfile on their own and do not expect to inherit a filedescriptor with a possibly placed lock.
If you need synchronisation among multiple threads, possibly in more than one process, you should use flock() based locks if your system supports them without emulation by fcntl(). Then every thread needs to open() the file rather than using dup()ed or fork()ed handles.
Edit 2022: An excellent write-up and additional thoughts here: https://lwn.net/Articles/586904/
I have a threaded server that can add/append/read files and relay data to the client.
If a file is being added, no other thread can append/read it. If a file is being appended, no threads can append/read it. If a file is being read, no other thread can append to it. However, if a file is being read, other files can read it.
Currently I have a mutex system that will do this, except it won't allow multiple reads.
To fix this, in the read method, I will change:
pthread_mutex_lock(&(fm->mutex));//LOCK
//do some things`
...
pthread_mutex_unlock(&(fm->mutex));
to
pthread_mutex_trylock(&(fm->mutex));//TRYLOCK [NonBlocking, so the thread can continue the read]
//do some things`
...
pthread_mutex_unlock(&(fm->mutex));
Question
How can I unlock the file without allowing the other methods (just append really) to begin writing to the file before all the other read()'s have finished?
Example
For example, if the reading thread that originally locked the file completes and unlocks the file and there are still other threads trying to read the file, then an appending thread gets the chance to lock the file and begin appending while the others are still reading, which is a no-no.
Idea
I want to keep a count of the number of threads currently reading a file. When a thread finishes, reduce the count. If the count is 0, meaning no threads are still reading, unlock the file. But, I'm worried that this would not be thread safe. If this is a viable solution, how could I make it thread safe? Another but, I believe only the original thread can successfully unlock the mutex.
It sounds like you may be looking for a read-write lock, which is provided by pthreads. It allows two modes of locking: a shared/read-lock mode, which can be locked by multiple threads at once, and an exclusive/write-lock mode, where the lock call won't return until all other threads (readers and writers) have given up their hold on the lock.
You could use a semaphore instead of the mutex (see this link about the differences). The semaphore does thread-safe synchronized counting for you.
You can live without an additional mutex to lock the file for writing if you limit the number of simultaneous read accesses to a (sufficient large) number N and require the semaphore to be increased by that number for write access. This way you can only gain write access if the number of readers is zero and all other readers will be locked out until your writer has finished.
Note that the POSIX documentation for pthread_mutex_lock() says:
If successful, the pthread_mutex_lock(), pthread_mutex_trylock(), and pthread_mutex_unlock() functions shall return zero; otherwise, an error number shall be returned to indicate the error.
Since you don't show your code testing the return values, you don't know whether your lock operations (in particular) succeeded or not.
Separately, since you want a read/write lock, why not use one:
pthread_rwlock_rdlock()
pthread_rwlock_wrlock()
pthread_rwlock_unlock()
pthread_rwlock_init()
pthread_rwlock_destroy()
There are four pthread_rwlockattr_*() functions and a total of 9 pthread_rwlock_*() functions; I only listed the most important functions in the family.
Can we use both fcntl locking and flock locking (not simultaneously) on a shared file descriptor, if we want to share a lock between threads/processes?
fcntl locks are per-process locks. So you cannot "share locks" between processes, but you can between threads of one process.
flock locks are per-file table entry. So if one process opens a file and then forks one of more times, each process' copy of the file descriptor will manipulate the same lock. (I don't think this is a very useful feature, though.)
The POSIX specification for fcntl() states:
All locks associated with a file for a given process shall be removed when a file descriptor for that file is closed by that process or the process holding that file descriptor terminates.
Is this operation of unlocking the file segment locks that were held by a terminated process atomic per-file? In other words, if a process had locked byte segments B1..B2 and B3..B4 of a file but did not unlock the segments before terminating, when the system gets around to unlocking them, are segments B1..B2 and B3..B4 both unlocked before another fcntl() operation to lock a segment of the file can succeed? If not atomic per-file, does the order in which these file segments are unlocked by the system depend on the order in which the file segments were originally acquired?
The specification for fcntl() does not say, but perhaps there is a general provision in the POSIX specification that mandates a deterministic order on operations to clean up after a process that exits uncleanly or crashes.
There's a partial answer in section 2.9.7, Thread Interactions with Regular File Operations, of the POSIX specification:
All of the functions chmod(), close(), fchmod(), fcntl(), fstat(), ftruncate(), lseek(), open(), read(), readlink(), stat(), symlink(), and write() shall be atomic with respect to each other in the effects specified in IEEE Std 1003.1-2001 when they operate on regular files. If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them.
So, for a regular file, if a thread of a process holds locks on segments of a file and calls close() on the last file descriptor associated with the file, then the effects of close() (including removing all outstanding locks on the file that are held by the process) are atomic with respect to the effects of a call to fcntl() by a thread of another process to lock a segment of the file.
The specification for exit() states:
These functions shall terminate the calling process with the following consequences:
All of the file descriptors, directory streams[, conversion descriptors, and message catalog descriptors] open in the calling process shall be closed.
...
Presumably, open file descriptors are closed as if by appropriate calls to close(), but unfortunately the specification does not say how open file descriptors are "closed".
The 2004 specification seems even more vague when it comes to the steps of abnormal process termination. The only thing I could find is the documentation for abort(). At least with the 2008 specification, there is a section titled Consequences of Process Termination on the page for _Exit(). The wording, though, is still:
All of the file descriptors, directory streams, conversion descriptors, and message catalog descriptors open in the calling process shall be closed.
UPDATE: I just opened issue 0000498 in the Austin Group Defect Tracker.
I don't think the POSIX specification stipulates whether the releasing of locks is atomic or not, so you should assume that it behaves as inconveniently as possible for you. If you need them to be atomic, they aren't; if you need them to be handled separately, they're atomic; if you don't care, some machines will do it one way and other machines the other way. So, write your code so that it doesn't matter.
I'm not sure how you'd write code to detect the problem.
In practice, I expect that the locks would be released atomically, but the standard doesn't say, so you should not assume.