I'm currently using WaitForSingleObject((HANDLE)handle,INFINITE) function to mutex-lock some parts of my code.
Now I have a situation where I do not want to lock it but just peek, if it is in the locked-state. Using POSIX I can do that with pthread_mutex_trylock() - when it fails, I know there is already a lock on this mutex.
So: how can this be done with WaitforSingleObject()-call? How can I find out if the related mutex is already locked?
I guess it has something to do with the dwMilliseconds parameter, but I don't understand how I can find out if it is locked or just returned because of an other lock...
WaitForSingleObject (family of functions) is used for effectively putting a thread to sleep while waiting on various types of Windows handles. Execution of the thread will wait until the function has returned. In the simplest case of using mutex, these functions also request a lock. The thread will keep the mutex locked until you call ReleaseMutex.
dwMilliseconds merely specifies the wait timeout. Normally you should use the constant INFINITE here. You can also pass the value 0 to dwMilliseconds to have the function check the status of the handle and immediately return and continue execution. If it returns WAIT_OBJECT_0 (or equivalent), you have the mutex lock. This is the equivalent to pthread_mutex_trylock.
In case you do specify a timeout, WaitForSingleObject will return a timeout status WAIT_TIMEOUT when it has not gotten the requested handle within the specified time period. In case of WaitForMultipleObjects, you also need to check the result to see which object you got.
Example from MSDN: https://learn.microsoft.com/en-us/windows/win32/sync/using-mutex-objects
There is no "peeking" a mutex's current state. Either you lock the mutex or you don't, there is no peek.
pthread_mutex_trylock() always returns immediately and does not block the calling thread whether the mutex is already locked or not. However, if the mutex was not locked and trylock is successful then the lock has been obtained and you must unlock it. You must check the return value to know which is the case.
To replicate the same behavior with WaitForSingleObject(), simply set the timeout to 0 so it exits immediately without blocking. If the mutex is not already locked and the wait is successful, the lock is obtained and you must unlock it. Again, you must check the return value.
Note: there is a subtle but important difference between a pthread mutex and a Win32 mutex. A Win32 mutex is always recursive. When a thread already has a lock to a Win32 mutex, it can safely relock the same mutex without blocking itself. An internal lock count is incremented each time the mutex is relocked, and the thread simply needs to unlock the mutex as many times as it (re)locked in order to release the mutex for other threads to lock. A pthread mutex, on the other hand, is recursive only if the mutex creator explicitly requests it when calling pthread_mutex_init(). So be carefully with your (re)locking to avoid deadlocking your code.
Related
Right before exiting, I call from the main() in the following order to:
pthread_cancel() other threads uses mtx which are "waiting" (They are waiting for other cond_variable and mutex. Maybe that's the problem?
pthread_cond_destroy(&cnd) (which is "coupled" whith mtx)
pthread_mutex_unlock(&mtx)
pthread_mutex_destroy(&mtx)
However, the last function results EBUSY. Each time another thread uses the mutex it almost immediately release it. Also, as mentioned, I kill all those threads before trying to destroy the mutex.
Why is it happening?
As per man pthread_mutex_destroy:
The pthread_mutex_destroy() function may fail if:
EBUSY
The implementation has detected an attempt to destroy the object referenced by mutex while it is locked or referenced (for example,
while being used in a pthread_cond_timedwait() or pthread_cond_wait())
by another thread.
Check if the mutex is not used by another thread when you try to destroy it.
pthread_cancel() other threads uses mtx which are "waiting" (They are waiting for other cond_variable and mutex.
Cancellation is running asynchronously to the cancelling process, that is pthread_cancel() might very well return before the thread to be cancelled ended.
This results in resources (mutexes, conditions, ...) used by the thread to be cancelled perhaps still being in use when immediately calling pthread_mutex_destroy() afterwards.
The only way to test whether cancellation succeeded it to call pthread_join()on the cancelled thread and expect it to return PTHREAD_CANCELED. This implies that the thread to be cancelled wasn't detached.
Here you see one possible issue with cancelling threads. There are others. Simply avoid all this by not using pthread_cancel(), but implement a proper design ending all threads in well defined manner.
Scenario 1: release mutex then wait
Scenario 2: wait and then release mutex
Trying to understand conceptually what it does.
If the mutex were released before the calling thread is considered "blocked" on the condition variable, then another thread could lock the mutex, change the state that the predicate is based on, and call pthread_cond_signal without the waiting thread ever waking up (since it's not yet blocked). That's the problem.
Scenario 2, waiting then releasing the mutex, is internally how any real-world implementation has to work, since there's no such thing as an atomic implementation of the necessary behavior. But from the application's perspective, there's no way to observe the thread being part of the blocked set without the mutex also being released, so in the sense of the "abstract machine", it's atomic.
Edit: To go into more detail, the real-world implementation of a condition variable wait generally looks like:
Modify some internal state of the condition variable object such that the caller is considered to be part of the blocked set for it.
Unlock the mutex.
Perform a blocking wait operation, with the special property that it will return immediately if the state of the condition variable object from step 1 has changed due to a signal from any other thread.
Thus, the act of "blocking" is split between two steps, one of which happens before the mutex is unlocked (gaining membership in the blocked set) and the other of which happens after the mutex is unlocked (possibly sleeping and yielding control to other threads). It's this split that's able to make the "condition wait" operation "atomic" in the abstract machine.
Suppose a condition variable is used in a situation where the signaling thread modifies the state affecting the truth value of the predicate and calls pthread_cond_signal without holding the mutex associated with the condition variable? Is it true that this type of usage is always subject to race conditions where the signal may be missed?
To me, there seems to always be an obvious race:
Waiter evaluates the predicate as false, but before it can begin waiting...
Another thread changes state in a way that makes the predicate true.
That other thread calls pthread_cond_signal, which does nothing because there are no waiters yet.
The waiter thread enters pthread_cond_wait, unaware that the predicate is now true, and waits indefinitely.
But does this same kind of race condition always exist if the situation is changed so that either (A) the mutex is held while calling pthread_cond_signal, just not while changing the state, or (B) so that the mutex is held while changing the state, just not while calling pthread_cond_signal?
I'm asking from a standpoint of wanting to know if there are any valid uses of the above not-best-practices usages, i.e. whether a correct condition-variable implementation needs to account for such usages in avoiding race conditions itself, or whether it can ignore them because they're already inherently racy.
The fundamental race here looks like this:
THREAD A THREAD B
Mutex lock
Check state
Change state
Signal
cvar wait
(never awakens)
If we take a lock EITHER on the state change OR the signal, OR both, then we avoid this; it's not possible for both the state-change and the signal to occur while thread A is in its critical section and holding the lock.
If we consider the reverse case, where thread A interleaves into thread B, there's no problem:
THREAD A THREAD B
Change state
Mutex lock
Check state
( no need to wait )
Mutex unlock
Signal (nobody cares)
So there's no particular need for thread B to hold a mutex over the entire operation; it just need to hold the mutex for some, possible infinitesimally small interval, between the state change and signal. Of course, if the state itself requires locking for safe manipulation, then the lock must be held over the state change as well.
Finally, note that dropping the mutex early is unlikely to be a performance improvement in most cases. Requiring the mutex to be held reduces contention over the internal locks in the condition variable, and in modern pthreads implementations, the system can 'move' the waiting thread from waiting on the cvar to waiting on the mutex without waking it up (thus avoiding it waking up only to immediately block on the mutex).
As pointed out in the comments, dropping the mutex may improve performance in some cases, by reducing the number of syscalls needed. Then again it could also lead to extra contention on the condition variable's internal mutex. Hard to say. It's probably not worth worrying about in any case.
Note that the applicable standards require that pthread_cond_signal be safely callable without holding the mutex:
The pthread_cond_signal() or pthread_cond_broadcast() functions may be called by a thread whether or not it currently owns the mutex that threads calling pthread_cond_wait() or pthread_cond_timedwait() have associated with the condition variable during their waits [...]
This usually means that condition variables have an internal lock over their internal data structures, or otherwise use some very careful lock-free algorithm.
The state must be modified inside a mutex, if for no other reason than the possibility of spurious wake-ups, which would lead to the reader reading the state while the writer is in the middle of writing it.
You can call pthread_cond_signal anytime after the state is changed. It doesn't have to be inside the mutex. POSIX guarantees that at least one waiter will awaken to check the new state. More to the point:
Calling pthread_cond_signal doesn't guarantee that a reader will acquire the mutex first. Another writer might get in before a reader gets a chance to check the new status. Condition variables don't guarantee that readers immediately follow writers (After all, what if there are no readers?)
Calling it after releasing the lock is actually better, since you don't risk having the just-awoken reader immediately going back to sleep trying to acquire the lock that the writer is still holding.
EDIT: #DietrichEpp makes a good point in the comments. The writer must change the state in such a way that the reader can never access an inconsistent state. It can do so either by acquiring the mutex used in the condition-variable, as I indicate above, or by ensuring that all state-changes are atomic.
The answer is, there is a race, and to eliminate that race, you must do this:
/* atomic op outside of mutex, and then: */
pthread_mutex_lock(&m);
pthread_mutex_unlock(&m);
pthread_cond_signal(&c);
The protection of the data doesn't matter, because you don't hold the mutex when calling pthread_cond_signal anyway.
See, by locking and unlocking the mutex, you have created a barrier. During that brief moment when the signaler has the mutex, there is a certainty: no other thread has the mutex. This means no other thread is executing any critical regions.
This means that all threads are either about to get the mutex to discover the change you have posted, or else they have already found that change and ran off with it (releasing the mutex), or else have not found they are looking for and have atomically given up the mutex to gone to sleep (and are guaranteed to be waiting nicely on the condition).
Without the mutex lock/unlock, you have no synchronization. The signal will sometimes fire as threads which didn't see the changed atomic value are transitioning to their atomic sleep to wait for it.
So this is what the mutex does from the point of view of a thread which is signaling. You can get the atomicity of access from something else, but not the synchronization.
P.S. I have implemented this logic before. The situation was in the Linux kernel (using my own mutexes and condition variables).
In my situation, it was impossible for the signaler to hold the mutex for the atomic operation on shared data. Why? Because the signaler did the operation in user space, inside a buffer shared between the kernel and user, and then (in some situations) made a system call into the kernel to wake up a thread. User space simply made some modifications to the buffer, and then if some conditions were satisfied, it would perform an ioctl.
So in the ioctl call I did the mutex lock/unlock thing, and then hit the condition variable. This ensured that the thread would not miss the wake up related to that latest modification posted by user space.
At first I just had the condition variable signal, but it looked wrong without the involvement of the mutex, so I reasoned about the situation a little bit and realized that the mutex must simply be locked and unlocked to conform to the synchronization ritual which eliminates the lost wakeup.
C Programming:
What happens when a thread tries to acquire a mutex lock, and fails to get it?
Does it go to sleep?
Will the thread be woken up when pthread_mutex_unlock(&mutex); is called?
Then try to obtain the lock again?
From the man page:
The pthread_mutex_lock() function locks mutex. If the mutex is already locked, the calling thread will block until the mutex becomes available.
So yes - your thread is blocked until the lock is available and it can obtain it.
Yes, it is a blocking call and will block until it gets the lock.
The non-blocking version is pthread_mutex_trylock(pthread_mutex_t *mutex) and will return EBUSY if someone else has the lock, or 0 if it got the lock. (Or some other error, of course)
Normally, pthread_mutex_lock cannot return until it acquires the lock, even if this means that it never returns (deadlock). There are a few notable exceptions though:
For recursive mutexes, it can return EAGAIN if the maximum reference count would be exceeded.
For error-checking mutexes, it can return EDEADLK if the thread tries to lock a mutex it already holds a lock on.
For robust mutexes, it can return EOWNERDEAD if another process died while holding the (shared) mutex. In this case, despite getting an error return, the caller holds the mutex lock and can mark the mutex-protected state valid again by calling pthread_mutex_consistent.
For robust mutexes whose owner died and for which the new owner called pthread_mutex_unlock without calling pthread_mutex_consistent first, it will return ENOTRECOVERABLE.
There may be a few cases I missed. Note that none of these apply to normal mutexes (PTHREAD_MUTEX_NORMAL type) without the robust attribute set, so if you only use normal mutexes, you can reasonably assume the call never returns without succeeding.
From the POSIX standard:
If the mutex is already locked, the calling thread shall block until the mutex becomes available.
(...)
If there are threads blocked on the mutex object referenced by mutex when pthread_mutex_unlock() is called, resulting in the mutex becoming available, the scheduling policy shall determine which thread shall acquire the mutex.
Where the "resulting in" clause is necessary because
(In the case of PTHREAD_MUTEX_RECURSIVE mutexes, the mutex shall become available when the count reaches zero and the calling thread no longer has any locks on this mutex.)
Suppose there are two threads, the main thread and say thread B(created by main). If B acquired a mutex(say pthread_mutex) and it has called pthread_exit without unlocking the lock. So what happens to the mutex? Does it become free?
nope. The mutex remaines locked. What actually happens to such a lock depends on its type, You can read about that here or here
If you created a robust mutex by setting up the right attributes before calling pthread_mutex_init, the mutex will enter a special state when the thread that holds the lock terminates, and the next thread to attempt to acquire the mutex will obtain an error of EOWNERDEAD. It is then responsible for cleaning up whatever state the mutex protects and calling pthread_mutex_consistent to make the mutex usable again, or calling pthread_mutex_unlock (which will make the mutex permanently unusable; further attempts to use it will return ENOTRECOVERABLE).
For non-robust mutexes, the mutex is permanently unusable if the thread that locked it terminates without unlocking it. Per the standard (see the resolution to issue 755 on the Austin Group tracker), the mutex remains locked and its formal ownership continues to belong to the thread that exited, and any thread that attempts to lock it will deadlock. If another thread attempts to unlock it, that's normally undefined behavior, unless the mutex was created with the PTHREAD_MUTEX_ERRORCHECK attribute, in which case an error will be returned.
On the other hand, many (most?) real-world implementations don't actually follow the requirements of the standard. An attempt to lock or unlock the mutex from another thread might spuriously succeed, since the thread id (used to track ownership) might have been reused and may now refer to a different thread (possibly the one making the new lock/unlock request). At least glibc's NPTL is known to exhibit this behavior.