In the course of commentary on a recent question, a subsidiary question arose about at what point a cancellation request for a pthreads thread with cancelability PTHREAD_CANCEL_DEFERRED can be expected to be acted upon. References to the standard and a bit of lawyering ensued. I'm not much concerned specifically about whether I was mistaken in my comments on that question, but I would like to be sure I understand POSIX's provisions correctly.
The most pertinent section of the standard says
Whenever a thread has cancelability enabled and a cancellation request has been made with that thread as the target, and the thread then calls any function that is a cancellation point [...], the cancellation request shall be acted upon before the function returns. If a thread has cancelability enabled and a cancellation request is made with the thread as a target while the thread is suspended at a cancellation point, the thread shall be awakened and the cancellation request shall be acted upon.
What, though, does it mean for a thread to be "suspended"? POSIX explicitly defines the term for processes, but not, as far as I can determine, for threads. On the other hand, POSIX documents thread suspension to be among the behaviors of a handful of functions, including, but not limited to, some of those related to synchronization objects. Should one then conclude that those serve collectively as the relevant definition of the term?
And as this all pertains to the question that spawned this line of inquiry, given that POSIX does not specify thread suspension as part of the behavior of read(), fread(), or any of the general file or stream I/O functions, if a thread is not making progress on account of being blocked on I/O, does that necessarily mean it is "suspended" for the purposes of cancellation?
A suspended thread is one that, as you say, is blocked on a socket read, waiting for a semaphore to become available, etc.
Given that POSIX implementations vary at the tricky edges, and that there is the potential for a thread to be blocked in a function that is not a cancellation point, it might be that relying on cancellation in code that is to be ported might be more trouble than it's worth.
I've never used it, I've always chosen to have code to explicitly instruct a thread to terminate (normally a message down a pipe or queue). This is very easy with a Communicating Sequential Processes or Actor Model system.
That way clean up can be done under one's own control, freeing memory, etc. as necessary. I've no idea whether a cancelled thread will clean up its memory (I suspect not), or whether there is the option for an at_exit() type thing (there may be). On the whole I think that application behaviour is more thoroughly controlled if there is only one single way a thread can exit.
==EDIT==
#JohnBollinger,
The language used If a thread has cancelability enabled and a cancellation request is made with the thread as a target while the thread is suspended at a cancellation point could be interpretted as IF a thread has cancelability enabled AND IF cancelled and IF implementation suspends blocked threads AND IF the thread is blocked THEN the thread shall be awakened.... In other words, they're leaving it up to the implementer of the POSIX subsystem.
Cygwin's implementation of select() does not (or at least did not) result in the thread being suspended. Instead it spawns a polling thread per file descriptor to test for signalable activity, due to the fundamental lack of anything quite like select() in Windows (it gets close, but no cigar. Win32 select() works on only sockets). Implementations of select() back in the 1980s often worked this way too.
It might be for reasons like this that POSIX is reluctant to clearly define when a thread is suspended. Historically many implementations of select() were like this, making it a minefield for a standards committee to say when a thread might or might not be suspended. Of course the complexities caused by select() would also apply to a process but as POSIX does define a suspended process it does seem odd that they couldn't / didn't extend the definition to threads.
It might be down to how threads are implemented; you can conceivably have a POSIX implementation that doesn't use OS threads (a bit like the early implementations of ADA back in the days when OSes didn't do threads at all), and in such an implementation a blocked thread might not be suspended (in the sense of taking no CPU cycles) at all.
Definition of suspend in the context of threads:
3.107 Condition Variable
A synchronization object which allows a thread to suspend execution, repeatedly, until some associated predicate becomes true. A thread whose execution is suspended on a condition variable is said to be blocked on the condition variable.
From: http://pubs.opengroup.org/onlinepubs/9699919799/
This is not a direct answer, just a definition – too large for a comment. Blocked == suspended.
read, fread, and friends are system calls and as such they will execute a context switch and execute from the kernel context until those functions complete. Interrupting a kernel context is outside the scope of pthreads thus they will not cause a cancellation.
I don't have a reference for it, but as far as I know, thread suspension in the context of Posix threads has to do with it's synchronization object's ( like futex's ).
Related
I'm experiencing an issue where a thread is somehow being preempted by a lower-priority thread, despite the fact that the higher-priority thread doesn't make any blocking calls. I've noticed that when the lower-priority thread does preempt the higher-priority thread, the higher-priority thread is in the middle of a system call. Specifically, ReleaseMutex and Sleep(0). Is it possible that the system call is doing something that would block the current thread and allow a lower priority thread to run?
Priority just means that there is some amount of preference for one thread over another. It can always be the case that a thread can't make forward progress for some reason and a lower-priority thread then pre-empts it.
Imagine, for example, if ReleaseMutex happens to wind up in some rare code path due to some strange edge case and the code for handling that edge case has paged out to disk. The thread that called ReleaseMutex is not ready-to-run until that code pages in, so a lower-priority thread can get the CPU.
I don't think that's a particularly likely scenario. But the point is that it's not guaranteed not to happen. Priority is not an exclusionary method or a synchronization mechanism. It's just a way of indicating what you prefer when the system happens to have a choice.
If this is causing you an issue, you have something very wrong in your design. If you're trying to use thread priorities as a way of guaranteeing particular behavior (rather than indicating preferences) you are handling exclusion entirely wrong.
I have read that TerminateThread() in WinAPI is dangerous to use.
Is pthread_kill() in Linux also dangerous to use?
Edit: Sorry I meant pthread_kill() and not pthread_exit().
To quote Sir Humphrey Appleby, the answer is "yes, and no".
In and of itself calling pthread_exit() is not dangerous and is called implicitly when your thread exits its method. However, there are a few "gotchas" if you call it manually.
All cleanup handlers are called when this is called. So if you call this method, then access some memory that the cleanup handlers have cleaned up, you get a memory error.
Similarly, after this is called, any local and thread-local variables for the thread become invalid. So if a reference is made to them you can get a memory error.
If this has already been called for the thread (implicitly or explicitly) calling it again has an undefined behaviour, and
If this is the last thread in your process, this will cause the process to exit.
If you are careful of the above (i.e. if you are careful to not reference anything about the thread after you have called pthread_exit) then it is safe to call call manually. However, if you are using C++ instead of C I would highly recommend using the std::thread class rather than doing it manually. It is easier to read, involves less code, and ensures that you are not breaking any of the above.
For more information type "man pthread_exit" which will essentially tell you the above.
Since the question has now been changed, I will write a new answer. My answer still remains "yes and no" but the reasons have changed.
pthread_kill is somewhat dangerous in that it shares the potential timing risks that is inherent in all signal handling systems. In addition there are complexities in dealing with it, specifically you have to setup a signal handler within the thread. However one could argue that it is less dangerous than the Windows function you mention. Here is why:
The Windows function essentially stops the thread, possibly bypassing the proper cleanup. It is intended as a last resort option. pthread_kill, on the other hand, does not terminate the thread at all. It simply sends a signal to the thread that the thread can respond to.
In order for this to do something you need to have registered in the thread what signals you want it to handle. If your goal is to use pthread_kill to terminate the thread, you can use this by having your signal handler set a flag that the thread can access, and having the thread check the flag and exit when it gets set. You may be able to call pthread_exit from the signal handler (I've never tried that) but it strikes me as being a bad idea since the signal comes asynchronously, and your thread is not guaranteed to still be running. The flag option I mention solves this provided that the flag is not local to the thread, allowing the signal handler to set it even if the target thread has already exited. Of course if you are doing this, you don't really need pthread_kill at all, as you can simply have your main thread set the flag at the appropriate time.
There is another option for stopping another thread - the pthread_cancel method. This method will place a cancel request on the target thread and, if the thread has been configured to be cancellable (you generally do this in the pthread_create, but you can also do it after the fact), then the next time the thread reaches a potential cancellation point (specified by pthread_testcancel but also automatically handled by many system routines such as the IO calls), it will exit. This is also safer than what Windows does as it is not violently stopping the thread - it only stops at well defined points. But it is more work than the Windows version as you have to configure the thread properly.
The Wikipedia page for "posix threads" describes some of this (but not much), but it has a pretty good "See also" and "References" section that will give you more details.
I've just started to study the pthread API. I've been using different books and websites, and judging from what they all report, pthread synchronization functions (e.g. those involving mutexes) all work both for a uniprocessor and multiprocessor environments. But none of these sources explicitly stated it, so I wanted to know if that's actually the case (of course I believe so, I just wanted to be 100% sure).
So, if two threads running on different CPUs called a lock (e.g. pthread_mutex_lock()) on the same mutex at the same time, would the execution of this routine be executed sequentially rather than in parallel? And after the first lock is over and the thread invoking it has private access to the critical section, does the lock executed by the other thread on another CPU cause the latter thread to suspend?
Yes, it does. The POSIX API is described in terms of requirements on implementations - for example, a pthread_mutex_lock() that returns zero or EOWNERDEAD must return with the mutex locked and owned by the calling thread. There's no exception for multiprocessor environments, so conforming implementations in multiprocessor environments must continue to make it work.
So, if two threads running on different CPUs called a lock (e.g.
pthread_mutex_lock()) on the same mutex at the same time, would the
execution of this routine be executed sequentially rather than in
parallel?
It's not specified how pthread_mutex_lock() works underneath, but from an application point of view you know that if it doesn't return an error, your thread has acquired the lock.
And after the first lock is over and the thread invoking it has
private access to the critical section, does the lock executed by the
other thread on another CPU cause the latter thread to suspend?
Yes - the specification for pthread_mutex_lock() says:
If the mutex is already locked by another thread, the calling thread
shall block until the mutex becomes available.
Suppose I have multiple threads blocking on a call to pthread_mutex_lock(). When the mutex becomes available, does the first thread that called pthread_mutex_lock() get the lock? That is, are calls to pthread_mutex_lock() in FIFO order? If not, what, if any, order are they in? Thanks!
When the mutex becomes available, does the first thread that called pthread_mutex_lock() get the lock?
No. One of the waiting threads gets a lock, but which one gets it is not determined.
FIFO order?
FIFO mutex is rather a pattern already. See Implementing a FIFO mutex in pthreads
"If there are threads blocked on the mutex object referenced by mutex when pthread_mutex_unlock() is called, resulting in the mutex becoming available, the scheduling policy shall determine which thread shall acquire the mutex."
Aside from that, the answer to your question isn't specified by the POSIX standard. It may be random, or it may be in FIFO or LIFO or any other order, according to the choices made by the implementation.
FIFO ordering is about the least efficient mutex wake order possible. Only a truly awful implementation would use it. The thread that ran the most recently may be able to run again without a context switch and the more recently a thread ran, more of its data and code will be hot in the cache. Reasonable implementations try to give the mutex to the thread that held it the most recently most of the time.
Consider two threads that do this:
Acquire a mutex.
Adjust some data.
Release the mutex.
Go to step 1.
Now imagine two threads running this code on a single core CPU. It should be clear that FIFO mutex behavior would result in one "adjust some data" per context switch -- the worst possible outcome.
Of course, reasonable implementations generally do give some nod to fairness. We don't want one thread to make no forward progress. But that hardly justifies a FIFO implementation!
I'm extending the functionality of a semaphore. I ran into a roadblock when I realized I don't know the implementation of an actual semaphore and to make sure my code ran correctly, I needed to know this.
I know a semaphore works by blocking threads that are waiting on it when they call sem_wait() and another thread currently has it locked. The thread is then blocked and then put into a wait list for that semaphore.
My question relates to what happens on a sem_post(). Is the next thread pulled off the waiting list, set as the locking thread, and allowed to be unblocked? Or is the scheme for posting completely different?
Thanks!
The next thread to unblock on it's sem_wait() will be whatever thread the OS decides is the next one to context switch into. Nobody makes any guarantee of ordering; it depends on your OS's scheduling strategy. It might be the thread that has been off the CPU for the longest, or the one that has been assigned the highest "priority", or the one that has historically had certain resource-usage statistics, or whatever.
Most likely, your current thread (the one that called sem_post()) will continue running for a while, until it either starts waiting for user input, blocks on another semaphore, or runs out of its os-allotted time slice. Then, the OS will switch in some totally unrelated process to run for a fraction of a second (probably Firefox or something), then go off and handle some network traffic, get itself a cup of tea, and, finally, when it gets around to it, pick whichever of your other threads it feels like, based on something like whether it feels based on past history that the particular thread is more CPU or I/O-bound.
In many OSes, priority is given to I/O-bound processes that haven't been around for very long. The theory is that new processes might be short-lived (if it's been around for five hours already, odds are it won't be finishing up in the next 1ms) so we might as well get them over with. I/O-bound processes are likely to continue to be I/O-bound, which means that chances are they are going to switch off the CPU shortly while waiting for other resources. Basically, the OS wants to find the process that it's going to be able to be done with ASAP, so it can get back to sipping its tea and running your malware.
Semaphores have two operations:
P() To acquire the semaphore (you seem to call this sem_wait)
V() To release the semaphore (you seem to call this sem_post)
Semaphores also have an integer associated to them, which is the number of concurrent threads allowed to pass P() without blocking. Other calls to P() will block until V() is called to free up spots.
That is the classic definition of a semaphore.
Edit: Semaphores do not make any guarantee of order. They don't have to actually use a queue or other FIFO structure. When only one thread is allowed at a time, when it calls V(), another (possibly random) thread will then return from its P() call and continue.
According to the IEEE standards, the behavior of POSIX semaphores:
If the semaphore value resulting from this operation is positive, then no threads were blocked waiting for the semaphore to become unlocked; the semaphore value is simply incremented.
If the value of the semaphore resulting from this operation is zero, then one of the threads blocked waiting for the semaphore shall be allowed to return successfully from its call to sem_wait(). If the Process Scheduling option is supported, the thread to be unblocked shall be chosen in a manner appropriate to the scheduling policies and parameters in effect for the blocked threads. In the case of the schedulers SCHED_FIFO and SCHED_RR, the highest priority waiting thread shall be unblocked, and if there is more than one highest priority thread blocked waiting for the semaphore, then the highest priority thread that has been waiting the longest shall be unblocked. If the Process Scheduling option is not defined, the choice of a thread to unblock is unspecified.
If the Process Sporadic Server option is supported, and the scheduling policy is SCHED_SPORADIC, the semantics are as per SCHED_FIFO above."