I would like to know exactly how the execution of asynchronous signal handlers works on Linux. First, I am unclear as to which thread executes the signal handler. Second, I would like to know the steps that are followed to make the thread execute the signal handler.
On the first matter, I have read two different, seemingly conflicting, explanations:
The Linux Kernel, by Andries Brouwer, ยง5.2 "Receiving signals" states:
When a signal arrives, the process is interrupted, the current registers are saved, and the signal handler is invoked. When the signal handler returns, the interrupted activity is continued.
The StackOverflow question "Dealing With Asynchronous Signals In Multi Threaded Program" leads me to think that Linux's behavior is like SCO Unix's:
When a signal is delivered to a process, if it is being caught, it will be handled by one, and only one, of the threads meeting either of the following conditions:
A thread blocked in a sigwait(2) system call whose argument does include the type of the caught signal.
A thread whose signal mask does not include the type of the caught signal.
Additional considerations:
A thread blocked in sigwait(2) is given preference over a thread not blocking the signal type.
If more than one thread meets these requirements (perhaps two threads are calling sigwait(2)), then one of them will be chosen. This choice is not predictable by application programs.
If no thread is eligible, the signal will remain ``pending'' at the process level until some thread becomes eligible.
Also, "The Linux Signals Handling Model" by Moshe Bar states "Asynchronous signals are delivered to the first thread found not blocking the signal.", which I interpret to mean that the signal is delivered to some thread having its sigmask not including the signal.
Which one is correct?
On the second matter, what happens to the stack and register contents for the selected thread? Suppose the thread-to-run-the-signal-handler T is in the middle of executing a do_stuff() function. Is thread T's stack used directly to execute the signal handler (i.e. the address of the signal trampoline is pushed onto T's stack and control flow goes to the signal handler)? Alternatively, is a separate stack used? How does it work?
These two explanations really aren't contradictory if you take into account the fact that Linux hackers tend to be confused about the difference between a thread and a process, mainly due to the historical mistake of trying to pretend threads could be implemented as processes that share memory. :-)
With that said, explanation #2 is much more detailed, complete, and correct.
As for the stack and register contents, each thread can register its own alternate signal-handling stack, and the process can choose on a per-signal basis which signals will be delivered on alternate signal-handling stacks. The interrupted context (registers, signal mask, etc.) will be saved in a ucontext_t structure on the (possibly alternate) stack for the thread, along with the trampoline return address. Signal handlers installed with the SA_SIGINFO flag are able to examine this ucontext_t structure if they like, but the only portable thing they can do with it is examine (and possibly modify) the saved signal mask. (I'm not sure if modifying it is sanctioned by the standard, but it's very useful because it allows the signal handler to atomically replace the interrupted code's signal mask upon return, for instance to leave the signal blocked so it can't happen again.)
Source #1 (Andries Brouwer) is correct for a single-threaded process. Source #2 (SCO Unix) is wrong for Linux, because Linux does not prefer threads in sigwait(2). Moshe Bar is correct about the first available thread.
Which thread gets the signal? Linux's manual pages are a good reference. A process uses clone(2) with CLONE_THREAD to create multiple threads. These threads belong to a "thread group" and share a single process ID. The manual for clone(2) says,
Signals may be sent to a thread group as a whole (i.e., a
TGID) using kill(2), or to a specific thread (i.e., TID) using
tgkill(2).
Signal dispositions and actions are process-wide: if an
unhandled signal is delivered to a thread, then it will affect
(terminate, stop, continue, be ignored in) all members of the
thread group.
Each thread has its own signal mask, as set by sigprocmask(2),
but signals can be pending either: for the whole process
(i.e., deliverable to any member of the thread group), when
sent with kill(2); or for an individual thread, when sent with
tgkill(2). A call to sigpending(2) returns a signal set that
is the union of the signals pending for the whole process and
the signals that are pending for the calling thread.
If kill(2) is used to send a signal to a thread group, and the
thread group has installed a handler for the signal, then the
handler will be invoked in exactly one, arbitrarily selected
member of the thread group that has not blocked the signal.
If multiple threads in a group are waiting to accept the same
signal using sigwaitinfo(2), the kernel will arbitrarily
select one of these threads to receive a signal sent using
kill(2).
Linux is not SCO Unix, because Linux might give the signal to any thread, even if some threads are waiting for a signal (with sigwaitinfo, sigtimedwait, or sigwait) and some threads are not. The manual for sigwaitinfo(2) warns,
In normal usage, the calling program blocks the signals in set via a
prior call to sigprocmask(2) (so that the default disposition for
these signals does not occur if they become pending between
successive calls to sigwaitinfo() or sigtimedwait()) and does not
establish handlers for these signals. In a multithreaded program,
the signal should be blocked in all threads, in order to prevent the
signal being treated according to its default disposition in a thread
other than the one calling sigwaitinfo() or sigtimedwait()).
The code to pick a thread for the signal lives in linux/kernel/signal.c (the link points to GitHub's mirror). See the functions wants_signal() and completes_signal(). The code picks the first available thread for the signal. An available thread is one that doesn't block the signal and has no other signals in its queue. The code happens to check the main thread first, then it checks the other threads in some order unknown to me. If no thread is available, then the signal is stuck until some thread unblocks the signal or empties its queue.
What happens when a thread gets the signal? If there is a signal handler, then the kernel causes the thread to call the handler. Most handlers run on the thread's stack. A handler can run on an alternate stack if the process uses sigaltstack(2) to provide the stack, and sigaction(2) with SA_ONSTACK to set the handler. The kernel pushes some things onto the chosen stack, and sets some of the thread's registers.
To run the handler, the thread must be running in userspace. If the thread is running in the kernel (perhaps for a system call or a page fault), then it does not run the handler until it goes to userspace. The kernel can interrupt some system calls, so the thread runs the handler now, without waiting for the system call to finish.
The signal handler is a C function, so the kernel obeys the architecture's convention for calling C functions. Each architecture, like arm, i386, powerpc, or sparc, has its own convention. For powerpc, to call handler(signum), the kernel sets the register r3 to signum. The kernel also sets the handler's return address to the signal trampoline. The return address goes on the stack or in a register by convention.
The kernel puts one signal trampoline in each process. This trampoline calls sigreturn(2) to restore the thread. In the kernel, sigreturn(2) reads some information (like saved registers) from the stack. The kernel had pushed this information on the stack before calling the handler. If there was an interrupted system call, the kernel might restart the call (only if the handler used SA_RESTART), or fail the call with EINTR, or return a short read or write.
Related
Platform is Linux/POSIX.
The signal is sent to a whole process, not a specific thread.
No signal is set to blocked, all default.
The process is multi-thread process.
From what I've googled, a signal may be handled by a random thread.
And when that signal's handler is executing, it's temporarily blocked until handler returns.
QUESTION: Multiple signals of different types reached simultaneously. Do their handler execute simultaneously on multiple thread or all of them go to one randomly picked thread (SUB-QUESTION: in this case a handler could interrupt another handler's execution started previously, so there could be a interrupt stack?) ? Or mixed? For instance there are 3 type of signals received but only 2 thread free (this is actually the first case).
EXAMPLE: SIGHUP, SIGINT, SIGTERM reached almost simultaneously. The program has two available thread to dispatch signal handler execution.
SIDE-QUESTION: If signal handlers run in parallel, I'll have to use mutex to synchronize them properly. Otherwise 'volatile sig_atomic_t' would be enough, right?
Expected: all signals go to one thread (randomly picked) despite of their different signal types, I haven't seen an example of using mutexes and atoms to synchronize signal handlers.
Your understanding is correct - unless a signal was directed
to a specific thread, there's no guarantee which thread will handle a signal.
See POSIX's Signal Generation and Delivery and pthreads(7):
POSIX.1 distinguishes the notions of signals that are directed
to the process as a whole and signals that are directed to
individual threads. According to POSIX.1, a process-directed
signal (sent using kill(2), for example) should be handled by
a single, arbitrarily selected thread within the process.
So it may be delivered & handled by the same thread that's currently handling another signal (in that case, the previous handler may be interrupted by the new signal). Or may be delivered to another signal.
You can block other signals while one is being handled using sa_mask field
of sigaction to avoid a signal handler being interrupted.
SIDE-QUESTION: If signal handlers run in parallel, I'll have to use mutex to synchronize them properly. Otherwise 'volatile sig_atomic_t' would be enough, right?
You almost certainly don't want to use mutex in a signal handler. There are only few functions that can be safely called from a signal handler (you can only call the functions that are async-signal-safe).
See signal-safty for more information.
If you can use volatile sig_atomic_t for whatever the purpose (do you need to co-ordinate execution of different signal handlers?), it should be preferred.
Expected: all signals go to one thread (randomly picked) despite of their different signal types, I haven't seen an example of using mutexes and atoms to synchronize signal handlers.
This is commonly done by blocking signals that you're interested in from main and fetching/handling them in a specific thread. See pthread_sigmask which also has an example on how to implement this.
Can someone explain why we should not call non async functions from signal handlers ? Like the exact sequence of steps that corrupt the programs while calling with such functions.
And, does signals always run on separate stack ? if so is it a separate context or it runs on the context of the signaled thread ?
Finally, in case of a multi-threaded system what happens when signal handler is executed and some other thread is signaled and calls the same signal handler ?
(I am trying to develop deep understanding of signals and its applications)
When a process receives a signal, it is handled in the context of the process. You should only use aync-safe functions or re-entrant functions from inside a signal handler. For instance, you cannot call a malloc() or a printf() within a signal handler. The reason being:
*) Lets assume your process was executing in malloc when you received the signal. So the global heap data structures are in an inconsistent state. Now if you acquire the heap lock from inside your signal handler and make changes you will further render the heap inconsistent.
*) Another possibility is if the heap lock has been acquired by your process when it received the signal, and then you call malloc() from your signal handler, it sees that lock is held and it waits infinitely to acquire the lock (infinitely because the thread that can release the lock will not run till the signal is completely handled).
2) Signals run in the context of the process. As for the signal stack you can look at this SO answer -> Do signal handers have a separate stack?
3) As for getting multiple instances of the same signal you can look at this link -> Signal Handling in UNIX where Rumple Stiltskin answers it well.
I know some Solaris. So I'm using that for details. LWP==Solaris for "thread" as in pthreads.
trap signals like SIGILL, are delivered to the thread that caused the trap. Asynchronous signals are delivered to the first active thread (LWP), or process that is not blocking that signal. A kernel module called aslwp() traverses the process-header table (has associated LWP's) looking for the first likely candidate to receive the asynch signal.
A signal stack lives in the kernel. I'm not sure what/how to answer your signal stack question.
One process may have several pending signals. Is that what you mean?
Each signal destined for a process is held there until the process switches context (or is forced) into the active state. This in part because you generally cannot incur a trap when the process context has been swapped out and the process does nothing cpu-wise. You certainly can incur asynch signals. But the process cannot "do anything" with any signal if it cannot run. So, at this point the kernel swaps the context back to active, and the signal is delivered via aslwp().
Realtime signals behave differently, and I'm letting it stay with that.
Try reading this:
developers.sun.com/solaris/articles/signalprimer.html
I have a program in Linux which is multithreaded. There are certain memory areas in which I'm interested to see if they have been written within a certain time period. For that I give only read access to those memory pages and install a signal handler for SIGSEGV. Now my question is, will each thread call the signal handler for itself. Say Thread 1 writes to some forbidden memory area, will it be the one to execute the signal handler?
First of all
Signal dispositions are process-wide;
all threads in a process share the
same disposition for each signal. If
one thread uses sigaction() to
establish a handler for, say, SIGINT,
then that handler may be invoked from
any thread to which the SIGINT is
delivered.
But read on
A signal may be directed to either the
process as a whole or to a specific
thread. A signal is thread-directed if
it is generated as the direct result
of the execution of a specific
hardware instruction within the
context of the thread (SIGBUS, SIGFPE, SIGILL, and SIGSEGV)
I am quoting from TLPI.
No, per the question title.
To the question body: For the particular signal that you are asking for, yes (otherwise: it depends). The thread causing a segfault will receive the signal.
See signal(7):
A signal may be generated (and thus pending) for a process as a whole (e.g.,
when sent using kill(2)) or for a specific thread (e.g., certain signals, such
as SIGSEGV and SIGFPE, generated as a consequence of executing a specific
machine-language instruction are thread directed [...].
Manual has said that setitimer is shared in the whole PROCESS and the SIGPROF is send to the PROCESS not to the thread.
But when I create the timer in my multithread PROCESS, unless I create independent stacks for every thread in the PROCESS to handler the signo, I will got some very serious errors in the sig handler. Through some debugging, I confirm that the stack(sole stack case) must have been reenterd.
So now I suspect that SIGPROFs may be send to multithread at the same time? Thanks!
I don't follow the details of your question but the general case is:
A signal may be generated (and thus pending) for a process as a whole (e.g., when sent using kill(2)) or for a specific thread (e.g., certain signals, such as SIGSEGV and SIGFPE, generated as a consequence of executing a specific machine-language instruction are thread directed, as are signals targeted at a specific thread using pthread_kill(3)). A process-directed signal may be delivered to any one of the threads that does not currently have the signal blocked. If more than one of the threads has the signal unblocked, then the kernel chooses an arbitrary thread to which to deliver the signal.
man (7) signal
You can block the signal for specific threads with pthread_sigmask and by elimination direct it to the thread you want to handle it.
According to POSIX, the alternate signal stack established with sigaltstack is per-thread, and is not inherited by new threads. However, I believe some versions of Linux and/or userspace pthread library code (at least old kernels with LinuxThreads and maybe some versions with NPTL too?) have a bug where the alternate stack is inherited, and of course that will lead to crashing whenever you use the alternate stack. Is there a reason you need alternate stacks? Normally the only purpose is to handle stack overflows semi-gracefully (allowing yourself some stack place to catch SIGSEGV and save any unsaved data before exiting). I would just disable it.
Alternatively, use pthread_sigmask to block SIGPROF in all threads but the main one. Note that, to avoid a nasty race condition here, you need to block it in the main thread before calling pthread_create so that the new thread starts with it blocked, and unblock it after pthread_create returns.
How do signals work in unix? I went through W.R. Stevens but was unable to understand. Please help me.
The explanation below is not exact, and several aspects of how this works differ between different systems (and maybe even the same OS on different hardware for some portions), but I think that it is generally good enough for you to satisfy your curiosity enough to use them. Most people start using signals in programming without even this level of understanding, but before I got comfortable using them I wanted to understand them.
signal delivery
The OS kernel has a data structure called a process control block for each process running which has data about that process. This can be looked up by the process id (PID) and included a table of signal actions and pending signals.
When a signal is sent to a process the OS kernel will look up that process's process control block and examines the signal action table to locate the action for the particular signal being sent. If the signal action value is SIG_IGN then the new signal is forgotten about by the kernel. If the signal action value is SIG_DFL then the kernel looks up the default signal handling action for that signal in another table and preforms that action. If the values are anything else then that is assumed to be a function address within the process that the signal is being sent to which should be called. The values for SIG_IGN and SIG_DFL are numbers cast to function pointers whose values are not valid addresses within a process's address space (such as 0 and 1, which are both in page 0, which is never mapped into a process).
If a signal handling function were registered by the process (the signal action value was neither SIG_IGN or SIG_DFL) then an entry in the pending signal table is made for that signal and that process is marked as ready to RUN (it may have been waiting on something, like data to become available for a call to read, waiting for a signal, or several other things).
Now the next time that the process is run the OS kernel will first add some data to the stack and changes the instruction pointer for that process so that it looks almost like the process itself has just called the signal handler. This is not entirely correct and actually deviates enough from what actually happens that I'll talk about it more in a little bit.
The signal handler function can do whatever it does (it is part of the process that it was called on behalf of, so it was written with knowledge about what that program should do with that signal). When the signal handler returns then the regular code for the process begins executing again. (again, not accurate, but more on that next)
Ok, the above should have given you a pretty good idea of how signals are delivered to a process. I think that this pretty good idea version is needed before you can grasp the full idea, which includes some more complicated stuff.
Very often the OS kernel needs to know when a signal handler returns. This is because signal handlers take an argument (which may require stack space), you can block the same signal from being delivered twice during the execution of the signal handler, and/or have system calls restarted after a signal is delivered. To accomplish this a little bit more than stack and instruction pointer changes.
What has to happen is that the kernel needs to make the process tell it that it has finished executing the signal handler function. This may be done by mapping a section of RAM into the process's address space which contains code to make this system call and making the return address for the signal handler function (the top value on the stack when this function started running) be the address of this code. I think that this is how it is done in Linux (at least newer versions). Another way to accomplish this (I don't know if this is done, but it could be) would be do make the return address for the signal handler function be an invalid address (such as NULL) which would cause an interrupt on most systems, which would give the OS kernel control again. It doesn't matter a whole lot how this happens, but the kernel has to get control again to fix up the stack and know that the signal handler has completed.
WHILE LOOKING INTO ANOTHER QUESTION I LEARNED
that the Linux kernel does map a page into the process for this, but that the actual system call for registering signal handlers (what sigaction calls ) takes a parameter sa_restore parameter, which is an address that should be used as the return address from the signal handler, and the kernel just makes sure that it is put there. The code at this address issues the I'm done system call (sigreturn)and the kernel knows that the signal handler has finished.
signal generation
I'm mostly assuming that you know how signals are generated in the first place. The OS can generate them on behalf of a process due to something happening, like a timer expiring, a child process dying, accessing memory that it should not be accessing, or issuing an instruction that it should not (either an instruction that does not exist or one that is privileged), or many other things. The timer case is functionally a little different from the others because it may occur when the process is not running, and so is more like the signals sent with the kill system call. For the non-timer related signals sent on behalf of the current process these are generated when an interrupt occurs because the current process is doing something wrong. This interrupt gives the kernel control (just like a system call) and the kernel generates the signal to be delivered to the current process.
Some issues that are not addressed in all of the above statements are multi core, running in kernel space while receiving a signal, sleeping in kernel space while receiving a signal, system call restarting and signal handler latency.
Here are a couple of issues to consider:
What if the kernel knows that a signal needs to be delivered to process X which is running on CPU_X, but the kernel learns about it while running on CPU_Y (CPU_X!=CPU_Y). So the kernel needs to stop the process from running on a different core.
What if the process is running in kernel space while receiving a signal? Every time a process makes a system call it enters kernel space and tinkers with data structures and memory allocations in kernel space. Does all of this hacking take place in kernel space too?
What if the process is sleeping in kernel space waiting for some other event? (read, write, signal, poll, mutex are just some options).
Answers:
If the process is running on another CPU the kernel, via cross CPU communication, will deliver an interrupt to the other CPU and a message for it. The other CPU will, in hardware, save state and jump to the kernel on the other CPU and then will do the delivery of the signal on the other CPU. This is all a part of trying not to execute the signal handler of the process on another CPU which will break cache locality.
If the process is running in kernel space it is not interrupted. Instead it is recorded that this process has received a signal. When the process exits kernel space (at the end of each system call), the kernel will setup the trampoline to execute the signal handler.
If the process, while running in kernel space, after having received a signal, reaches a sleep function, then that sleep function (and this is common to all sleep functions within the kernel) will check if the process has a signal pending. If it is so, it will not put the process to sleep and instead will cancel all that has been done while coming down into the kernel, and will exit to user space while setting up a trampoline to execute the signal handler and then restart the system call. You can actually control which signals you want to interrupt system calls and which you do not using the siginterrupt(2) system call. You can decide if you want system calls restartable for a certain signal when you register the signal using sigaction(2) with the SA_RESTART flag. If a system call is issued and is cut off by a signal and is not restarted automatically you will get an EINTR (interrupted) return value and you must handle that value. You can also look at the restart_syscall(2) system call for more details.
If the process is already sleeping/waiting in kernel space (actually all sleeping/waiting is always in kernel space) it is woken from the sleep, kernel code cleans up after itself and jump to signal handler on return to user space after which the system call is automatically restarted if the user so desired (very similar to previous explanation of what happens if the process is running in kernel space).
A few notes about why all of this is so complex:
You cannot just stop a process running in kernel space since the kernel developer allocates memory, does things to data structures and more. If you just take the control away you will corrupt the kernel state and cause a machine hang. The kernel code must be notified in a controlled way that it must stop its running, return to user space and allow user space to handle the signal. This is done via the return value of all (well, almost all) sleeping functions in the kernel. And kernel programmers are expected to treat those return values with respect and act accordingly.
Signals are asynchronous. This means that they should be delivered as soon as possible. Imagine a process that has only one thread, went to sleep for hour, and is delivered a signal. Sleep is inside the kernel. So you except the kernel code to wake up, clean up after itself, return to user space and execute the signal handler, possibly restarting the system call after the signal handler finished. You certainly do not expect that process to only execute the signal handler an hour later. Then you expect the sleep to resume. Great trouble is taken by the user space and kernel people to allow just that.
All in all signals are like interrupt handlers but for user space. This is a good analogy but not perfect. While interrupt handlers are generated by hardware some signal handlers originate from hardware but most are just software (signal about a child process dying, signal from another process using the kill(2) syscall and more).
So what is the latency of signal handling?
If when you get a signal some other process is running then it up to the kernel scheduler to decide if to let the other process finish its time slice and only then deliver the signal or not. If you are on a regular Linux/Unix system this means that you could be delayed by 1 or more time slices before you get the signal (which means milliseconds which are equivalent to eternity).
When you get a signal, if your process is high-priority or other processes already got their time slice you will get the signal quite fast. If you are running in user space you will get it "immediately", if you are running in kernel space you will shortly reach a sleep function or return from kernel in which case when you return to user space your signal handler will be called. That is usually a short time since not a lot of time is spent in the kernel.
If you are sleeping in the kernel, and nothing else is above your priority or needs to run, the kernel thread handling your system call is woken up, cleans up after all the stuff it did on the way down into the kernel, goes back to user space and executes your signal. This doesn't take too long (were talking microseconds here).
If you are running a real time version of Linux and your process has the highest real time priority then you will get the signal very soon after it is triggered. Were talking 50 microseconds or even better (depends on other factors that I cannot go into).
Think of the signal facility as interrupts, implemented by the OS (instead of in hardware).
As your program merrily traverses its locus of execution rooted in main(), these interrupts can occur, cause the program to be dispatched to a vector (handler), run the code there, and then return to the location where it got interrupted.
These interrupts (signals) can originate from a variety of sources e.g. hardware errors like accessing bad or misaligned addresses, death of a child process, user generated signals using the kill command, or from other processes using the kill system call. The way you consume signals is by designating handlers for them, which are dispatched by the OS when the signals occur. Note that some of these signals cannot be handled, and result in the process simply dying.
But those that can be handled, can be quite useful. You can use them for inter process communication i.e. one process sends a signal to another process, which handles it, and in the handler does something useful. Many daemons will do useful things like reread the configuration file if you send them the right signal.
Signal are nothing but an interrupt in the execution of the process. A process can signal itself or it can cause a signal to be passed to another process. Maybe a parent can send a signal to its child in order to terminate it, etc..
Check the following link to understand.
https://unix.stackexchange.com/questions/80044/how-signals-work-internally
http://www.linuxjournal.com/article/3985
http://www.linuxprogrammingblog.com/all-about-linux-signals?page=show