There's a conflict between man pages which I don't understand.
man 7 pthreads says that:
POSIX.1-2001 and POSIX.1-2008 require that all functions
specified in the standard shall be thread-safe, except for the
following functions:
and exit() isn't in the thread-safety exception list.
However, man 3 exit says:
The exit() function uses a global variable that is not protected, so it is not thread-safe.
By googling it seems that exit() is actually thread unsafe. So what's wrong with my understanding of man pages? Why is exit() not listed as thread unsafe in man 7 pthreads?
Why is exit() not listed as thread unsafe in man 7 pthreads?
That is a question for the documentation authors. But I can say why it should never be considered thread safe (from a userspace/c language perspective). It is explicitly listed as: well it's complicated.
You can't legally call exit twice, so that takes care of the data race. Because even if you did it's already explictly undefined behavior to call exit or quick_exit anywhere in the application again after one of the two has been called.
Once you call exit you've basically said to the runtime "Hey I need to tear down the house" the house being your application. The runtime then runs atexit handlers per the standard.
Also worth noting is that atexit handlers are not guaranteed thread safe but atexit itself is, so if threads have registered their own atexit handlers those will get called.
So what does all this mean:
exit is not thread safe. But if you're tearing down the house does it really matter if the house was constructed to code anymore? Once you've called exit and the atexit handlers have finished then the kernel kills the threads, closes any handles, and reclaims everything. Worrying about closing the drapes when the bulldozer is coming through the wall is kind of pointless.
The C standard disallows exit from being called more than once. Section 7.22.4.4 regarding the exit function paragraph 2 states:
The exit function causes normal program termination to occur. No
functions registered by the at_quick_exit function are called.
If a program calls the exit function more than once, or calls the quick_exit function in addition to the exit function, the
behavior is undefined.
Additionally, the POSIX man page for exit, i.e. man 3p exit also states this:
If a function registered by a call to atexit() fails to return, the
remaining registered functions shall not be called and the rest of the
exit()
processing shall not be completed. If exit() is called more than once, the behavior is undefined.
So the function is not thread safe by definition.
Perhaps the answer is in https://man7.org/linux/man-pages/man3/pthread_exit.3.html
You call pthread_exit(). Not exit()
Eventually, when the last thread exits, exit is called for you.
exit is not thread-safe for these reasons:
it abruptly terminates all the threads. How can something which kills all the threads be "thread-safe"?
exit must not be called more than once; that is undefined behavior. Therefore the basic thread-safety question "can this function be called from two threads at around the same time" is not answerable in the affirmative.
exit can execute arbitrary application code: the atexit handlers. According to POSIX, calling the atexit handlers is the first thing that exit does, and that means this happens while threads are running. These handlers can be badly written such that they lack thread safety, which means that their parent function, exit, is rendered unsafe.
Related
This question already has answers here:
How to avoid using printf in a signal handler?
(8 answers)
Closed 2 years ago.
I have a code that looks like this:
//global variables
void signal_handler() {
//deallocation of global variables
free(foo);
close(foo_2);
exit(0);
}
int main () {
signal(SIGINT, signal_handler);
//irrelevant code
}
As you can see, I changed the CTRL+C interruption to execute the signal_handler function once instead of killing the process right away. I read somewhere that some functions like might be free are not async-safe and would NOT execute in the signal_handler but I'm not sure about that.
Can I execute functions like free, close, exit or even pthread_join in a signal handler?
No. Only functions listed in man 7 signal-safety are safe to call inside a signal handler.
close is listed and should be safe. free is not. For reasons why you would have to look at its source code (it contains locks). exit is not safe because it can call arbitrary cleanup handlers. You have _exit which exits abruptly without the cleanup.
You techincally can compile a program that calls such functions in a signal handler, nothing stops you from doing that. However it will result in undefined behavior if the function you are trying to execute is not async-signal-safe. It's not like unsafe function would just "NOT execute" as you say, they very well could, but that'd still be undefined behavior.
A list of async-signal-safe functions is documented in man 7 signal-safety. The close() function is safe, while free() and phtread_join() are not. The exit() function is also not safe to call from a signal handler, if you wish to exit from such context you will have to do so using _exit() instead.
The only way to safely call a function that is not async-signal-safe when receiving a signal is to "remember" that you have to call it (for example setting a global variable) and then do so after returning from the signal handler.
Short answer is no:
7.1.4 Use of library functions
...
4 The functions in the standard library are not guaranteed to be reentrant and may modify
objects with static or thread storage duration.188)
188) Thus, a signal handler cannot, in general, call standard library functions
C 2011 Online Draft
Real-world example of the consequences - I worked on a system that communicated with an Access database. There was a signal handler that tried to write an error message to the console with fprintf, but somehow during the signal handling process stderr got mapped to the .mdb file that stored the database, overwriting the header and ruining the database beyond repair.
There's honestly not a whole lot you can do in a signal handler other than set a flag to be checked elsewhere.
Can I execute free() or close() in a signal handler?
You definitely should not. See signal(7) and signal-safety(7)
In practice, it might work like you want perhaps more than half of the time. IIRC, the GCC compiler is doing like you want to do, and it usually works.
A better approach is to use some write(2) to a pipe(7) (from inside your signal handler) and from time to time check that pipe (in your main program) with poll(2) or related things.
Or you could set some volatile sigatomic_t flag; (perhaps it should be also _Atomic) in your signal handler, and check that flag elsewhere (in the main program, outside of signal handlers).
Qt is explaining that better than I could do in a few minutes.
On Linux, see also signalfd(2) and eventfd(2).
I'm debugging a sort of weird issue where it looks like a thread that's killed in an atexit handler is accessing a shared library, and is segfaulting because that shared library is unloaded before the exit handler runs. I'm not sure this is actually the issue, but it's a hunch on what might be happening.
When a process terminates (main exits or exit() is called), is the atexit handler the immediate next thing to run? My mind says so, but the segfault I'm seeing seems to say otherwise.
Is there any difference (with regards to exit handling) between main returning (either end of function or with return) and calling exit() directly?
When a process terminates (main exits or exit() is called), is the atexit handler the immediate next thing to run? My mind says so, but the segfault I'm seeing seems to say otherwise.
Not necessarily, you're guaranteed that atexit handlers will run. But that's it, atexit handlers may even be called concurrently with other things. While you're using c remember that c++ may be in the same process. C++ says that atexit may be called concurrently to destructors being run for objects of static duration. This means that atexit is very dangerous and you need to ensure you're being very careful what you call.
Is there any difference (with regards to exit handling) between main returning (either end of function or with return) and calling exit() directly?
According to the documentation: No.
The safest thing to do when tearing down the house: Nothing. Just get out and let the house be torn down. Closing the drapes on the way out isn't worth the time so to speak.
I know that atexit is used to register a function handler. Then when an exit in the code occur that function is called. But what if an exit occur inside the function handler?
I was expecting an infinite loop but in reality the program exit normally. Why?
void handler(){
printf("exit\n");
exit(1);
}
int maint(int argc, char *argv[]) {
atexit(handler);
exit(1);
}
The behavior is undefined.
7.22.4.4 The exit function
2 The exit function causes normal program termination to occur. No
functions registered by the at_quick_exit function are called. If a
program calls the exit function more than once, or calls the
quick_exit function in addition to the exit function, the behavior is
undefined.
Calling exit in an at_exit handler (that is being run during the normal processing of exit) is definitely a second call to exit.
Exiting normally is a possible behavior, but seeing as anything can happen (the nature of the behavior being undefined), it could very well result in catastrophe. Best not to do it.
As you have been pointed to, the behaviour is undefined.... but despite of that, trying to justify your observed behaviour, the library writers normally tend to cope with strange programmer behaviours (like the at least strange of calling exit() while the program is in exit()) I'll say:
It is possible that the exit(3) function, before calling any of the exit handlers, just unregisters it from the list of signal handlers. This would make the exit(2) function to call each exit handler only once and not to call the handler recursively. Just try to register it again to see what happens would be a good exercise.
It is possible that the exit function, marks itself as being run and if called inside a handler, just return, as if nothing happens.
It is possible your expected behaviour that could lead to a stak overflow (no pun here :))
It is possible to ...
Whatever happens is part of the U.B. commented in other answers, but for a library that tries to extend on the standard and behave normally, the probably best behaviour is to avoid recursive calls in exit handlers in some of the ways proposed.
On the other side, you had better not to use this feature (let's call so) in your programs, because, as it is not endorsed by the standard, can lead you to trouble if you port your programs elsewhere in the future.
You probably think on exit(3) as a function that is never to be called twice (apart from recursively, like you expose) but think you have several threads on your program and two of them call the exit(3) function at the same time.
The probable best behaviour is to have some kind of semaphore that allows the handlers to be protected from mutual access... but the best way to have the list of handlers for short time compromised, is to unlink one handler from the list (let's consider the list a queue, where each thread comes and takes a handler) they get the handler, unlock the queue and then execute it. This can lead in each handler being executed by one thread of the ones that have called exit(). The first thing an implementor faces it how to deal with several threads calling exit() at the same time.
POSIX.1 says that the result of calling exit(3) more than once (i.e., calling exit(3) within a function registered using atexit()) is undefined. On some systems (but not Linux), this can result in an infinite recursion; portable programs should not invoke exit(3) inside a function registered using atexit().
found here
When our UNIX/C program needs an emergency exit, we use exit (3) function and install atexit (3) handlers for emergency clean-ups. This approach worked fine until our application got threaded, at which point atexit() handlers stopped to work predictably.
We learned by trial an error that threads may already be dead in atexit() handler, and their stacks deallocated.
I failed to find a quote in the standard linking thread disappearance with atexit(): threads cease to exist after return from main(), but is it before invocation of atexit() or after? What's the actual practice on Linux, FreeBSD and Mac?
Is there a good pattern for emergency cleanup in a multi-threaded program?
Posix Standard
It doesn't seem to be defined by Posix whether atexit handlers are called before or after threads are terminated by exit.
There are two (or three) ways for a process to terminate "normally".
All threads terminate. When the last thread exits, either by returning or calling pthread_exit, atexit handlers are run. In this case there are no other threads. (This is platform dependent. Some platforms may terminate other threads if the main thread terminates other than by exit, others do not).
One thread calls exit. In this case, atexit handlers will be run and all threads terminated. Posix doesn't specify in what order.
main returns. This is more-or-less equivalent to calling exit() as the last line of main, so can be treated as above.
OS Practice
In Linux, the documentation https://linux.die.net/man/2/exit
says threads are terminated by _exit calling exit_group, and that _exit is called after atexit handlers. Therefore in Linux on calling exit any atexit handlers are run before threads are terminated. Note that they are run on the thread calling exit, not the thread that called atexit.
On Windows the behaviour is the same, if you care.
Patterns for emergency cleanup.
The best pattern is: Never be in a state which requires emergency cleanup.
There is no guarantee that your cleanup will run because
you could have a kill -9 or
a power outage.
Therefore you need to be able to recover in that scenario.
If you can recover from a that, you can also recover from abort, so you can use abort for your emergency exit.
If you can't do that, or if you have "nice-to-have" cleanup you want to do, atexit handlers should be fine provided you first gracefully stop all threads in the process to prevent entering an inconsistent state while doing cleanup.
The man page for atexit(3) says the following:
POSIX.1-2001 says that the result of calling exit(3) more than once (i.e., calling exit(3) within a function registered using atexit()) is undefined. On some systems (but not Linux), this can result in an infinite recursion; portable programs should not invoke exit(3) inside a function registered using atexit().
However, I'm interested in modifying the exit code in a finalizer for my program. The only way I've managed to do this is by calling exit() from within my finalization function, but the man page explicitly warns against that.
Is there any practical danger against doing this? Are there any implementations where that approach might cause problems? Even better, is there another way of doing this?
You can call _exit() instead.
Within the Notes section of the man page:
The function _exit() is like exit(), but does not call any functions registered with atexit() or on_exit().
This should avoid the "recursive" issue that is being warned about in the POSIX spec. If you are somehow able to guarantee that your "exit code changing" exit handler runs last, this should work perfectly, modulo the caveats also listed in the Notes:
Whether it flushes standard I/O buffers and removes temporary files created with tmpfile(3) is implementation-dependent. On the other hand, _exit() does close open file descriptors, and this may cause an unknown delay, waiting for pending output to finish. If the delay is undesired, it may be useful to call functions like tcflush(3) before calling _exit(). Whether any pending I/O is canceled, and which pending I/O may be canceled upon _exit(), is implementation-dependent.