so I cannot seem to find solid info on whether assert is useable in a mulththreaded context.
logically to me it seems if an assertion fails the thread get shutdown but not the other threads?
or does the entire process get killed?
so basically my question. is it safe to use assert in a multithreaded environment without leaking resources?
if you see the man page of assert(), it clearly states,
The purpose of this macro is to help the programmer find bugs in his
program. The message "assertion failed in file foo.c, function
do_bar(), line 1287" is of no help at all to a user.
This means, it's only useful [and should be used] in a developing environment, not in production software. IMO, in development stage, you need not to worry about leaks caused by assert(). YMMV.
Once you finished debugging your code, you can simply switch off the assert() functionality by defining [#define] NDEBUG.
I'd say more than yes. If I'd see a multithreaded code without asserts I'd not trust it. If you simplify a bit its implementations to something like:
#define assert(x) if( !(x) ) abort()
You'll see that it does nothing special for thread-safety or thread-specific. It's your responsibility to provide race-free condition and if the assertion fails, the whole process is aborted.
The entire process gets killed. Assert will send the expression, source filename and line number to stderr and then call abort(). Abort() terminates the entire process.
Related
Is there a difference between the first thread and other threads created during runtime. Because I have a program where to abort longjmp is used and a thread should be able to terminate the program (exit or abort don't work in my case). Could I safely use pthread_kill_other_threads_np and then longjmp?
I'm not sure what platform you're talking about, but pthread_kill_other_threads_np is not a standard function and not a remotely reasonable operation anymore than free_all_malloced_memory would be. Process termination inherently involves the termination of all threads atomically with respect to each other (they don't see each other terminate).
As for longjmp, while there is nothing wrong with longjmp, you cannot use it to jump to a context in a different thread.
It sounds like you have an XY problem here; you've asked about whether you can use (or how to use) particular tools that are not the right tool for whatever it is you want, without actually explaining what your constraints are.
According to the man page (2) the exit function is not thread safe : MT-Unsafe race:exit, this is because this function tries to clean up resources (flush data to the disk, close file descriptors, etc...) by calling callbacks registered using on_exit and atexit. And I want my program to do that ! (one of my thread keeps a fd open during the whole program's lifespan so _exit is not an option for me because I want all the data to be written to the output file)
My question is the following : if I'm being careful and I don't share any sensible data (like a fd) between my threads, is it "acceptable" to call exit in a multi-threaded program ? Note that I'm only calling exit if an unrecoverable error occurs. Yet, I can't afford having a segfault while the program tries to exit. The thing is, an unrecoverable error can happen from any thread...
I was thinking about using setjmp/longjmp to kill my threads "nicely" but this would be quite complex to do and would require many changes everywhere in my code.
Any suggestions would be greatly appreciated. Thanks ! :)
EDIT : Thanks to #Ctx enlightenment, I came up with the following idea :
#define EXIT(status) do { pthread_mutex_lock(&exit_mutex); exit(status); } while(0)
Of course the exit_mutex must be global (extern).
The manpage states that
The exit() function uses a global variable that is not protected, so it is not thread-safe.
so it won't help, if you are being careful in any way.
But the problem documented is a race condition: MT-Unsafe race:exit
So if you make sure, that exit() can never be called concurrently from two threads, you should be on the safe side! You can make this sure by using a mutex for example.
A modern cross-platform C++ solution could be:
#include <cstdlib>
#include <mutex>
std::mutex exit_mutex;
[[noreturn]] void exit_thread_safe(const int status)
{
exit_mutex.lock();
exit(status);
}
The mutex ensures that exit is never called by 2 (or more) different threads.
However, I still question the reason behind even caring about this. How likely is a multi-threaded call to exit() and which bad things can even realistically happen?
EDIT:
Using std::quick_exit avoids the clang diagnostic warning.
It can't be done: even if no data is shared between threads at first, data must be shared between a thread and its cleanup function. The function should run only after the thread has stopped or reached a safe point.
I am new to C programming. I used to think using exit() was the cleanest way of process termination (as it is capable of removing temporary files, closing open files, normal process termination...), but when I tried man exit command on the terminal (Ubuntu 16.04.5, gcc 5.4.0) I saw the following line:
The exit() function uses a global variable that is not protected, so
it is not thread-safe.
After that I tried to make some research about better replacement for exit() (to change my programming behavior from the beginning). While doing that I faced with this question in which side effects of exit() is mentioned and it is suggested to use atexit() properly to solve the problem (at least partially).
There were some cases in which using abort() was preferred over exit(). On top of that, this question suggests that atexit() might also be harmful.
So here are my questions:
Is there any general and better way of process terminating (which is guaranteed to clean like exit() and is not harmful for the system at any case)?
If the answer to the first question is NO!, what is the best possible way of process terminating (including the cases in which they are most useful)?
what is the best possible way of process terminating
If going single threaded just use exit(), as your code is not going multi-threaded.
Else make sure all but one thread have ended before the last thread and then safely call exit() because of 1. above.
Given that power/hardware fails can happen at any time, the imposs.. extreme difficulty of reliably terminating threads with user code and the chaotic nature of the use of memory pools etc. in many non-trivial multithreaded apps, it is better to design apps and systems that can clean temp files etc. on start-up, rather than trying to micro-manage shutdown.
'Clean up all the resources you allocate before you exit' sounds like good advice in a classroom or lecture, but quickly becomes a whole chain of albatross round your neck when faced with a dozen threads, queues and pools in a continually changing dynamic system.
If you can, if you are running under a non trivial OS, let it do its job and clean up for you. It's much better at it than your user code will ever be.
There is question about using exit in C++. The answer discusses that it is not good idea mainly because of RAII, e.g., if exit is called somewhere in code, destructors of objects will not be called, hence, if for example a destructor was meant to write data to file, this will not happen, because the destructor was not called.
I was interested how is this situation in C. Are similar issues applicable also in C? I thought since in C we don't use constructors/destructors, situation might be different in C. So is it ok to use exit in C? For example I have seen following functions sometimes used in C:
void die(const char *message)
{
if(errno) {
perror(message);
} else {
printf("ERROR: %s\n", message);
}
exit(1);
}
Rather than abort(), the exit() function in C is considered to be a "graceful" exit.
From C11 (N1570) 7.22.4.4/p2 The exit function (emphasis mine):
The exit function causes normal program termination to occur.
The Standard also says in 7.22.4.4/p4 that:
Next, all open streams with unwritten buffered data are flushed, all
open streams are closed, and all files created by the tmpfile function
are removed.
It is also worth looking at 7.21.3/p5 Files:
If the main function returns to its original caller, or if the exit
function is called, all open files are closed (hence all output
streams are flushed) before program termination. Other paths to
program termination, such as calling the abort function, need not
close all files properly.
However, as mentioned in comments below you can't assume that it will cover every other resource, so you may need to resort to atexit() and define callbacks for their release individually. In fact it is exactly what atexit() is intended to do, as it says in 7.22.4.2/p2 The atexit function:
The atexit function registers the function pointed to by func, to be
called without arguments at normal program termination.
Notably, the C standard does not say precisely what should happen to objects of allocated storage duration (i.e. malloc()), thus requiring you be aware of how it is done on particular implementation. For modern, host-oriented OS it is likely that the system will take care of it, but still you might want to handle this by yourself in order to silence memory debuggers such as Valgrind.
Yes, it is ok to use exit in C.
To ensure all buffers and graceful orderly shutdown, it would be recommended to use this function atexit, more information on this here
An example code would be like this:
void cleanup(void){
/* example of closing file pointer and free up memory */
if (fp) fclose(fp);
if (ptr) free(ptr);
}
int main(int argc, char **argv){
/* ... */
atexit(cleanup);
/* ... */
return 0;
}
Now, whenever exit is called, the function cleanup will get executed, which can house graceful shutdown, clean up of buffers, memory etc.
You don't have constructors and destructors but you could have resources (e.g. files, streams, sockets) and it is important to close them correctly. A buffer could not be written synchronously, so exiting from the program without correctly closing the resource first, could lead to corruption.
Using exit() is OK
Two major aspects of code design that have not yet been mentioned are 'threading' and 'libraries'.
In a single-threaded program, in the code you're writing to implement that program, using exit() is fine. My programs use it routinely when something has gone wrong and the code isn't going to recover.
But…
However, calling exit() is a unilateral action that can't be undone. That's why both 'threading' and 'libraries' require careful thought.
Threaded programs
If a program is multi-threaded, then using exit() is a dramatic action which terminates all the threads. It will probably be inappropriate to exit the entire program. It may be appropriate to exit the thread, reporting an error. If you're cognizant of the design of the program, then maybe that unilateral exit is permissible, but in general, it will not be acceptable.
Library code
And that 'cognizant of the design of the program' clause applies to code in libraries, too. It is very seldom correct for a general purpose library function to call exit(). You'd be justifiably upset if one of the standard C library functions failed to return just because of an error. (Obviously, functions like exit(), _Exit(), quick_exit(), abort() are intended not to return; that's different.) The functions in the C library therefore either "can't fail" or return an error indication somehow. If you're writing code to go into a general purpose library, you need to consider the error handling strategy for your code carefully. It should fit in with the error handling strategies of the programs with which it is intended to be used, or the error handling may be made configurable.
I have a series of library functions (in a package with header "stderr.h", a name which treads on thin ice) that are intended to exit as they're used for error reporting. Those functions exit by design. There are a related series of functions in the same package that report errors and do not exit. The exiting functions are implemented in terms of the non-exiting functions, of course, but that's an internal implementation detail.
I have many other library functions, and a good many of them rely on the "stderr.h" code for error reporting. That's a design decision I made and is one that I'm OK with. But when the errors are reported with the functions that exit, it limits the general usefulness the library code. If the code calls the error reporting functions that do not exit, then the main code paths in the function have to deal with error returns sanely — detect them and relay an error indication to the calling code.
The code for my error reporting package is available in my SOQ (Stack Overflow Questions) repository on GitHub as files stderr.c and stderr.h in the src/libsoq sub-directory.
One reason to avoid exit in functions other than main() is the possibility that your code might be taken out of context. Remember, exit is a type of non local control flow. Like uncatchable exceptions.
For example, you might write some storage management functions that exit on a critical disk error. Then someone decides to move them into a library. Exiting from a library is something that will cause the calling program to exit in an inconsitent state which it may not be prepared for.
Or you might run it on an embedded system. There is nowhere to exit to, the whole thing runs in a while(1) loop in main(). It might not even be defined in the standard library.
Depending on what you are doing, exit may be the most logical way out of a program in C. I know it's very useful for checking to make sure chains of callbacks work correctly. Take this example callback I used recently:
unsigned char cbShowDataThenExit( unsigned char *data, unsigned short dataSz,unsigned char status)
{
printf("cbShowDataThenExit with status %X (dataSz %d)\n", status, dataSz);
printf("status:%d\n",status);
printArray(data,dataSz);
cleanUp();
exit(0);
}
In the main loop, I set everything up for this system and then wait in a while(1) loop. It is possible to make a global flag to exit the while loop instead, but this is simple and does what it needs to do. If you are dealing with any open buffers like files and devices you should clean them up before close for consistency.
It is terrible in a big project when any code can exit except for coredump. Trace is very import to maintain a online server.
I have a program that has to perform certain tasks before it finishes. The problem is that sometimes the program crashes with an exception (like database cannot be reached, etc).
Now, is there any way to detect an abnormal termination and execute some code before it dies?
Thanks.
code is appreciated.
1. Win32
The Win32 API contains a way to do this via the SetUnhandledExceptionFilter function, as follows:
LONG myFunc(LPEXCEPTION_POINTERS p)
{
printf("Exception!!!\n");
return EXCEPTION_EXECUTE_HANDLER;
}
int main()
{
SetUnhandledExceptionFilter((LPTOP_LEVEL_EXCEPTION_FILTER)&myFunc);
// generate an exception !
int x = 0;
int y = 1/x;
return 0;
}
2. POSIX/Linux
I usually do this via the signal() function and then handle the SIGSEGV signal appropriately. You can also handle the SIGTERM signal and SIGINT, but not SIGKILL (by design). You can use strace() to get a backtrace to see what caused the signal.
There are sysinternals forum threads about protecting against end-process attempts by hooking NT Internals, but what you really want is either a watchdog or peer process (reasonable approach) or some method of intercepting catastrophic events (pretty dicey).
Edit: There are reasons why they make this difficult, but it's possible to intercept or block attempts to kill your process. I know you're just trying to clean up before exiting, but as soon as someone releases a process that can't be immediately killed, someone will ask for a method to kill it immediately, and so on. Anyhow, to go down this road, see above linked thread and search some keywords you find in there for more. hook OR filter NtTerminateProcess etc. We're talking about kernel code, device drivers, anti-virus, security, malware, rootkit stuff here. Some books to help in this area are Windows NT/2000 Native API, Undocumented Windows 2000 Secrets: A Programmer's Cookbook, Rootkits: Subverting the Windows Kernel, and, of course, Windows® Internals: Fifth Edition. This stuff is not too tough to code, but pretty touchy to get just right, and you may be introducing unexpected side-effects.
Perhaps Application Recovery and Restart Functions could be of use? Supported by Vista and Server 2008 and above.
ApplicationRecoveryCallback Callback Function Application-defined callback function used to save data and application state information in the event the application encounters an unhandled exception or becomes unresponsive.
On using SetUnhandledExceptionFilter, MSDN Social discussion advises that to make this work reliably, patching that method in-memory is the only way to be sure your filter gets called. Advises to instead wrap with __try/__except. Regardless, there is some sample code and discussion of filtering calls to SetUnhandledExceptionFilter in the article "SetUnhandledExceptionFilter" and VC8.
Also, see Windows SEH Revisited at The Awesome Factor for some sample code of AddVectoredExceptionHandler.
It depends what do you do with your "exceptions". If you handle them properly and exit from program, you can register you function to be called on exit, using atexit().
It won't work in case of real abnormal termination, like segfault.
Don't know about Windows, but on POSIX-compliant OS you can install signal handler that will catch different signals and do something about it. Of course you cannot catch SIGKILL and SIGSTOP.
Signal API is part of ANSI C since C89 so probably Windows supports it. See signal() syscall for details.
If it's Windows-only, then you can use SEH (SetUnhandledExceptionFilter), or VEH (AddVectoredExceptionHandler, but it's only for XP/2003 and up)
Sorry, not a windows programmer. But maybe
_onexit()
Registers a function to be called when program terminates.
http://msdn.microsoft.com/en-us/library/aa298513%28VS.60%29.aspx
First, though this is fairly obvious: You can never have a completely robust solution -- someone can always just hit the power cable to terminate your process. So you need a compromise, and you need to carefully lay out the details of that compromise.
One of the more robust solutions is putting the relevant code in a wrapper program. The wrapper program invokes your "real" program, waits for its process to terminate, and then -- unless your "real" program specifically signals that it has completed normally -- runs the cleanup code. This is fairly common for things like test harnesses, where the test program is likely to crash or abort or otherwise die in unexpected ways.
That still gives you the difficulty of what happens if someone does a TerminateProcess on your wrapper function, if that's something you need to worry about. If necessary, you could get around that by setting it up as a service in Windows and using the operating system's features to restart it if it dies. (This just changes things a little; someone could still just stop the service.) At this point, you probably are at a point where you need to signal successful completion by something persistent like creating a file.
I published an article at ddj.com about "post mortem debugging" some years ago.
It includes sources for windows and unix/linux to detect abnormal termination. By my experience though, a windows handler installed using SetUnhandledExceptionFilter is not always called. In many cases it is called, but I receive quite a few log files from customers that do not include a report from the installed handlers, where i.e. an ACCESS VIOLATION was the cause.
http://www.ddj.com/development-tools/185300443