Background: I'm fuzzing a long-lived process with afl-fuzz by passing to it the filename to process from a stub that afl-fuzz runs for each sample.
When the long-lived process crashes via SIGSEGV, I want the stub to also generate a SIGSEGV, so that afl-fuzz will mark the sample as interesting.
Will calling kill(stub_pid, SIGSEGV) from the long-lived process's SIGSEGV handler work ?
Will calling kill(stub_pid, SIGSEGV) from the long-lived process's SIGSEGV handler work ?
If a process ends up in a SIGSEGV-handler something very bad happened, which might include a completely destroyed stack and/or memory management.
It is not a good idea to rely on anything any more at this point, but just that the process goes down.
Trying to invoke any functionally beyond this point is likely to fail, that is unreliable.
A much safer approach to this would be to have the calling process monitor its child, and if the child happens to terminated unexpected (typically via SIGSEGV) start the appropriate actions.
Have a look at signal handling inside shell scripts (seach-key: "trap"), as such a script might be the parent to the process you want to monitor.
not recommended to do this through SIGSEGV but you can do this if you have proper permission.
Instead of wondering how to cause a segmentation fault in your program so that AFL would notice something odd, just call abort(). SIGABRT is caught by AFL as well and is much easier to trigger.
Related
I want to create an Units test framwork, but to provide a good reporting I need to catch SIGABRT, SIGSEGV and probably others signals to prevent my process from being killed (And so, to be able to continue the tests processing)...
But I don't know how to do this and so, I need information:
SIGABRT is a thread direct signal ?
What happens if I only use the main thread to catch the SIGABRT (or SIGSEGV) signal? Could the thread that called abort return from its call (I hope not) ?
If you have any useful documents, links or tutorial, I'm interested. It's for a C code using pthreads.
Thanks for your help
I need to catch SIGABRT, SIGSEGV and probably others signals to prevent my process from being killed
This is an exercise in futility. After SIGABRT or SIGSEGV is raised, you (in general) have no idea about the state of the process -- it may have corrupted heap, stack, global data internal to your test framework, global data internal to the C runtime system, etc. etc. Continuing such process is exceedingly likely to continue crashing at random (correct) places in the code.
The only sane way to handle this in a test framework is to fork and have the parent process handle child error exits, report them and continue running additional tests.
SIGABRT is a thread direct signal ?
There is no such thing as "direct signal". SIGABRT may be sent to the process from outside, or it can be raised inside the process.
What happens if I only use the main thread to catch the SIGABRT (or SIGSEGV) signal?
SIGSEGV and SIGABRT (when not sent from outside) is sent to the thread which caused the invalid memory operation (or raised it).
In addition, there is no way to "only use main thread" -- sigaction is global across all threads (though you can set a thread-specific signal mask).
My project language is C and will just run one process and multiple threads.
One of thread always crash, it will case process crash.
The stability of the process is very very important. So I want to know whether it is possible to "isolation" this thread. For example, if we can intercept SIGSEGV signal, we can just restart this thread.
If one thread does something that could cause a crash it might be affecting another thread since they share the same memory space, so there's no way to isolate them.
You need to fix your program so it doesn't crash in the first place. Start by using a memory checker such as valgrind.
The stability of the process is very very important.
Then it is very very important that you find and fix the cause of the SIGSEGV rather than "papering over the cracks" by trying to recover crashed threads.
Why?
Because the root cause of the SIGSEGV could be bug that will be retriggered in the restarted thread, leading to endless crashing/restarting of threads.
Because the root cause of the SIGSEGV could be a problem in a different thread ... which could continue triggering the problem.
Because in the execution steps leading up to the SIGSEGV, the thread could have corrupted shared data structures or done other things that may cause other threads to crash, get stuck or behave incorrectly in other ways.
Depends on the cause of the crash. It could be memory related resulting in corrupt data thus leading to the process crash.
If the program does crash and it is critical, use Monit to restart it.
Is there a possibility to backtrace a location where child process crashed in Linux using C/C++ code?
What I want to do is the following:
fork a new child process and retrieve it's PID
wait for forked child process to crash ... probably using signal handler for SIGCHLD, or using waitpid()/waitid()
retrieve stack trace of child at location where it crashed
This would make parent process act similar to debugger when attached proces crashes.
You can assume that child process is compiled with debug symbols and parent process has root permissions.
What is the simplest way to achieve such functionality?
It is much simpler in Linux to use the libSegFault library provided as part of the GNU C library. On my system, it is installed in /lib/x86_64-linux-gnu/libSegFault.so.
All you need to do is to set SEGFAULT_SIGNALS environment variable to all (so you can catch all crash causes the library supports), optionally SEGFAULT_OUTPUT_NAME to point to the file the stack trace is written to (default is to standard error), and LD_PRELOAD to point to the segfault library. As long as the process does not modify these environment variables, they apply to all child processes as well.
For example, if ./yourprog was the program that forks a child that crashes, and you want the stack trace to ./yourprog.stacktrace, run
SEGFAULT_SIGNALS=all \
SEGFAULT_OUTPUT_NAME=./yourprog.stacktrace \
LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so \
./yourprog
or all in one line without the backslashes (\).
The only downside is that each crash overwrites the existing file, so you'll only see the latest one. If you have /proc mounted, then the crash dump includes both a backtrace and the memory map of the process at the crash moment.
If you insist on doing it in your own C program, I recommend you first take a look at the libSegFault sources.
The point is, the stack trace must be dumped by the process itself; it is not accessible to the parent. To do that, you inject code into the child process using e.g. LD_PRELOAD environment variable (which is one of the dynamic linker control variables in Linux). (Note that the stack tracing etc. is done in a signal handler context, so only async-signal-safe functions should be used.)
For example, the parent process can create a pipe, and move its write end to a specific descriptor in the child process before executing the target process, with your helper preload library path in LD_PRELOAD.
The helper preload library interposes signal(), sigaction(), and possibly sigprocmask(), sigwait(), sigwaitinfo(), pthread_sigmask(), to ensure the helper librarys crash dump signal handlers are executed when such a signal is delivered (SIGSEGV, SIGBUS, SIGILL, possibly SIGTRAP). The signal handler does the stack dump (and prints /proc/PID/maps), then sets the signal disposition to default, and re-raises the signal (using raise()).
Essentially, it boils down to doing the same as above libSegFault, except with your own C code.
If you don't want to inject code to the child process, or managing the signal handlers is too complicated, you can use ptrace instead.
When the tracee is killed by a signal (other than SIGKILL), the thread receiving the signal is stopped first ("signal-delivery-stop"), so the tracer can examine its stack (and memory map of the tracee), before letting the child process continue/die.
In practice, ptracing is more invasive, as there are many events that cause the tracees threads to stop. It is also much more complicated for multithreaded processes than the LD_PRELOAD approach, because ptrace can control individual threads in the tracee; there are much more details to get right.
I have a test program for part of a C library where I handle signals. There is a lot of code in this test program - so much that I feel that posting it here would only make it more difficult to answer my question.
Basically what's happening is that I am forcing the system to send me a SIGSEGV inside a background thread. I handle that signal, and then the test is over. The main thread will be waiting for this and part of the signal handler is to restart the main thread. When the main thread restarts I restore the default signal handler by signal(SIGSEGV, SIG_DFL) and then I get another SIGSEGV.
I understand that I have a lot of code and this could very well be my fault, but is there a common explanation of why a SIGSEGV would fire twice? It is not happening when I cause other signals (though I haven't tried all of them).
To clarify I am on OS X, writing in C, compiling with gcc.
If you have any problem inside the signal handler, for example, memory overwrite, trying to access beyond the allocated area, then a new signal will be raised. Check the following page for details.
http://www.gnu.org/software/libc/manual/html_node/Signals-in-Handler.html
You should also use reentrant functions inside the signal handlers, For example localtime_r instead of localtime.
Moreover it is recommended to use sigaction() instead of signal().
More details here: What is the difference between sigaction and signal?
What happens when two threads of the same process running on different logical cpu hit a seg fault?
Default action is for the process to exit. If you handle the segfault, I suppose you could try to arrange for just the thread where it happened to terminate. However, since the only things which cause a segfault to occur naturally (as opposed to raise or kill) stem from undefined behavior, the program is in an indeterminate state and you can't rely on being able to recover anything.
Normal handling of a Segmentation Fault involves the termination of the process. That means that both of them are terminated.
I think the default action on all major OSes is to terminate the process. However, you could conceivably install (e.g using signal) an alternate handler that only terminated the thread. Of course, once you have a segmentation fault, behavior typically becomes undefined, and attempting to continue is risky.
Signals generated due to illegal execution are handled synchronously by the kernel. So even if both the threads generate seg fault at the same time, only one gets thru'.
The segfault handler is called for the process. If you don't do anything special, the OS will kill the process and reclaim its resources.