Debugging a program that uses SIGINT with gdb - c

I frequently work with PostgreSQL for debugging, and it uses SIGINT internally for some of its inter-backend signalling.
As a result when running certain backends under gdb execution tends to get interrupted a lot. One can use the signal command to make sure SIGINT is passed to the program and that it is not captured by gdb... but then gdb doesn't respond to control-C on the command line, since that sends SIGINT.
If you run:
handle SIGINT noprint nostop pass
gdb will complain
SIGINT is used by the debugger.
Are you sure you want to change it? (y or n) y
Is there any way to get gdb to use a different interrupt signal? Or any alternative method that'd let me have gdb ignore SIGINT?
(This isn't an issue for most PostgreSQL backend debugging, but it's a pain with background workers and autovacuum).

Readers who end up on this page (as I did) with a slightly different variation of this problem, would perhaps be more interested in this question:
Debugging a segmentation fault when I do ctrl c
... and its answer, which is:
send SIGINT from inside gdb itself:
(gdb) signal 2
(Normally I would post the link as a simple comment under the OP's question on this page, but since there are already 7 comments, comments are being hidden/buried.)
If you read all the details of the OP's question here, then it is obvious that my answer is not correct for OP.
However, my answer is correct for many situations that could be described by the same title: "Debugging a program that uses SIGINT with gdb"

On UNIX-like systems, you can distinguish a tty-initiated SIGINT from one sent by kill by looking at the si_pid element in the siginfo struct. If the pid is 0, it came from a tty.
So you could do something like this:
catch signal SIGINT
commands
if $_siginfo._sifields._kill.si_pid == 0
print "Received SIGINT from tty"
else
printf "Received SIGINT from %d; continuing\n", $_siginfo._sifields._kill.si_pid
signal SIGINT
end
end

This part of gdb is a bit tricky, both due to its history and also due to the various modes of operation it supports.
One might think that running gdb in a separate terminal and only using attach would help it do the right thing, but I don't think it is that easy.
One way forward might be to only use async execution when debugging, and then use a command to interrupt the inferior. Something like:
(gdb) attach 5555
... attaches
(gdb) continue &
... lots of stuff happens
(gdb) interrupt -a
Depending on your version of gdb you might need to set target-async for this to work.

Related

How to change the meaning of SIGTERM in C program

I've recently had a problem with signals. I'd like to write a program in C which would print anything after a signal is sent to the process. For example: If I send SIGTERM to my process (which is simply running program), I want the program to print out for example, "killing the process denied" instead of killing the process. So how to do that? How to force process to catch and change the meaning of such signal. Also I have a question if there is any possibility to kill the init process (I know it's kind of a stupid question, but I was wondering how linux deals with such a signal, and how would it technically look if I type: sudo kill -9 1.
Don't use the signal handler to print. You can set a variable of type volatile sig_atomic_t instead, and have your main thread check this (see this example).
When your main thread has nothing else to do (which should be most of the time), let it block on a blocking function call (e.g. sleep()) that will wake up immediately when the signal is received (and set errno to EINTR).
C++ gotcha: Unlike the C sleep() function, std::this_thread::sleep_for() (in recent versions of glibc) does not wake up when a signal is received.
Regarding if it's possible to kill pid 1, see this question. The answer seems to be no, but I remember that Linux got very grumpy once I booted with init=/bin/bash and later exited this shell – had to hard reboot.
If you're looking for trouble, better kill pid -1.

Re-propagate a Caught Signal in Linux

I am trying to use sigwait() for SIGSEGV and other signals and then do something (says writing the timestamp to a file). Then, I would like the program to behave as if the signal were never caught (e.g. a coredump is generated when SIGSEGV happens -- assuming the system is configured for coredump).
I read through couple manpages, and I don't see an obvious way to do so.
Any suggestion?
Thanks in advance!
You can register a signal handler that won't cause the program to crash. Once you receive the signal, you'll enter the signal handler, and after executing the signal handler (write to file you said), you'll reenter the program where you left off.
If you want a core dump, you can use the linux core(5) command and it will create a core dump in your signal handler without ending your process.
http://www.alexonlinux.com/signal-handling-in-linux#signal_masks
I believe you can not handle few signals the way you want to for example
SIGKILL
SIGSEGV
as manual itself states The signals SIGKILL and SIGSTOP cannot be caught or ignored
check below links
SIGNAL(7) and SIGNAL(2) These links should provide good enough details on signal handling in linux
You can however register a signal handler to handle few other signals
to list signals in operating system like linux you can use below command
kill -l
and as far as generating core dump is conserned its generated by operating system click here for more info
check the example code here

Uninterruptable process in Windows(or Linux)?

Is there any way to make a program that cannot be interrupted (an uninterrupted program)? By that, I mean a process that can't be terminated by any signal, kill command, or any other key combinations in any System: Linux, windows etc.
First, I am interested to know whether it's possible or not. And if yes, upto what extend it is possible?
I mostly write code in C, C++, and python; but I don't know any of such command(s) available in these programming languages.
Is it possible with assembly language, & how ? Or in high level language c with embedded assembly code(inline assembly)?
I know some signals are catchable some are not like SIGKILL and SIGSTOP.
I remember, when I was use to work on Windows-XP, some viruses couldn't be terminated even from Task Manager. So I guess some solution is possible in low level languages. maybe by overriding Interrupt Vector Table.
Can we write an uninterrupted program using TSRs(Hooking)? Because TSR can only removed when the computer is rebooted or if the TSR is explicitly removed from memory. Am I correct?
I couldn't find any thing on Google.
Well, possibly one can write a program which doesn't respond for most signals like SIGQUIT, SIGHUP etc. - each kind of "kill" is actually a kind of signal sent to program by kernel, some signals means for the kernel that program is stuck and should be killed.
Actually the only unkillable program is kernel itself, even init ( PID 1 ) can be "killed" with HUP ( which means reload ).
Learn more about signal handling, starting with kill -l ( list signals ) command.
Regarding Windows ( basing on "antivirus" tag ) - which actually applies to linux too - if you just need to run some antivirus user is unable to skip/close, it's permission problem, I mean program started by system, and non-administrative user without permission to kill it, won't be able to close/exit it anyway. I guess lameusers on Windows all over the world would start "solving" any problems they have by trying to close antivirus first, just if it would be possible :)
On Linux, it is possible to avoid being killed by one of two ways:
Become init (PID 1). init ignores all signals that it does not catch, even normally unblockable ones like SIGSTOP and SIGKILL.
Trigger a kernel bug, and get your program stuck in D (uninterruptible wait) state.
For 2., one common way to end up in D state is to attempt to access some hardware that is not responding. Particularly on older versions of Linux, the process would become stuck in kernel mode, and not respond to any signals until the kernel gave up on the hardware (which can take quite some time!). Of course, your program can't do anything else while it's stuck like this, so it's more annoying than useful, and newer versions of Linux are starting to rectify this problem by dividing D state into a killable state (where SIGKILL works) and an unkillable state (where all signals are blocked).
Or, of course, you could simply load your code as a kernel module. Kernel modules can't be 'killed', only unloaded - and only if they allow themselves to be unloaded.
You can catch pretty-much any signal or input and stay alive through it, the main exception being SIGKILL. It is possible to prevent that from killing you, but you'd have to replace init (and reboot to become the new init). PID 0 is special on most Unixes, in that it's the only thing that can't be KILL'd.

Debugging signal handling in multithreaded application

I have this multithreaded application using pthreads. My threads actually wait for signals using sigwait. Actually, I want to debug my application, see which thread receives which signal at what time and then debug it. Is there any method, I can do this. If I directly run my program, then signals are generated rapidly and handled by my handler threads. I want to see which handler wakes up from the sigwait call and processes the signal and all.
The handy strace utility can print out a huge amount of useful information regarding system calls and signals. It would be useful to log timing information or collect statistics regarding the performance of signal usage.
If instead you are interested in getting a breakpoint inside of an event triggered by a specific signal, you could consider stashing enough relevant information to identify the event in a variable and setting a conditional breakpoint.
One of the things you may try with gdb is set breakpoints by thread (e.g. just after return from sigwait), so you know which thread wakes up:
break file.c thread [thread_nr]
Don't forget to tell gdb to pass signals to your program e.g.:
handle SIGINT pass
You may want to put all of this into your .gdbinit file to save yourself a lot of typing.
Steven Schlansker is definitely right: if that happens to significantly change timing patterns of your program (so you can see that your program behaves completely different under debugger, than "in the wild") then strace and logging is your last hope.
I hope that helps.

Preventing MSYS 'bash' from killing processes that trap ^C

I have a console-mode Windows application (ported from Unix) that was originally designed to do a clean exit when it received ^C (Unix SIGINT). A clean exit in this case involves waiting, potentially quite a long time, for remote network connections to close down. (I know this is not the normal behavior of ^C but I am not in a position to change it.) The program is single-threaded.
I can trap ^C with either signal(SIGINT) (as under Unix) or with SetConsoleCtrlHandler. Either works correctly when the program is run under CMD.EXE. However, if I use the "bash" shell that comes with MSYS (I am using the MinGW environment to build the program, as this allows me to reuse the Unix makefiles) then the program is forcibly terminated some random, short time (less than 100 milliseconds) after the ^C. This is unacceptable, since as I mentioned, the program needs to wait for remote network connections to close down.
It is very likely that people will want to run this program under MSYS bash. Also, this effect breaks the test suite. I have not been able to find any way to work around the problem either from within the program (ideal) or by settings on the shell (acceptable). Can anyone recommend anything?
I had the exact same problem - I had written a program with a SIGINT/SIGTERM handler. That handler did clean-up work which sometimes took awhile. When I ran the program from within msys bash, ctrl-c would cause my SIGINT handler to fire, but it would not finish - the program was terminated ("from the outside", as it were) before it could complete its clean-up work.
Building on phs's answer, and this answer to a similar question: https://stackoverflow.com/a/23678996/2494650, I came up with the following solution. It's insanely simple, and it might have some side-effects that I've yet to discover, but it fixed the problem for me.
Create a ~/.bashrc file with the following line:
trap '' SIGINT
That's it. This traps the sigint signal and prevents msys bash from terminating your program "from the outside". However, it somehow still lets the SIGINT signal through to your program, allowing it to do its graceful cleanup/shutdown. I can't tell you exactly why it works this way, but it does - at least for me.
Good luck!
This could be due to the infamous mintty "Input/Output interaction with alien programs" problem (aka mintty issue #56). In this case it is manifesting as Ctrl-C abruptly killing the program rather than being passed down to the program as a signal to be caught and handled. Evidence for this theory is based on zwol's extensive explanation: "console-mode Windows application", "[application is] designed to do a clean exit when it received ^C", "[application] works correctly when the program is run under CMD.EXE" but "[when using the terminal] that comes with MSYS [...] program is forcibly terminated" (at the time of writing (2018) MSYS defaults to using mintty as its terminal).
Unfortunately mintty isn't a full Windows console replacement and various behaviours expected by "native" Windows programs are not implemented. However, you might have some joy wrapping such native programs in winpty when running them within mintty...
Other questions also describe this behaviour: see https://superuser.com/questions/606201/how-to-politely-kill-windows-process-from-cygwin and https://superuser.com/questions/1039098/how-to-make-mintty-close-gracefully-on-ctrl-c .
Arg - 5 minute edit on comment. Here's what I wanted to write:
As a workaround, instead of trying to trap the CTRL-C event which is also being propagated to the shell I'd propose turning off the ENABLED_PROCESSED_INPUT on stdin so that CTRL-C is reported as a keyboard input instead of as a signal:
DWORD mode;
HANDLE hstdin = GetStdHandle(STD_INPUT_HANDLE);
GetConsoleMode(hstdin, &mode);
SetConsoleMode(hstdin, mode & ~ENABLE_PROCESSED_INPUT); /* disable CTRL-C processing as a signal */
You could then process keyboard input in your main thread while the rest of the program does its thing in a separate thread and set an event to cleanup when CTRL-C is received.
When you run your program with MSYS bash, do you run the executable directly, or is there a wrapping (bash) shell script?
If so, it may be registering a custom Ctrl-C handler with the trap command (that does a sleep followed by a kill.) If such a thing exists, alter or remove it.
If there is no trap registered, or there is no wrapping script, consider making such a script and adding your own trap to override the default behavior. You can see an example of how to use it here or on bash's man page (in the SHELL BUILTINS section).
Ctrl-C is SIGINT? I thought Ctrl-Z was SIGINT, but Ctrl-C is SIGTERM. Check that.
Do you have a CYGWIN environment setting (in control panel/environment variables)? Try setting CYGWIN=notty and restart open a new MSYS bash shell - does the problem persist?

Resources