I set a lot of breakpoints in lldb for a C language based application I installed on my MacOS. The breakpoints were mostly set in the same function in the application. However, the next day that I went back to the application to continue working on it, and I started setting breakpoints again in the same function, a problem arose in that the break didn't occur inside the application function, but rather in one of the underlying libraries of the application and it keeps doing this over and over again everytime I try to break in the function (i.e. it's stopping in the underlying library) and I'm not able to reach the desired function by stepping (every time I step, it just steps forward in the underlying library).
Update:
The function I am setting the breakpoint in is called from within a signal handler. For example when I send a SIGINT signal, the signal handler calls some functions to cleanup in the application, and I am setting the breakpoint on one of those functions that cleanup. Sometimes, LLDB stops in the function that I set the breakpoint in (with stop reason = breakpoint 1.1) , sometimes it stops in the underlying/included event handling library with stop reason = signal SIGSTOP and, if the latter, if I press "c" (to continue onto the breakpoint in the application hopefully and out of the event handling library), only sometimes does it let me continue onto the desired breakpoint, othertimes it just says "Process 41524 resuming" and I can never get to the desired breakpoint.
Ah, then I don't think the problem was with breakpoints, but with whether your signal handler was actually getting called.
Most debuggers have some way to control what happens when a signal is received. In lldb this is done through the process handle command. For instance:
(lldb) process handle SIGSTOP
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGSTOP false true true
That means lldb will stop when your process is given a SIGSTOP, and will notify you about the SIGSTOP, but will NOT pass the SIGSTOP on to the program you are debugging (and thus your handler will not get called for SIGSTOP.) process handle with no arguments will give you the list of behaviors for all signals.
We don't pass SIGSTOP by default because it is used by the debugger for its own purposes, and so you might get calls to your handler that didn't come from "real" SIGSTOP's. The same is true, for the same reason, of SIGINT:
(lldb) process handle SIGINT
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGINT false true true
You can easily change this behavior, for instance for SIGINT:
(lldb) process handle SIGINT -p true
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGINT true true true
Then the debugger will pass the SIGINT on to the process, and it will stop in your handler.
As mentioned in the troubleshooting guide, adding a target.inline-breakpoint-strategy setting to the .lldbinit file seemed to fix the problem
"settings set target.inline-breakpoint-strategy always" >> ~/.lldbinit
Update: problem not fixed, see OP, so this is not a good solution (AFAIK)
Related
The following code is from an example in the GNU manual about signal handling
...
if (signal(SIGINT, termination_handler) == SIG_IGN)
signal(SIGINT, SIG_IGN);
...
The manual says,
Note that if a given signal was previously set to be ignored, this code avoids altering that setting. This is because non-job-control shells often ignore certain signals when starting children, and it is important for the children to respect this.
I don't understand why ignored signals should be kept as ignored when non-job-control shells make some signals ignored before starting my process. (What does it mean by "non-job-control shells"?)
I found a similar part in my book, 'The Linux System Programming' written by Robert Love. The book says,
When a process is first executed, all signals are set to their default actions, unless the parent process (the one executing the new process) is ignoring them; in this case, the newly created process will also ignore those signals. Put another way, any signal caught by the parent is reset to the default action in the new process, and all other signals remain the same. This makes sense because a freshly executed process does not share the address space of its parent, and thus any registered signal handlers may not exist.
This behavior on process execution has one notable use: when the shell executes a process "in the background" (or when another background process executes another process), the newly executed process should ignore the interrupt and quit characters.
Thus, before a shell executes a background process, it should set SIGINT and SIGQUIT to SIG_IGN. It is therefore common for programs that handle these signals to first check to make sure they are not ignored.
I tried registering a signal handler for SIGINT (without checking ignored or not) and running it in the background (just to see what happens if I do what they're saying not to do).
$ ./my_program &
But nothing special happened.
What could be happen when I register signal handlers for previously ignored signals?
1st foray into using pthreads to create a multithreaded aplication
I'm trying to debug with gdb but getting some strange unexpected behaviour
Trying to ascertain whether its me or gdb at fault
Scenario:
Main thread creates a child thread.
I place a breakpoint on a line in the child thread fn
gdb stops on that breakpoint no problem
I confirm there are now 2 threads with info threads
I also check that the 2nd thread is starred, i.e. it is the current thread for gdbs purposes
Here is the problem, when I now hit n to step through to the next line in the thread fn, the parent thread (thread 1) simply resumes and completes and gdb exits.
Is this the correct behaviour?
How can I step through the thread fn code that is being executed in the 2nd thread line by line with gdb?
In other words, even though thread 2 is confirmed as the current thread by gdb, when I hit n, it seems to be the equivalent of hitting c in the parent thread, i.e. the parent thread (thread 1) just resumes execution, completes and exits.
At a loss as to how to debug multiple threads with gdb behaving as it is currently
I am using gdb from within emacs25, i.e. M-x gud-gdb
What GDB does here depends on your settings, and also your system (some vendors patch this area).
Normally, in all-stop mode, when the inferior stops, GDB stops all the threads. This gives you the behavior that you'd "expect" -- you can switch freely between threads and see what is going on in each one.
When the inferior continues, including via next or step, GDB lets all threads run. So, if your second thread doesn't interact with your first thread in any way (no locks, etc), you may well see it exit.
However, you can control this using set scheduler-locking. Setting this to on will make it so that only the current thread can be resumed. And, setting it to step will make it so that only the current thread can be resumed by step and next, but will let all threads run freely on continue and the like.
The default mode here is replay, which is basically off, except when using record-and-replay mode. However, the Fedora GDB is built with the default as step; I am not sure if other distros followed this, but you may want to check.
Yes, this is correct behaviour of gdb. You are only debugging currently active thread, other threads are executing normally behind the scenes. Think about it, how else would you move other threds?
But your code has a bug. Your parent thread should not exit before child thread is done. The best way to do this is to join child thread in the main thread before exiting.
I frequently work with PostgreSQL for debugging, and it uses SIGINT internally for some of its inter-backend signalling.
As a result when running certain backends under gdb execution tends to get interrupted a lot. One can use the signal command to make sure SIGINT is passed to the program and that it is not captured by gdb... but then gdb doesn't respond to control-C on the command line, since that sends SIGINT.
If you run:
handle SIGINT noprint nostop pass
gdb will complain
SIGINT is used by the debugger.
Are you sure you want to change it? (y or n) y
Is there any way to get gdb to use a different interrupt signal? Or any alternative method that'd let me have gdb ignore SIGINT?
(This isn't an issue for most PostgreSQL backend debugging, but it's a pain with background workers and autovacuum).
Readers who end up on this page (as I did) with a slightly different variation of this problem, would perhaps be more interested in this question:
Debugging a segmentation fault when I do ctrl c
... and its answer, which is:
send SIGINT from inside gdb itself:
(gdb) signal 2
(Normally I would post the link as a simple comment under the OP's question on this page, but since there are already 7 comments, comments are being hidden/buried.)
If you read all the details of the OP's question here, then it is obvious that my answer is not correct for OP.
However, my answer is correct for many situations that could be described by the same title: "Debugging a program that uses SIGINT with gdb"
On UNIX-like systems, you can distinguish a tty-initiated SIGINT from one sent by kill by looking at the si_pid element in the siginfo struct. If the pid is 0, it came from a tty.
So you could do something like this:
catch signal SIGINT
commands
if $_siginfo._sifields._kill.si_pid == 0
print "Received SIGINT from tty"
else
printf "Received SIGINT from %d; continuing\n", $_siginfo._sifields._kill.si_pid
signal SIGINT
end
end
This part of gdb is a bit tricky, both due to its history and also due to the various modes of operation it supports.
One might think that running gdb in a separate terminal and only using attach would help it do the right thing, but I don't think it is that easy.
One way forward might be to only use async execution when debugging, and then use a command to interrupt the inferior. Something like:
(gdb) attach 5555
... attaches
(gdb) continue &
... lots of stuff happens
(gdb) interrupt -a
Depending on your version of gdb you might need to set target-async for this to work.
My UI is in a DLL. Right now, both the DLL and the EXE that uses it are compiled as console programs so I can use stdout and stderr for debugging and error reporting during development. One of the things is that I have an uninit() function that makes sure the DLL isn't leaking memory.
As a result, I have a control handler set up by the DLL such that CTRL_LOGOFF_EVENT and CTRL_SHUTDOWN_EVENT simulate the user clicking the Quit option from the File menu: it does PostQuitMessage(0), with the cleanup code happening after the message pump returns.
I know that normally CTRL_SHUTDOWN_EVENT cannot be ignored, and that the program will terminate after the handler routine returns, regardless of what it returns. But according to MSDN,
Note that a third-party library or DLL can install a console control handler for your application. If it does, this handler overrides the default handler, and can cause the application to exit when the user logs off.
If I am reading this correctly, this says that a control handler that is installed by a DLL overrides the handler that causes my program to quit when the handler function returns. Am I wrong about that? My DLL's handler function simply returns TRUE, which I assume will further stop any other defaults from running given the blurb above.
Why? I'm noticing weird behavior:
On Windows Vista, the program closes regardless of what I do. In this case, I'm wondering if the blurb is wrong and that the handler that terminates the process is still running. This happens regardless of whether I have called ShutdownBlockReasonCreate().
On Windows 7, however, it seems that my program's main window gets a WM_QUERYENDSESSION, and Windows responds to it accordingly. That means that if I say "no, don't quit yet (don't call PostQuitMessage(0))" in my Quit function, Windows pops up the "an application is preventing shutdown" screen saying my main window is preventing shutdown. In that case, the blurb above appears to be correct, as the program is not quitting on return from the console handler (if it's even being called!).
If I instead say "yes, call PostQuitMessage(0), the program quits normally. However, I lose the debugging output on stdout and stderr, so I can't tell if it really is quitting normally or not. Invoking my program as
new.exe > out.txt 2> err.txt
on cmd.exe produces two empty files; I don't know why the output isn't saving on system shutdown (and Googling doesn't turn up any information).
So can someone help clear up my confusion so I can implement this (including ShutdownBlockReasonCreate()) properly? Thanks.
When you return TRUE from the handler you registered, Windows immediately terminates the process. When you return FALSE, the previous handler gets called. Ultimately that will be the default handler, it immediately terminates the process.
So what you have to do is not return and block until you are happy. That requires synching with the thread that's pumping the message loop. You'd use an event, the pumping thread can call SetEvent() after its message loop and your handler can call WaitForSingleEvent() to block after it called PostQuitMessage().
It is however a threading race, your UI thread was probably started by main() and the CRT is going to terminate the program when main() returns. Which one will get there first is a unpredictable.
Having the feeling you are doing something wrong? Well, you are. A console window just isn't a very good way to display debug output. Not sure why you are doing this but I know your tool-chain is unusual, I can never get any of your code snippets to compile and run. The proper way is OutputDebugString(). That function talks to your debugger and gets it to display text. Even if your debugger isn't capable of displaying such text, you can still fallback to SysInternals' DebugView utility.
Your are probably using printf() and won't enjoy fixing all your debug statements, simply write your own version of that links ahead of the CRT, use vprintf() and OutputDebugStringA().
I have this multithreaded application using pthreads. My threads actually wait for signals using sigwait. Actually, I want to debug my application, see which thread receives which signal at what time and then debug it. Is there any method, I can do this. If I directly run my program, then signals are generated rapidly and handled by my handler threads. I want to see which handler wakes up from the sigwait call and processes the signal and all.
The handy strace utility can print out a huge amount of useful information regarding system calls and signals. It would be useful to log timing information or collect statistics regarding the performance of signal usage.
If instead you are interested in getting a breakpoint inside of an event triggered by a specific signal, you could consider stashing enough relevant information to identify the event in a variable and setting a conditional breakpoint.
One of the things you may try with gdb is set breakpoints by thread (e.g. just after return from sigwait), so you know which thread wakes up:
break file.c thread [thread_nr]
Don't forget to tell gdb to pass signals to your program e.g.:
handle SIGINT pass
You may want to put all of this into your .gdbinit file to save yourself a lot of typing.
Steven Schlansker is definitely right: if that happens to significantly change timing patterns of your program (so you can see that your program behaves completely different under debugger, than "in the wild") then strace and logging is your last hope.
I hope that helps.