GDB normally stops all threads if a breakpoint is reached (or Ctrl+C is pressed in the GDB shell). I'm aware that commands like scheduler-locking and schedule-multiple exists, but I see no possibility to let a defined thread run in the background while another is debugged.
You can use set target-async on to enable asynchronous mode, if your target supports it. Then, you can specify background execution with commands. For example,
continue&
can be used to run a single thread, and
interrupt [-a]
to suspend execution of a single thread, or the whole program.
In conjunction with non-stop mode, you can examine a single thread while others continue to run in the background:
# If using the CLI, pagination breaks non-stop.
set pagination off
# Finally, turn it on!
set non-stop on
# Before debugging is started!
Related
I can't attach to the C application process because it ends only milliseconds after being called. And the caller is an exe that I don't have the source code.
It's an annoying method but you can set a bbreakpoint in your sub-program, use CreateProcess (or one of its variants) to start the program with CREATE_SUSPENDED in the dwCreationFlags argument. Next attach to the suspended process, and finally resume the suspended program's main thread using the Debug -> Windows -> Threads window. The breakpont is necessary because otherwise the program will simply run to completion once the thread is resumed.
Or use trace debugging (although if it's not a console program you'll probably need to write to a log file).
I set a lot of breakpoints in lldb for a C language based application I installed on my MacOS. The breakpoints were mostly set in the same function in the application. However, the next day that I went back to the application to continue working on it, and I started setting breakpoints again in the same function, a problem arose in that the break didn't occur inside the application function, but rather in one of the underlying libraries of the application and it keeps doing this over and over again everytime I try to break in the function (i.e. it's stopping in the underlying library) and I'm not able to reach the desired function by stepping (every time I step, it just steps forward in the underlying library).
Update:
The function I am setting the breakpoint in is called from within a signal handler. For example when I send a SIGINT signal, the signal handler calls some functions to cleanup in the application, and I am setting the breakpoint on one of those functions that cleanup. Sometimes, LLDB stops in the function that I set the breakpoint in (with stop reason = breakpoint 1.1) , sometimes it stops in the underlying/included event handling library with stop reason = signal SIGSTOP and, if the latter, if I press "c" (to continue onto the breakpoint in the application hopefully and out of the event handling library), only sometimes does it let me continue onto the desired breakpoint, othertimes it just says "Process 41524 resuming" and I can never get to the desired breakpoint.
Ah, then I don't think the problem was with breakpoints, but with whether your signal handler was actually getting called.
Most debuggers have some way to control what happens when a signal is received. In lldb this is done through the process handle command. For instance:
(lldb) process handle SIGSTOP
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGSTOP false true true
That means lldb will stop when your process is given a SIGSTOP, and will notify you about the SIGSTOP, but will NOT pass the SIGSTOP on to the program you are debugging (and thus your handler will not get called for SIGSTOP.) process handle with no arguments will give you the list of behaviors for all signals.
We don't pass SIGSTOP by default because it is used by the debugger for its own purposes, and so you might get calls to your handler that didn't come from "real" SIGSTOP's. The same is true, for the same reason, of SIGINT:
(lldb) process handle SIGINT
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGINT false true true
You can easily change this behavior, for instance for SIGINT:
(lldb) process handle SIGINT -p true
NAME PASS STOP NOTIFY
=========== ===== ===== ======
SIGINT true true true
Then the debugger will pass the SIGINT on to the process, and it will stop in your handler.
As mentioned in the troubleshooting guide, adding a target.inline-breakpoint-strategy setting to the .lldbinit file seemed to fix the problem
"settings set target.inline-breakpoint-strategy always" >> ~/.lldbinit
Update: problem not fixed, see OP, so this is not a good solution (AFAIK)
1st foray into using pthreads to create a multithreaded aplication
I'm trying to debug with gdb but getting some strange unexpected behaviour
Trying to ascertain whether its me or gdb at fault
Scenario:
Main thread creates a child thread.
I place a breakpoint on a line in the child thread fn
gdb stops on that breakpoint no problem
I confirm there are now 2 threads with info threads
I also check that the 2nd thread is starred, i.e. it is the current thread for gdbs purposes
Here is the problem, when I now hit n to step through to the next line in the thread fn, the parent thread (thread 1) simply resumes and completes and gdb exits.
Is this the correct behaviour?
How can I step through the thread fn code that is being executed in the 2nd thread line by line with gdb?
In other words, even though thread 2 is confirmed as the current thread by gdb, when I hit n, it seems to be the equivalent of hitting c in the parent thread, i.e. the parent thread (thread 1) just resumes execution, completes and exits.
At a loss as to how to debug multiple threads with gdb behaving as it is currently
I am using gdb from within emacs25, i.e. M-x gud-gdb
What GDB does here depends on your settings, and also your system (some vendors patch this area).
Normally, in all-stop mode, when the inferior stops, GDB stops all the threads. This gives you the behavior that you'd "expect" -- you can switch freely between threads and see what is going on in each one.
When the inferior continues, including via next or step, GDB lets all threads run. So, if your second thread doesn't interact with your first thread in any way (no locks, etc), you may well see it exit.
However, you can control this using set scheduler-locking. Setting this to on will make it so that only the current thread can be resumed. And, setting it to step will make it so that only the current thread can be resumed by step and next, but will let all threads run freely on continue and the like.
The default mode here is replay, which is basically off, except when using record-and-replay mode. However, the Fedora GDB is built with the default as step; I am not sure if other distros followed this, but you may want to check.
Yes, this is correct behaviour of gdb. You are only debugging currently active thread, other threads are executing normally behind the scenes. Think about it, how else would you move other threds?
But your code has a bug. Your parent thread should not exit before child thread is done. The best way to do this is to join child thread in the main thread before exiting.
I have this multithreaded application using pthreads. My threads actually wait for signals using sigwait. Actually, I want to debug my application, see which thread receives which signal at what time and then debug it. Is there any method, I can do this. If I directly run my program, then signals are generated rapidly and handled by my handler threads. I want to see which handler wakes up from the sigwait call and processes the signal and all.
The handy strace utility can print out a huge amount of useful information regarding system calls and signals. It would be useful to log timing information or collect statistics regarding the performance of signal usage.
If instead you are interested in getting a breakpoint inside of an event triggered by a specific signal, you could consider stashing enough relevant information to identify the event in a variable and setting a conditional breakpoint.
One of the things you may try with gdb is set breakpoints by thread (e.g. just after return from sigwait), so you know which thread wakes up:
break file.c thread [thread_nr]
Don't forget to tell gdb to pass signals to your program e.g.:
handle SIGINT pass
You may want to put all of this into your .gdbinit file to save yourself a lot of typing.
Steven Schlansker is definitely right: if that happens to significantly change timing patterns of your program (so you can see that your program behaves completely different under debugger, than "in the wild") then strace and logging is your last hope.
I hope that helps.
I have a console-mode Windows application (ported from Unix) that was originally designed to do a clean exit when it received ^C (Unix SIGINT). A clean exit in this case involves waiting, potentially quite a long time, for remote network connections to close down. (I know this is not the normal behavior of ^C but I am not in a position to change it.) The program is single-threaded.
I can trap ^C with either signal(SIGINT) (as under Unix) or with SetConsoleCtrlHandler. Either works correctly when the program is run under CMD.EXE. However, if I use the "bash" shell that comes with MSYS (I am using the MinGW environment to build the program, as this allows me to reuse the Unix makefiles) then the program is forcibly terminated some random, short time (less than 100 milliseconds) after the ^C. This is unacceptable, since as I mentioned, the program needs to wait for remote network connections to close down.
It is very likely that people will want to run this program under MSYS bash. Also, this effect breaks the test suite. I have not been able to find any way to work around the problem either from within the program (ideal) or by settings on the shell (acceptable). Can anyone recommend anything?
I had the exact same problem - I had written a program with a SIGINT/SIGTERM handler. That handler did clean-up work which sometimes took awhile. When I ran the program from within msys bash, ctrl-c would cause my SIGINT handler to fire, but it would not finish - the program was terminated ("from the outside", as it were) before it could complete its clean-up work.
Building on phs's answer, and this answer to a similar question: https://stackoverflow.com/a/23678996/2494650, I came up with the following solution. It's insanely simple, and it might have some side-effects that I've yet to discover, but it fixed the problem for me.
Create a ~/.bashrc file with the following line:
trap '' SIGINT
That's it. This traps the sigint signal and prevents msys bash from terminating your program "from the outside". However, it somehow still lets the SIGINT signal through to your program, allowing it to do its graceful cleanup/shutdown. I can't tell you exactly why it works this way, but it does - at least for me.
Good luck!
This could be due to the infamous mintty "Input/Output interaction with alien programs" problem (aka mintty issue #56). In this case it is manifesting as Ctrl-C abruptly killing the program rather than being passed down to the program as a signal to be caught and handled. Evidence for this theory is based on zwol's extensive explanation: "console-mode Windows application", "[application is] designed to do a clean exit when it received ^C", "[application] works correctly when the program is run under CMD.EXE" but "[when using the terminal] that comes with MSYS [...] program is forcibly terminated" (at the time of writing (2018) MSYS defaults to using mintty as its terminal).
Unfortunately mintty isn't a full Windows console replacement and various behaviours expected by "native" Windows programs are not implemented. However, you might have some joy wrapping such native programs in winpty when running them within mintty...
Other questions also describe this behaviour: see https://superuser.com/questions/606201/how-to-politely-kill-windows-process-from-cygwin and https://superuser.com/questions/1039098/how-to-make-mintty-close-gracefully-on-ctrl-c .
Arg - 5 minute edit on comment. Here's what I wanted to write:
As a workaround, instead of trying to trap the CTRL-C event which is also being propagated to the shell I'd propose turning off the ENABLED_PROCESSED_INPUT on stdin so that CTRL-C is reported as a keyboard input instead of as a signal:
DWORD mode;
HANDLE hstdin = GetStdHandle(STD_INPUT_HANDLE);
GetConsoleMode(hstdin, &mode);
SetConsoleMode(hstdin, mode & ~ENABLE_PROCESSED_INPUT); /* disable CTRL-C processing as a signal */
You could then process keyboard input in your main thread while the rest of the program does its thing in a separate thread and set an event to cleanup when CTRL-C is received.
When you run your program with MSYS bash, do you run the executable directly, or is there a wrapping (bash) shell script?
If so, it may be registering a custom Ctrl-C handler with the trap command (that does a sleep followed by a kill.) If such a thing exists, alter or remove it.
If there is no trap registered, or there is no wrapping script, consider making such a script and adding your own trap to override the default behavior. You can see an example of how to use it here or on bash's man page (in the SHELL BUILTINS section).
Ctrl-C is SIGINT? I thought Ctrl-Z was SIGINT, but Ctrl-C is SIGTERM. Check that.
Do you have a CYGWIN environment setting (in control panel/environment variables)? Try setting CYGWIN=notty and restart open a new MSYS bash shell - does the problem persist?