Is it possible that we can dump all the data structure of a Daemon as a core file (Or any gdb compatible file) for offline analysis/debugging when Daemon is actually not running on a platform(for us Routers). IF so, how can we dump it.
Programming language : C
If you attach gdb to your running process gdb -p <pid> then you can use the gdb command generate-core-file to write out a standard core file.
Later, when the daemon is stopped, you can debug with gdb -c <core-file> <daemon-binary>.
Another option is to use gcore command for a process with gdb. It can be used as gcore <pid> or gcore pgrep <appname> . This will generate coredump of the format core.pid which inturn can be read by gdb and debugged. The command gcore -s enables to stop the process while gathering the core image and resume when it is done(This is the recommended approach). The advantage with gcore is that you can get the memory image of the process and its states without killing the process.
In case of multiprocess application, the coredumps can get overwritten if saved in the same name, so ensure to use sysctl -w kernel.core_pattern=/tmp/core_%p_%e so that the core dump is traceable with process id and the application(executable).
Here is a good link as far as the daemon cores is concerned - http://www.bonsai.com/wiki/howtos/debugging/daemon_core/
Related
I am using macOS Big Sur, v11.5.2, and GDB v10.2
I have seen similar issues before, but with no clear answer.
I am trying to run a program in GDB with input redirection
sudo gdb ./executable
r < input.txt
But when I do this the shell hangs, waiting for manual input instead of reading input from the file. I have gone through the process of creating code signing certificate for GDB, running as sudo, but the same issue occurs.
I also know about MacOS's SIP and how that interferes. This link, coming from this StackOverflow post doesn't seem to offer any solutions. At least any solution that maintains the ability to use input/output redirection in GDB.
If anyone has an answer and can clearly articulate it here, I would greatly appreciate it.
Caveat: This isn't so much a complete solution as some workarounds and suggestions [that are too big to fit into a comment].
There's basically two issues:
Does gdb's r command [under macOS] allow input redirection?
Can gdb be run under sudo?
So, let's start by eliminating sudo. Does the following work:
gdb ./executable
r < input.txt
If not, either the run command's input redirection does not work or there is a bug in the executable.
Side note: For many years, I did not know that run could take I/O redirection [silly me :-)].
What I usually do/did is modify the program to allow input either from stdin or from an argument [just like cat]. Then, I'd do:
gdb ./executable
set args input.txt
run
The other issue is whether gdb can be run under sudo.
In the past, I've tried to debug a setuid executable as an ordinary user [if possible]. This eliminates the complexity of trying to run gdb and the executable as root. Sometimes this involves creating a test directory that mirrors/mimics the necessary file hierarchy [but the files are owned by the non-root user] and using (e.g.) chroot or equivalent
Much of the debug can be done this way without root privileges. For the parts that do require root, I often resort to printf debug messages--YMMV.
I'm not familiar with macOS, so I don't know how [the aformentioned] SIP will respond to the following ...
Another trick I've used is to make the executable [and a copy of gdb] into setuid programs:
cp /usr/bin/gdb /home/me/mygdb
sudo chown root /home/me/mygdb
sudo chmod 6741 /home/me/mygdb
sudo chown root /home/me/myprogram
sudo chmod 6741 /home/me/myprogram
Then, I run the modified programs. This can bypass some of the issues with using sudo directly on the target programs.
Of course, I try to make sure that there is no access to the containing directory by others. And, I remove the files when done testing.
I am trying to figure out semihostong on ARM (STM32042). I can see printf output if I run openocd directly from a command line and connect to it from gdb over TCP. But if I launch openocd from inside gdb, the output goes to some big /dev/null in the sky.
If I launch openocd myself with
$ openocd -f interface/stlink-v2-1.cfg -f target/stm32f0x.cfg
and connect to it from gdb like so
(gdb) target remote localhost:3333
I see printf output in the opeocd terminal. But if I launch openocd from within gdb as this link suggests,
(gdb) target remote | openocd -f interface/stlink-v2-1.cfg -f target/stm32f0x.cfg -c "gdb_port pipe"
I see all the openocd debug messages (e.g., xPSR: 0xc1000000 pc: 0x08001648 msp: 0x20001800, semihosting), but not my printfs.
It turned out slightly more complicated than I wanted. Apparently, it is impossible to launch GDB and OpenOCD in a pipe; they need bidirectional communication over a socket. Because it takes two separate command invocations, and because the configuration for GDB is more than a one-liner, I pulled this into a little repository of its own:
https://github.com/arikrupnik/semihosting-ut
The repo has code that routes stdout to the console, but it does more than that. I've been using it extensively for unit tests on the ARM target for http://www.dish.tc, and have found it very useful. I'm happy to answer questions about it.
I have written a program in C for an embedded device on a Debian-based linux.
One of the devices got a segmentation fault after 8 days running, so it is not a very frequent bug that I can track fast with gdb.
A few years ago I know that I used gdb (with gdbserver, I guess) running the program detached from the shell so I could leave the device running and check every day if something bad had happened, but I don't remember how I did that!
I have tried with gdbserver, connecting to it with gdb, but it stops debugging when I close the connection.
Do you know how to achieve this?
You'll have to detach, not quit gdb like that.
Since you're only in for post-mortems, anyway, I'd however recommend something completely different:
Enable core dumping; as root, run
> sudo -s ##become root
$ echo "* soft core unlimited" >> /etc/security/limits.conf
$ su -l <user that is running the crashing program>
$ ulimit -c unlimited
$ program
##wait for crash
(that lifts any restriction, including the default 0B restriction, on the maximum core dump file size).
Then, after a crash, find the core.* coredump file, and open it with gdb. Voila, the state of the program when crashing is restored in gdb and you can do pretty much anything that you could do with a gdb attached at crash time.
I checked the ulimit so it shouldn't be a problem
ulimit -c
unlimited
What else can I check?
If you are getting no coredump, there are a few possibilities.
ulimit : are you sure that the process has core unlimited? Your shell may, but what started the process?
Directory permissions : The coredump would be in the cwd. If the process does not have write permission to the cwd then it cannot dump the core.
SELinux can apparently cause issues. Check /var/log/messages for warnings if you are on a Linux box and have it enabled.
Does your process have some traphandler or a wrapper that is preventing a coredump? (Note that SIGSEGV cannot be trapped but a wrapper process might catch it)
My guess is #2, though you've not given us much in the way of details to work with.
I'm helping to write a program right now, and on windows, the program works fine. On the mac, after changing the framework directory to the only suitable location, I'm getting this error:
run:
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007fff92309bb2, pid=12438, tid=44807
JRE version: 7.0_10-b18
Java VM: Java HotSpot(TM) 64-Bit Server VM (23.6-b04 mixed mode bsd-amd64 compressed oops)
Problematic frame:
C [CoreFoundation+0x1bb2] CFDictionaryGetValue+0x12
Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
An error report file with more information is saved as:
/Downloads/BiscuitAnimator-Mac-master/hs_err_pid12438.log
If you would like to submit a bug report, please visit:
http://bugreport.sun.com/bugreport/crash.jsp
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.
Java Result: 134
BUILD SUCCESSFUL (total time: 4 seconds)
How do I use the ulimit -c unlimited command/set up core dumping in NetBeans?
If you have some bug (unexpected error scenarios) in your program and it is resulting in crashing your program, you should be able to debug why the program has crashed.
For this purpose the Operating system will copy process stack into a file called core dump file.
ulimit , is a system command.
ulimit -c unlimited, command used to set the core dump file size to unlimited.
Instead of unlimited you can specify the maximum limit of the core dump file size also.
for more information type
man ulimit