Live monitoring variables of compiled (c) code - c

I'm running a headless aarch64 based embedded linux system. on this system there is a main program running which is a compiled bit o C code. However I would like to be able to monitor this program while it continues doing its thing.
During my education I worked with the XCP-protocol to monitor systems and adjust them on the fly. But that is hands on, I would like to automate it with python or javascript so I can handle the data for other purposes.
So in short, there's a compiled bit of C-code running on a Linux system and I want to be able to see the value of its variables with something like javascript or python.
I did some testing with GDB but that seems to pause the execution of the program which can't happen, maybe only for a quick bit when its starting but not after.
I've found this previous post:
How can you debug a process using gdb without pausing it?
However when i type continue after setting all of the related settings that I could find, it doesnt let me look up the value of a variable anymore using a command like p .

Related

Is there a "TUI" mode for standalone lldb?

Since gdb is getting onerous to work with on a Mac these days (at least I feel like I am fighting uphill against Apple), I've started to play around with lldb.
Is there an equivalent mode to gdb -tui that shows a nice, persistent view of the source and where you are in it when running lldb standalone from the command line? Obviously, in Xcode, there is such a display, but I deploy most of my code to Linux boxes eventually and would prefer to use the same development environment on both platforms (i.e., vim, Makefiles, autotools, etc.).
In top of tree svn source lldb there's a very new command line GUI mode that's been added by Greg Clayton. It's pretty neat with separate windows for the source, local variables, stack, threads, registers, etc. No console window yet. It's pretty cool. See http://lldb.llvm.org/build.html for how to build lldb from source if you'd like to check it out. To enter the GUI mode, just type gui from the lldb command line when you're debugging.
UPDATE: this gui feature is included in the lldb that ships in Xcode 6 and later. It's a nights-and-weekends side project that isn't seeing a lot of active development right now but give it a try.
Sadly, no.
However, I did came across this:
https://github.com/snarez/voltron
If you are familiar with screen/tmux it works pretty good.
Type gui in lldb's prompt after you're at an interesting breakpoint. It has a small tutorial the first time it's opened and built-in help.
I'm having good luck with a free GUI facade from Affinic. It's not perfect (or, at least, my understanding of it is not perfect). I must occasionally restart it and sometimes it gets unsynchronized from particularly gnarly / fancy C++ code, e.g., code that does a lot of implicit stuff in constructors. But overall it has saved me a lot of time over the old-fashioned way at the command line.
I have not been able to get the ncurses GUI mode in the LLDB CLI to work, and I have not tried Voltron.

Profiling C program in XCode

I'm using XCode to develop a C command line utility and I'm attempting to using XCode's profiling capabilities to track program allocations and possibly memory leaks. I can attach an allocation tracker utility just fine to the program itself and it works, the problem is I cannot interact with the program from this point and it's just stuck in its initialized and waiting state (the program is definitely running in the background somewhere I just cannot get to it). I've tried tweaking various settings to no avail, any ideas would be greatly appreciated, thanks.
If you launch Instruments outside of Xcode, you should be able to use the pull-down list above 'Target' and 'Attach to Process' to profile any already-running program. So one option — given that the way you describe your program makes it sound interactive — is to launch your utility in a terminal then to attach Instruments to it.
E.g. vi isn't symbolicated but running Instruments against it has just revealed that when in insert mode, it spends about 14% of its time in write and 4.5% in strcmp (albeit that with something like vi the processing is so minuscule that there's bound to be sampling error in there).

Following, and saving, the flow of code

I was wondering if there is any way of compiling a program (my own program, or an open source program), with which I can follow the flow of that program when I execute it. Ideally, I would like to output the specific methods which the program goes through when it executes. Each time it calls a specific method, I would like to output that it has done so, which I would like to save to a file for later analysis.
For example, I am trying to better understand the flow within KVM (an open source hypervisor) but there are obviously many lines of code, and would be impossible for me to know where the code goes unless I dedicated possibly weeks to finding out.
The code I am looking at is written mostly in C, but also uses other languages. Any ideas please?
KVM is a subsystem of Linux kernel, so you should use ftrace (http://lwn.net/Articles/322666/) for tracing kernel-space code.

What kind of bug hangs Linux?

I'm trying to debug a simple cross-platform commandline program (a C parser, itself written in C) and running into something strange.
On Windows, when I run it on a small dataset (the source code of glib) it completes successfully, and when I run it on a large dataset (the source code of the Linux kernel) it exits with an out of memory error. I'm not sure whether the latter is a bug in my code or just a consequence of not just having optimized the memory consumption yet, so I've been trying to run it on Linux so I can get some feedback from valgrind.
On Linux (Ubuntu 11.04 x64 in VirtualBox), when I run my program on a small dataset it completes successfully, and when I run it on a large dataset Linux locks up hard enough I have to reset the entire virtual box (mouse pointer still moves but other than that it's completely unresponsive; Windows task manager says the virtual box is using one hundred percent of a CPU core but not allocating memory).
I wouldn't have expected a bug in my code to crash Linux unless I was writing something like a device driver, and when I try simple test cases that allocate too much memory, go into an infinite loop or both, Linux can handle them just fine. What kind of bug should I be looking for, or what am I missing?
On Linux (Ubuntu 11.04 x64 in VirtualBox)
Probably you haven't reserved enough memory to your virtual machine.
This is most likely an infinite loop (easily done in a parser), which could easily take up 100% cpu or 100% ram.
Attach a debugger!
e.g. gdb
http://www.gnu.org/s/gdb/
gdb comes with gcc on Ubuntu etc...
Here's a how-to: http://www.unknownroad.com/rtfm/gdbtut/gdbtoc.html
EDIT: just saw you already tried gdb. So, try running strace on it, it might give you a hint.
Further to that, try adding log messages to see how far the program gets (primitive, but it'll work eventually!)

How can a C program produce a core dump of itself without terminating?

I want a C program to produce a core dump under certain circumstances. This is a program that runs in a production environment and isn't easily stopped and restarted to adjust other kinds of debugging code. Also, since it's in a production environment, I don't want to call abort(). The issues under investigation aren't easily replicated in a non-production environment. What I'd like is for the program, when it detects certain issues, to produce a core dump on its own, preferably with enough information to rename the file, and then continue.
void create_dump(void)
{
if(!fork()) {
// Crash the app in your favorite way here
*((void*)0) = 42;
}
}
Fork the process then crash the child - it'll give you a snapshot whenever you want
Another way might be to use the Google Coredumper library. This creates a similar result to the fork+abort technique but plays nicer with multithreaded apps (suspends all threads for a little while before forking so that they don't make a mess in the child).
Example:
#include <google/coredumper.h>
...
WriteCoreDump('core.myprogram');
/* Keep going, we generated a core file,
* but we didn't crash.
*/
Sun describes how to get a core file on Solaris, HP-UX, Redhat, and Windows here.
Solaris has the gcore program. HP-UX may have it.
Otherwise use gdb and its gcore commmand.
Windows has win-dbg-root\tlist.exe and win-dbg-root\adplus.vbs
Do you really want a core, or just a stacktrace ?
If all you want is a stacktrace you could take a look at the opensource here and try and integrate the code from there, or maybe just calling it from the command line is enough.
I believe some code in the gdb project might also be useful.
Another think you might want to do is to use gdb to attach to a running process.
$ gdb /path/to/exec 1234 # 1234 is the pid of the running process
The source code to produce a core dump is in 'gcore', which is part of the gdb package.
Also, the Sun has gcore.
Also, you have to have a separate process running the core dump, as the current process must be suspended. You'll find the details in the gcore source, or you can just run your platform's gcore with your process as the target.

Resources