How to use gdb stacktrace with run time generated machine code? - c

Question

I've inherited some clever x64 machine code for GNU/Linux that creates a machine code wrapper around a c-function call. I guess that in higher language terms the code might be called a decorator or a closure. The code is functioning well, but with the unfortunate artifact that when the wrapper is called, it gobbles the stack trace in gdb.
From what I have learned from the net gdb uses https://en.wikipedia.org/wiki/DWARF as a guide for separating the stack frames in the stack. This works well for static code, but obviously code generated and called at run time isn't registered in the DWARF framework.
My question is if there is any way to rescue the stack trace in this situation?
Here is some similar c-code that shows the problem.
typedef int (*ftype)(int x);
int wuz(int x) { return x + 7; }
int wbar(int x) { return wuz(x)+5; }
int main(int argc, char **argv)
{
const unsigned char wbarcode[] = {
0x55 , // push %rbp
0x48,0x89,0xe5 , // mov %rsp,%rbp
0x48,0x83,0xec,0x08 , // sub $0x8,%rsp
0x89,0x7d,0xfc , // mov %edi,-0x4(%rbp)
0x8b,0x45,0xfc , // mov -0x4(%rbp),%eax
0x89,0xc7 , // mov %eax,%edi
0x48,0xc7,0xc0,0xf6,0x04,0x40,00, // mov $0x4004f6,%rax
0xff,0xd0, // callq *%rax
0x83,0xc0,0x05 , // add $0x5,%eax
0xc9 , // leaveq
0xc3 // retq
};
int wb = wbar(5);
ftype wf = (ftype)wbarcode;
int fwb = wf(5);
}
Compile it by:
gcc -g -o mcode mcode.c
execstack -s mcode
and run it in gdb by:
$ gdb mcode
(gdb) break wuz
If we disassemble wbar we get something very similar to the byte sequence in wbarcode[]. The only difference is that I changed the calling convention for calling wuz().
(gdb) disas/r wbar
Dump of assembler code for function wbar:
0x0000000000400505 <+0>: 55 push %rbp
0x0000000000400506 <+1>: 48 89 e5 mov %rsp,%rbp
0x0000000000400509 <+4>: 48 83 ec 08 sub $0x8,%rsp
0x000000000040050d <+8>: 89 7d fc mov %edi,-0x4(%rbp)
0x0000000000400510 <+11>: 8b 45 fc mov -0x4(%rbp),%eax
0x0000000000400513 <+14>: 89 c7 mov %eax,%edi
0x0000000000400515 <+16>: e8 dc ff ff ff callq 0x4004f6 <wuz>
0x000000000040051a <+21>: 83 c0 05 add $0x5,%eax
0x000000000040051d <+24>: c9 leaveq
0x000000000040051e <+25>: c3 retq
End of assembler dump.
If we now run the program it will stop twice in wuz(). The first time
through our c-call and we can ask for a stack trace through bt.
Breakpoint 3, wuz (x=5) at mcode.c:2
=> 0x00000000004004fd <wuz+7>: 8b 45 fc mov -0x4(%rbp),%eax
0x0000000000400500 <wuz+10>: 83 c0 07 add $0x7,%eax
0x0000000000400503 <wuz+13>: 5d pop %rbp
0x0000000000400504 <wuz+14>: c3 retq
(gdb) bt
#0 wuz (x=5) at mcode.c:2
#1 0x000000000040051a in wbar (x=5) at mcode.c:3
#2 0x00000000004005b0 in main (argc=1, argv=0x7fffffffe528) at mcode.c:20
This is a normal stack trace showing that we got from main() → wbar() → wuz().
But if we now continue we reach wuz() a second time, and we again
request a stack trace:
(gdb) c
Continuing.
Breakpoint 3, wuz (x=5) at mcode.c:2
=> 0x00000000004004fd <wuz+7>: 8b 45 fc mov -0x4(%rbp),%eax
0x0000000000400500 <wuz+10>: 83 c0 07 add $0x7,%eax
0x0000000000400503 <wuz+13>: 5d pop %rbp
0x0000000000400504 <wuz+14>: c3 retq
(gdb) bt
#0 wuz (x=5) at mcode.c:2
#1 0x00007fffffffe419 in ?? ()
#2 0x0000000500000001 in ?? ()
#3 0x00007fffffffe440 in ?? ()
#4 0x00000000004005c6 in main (argc=0, argv=0xffffffff) at mcode.c:22
Backtrace stopped: frame did not save the PC
Even though we have done the same two hierarchical calls, we get a
stack trace that contains the wrong frames. In my original inherited
wrapper code the situation was even worse, as the the stack trace
ended after 5 frames with the top level having address 0.
So the question is again, is there any extra code that can be added to
wbarcode[] that will cause gdb to output a valid stacktrace? Or is
there any other run time technique that may be used to make gdb
recognize the stack frames?

Answer 1

On some architectures, you can just make the frame have the layout that is expected by gdb's default unwinder for that port. However, this isn't available on all architectures. Reading the x86-64 port (see gdb/amd64-tdep.c, in particular the function amd64_frame_cache_1), I think here gdb wants to know the function bounds, so it can try to analyze the prologue. But, the function bounds come from the (ELF) symbol table, so you're out of luck there.
There's still a way, though. Due to the recent (in gdb terms) rise of JIT compilers, gdb provides three other ways to deal with this problem.
One way is that your program can emit a special ELF object (really any object format that gdb understands, IIRC) in memory, and call a runtime hook to inform gdb of its existence. gdb will read this object, including any debug information it contains. This approach is rather heavy, but gives access to most of gdb's capabilities -- you can specify not just the unwinding but also types, local variables, etc.
A second way is somewhat similar. Your program still calls a special hook. However, you also provide a plugin that is loaded by gdb. This plugin can read symbols and other information from the inferior, but in this case the symbols and unwind information don't have to be in any particular format.
The final way (new in gdb 7.10) is that you can write an unwinder in Python. When working on my JIT unwinder, I chose this approach because it is simple to debug, simple to deploy, reasonably flexible, and does not require any particular changes in the inferior.
These methods are all documented in the gdb manual. In some cases, though, I think the documentation leaves a bit to be desired. You may have to find some example code or dig into the gdb sources to really understand how it's supposed to work.