I'm having some trouble stepping over in GDB. I've built an example program from the ffmpeg library with debug symbols on and stripping off. Although I configured the ffmpeg library to static and explicitly disabled shared, it looks like the program I'm debugging is linking dynamically, since its file size is only 99kB. I don't know that this is the issue but thought to mention it.
After I set and hit a breakpoint in av_seek_frame, I use the 'next' command to step over. However, this steps into the first function within av_seek_frame(), as you can see below. Furthermore if a do a second 'next', the backtrace looses track of where it is. Am I set up wrong? How can I step over? I should note I double checked that 'set step-mode off' is off as the default (as I believe this will break at the first piece of code without debug info.)
Breakpoint 1, av_seek_frame (s=0x16429000, stream_index=0, timestamp=29727438, flags=0) at l
(gdb) list
1648
1649 return 0;
1650 }
1651
1652 int av_seek_frame(AVFormatContext *s, int stream_index, int64_t timestamp, int flags
1653 {
1654 int ret;
1655 AVStream *st;
1656
1657 ff_read_frame_flush(s);
(gdb) next
ff_read_frame_flush (s=0x16429000) at libavformat/utils.c:1248
(gdb) list
1243
1244 /**
1245 * Flush the frame reader.
1246 **/
1247 void ff_read_frame_flush(AVFormatContext *s)
1248 {
1249 AVStream *st;
1250 int i, j;
1251
1252 flush_packet_queue(s);
(gdb) next
ff_read_frame_flush (s=0x16429000) at libavformat/utils.c:1252
(gdb) where
#0 ff_read_frame_flush (s=0x16429000) at libavformat/utils.c:1252
#1 0x00000000 in ?? ()
If you're not sure whether or not your binary is statically linked, you can check it with ldd and see a message like this:
% ldd ffmpeg
not a dynamic executable
Next, make sure that you give gdb the full path to the executable so that you're not accidentally picking up a binary installed elsewhere on the system that happens to be in your PATH.
Most likely you're loading the wrong binary. Even without using --disable-stripping and --disable-optimizations I can use gdb fine using both step and next commands. You don't need to use --disable-stripping because inside gdb you can use the ffmpeg_g binary (or if you happen to run the ffmpeg binary you can load the symbols from it using file ffmpeg_g).
For debugging purposes it is nice to use the --disable-optimizations so that you don't get value optimized out when inspecting variables, but strictly speaking you don't need to use the option to get emacs/gdb to behave... I have no problems stepping through the code when optimizations are used.
There is one things to keep in mind, though, when setting breakpoints with gud/gdb inside of Emacs that might lead to confusion: the gud-break command uses only the base part of the filename for setting breakpoints, not the absolute path to it, which in the case of ffmpeg means that if, for example, you set a breakpoint in utils.c it might not work correctly depending on the value of the source code search paths that you have set in gdb because ffmpeg has multiple files named utils.c in different paths (in fact, there are a total of 5 utils.c files, one in each of the lib* subdirectories) . By default the search path is set to $cdir:$cwd, but if you have it set to something like /path/to/ffmpeg:$cdir:$cwd and you try to set a breakpoint in the utils.c of libavformat, it might find the one in libavutil-- in which case if you're lucky it will complain that the line where you want to set a breakpoint does not exist (because the one in libavutil is shorter), or it might set a breakpoint on the line you want, but in the wrong utils.c.
This issue with gud/gdb should be considered a bug. When I get a moment I'll submit a patch for gud-break/gud-format-command to fix the problem.
Related
I am looking for help with GDB to reverse engineer shared library written in C that is preloaded in /etc/ld.so.preload.
Current library hooks accept() call if source port is correct it returns reverse shell back to user.
Strings command doesn't give out source port, so my target is to try to find it within GDB.
Program consist of two files headers.h where I have my definitions and variable #define SECRET_PORT 11111
source.c contains accept hook with reverse shell.
My problem is I cannot figure out a way how to retrieve PORT within GDB - I can load mylib.so within gdb and run: info functions to see whats inside - I can see accept function but when I try to disass accept I only get instructions that I barely can understand.
Problem when I run mylib it gives out SIGSEGV (maybe thats the reason I cannot see variables) there is no main function where to set break and if I do set it on function accept is still gives SIGSEGV error.
I tested with starti instead of run then I got Program stopped 0xSOMEADRESGOESHERE in deregister_tm_clones() I don't even know if this is correct way to test .so file. maybe there are some oser switches.
Im thinking I need to find a way how to set BP in HTONS() checking function where if statement compares source port and extract values from there but so far no luck.
p.s. when mylib is loaded in gdb there is message No debugging symbols found. So I cannot run like list accept or anything like that to view a source.
Compilation code gcc -Wall -shared -fPIC mylib.c -o mylib.so -ldl
Im thinking I need to find a way how to set BP in HTONS() checking function where if statement compares source port and extract values from there
You don't need to do that -- the instructions will be the same whether you run the application, or disassemble the function without running.
Compilation code ...
So you are trying to reverse-engineer the library for which you have a source?
That makes it very easy to find the constant you are looking for.
Start by setting the constant to easily recognizable value, e.g. 0x12131415. Compile the library and disassemble it. Look for your constant.
If you don't see it, save the disassembled output, and rebuild the library with a different value, e.g. 0xA1B1C1D1. Disassemble it again and compare to previous disassembled output. It should be easy to spot the difference.
P.S. If you really want to debug this library with a live process, do this:
gdb ./myprog
(gdb) set env LD_PRELOAD /path/to/mylib.so
(gdb) run
At this point, you should be able to set breakpoints and observe your library "in action".
Ok managed solve this with a help
when running GDB on shared library You will have to check hex value for 11111 and it should be 2B67 so in registers this will become something like 0x2b67 & it will be passed to htons() as check for source port.
So let's assume I didn't have the source code I could still run: gdb -q *.so
then: info functions and see with disass functionNameGoesHere where some accept / htons calls are made. Correct value should be found right above htons line.
Then decoded hex to dec and thats how You can find it.
This took some while to figure out as I coudn't set BP's.
Again thanks for input from community! Cheers
I am trying to debug my application which use one static builded library.
I want to set break points in my library so i tried to set it using below command :
break TS.cpp:600(FIle name:line no)
but it says
No source file named TS.cpp.
Make breakpoint pending on future shared library load?(y or [n])
so I presses y here (I came to know after browsing internet) but after pressing y gdb is not stopping at my break point and it completed executing program.
Why GDB is not stopped at my break point??
Any input is highly appreciated.
No source file named TS.cpp
This means one of two things:
either the file TS.cpp was not compiled with -g (or equivalently TS.o has been stripped), or
the file TS.o was not linked into the application.
Since you are seeing prints from that source, it's a safe bet that #1 is the actual root cause.
info sources command shows only my application.c and not the files of my library
That is another confirmation that #1 is the root cause.
The problem in your case is with source mapping. It normally happens when application is compiled at some other machine and you are debugging it on some other machine where source location is different.
You can specify source path using directory command of gdb. e.g. if your sources are in /home/taimoor/testApp/src, you can do following:
(gdb) directory /home/taimoor/testApp/src
I am currently debugging a Kernel module and to this purpose, I built the whole kernel with debug information (produces kallsyms, etc ...).
When I try nm my_module.ko, I get the list of symbols included by my module. All is allright except that some symbols are kind of missing as they do not appear in the symbol list. My feeling about this is that the related functions are being automatically inlined.
Anyway, when running the kernel with qemu-kgdb/gdb, I am able to see that the "missing" function is called. This means the compiler did not wipe it out because it was never used in any code path (hence my "feeling").
Since the symbol does not appear, I can't set a breakpoint on it and gdb won't unroll it so that I can see the running code path - understand I don't know how to tell gdb to unroll it.
Unfortunately, I want to see this part of the code path ... How can I do so ?
EDIT : As suggested in Tom's answer, I tried using the file:line syntax as below :
My code file looks like this :
int foo(int arg) // The function that I suspect to be inlined - here is line 1
{
/* Blabla */
return 42;
}
void foo2(void)
{
foo(0); // Line 9
}
I tried b file.c:1, and the breakpoint was hit but the foo() function is not unrolled.
Of course, I am producing debug symbols, since I also set a breakpoint to foo2 to check what happened (which worked well).
You don't say what version of gdb you are using.
Very old versions of gdb don't have any support for inline functions. This was true for 6.8 and maybe even 7.0 -- I don't recall. You can look at the NEWS file for your gdb to see.
Then there were some versions of gdb that supported breakpoints on inline functions, but only using the "file:line" syntax. So what you would do is look up the function in your editor, and find its line number and enter, e.g.:
(gdb) break myfile.c:777
Even more recent versions of gdb, starting with 7.4 or 7.5 (I forget) will handle "break function" just fine if "function" was inlined.
All of this only works if you have debuginfo available. So if you tried this, and it failed, either you have an older gdb, or you forgot to use -g.
There's no good way inside gdb to see what objects in a compilation were missing -g. You can see it pretty easily from the shell, though, by running "readelf -WS" on the .o files, and looking for files that don't have a .debug_info section.
Setting a breakpoint to the signature line of the function did not work. But setting one to the line of an instruction of the inlined function solved the problem for me. For instance, considering the following function inline_foo, found in myfile.c:
inline int inline_foo(int arg) // l.1
{
int a_var = 0;
do_smth(&a_var);
do_some_other_thing(); // l.5
if (a_var) {
a_var = blob();
} else {
a_var = blub();
return a_var; // l.10
}
I was trying b myfile.c:1, which did not appear to work. But if I tried b myfile.c:3 instead, the breakpoint was well handled by GDB.
Since the technique is the same as the one described previously by Tom, I'll accept his answer.
Consider the following Linux kernel dump stack trace; e.g., you can trigger a panic from the kernel source code by calling panic("debugging a Linux kernel panic");:
[<001360ac>] (unwind_backtrace+0x0/0xf8) from [<00147b7c>] (warn_slowpath_common+0x50/0x60)
[<00147b7c>] (warn_slowpath_common+0x50/0x60) from [<00147c40>] (warn_slowpath_null+0x1c/0x24)
[<00147c40>] (warn_slowpath_null+0x1c/0x24) from [<0014de44>] (local_bh_enable_ip+0xa0/0xac)
[<0014de44>] (local_bh_enable_ip+0xa0/0xac) from [<0019594c>] (bdi_register+0xec/0x150)
In unwind_backtrace+0x0/0xf8 what does +0x0/0xf8 stand for?
How can I see the C code of unwind_backtrace+0x0/0xf8?
How to interpret the panic's content?
It's just an ordinary backtrace, those functions are called in reverse order (first one called was called by the previous one and so on):
unwind_backtrace+0x0/0xf8
warn_slowpath_common+0x50/0x60
warn_slowpath_null+0x1c/0x24
ocal_bh_enable_ip+0xa0/0xac
bdi_register+0xec/0x150
The bdi_register+0xec/0x150 is the symbol + the offset/length there's more information about that in Understanding a Kernel Oops and how you can debug a kernel oops. Also there's this excellent tutorial on Debugging the Kernel
Note: as suggested below by Eugene, you may want to try addr2line first, it still needs an image with debugging symbols though, for example
addr2line -e vmlinux_with_debug_info 0019594c(+offset)
Here are two alternatives for addr2line. Assuming you have the proper target's toolchain, you can do one of the following:
Use objdump:
locate your vmlinux or the .ko file under the kernel root directory, then disassemble the object file :
objdump -dS vmlinux > /tmp/kernel.s
Open the generated assembly file, /tmp/kernel.s. with a text editor such as vim. Go to
unwind_backtrace+0x0/0xf8, i.e. search for the address of unwind_backtrace + the offset. Finally, you have located the problematic part in your source code.
Use gdb:
IMO, an even more elegant option is to use the one and only gdb. Assuming you have the suitable toolchain on your host machine:
Run gdb <path-to-vmlinux>.
Execute in gdb's prompt: list *(unwind_backtrace+0x10).
For additional information, you may checkout the following resources:
Kernel Debugging Tricks.
Debugging The Linux Kernel Using Gdb
In unwind_backtrace+0x0/0xf8 what the +0x0/0xf8 stands for?
The first number (+0x0) is the offset from the beginning of the function (unwind_backtrace in this case). The second number (0xf8) is the total length of the function. Given these two pieces of information, if you already have a hunch about where the fault occurred this might be enough to confirm your suspicion (you can tell (roughly) how far along in the function you were).
To get the exact source line of the corresponding instruction (generally better than hunches), use addr2line or the other methods in other answers.
i've written two simple programs:
int main(int ac, char **argv ) {
execv( "/home/me/Desktop/execvtest2", argv );
}
and
int main(int ac, char **argv ) {
execv( "/home/me/Desktop/execvtest1", argv );
}
I've compiled them with
gcc -g to the according outputfiles.
I'm running Ubuntu 10.10 using gcc (Ubuntu/Linaro 4.4.4-14ubuntu5.1) 4.4.5.
When I'm debuging the first program with GNU gdb (GDB) 7.2-ubuntu, I can step until the first execv statement, but then the two files just keep running. Even if I set the follow-exec-mode to new, I can't step into the second program.
When I set catch exec, gdb stops at each call to execv (some how without linked source for the second program, and I'm not able to quit gdb, as it kind of hangs!?), but I'm not able to step over the call into the "new" (as exec replaces the process) inferior program.
So how can this be done? There must be a way to step into the new process right? Am I doing something wrong?
Cheers
you can use "catch" command.
this will give you chance to put some break points
after you exec
I've been doing something very similar to what you are doing for one of my classes. It is a bit hackish and if you're trying to get things like register values it may mess things up. According to GDB's documentation you can change the symbol file while maintaining the execution file. To do this, simply use the command symbol-file file2.
Note that this must be a binary file compiled with the GDB flag (-g in GCC). After you've loaded this symbol file, you will not be able to break or see any of the lines for the original execution file. However, you may set break points for the new symbol file i.e. break file2.c:40 and then step through execution just as before. It is a bit hackish and may not work perfectly because you are essentially catching the execution of a new process and mapping it to the symbol table of it's binary file, without using that binary file to run it directly. I haven't had stellar results but you can see the intermediate values this way.
Another thing, in order to return to debugging the original file you will have to do symbol-file file to reload it's symbol table.