Gdbserver, remote debugging, printf() output not seen on the host - c

I am running gdbserver on a remote armv5 target with the following command line:
./gdbserver --debug --remote-debug --multi :9000
The host machine is x86 Linux. I am trying to remote-debug a sample hello_world.c program, pretty much a int main(int argc, char** argv) { printf("hello world\n"); exit(0); };
I am doing the following on the host machine:
./gdb
(gdb)set sysroot /home/username
(gdb)set verbose on
(gdb)target extended-remote 192.168.1.217:9000
(gdb)file hello_world
(gdb)set remote exec-file /upgrade/hello_world
(gdb)r
The output is as follows:
Starting program: /home/username/hello_world
Loaded symbols for /lib/ld-linux.3.so
Loaded symbols for /lib/libc.6.so
Breakpoint 1, main (argc=1, argv=0xbe90cdd4) at hello_world.c:50
50 int parm=7;
(gdb) c
Continuing.
[Inferior 1 (process 16721) exited normally]
Notice that there is no printf() output on the host machine. It just stops on a breakpoint and then runs to completion when I hit 'c'.
I expect the \n in the printf to flush the buffer and produce output immediately, but for some reason I do not see it in the host gdb window.
Any thoughts ?

Related

gdb is miscounting args containing '#'

I run my program in gdb, withe the arguments -b # +L -m, but when I break main, argc is reported as 2 and the last argument available is "-b". However, using show args in GDB gives me Argument list to give program being debugged when it is started is "-b # +L -m", which is accurate.
Starting program: /home/matt/.../args -b # +L -m
Breakpoint 1, main (argc=2, argv=0x7fffffffe028) at args.c:129
Is # treated specially in an argument, even in GDB (not bash)?
I assume you are running on Linux. gdb on Linux by default will invoke the shell to start your program. Therefore everything after the # is still being interpreted by the shell as a comment. You can change that behaviour in gdb by telling it not to use the shell to start the program via the set startup-with-shell off command.
Here is an example program that just prints argc.
#include <stdio.h>
int main(int argc, char **argv)
{
printf("%d\n", argc);
return 0;
}
And here are the gdb runs with and without the shell receptively.
(gdb) r -b # +L -m
Starting program: /tmp/a.out -b # +L -m
2
[Inferior 1 (process 28385) exited normally]
(gdb) set startup-with-shell off
(gdb) r -b # +L -m
Starting program: /tmp/a.out -b # +L -m
5
[Inferior 1 (process 28443) exited normally]
(gdb)
Refer to the gdb manual for more details on program startup.

gdb showing different address than in code

I am trying to implement a buffer overflow attack and I need to know the address of my buffer that I am trying to overflow.
The address that is displayed using GDB is different than if I just did this in the code:
Exact code:
#include<stdio.h>
int main() {
char buffer[20];
printf("%p\n", buffer); // 0xbffff320
return 0;
}
However, in gdb if I do:
p &buffer
I get: 0xbffff330
Why is there a difference and will it mess up my buffer overflow attack?
I have ALSR and stack guard disabled.
Thanks.
EDIT 1: Even when I step through gdb and it encounters the print line, I get 0xbffff320 as the address
EDIT 2:
Environment: Ubuntu Linux 9 image running in virtual box on windows 7.
The gdb version: 6.8-debian
Compiled using GCC such as: gcc -g -fno-stack-protector filename.c
execute immediately: ./a.out
address printed: 0xbffff320
Then open in debugger like this: gdb ./a.out
then enter b main
then run
then p &buffer
Then address is 0xbffff330
Edit 3:
This is the gdb log to reproduce behavior:
$ gdb ./a.out
b main
run
p &buffer /* address here is different than what is shown if I run executable */
step through program to printf statement /* address here is same as p &buffer but different than what is printed when program is ran */
The question, as I understand it, is why the address of a local variable in main is different when the program is started from the shell versus when it is started from gdb.
Here's a sample program to show the difference:
mp#ubuntu:~$ cat s.c
#include<stdio.h>
int main(int argc, char **argv) {
char buffer[20];
system("env");
printf("%s %p\n", argv[0], buffer);
return 0;
}
We'll run it in a clean environment. (I also disabled ASLR).
mp#ubuntu:~$ env -i sh
$ ./s
PWD=/home/mp
./s 0xbffffe48
$ gdb ./s
(gdb) run
Starting program: /home/mp/s
COLUMNS=80
PWD=/home/mp
LINES=42
/home/mp/s 0xbffffe08
The output from gdb's print &buffer command is the same as the program's idea of the address, but they're both different from when the program was run in the shell.
(gdb) b 6
Breakpoint 1 at 0x804849c: file s.c, line 6.
(gdb) run
Starting program: /home/mp/s
COLUMNS=80
PWD=/home/mp
LINES=42
Breakpoint 1, main (argc=1, argv=0xbffffed4) at s.c:6
6 printf("%s %p\n", argv[0], buffer);
(gdb) p &buffer
$1 = (char (*)[20]) 0xbffffe08
(gdb) n
/home/mp/s 0xbffffe08
8 return 0;
There are a couple of things contributing to the difference:
gdb is invoking the program with an absolute pathname, so the argv array is bigger.
gdb sets (or in this case, adds) two environment variables. This is done in readline/shell.c:sh_set_lines_and_columns(). So the environ array is bigger.
To remove those two variables from the environment, you can use unset environment, or set exec-wrapper to run env -u .... That way, the program's addresses under gdb are the same as when it's run in the shell (if we use an absolute pathname).
$ `pwd`/s
PWD=/home/mp
/home/mp/s 0xbffffe28
$ gdb `pwd`/s
(gdb) set exec-wrapper env -u LINES -u COLUMNS
(gdb) run
Starting program: /home/mp/s
PWD=/home/mp
/home/mp/s 0xbffffe28
Your array object in your system is stored in the stack. At the top of your stack there is, among other, the environment. When you run your program with gdb, gdb will provide a different environment (the env var and their value) which explains the addresses difference.
You can check the difference by running show environment in gdb and by comparing the output with set command in your shell.
Found out that this is expected behavior in old versions of GDB (mine is 6.8-debian), and if you construct your buffer overflow attack properly you can work around this behavior and it won't be a problem.
For the moment, the only reasons I can imagine are :
you tried to print &buffer after your program terminated. Solution: try setting a breakpoint on main, run, next to execute printf, and print &buffer.
you first ran your program outside gdb, then ran it inside gdb but forgot to execute the printf line with next.
a bug in your version of gdb
a bug in your version of gcc (gcc might produce incorrect debug info: see 1 and 2)

How to debug multi-threaded Raspberry Pi code executed in qemu

I am trying to debug code written for Raspberry Pi on my Ubuntu laptop. I am using:
the Raspberry Pi Toolchain and (partial ?) root fs obtained with git clone git://github.com/raspberrypi/tools.git executed in ~/rpi
qemu virtualizer installed from Ubuntu repositiories - qemu-arm reports
version 2.0.0
the following test case:
#include <pthread.h>
#include <stdio.h>
static void *thread(void *arg)
{
static char szReturn[] = "Me too";
puts((char *)arg);
puts("Hello, I'm Thread");
return szReturn;
}
int main(int argc, char *argv[])
{
pthread_t tid;
char szGreeting[] = "Hello, I'm Main", *reply;
if (pthread_create(&tid, NULL, &thread, szGreeting) != 0)
{
perror("pthread_create says: ");
return 1;
}
puts("Nice to meet you");
if (pthread_join(tid, (void **)&reply) == 0)
puts(reply);
return 0;
}
After adding ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin to the PATH (I am using 64 bit Ubuntu), I do (from the directory in which the test code resides):
arm-linux-gnueabihf-gcc -g -O0 test.c -o test -lpthread
qemu-arm -L ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc ./test
I get the expected output, so the thread creation and execution works.
However, if I do:
qemu-arm -g 1234 -L ~/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc ./test in one terminal, to launch qemu and have it waiting for the debugger to connect to it
arm-linux-gnueabihf-gdb -ex="target remote localhost:1234" -x="set breakpoint pending on" -ex="break main" ./test in another terminal, to start debugging the program with gdb
I get in gdb, after 'continue' I reach main, I set breakpoints to thread, 21 and 24, I next OK to the ptread_create call, so debugging the main thread works, but:
when it executes pthread_create, instead of hitting one of the 3 breakpoints (the thread, the create failure or the create success cases), gdb reports "inferior terminated' and the session ends
back in the qemu terminal though, the program has generated the normal output, so the thread was created and executed, and the whole program worked as expected
So gdb can not debug multithreaded programs in this configuration/setup?! What am I missing?
Could the fact that I am running Ubuntu in a Parallels virtual machine on an OSX machine have something to do with it? I don't have a dedicated Ubuntu machine to test, but I doubt it.
Update: On a native Fedora system the behavior is even worse: the breakpoints are also not taken, but it also crashes in puts and pthread_join.
Thanks in advance

GDB breakpoint when executing external program through system()

I have the function system() call a separate script that has already been compiled. But I'd like to be able to set a breakpoint in functions within THAT specific file.
So:
File A:
system("./fileB");
File B:
void main() {
/* etc */
}
I'd like to be able to set a breakpoint at main after the system command is called.
Any help would be appreciated!
Newer versions of GDB (7.1+) can debug multiple programs at once and can indeed support this:
run-program.c
#include <stdlib.h>
int main()
{
system("./program-i-want-to-debug");
return 0;
}
program-i-want-to-debug.c
#include <stdio.h>
int main()
{
printf("Hello, World\n");
return 0;
}
run-program.gdb
set detach-on-fork off
set target-async on
set pagination off
set non-stop on
add-inferior -exec program-i-want-to-debug
break program-i-want-to-debug.c:5
file run-program
run
inferior 3
backtrace
Sample session
$ gdb -q -x run-program.gdb
Added inferior 2
Breakpoint 1 at 0x400441: file program-i-want-to-debug.c, line 5.
[New process 20297]
process 20297 is executing new program: /usr/bin/bash
process 20297 is executing new program: /home/scottt/Dropbox/stackoverflow/program-i-want-to-debug
Reading symbols from /home/scottt/Dropbox/stackoverflow/program-i-want-to-debug...done.
Breakpoint 1, main () at program-i-want-to-debug.c:5
5 printf("Hello, World\n");
[Switching to inferior 3 [process 20297] (/home/scottt/Dropbox/stackoverflow/program-i-want-to-debug)]
[Switching to thread 2 (process 20297)]
#0 main () at program-i-want-to-debug.c:5
5 printf("Hello, World\n");
#0 main () at program-i-want-to-debug.c:5
Obviously you'd want to compile the programs with debug info (gcc -g).
Maybe I do not catch your point. It seems that starting gdb debug on File A and setting a breakpoint on "FileB:line of main" resolve your problem.

How to debug the entry-point of fork-exec process in GDB?

I have a C linux application (A) that spawns another process (P) when it is started. When I want to debug P I start A as usual and I connect with ddd/gdb to P.
Problems appear when I want to debug the entry-point (start of main) of P. If I follow the usual approach when I connect the debugger to P is already to late. The solution I've found was to insert a sleep at the begining of the main of P so I have time to connect with gdb but this is not a very elegant solution.
I've also tried using asm("int $3") but it doesn't seems to work.
Do you have any idea how I could solve this problem? (preferably without altering the code of A or P)
You should use this option:
set follow-fork-mode mode
Where mode is one of parent, child or ask.
To follow the parent (this is the default) use:
set follow-fork-mode parent
To follow the child:
set follow-fork-mode child
To have the debugger ask you each time:
set follow-fork-mode ask
So basically you'd start out connecting gdb to A, then set gdb to follow the child, and then when A spawns P, gdb will connect to P and detach from A.
In addition to the Nathan Fellman's answer, catchpoints come in handy, i.g.:
catch exec
Catchpoint works as a breakpoint. Every time a call to exec() syscall is detected, GDB stops. This allows you to set any breakpoint (i.g. break main) in any newly loaded executable before continuing. Another catchpoint catch fork works similarly for fork() syscall detection.
It is especially convenient:
when both parent and child has to be followed (set detach-on-fork off);
when parent processes forks often loading various executables.
exec part with file + break main
The fork was part was explained at: https://stackoverflow.com/a/377295/895245
Now for the exec:
a.c:
#include <unistd.h>
int main(void) {
execl("./b", "./b", "ab", "cd", (char*)NULL);
return 1;
}
b.c:
#include <stdio.h>
int main(int argc, char **argv ) {
printf("%s\n", argv[0]);
printf("%s\n", argv[1]);
}
Then:
gcc -g a.c -o a
gcc -g b.c -o b
gdb -nh -q a
Now on the interactive session:
Reading symbols from a...done.
(gdb) start
Temporary breakpoint 1 at 0x4004ea: file a.c, line 4.
Starting program: /home/ciro/test/gdb-exec/a
Temporary breakpoint 1, main () at a.c:4
4 execl("./b", "./b", "ab", "cd", (char*)NULL);
(gdb) file b
A program is being debugged already.
Are you sure you want to change the file? (y or n) y
Load new symbol table from "b"? (y or n) y
Reading symbols from b...done.
(gdb) b main
Breakpoint 2 at 0x4004f5: file b.c, line 4.
(gdb) n
Breakpoint 2, main (argc=0, argv=0x7fffffffa570) at b.c:4
4 printf("%s\n", argv[1]);
(gdb) n
process 4877 is executing new program: /home/ciro/test/gdb-exec/b
Breakpoint 2, main (argc=3, argv=0x7fffffffa598) at b.c:4
4 printf("%s\n", argv[1]);
(gdb) n
ab
5 printf("%s\n", argv[2]);
(gdb) n
cd
6 }
(gdb)
You just have to make sure that you go up to the exec before running file, possibly with a b execl, since after that you will be using symbols from the new file.
Tested in Ubuntu 14.04, gdb 7.7.1.
You should be able to do this by making use of gdb's remote debugging features, specifically gdbserver. In effect, launch (P) using gdbserver. These links have more detailed info:
Using gdbserver
GDB Remote Debugging
set a break point at main(), it will also break at main() of the execed program.

Resources