I'm debugging an application write in ansi C, a multiple threads program.
Sometime, in the main thread cause a SIGSEGV fault.
(gdb) backtrace full
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#1 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) info registers
rax 0x1 1
rbx 0x0 0
rcx 0x0 0
rdx 0x2 2
rsi 0x458e7aa0 1166965408
rdi 0x0 0
rbp 0x0 0x0
rsp 0x458e7b60 0x458e7b60
r8 0x458e7b20 1166965536
r9 0x0 0
r10 0x0 0
r11 0x206 518
r12 0x2aaaac400e70 46912522686064
r13 0x2aaaac514090 46912523813008
r14 0x1 1
r15 0x18505f10 407920400
rip 0x0 0
eflags 0x10206 [ PF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x63 99
gs 0x0 0
fctrl 0x37f 895
fstat 0x0 0
ftag 0xffff 65535
fiseg 0x0 0
fioff 0x0 0
foseg 0x0 0
fooff 0x0 0
fop 0x0 0
mxcsr 0x1f80 [ IM DM ZM OM UM PM ]
(gdb)
This information is from core file, I'm not very family with debug in Linux environment, Is there anything I can do to find where's the problem?
Edit: all of source files are compiled with flag as follow
gcc -O0 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"src/redisconnector.d" -MT"src/redisconnector.d" -o"src/redisconnector.o" "../src/redisconnector.c"
Your RIP points to 0. It's probably caused by a stack overflow. Your RBP is also 0, so the backtrace gdb command will tell you nothing.
Recompile the application with "-g" option;
Use Gdb not with core files, but to run entire application:
gdb --args ./application application_options
then "run" command of gdb.
Running from gdb will detect SIGSEGV, and gdb will be focused on failed thread.
Well, first you'll need to compile with debugging enabled so that your backtrace has something usable. The flag is gcc -g
Related
I wrote the following C programm
#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
void main(){
char *variable = "CodeName0...";
while (1){
printf("%s\n%d\n","hello from gencore code source",getpid());
sleep(2);
}
exit(0);
}
after running it I sent a SIGABRT signal using the following command
kill -6 <process Id from code execution>
this generated a core dump file under the name core inside the same directory
I want to read the content of cpu registers ( IP ... ) but I didn't manage to do it
I'm runnig parrot OS, this is the output of uname -a
Linux Parrot 5.16.0-12parrot1-amd64 #1 SMP PREEMPT Debian 5.16.12-2parrot1 (2022-03-11) x86_64 GNU/Linux
I tried using gdb but I'm getting this error
"0x7fffd7f6de10s": not in executable format: file format not recognized
I would like to know why this is the case ( file format not recognized )
Not really sure how to explain it any further:
$ gdb <binary> <core>
i all-registers
where <binary> is your binary, ideally compiled with -g3, and <core> is your core file that was dumped. You may need to navigate to different frame (bt and f <number> where <number> is the frame number). all-registers is a little verbose but here is i register output:
rax 0xfffffffffffffdfc -516
rbx 0xffffffffffffff80 -128
rcx 0x7fa81f0c650a 140360052139274
rdx 0x7ffc19fe2020 140720744570912
rsi 0x0 0
rdi 0x0 0
rbp 0x0 0x0
rsp 0x7ffc19fe1fe0 0x7ffc19fe1fe0
r8 0x0 0
r9 0x27 39
r10 0x7ffc19fe2020 140720744570912
r11 0x246 582
r12 0x562d5cc38070 94752829833328
r13 0x0 0
r14 0x0 0
r15 0x0 0
rip 0x7fa81f0c650a 0x7fa81f0c650a <__GI___clock_nanosleep+42>
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
This question already has answers here:
GDB complaining about missing raise.c
(1 answer)
How to debug standard c library functions like printf?
(3 answers)
Closed 2 years ago.
I am using the Intel CMT-CAT distribution (cache management technology - cache allocation technology) at https://github.com/intel/intel-cmt-cat on Ubuntu 18.04.
One of the programs in the distribution (_pqos_api_lock) calls lockf.c from the C std library glibc. Lockf produces a segmentation fault (SIGSEGV). Further debugging shows that it segfaults at the point where rdx switches from 0 to 0x7fffffffe0c8 (140737488347336).
Here is the output from my gdb session:
Breakpoint 1, lockf (fd=4, cmd=cmd#entry=1, len=len#entry=0) at lockf.c:36
36 lockf.c: No such file or directory.
(gdb) ni
39 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
36 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
42 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
36 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
36 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
0x00007ffff78f07bb 36 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
0x00007ffff78f07c0 36 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
46 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
39 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x0 0
(gdb) ni
0x00007ffff78f07c8 39 in lockf.c
(gdb) i r rdi
rdi 0x4 4
(gdb) i r rsi
rsi 0x1 1
(gdb) i r rdx
rdx 0x7fffffffe0c8 140737488347336
(gdb) ni
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff78f07c8 in lockf (fd=4, cmd=cmd#entry=1, len=len#entry=0)
at lockf.c:39
39 in lockf.c
At each new instruction, I show the state of rdi, rsi and rdx (the three registers used for a call to lockf). I am able to see debug line numbers in lockf, but not the source code lines. I want to see the source code lines to understand why I get this error.
I downloaded source for lockf.c from https://code.woboq.org/userspace/glibc/io/lockf.c.html, but the line numbers do not appear to match what's installed on my system. WIthout the source lines of the lockf.c version installed locally, I can't debug why it segfaults.
The libc6-dbg package is already installed and it's the latest version (according to Ubuntu). The installed version of glibc is 2.27-3:
$ ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
So to sum it up, given the scenario described above, how can I see the source code lines in lockf.c (or any other glibc function) from a debug session in gdb.
Finally, I'm calling these programs from a shared object, but I don't think that makes a difference in this case.
UPDATE:
At the point where gdb goes into lockf.c, I issued info source:
info source
Current source file is lockf.c
Compilation directory is /build/glibc-OTsEL5/glibc-2.27/io
Source language is c.
Producer is GNU C11 7.3.0 -mtune=generic -march=x86-64 -g -O2 -O3 -std=gnu11 -fgnu89-inline -fmerge-all-constants -frounding-math -fstack-protector-strong -fPIC -fexceptions -ftls-model=initial-exec -fstack-protector-strong.
Compiled with DWARF 2 debugging format.
Does not include preprocessor macro info.
But there is no /build/glibc-OTsEL5/glibc-2.27/io on my Ubuntu 18.04.
I am doing an exercise for a class to hook some system calls on a running kernel using a linux kernel module and using my new_sys_call() in place of the old one. I am developing on CentOS 7 and using a stock kernel-3.10.0-862.3.2. I was able to successfully hook execve and I can access parameter values but I cannot seem to access the rename syscall parameter values. I realize this is probably not a practical real world approach but I am trying to learn more about kernel development and how system calls work.
Here's an example of the hooking method definitions:
/*
Original Syscall Prototype
*/
asmlinkage long (*orig_sys_chdir_fn)(const char __user *filename);
/*
new syscall
*/
asmlinkage long new_sys_chdir(const char __user *filename) {
// do stuff
return orig_sys_chdir_fn(filename);
}
We obtain the stub chdir address with this code:
orig_stub_chdir_addr = ((unsigned long *)(sys_call_table_addr))[__NR_chdir];
Thread 357 hit Breakpoint 3, new_sys_rename (oldname=0xffffff9c <error: Cannot access memory at address 0xffffff9c>,
newname=0xffffff9c <error: Cannot access memory at address 0xffffff9c>) at /home/ulys/syshooks/kmod/sysrename.c:19
19 asmlinkage long new_sys_rename(const char __user *oldname, const char __user *newname) {
(gdb) info registers
rax 0xffffffff8122dbd0 -2128421936
rbx 0x0 0
rcx 0x7ffd0faf9480 140724866618496
rdx 0xffffff9c 4294967196
rsi 0xffffff9c 4294967196
rdi 0xffffff9c 4294967196
rbp 0xffff88004183bf48 0xffff88004183bf48
rsp 0xffff88004183bf40 0xffff88004183bf40
r8 0x0 0
r9 0x1 1
r10 0x7ffd0faf7320 140724866609952
r11 0x202 514
r12 0x0 0
r13 0x0 0
r14 0x0 0
r15 0x0 0
rip 0xffffffffc0540b20 0xffffffffc0540b20 <new_sys_rename>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x10 16
ss 0x18 24
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0xb 11
(gdb) info stack
#0 new_sys_rename (oldname=0xffffff9c <error: Cannot access memory at address 0xffffff9c>,
newname=0xffffff9c <error: Cannot access memory at address 0xffffff9c>) at /home/ulys/syshooks/kmod/sysrename.c:19
#1 0xffffffff8122dbee in gather_surplus_pages (delta=-2123237509, h=0x0 <irq_stack_union>) at mm/hugetlb.c:1764
#2 hugetlb_acct_memory (h=0x0 <irq_stack_union>, delta=-2123237521) at mm/hugetlb.c:3149
#3 0xffffffff8171f82f in inet_netconf_fill_devconf (skb=0xffffffff8171f77b <inet_netconf_fill_devconf+123>,
ifindex=<optimized out>, devconf=<optimized out>, portid=<optimized out>, seq=<optimized out>, flags=<optimized out>,
type=<error reading variable: Cannot access memory at address 0x10>, event=80) at net/ipv4/devinet.c:1786
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) x/s 0x7ffd0faf9480
0x7ffd0faf9480: "/tmp/test2"
You can see the bogus addresses at 0xffffff9c which cannot be read by copy_from_user or used by anything in the new_sys_rename function. You can see that the newname is in the register rcx but I can't locate oldname. I have noticed this on several different filename based syscalls such as chmod and chdir as well.
On chdir, I receive similar results except I can't find filename in any register.
Thread 304 hit Breakpoint 1, new_sys_chdir (
filename=0xffffff9c <error: Cannot access memory at address 0xffffff9c>)
at /home/ulys/syshooks/kmod/syschdir.c:17
17 asmlinkage long new_sys_chdir(const char __user *filename) {
(gdb) info registers
rax 0x0 0
rbx 0x0 0
rcx 0xffff880055d47f18 -131939955343592
rdx 0x3 3
rsi 0x3 3
rdi 0xffffff9c 4294967196
rbp 0xffff880055d47f48 0xffff880055d47f48
rsp 0xffff880055d47f10 0xffff880055d47f10
r8 0xc919d0 13179344
r9 0xcba2d0 13345488
r10 0x7f7147cfd030 140124512833584
r11 0x246 582
r12 0x3 3
r13 0xc8f320 13169440
r14 0x0 0
r15 0x0 0
rip 0xffffffffc0561000 0xffffffffc0561000 <new_sys_chdir>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x10 16
ss 0x18 24
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0xb 11
(gdb) x/s 0xffff880055d47f18
0xffff880055d47f18: "o\367q\201\377\377\377\377{\367q\201\377\377\377\377\225\"", <incomplete sequence \356>
(gdb) x/s 0xc919d0
0xc919d0: " "
(gdb) x/s 0xc919d0
0xc919d0: " "
(gdb) x/s 0x7f7147cfd030
0x7f7147cfd030: "f\017s\372\005f\017t\321f\017\370\320fD\017\327\312\323\352A\323\351D)\312\017\205\v\004"
(gdb) info stack
#0 new_sys_chdir (filename=0xffffff9c <error: Cannot access memory at address 0xffffff9c>)
at /home/ulys/syshooks/kmod/syschdir.c:17
#1 0xffffffff81218b3e in move_page_tables (vma=<optimized out>, old_addr=3, new_vma=<optimized out>,
new_addr=18446744071586314095, len=<optimized out>, need_rmap_locks=<optimized out>) at mm/mremap.c:187
#2 0xffffffff8171f82f in inet_netconf_fill_devconf (skb=0xffffffff8171f77b <inet_netconf_fill_devconf+123>,
ifindex=<optimized out>, devconf=<optimized out>, portid=<optimized out>, seq=<optimized out>, flags=<optimized out>,
type=<error reading variable: Cannot access memory at address 0x10>, event=80) at net/ipv4/devinet.c:1786
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
It appears to work just fine on execve. Here you can see similar output but I have no problems accessing any of execve's filename or argv strings.
Thread 335 hit Breakpoint 2, new_sys_execve (filename=0x55c85e0d7af0 "/usr/libexec/postfix/pickup", argv=0x55c85e0d3d00,
envp=0x55c85e0d3d00) at /home/ulys/syshooks/kmod/sysexecve.c:21
21 const char __user * const __user * envp) {
(gdb) info registers
rax 0xffffffff8171fd80 -2123235968
rbx 0x0 0
rcx 0xffffffff 4294967295
rdx 0x55c85e0d3d00 94319059746048
rsi 0x55c85e0d3d00 94319059746048
rdi 0x55c85e0d7af0 94319059761904
rbp 0x0 0x0 <irq_stack_union>
rsp 0xffff88005323ff50 0xffff88005323ff50
r8 0x30 48
r9 0x0 0
r10 0xffffffff 4294967295
r11 0x206 518
r12 0x0 0
r13 0x0 0
r14 0x0 0
r15 0x0 0
rip 0xffffffffc053f880 0xffffffffc053f880 <new_sys_execve>
eflags 0x10282 [ SF IF RF ]
cs 0x10 16
ss 0x18 24
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0xb 11
(gdb) info stack
#0 new_sys_execve (filename=0x55c85e0d7af0 "/usr/libexec/postfix/pickup", argv=0x55c85e0d3d00, envp=0x55c85e0d3d00)
at /home/ulys/syshooks/kmod/sysexecve.c:21
#1 0xffffffff8171fdc8 in rtm_to_ifaddr (pprefered_lft=<synthetic pointer>, pvalid_lft=<synthetic pointer>,
nlh=0x0 <irq_stack_union>, net=0x0 <irq_stack_union>) at net/ipv4/devinet.c:801
#2 inet_rtm_newaddr (skb=0x0 <irq_stack_union>, nlh=0x0 <irq_stack_union>) at net/ipv4/devinet.c:854
Backtrace stopped: Cannot access memory at address 0x8
So why are some of these variables protected from kernel space? Where can they be found and is there another kernel function that can be called to get the values?
Look this:
(gdb) info reg
rax 0x7ffff00008c0 140737219922112
rbx 0x7ffff7fe9700 140737354045184
rcx 0x33d9e182a0 222698767008
rdx 0x7ffff7fe9a10 140737354045968
rsi 0x7ffff00008c0 140737219922112
rdi 0x0 0
rbp 0x7ffff7fe8e50 0x7ffff7fe8e50
rsp 0x7ffff7fe8e30 0x7ffff7fe8e30
r8 0x400afb 4197115
r9 0x0 0
r10 0x7ffff7fe8bb0 140737354042288
r11 0x33d9c0c480 222696621184
r12 0x33d9c11200 222696641024
r13 0x7ffff7fe99c0 140737354045888
r14 0x0 0
r15 0x3 3
rip 0x4008c1 0x4008c1 <write_to_thread_log(char const*)+29>
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
My question is:
Which interface can realize this function in Linux?
My platform ：Linux 2.6.32-279.el6.x86_64 x86_64 GNU/Linux
Many thanks for your help.
The ptrace system call is the standard debugging facility in Linux, which GDB uses to interact with other processes being debugged. GDB probably gets this information with the PTRACE_GETREGS or PTRACE_GETREGSET requests.
ptrace man page
This line is causing segfault to me:
30 printf("st_name:\t%s\n", &p_str_tab[p->st_name]);
I've tried to trace it down in gdb:
(gdb) p p_str_tab[p->st_name]
$11 = 0 '\000'
(gdb) p &p_str_tab[p->st_name]
$12 = 0x2aaaaaab0000 ""
(gdb) x/16s 0x2aaaaaab0000
0x2aaaaaab0000: ""
0x2aaaaaab0001: ".symtab"
0x2aaaaaab0009: ".strtab"
(gdb) call printf("st_name:\t%s\n", 0x2aaaaaab0000)
Program received signal SIGSEGV, Segmentation fault.
0x00000034f4042729 in vfprintf () from /lib64/libc.so.6
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
I can print the memory with gdb's x command ,but if I use printf,segmentation fault.
Why?
UPDATE as required in comment:
(gdb) x/1i $rip
0x34f4042729 <vfprintf+57>: mov 0xc0(%rdi),%eax
(gdb) info reg
rax 0x54 84
rbx 0x34f3e1bbc0 227429956544
rcx 0x0 0
rdx 0xffffffffffffffb0 -80
rsi 0x401b08 4201224
rdi 0x600908 6293768
rbp 0x7fffffffe6e0 0x7fffffffe6e0
rsp 0x7fffffffe040 0x7fffffffe040
r8 0x2aaaaaabf210 46912496202256
r9 0x34f4351780 227435419520
r10 0x1238 4664
r11 0x648 1608
r12 0x0 0
r13 0x7fffffffe9c0 140737488349632
r14 0x0 0
r15 0x0 0
rip 0x34f4042729 0x34f4042729 <vfprintf+57>
eflags 0x10202 [ IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
fctrl 0x37f 895
fstat 0x0 0
ftag 0xffff 65535
---Type <return> to continue, or q <return> to quit---
fiseg 0x0 0
fioff 0x0 0
foseg 0x0 0
fooff 0x0 0
fop 0x0 0
mxcsr 0x1f80 [ IM DM ZM OM UM PM ]
You might want to check whether you're overflowing the stack.
The faulting instruction mov 0xc0(%rdi),%eax represents something like eax = rdi->member where member is at offset 0xc0. Without seeing more disassembly it's hard to know what that is for sure, but it seems likely that it's stdout or something inside stdout. It's not likely that the faulting instruction is dereferencing your input string.
Have you done anything unusual to stdout? A brute force approach would be to sprinkle printf everywhere (of what it probably doesn't matter) and see where it starts crashing. Just before that is where something got corrupted.
Must be a pointer overran issue ,try valgrind.