analyzing .DSYM file using lldb - c

I wrote a simple c program that makes use of the assert() call. I'd like to analyze it using lldb.
OS in use: OS X Mavericks
Compiler used to compile:
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
The -g compiler option generated a .DSYM directory. I wanted to know how to how to analyze this core using lldb.
PS: I have compiled using the -g option (clang -g test.c)

Start lldb and then execute the command
target create --core /cores/core.NNNN
where "/cores/core.NNNN" is your core file. A simple example:
$ lldb
(lldb) target create --core /cores/core.5884
Core file '/cores/core.5884' (x86_64) was loaded.
Process 0 stopped
* thread #1: tid = 0x0000, 0x00007fff8873c866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGSTOP
frame #0: 0x00007fff8873c866 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill + 10:
-> 0x7fff8873c866: jae 0x7fff8873c870 ; __pthread_kill + 20
0x7fff8873c868: movq %rax, %rdi
0x7fff8873c86b: jmpq 0x7fff88739175 ; cerror_nocancel
0x7fff8873c870: ret
(lldb) bt
* thread #1: tid = 0x0000, 0x00007fff8873c866 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGSTOP
frame #0: 0x00007fff8873c866 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff85de835c libsystem_pthread.dylib`pthread_kill + 92
frame #2: 0x00007fff87554bba libsystem_c.dylib`abort + 125
frame #3: 0x00007fff8751ea5f libsystem_c.dylib`__assert_rtn + 321
frame #4: 0x000000010c867f59 a.out`main(argc=1, argv=0x00007fff53398c50) + 89 at prog.c:7
frame #5: 0x00007fff872b65fd libdyld.dylib`start + 1
(lldb) frame select 4
frame #4: 0x000000010c867f59 a.out`main(argc=1, argv=0x00007fff53398c50) + 89 at prog.c:7
4 int main(int argc, char **argv)
5 {
6 int i = 0;
-> 7 assert(i != 0);
8 return 0;
9 }
10
(lldb) p i
(int) $0 = 0

At the command prompt, in the same directory where you have the symbols directory, type
lldb program-name
then use the commands you want as in this official gdb to lldb command map:
lldb-gdb

Related

stack not 16 byte aligned error when mocking exit with mimick

I'm using Mimick to mock the exit function, but I get a stack not 16 byte aligned error.
Here's a reduced code example:
#include <stdlib.h>
#include <mimick.h>
mmk_mock_define(exit_mock, void, int);
int main(void) {
mmk_mock("exit#self", exit_mock);
exit(EXIT_FAILURE);
mmk_reset(exit);
return 0;
}
Compiled with the following on macOS 11:
clang -I ./include -g -rpath ./lib/ -Wl,-segalign,1000 -L ./lib/ -l mimick -o test test.c
Where mimick is already compiled and installed to ./lib and ./include.
Running with: lldb ./test
lldb gives me:
(lldb) target create "test"
Current executable set to '/Users/camdennarzt/Developer/C/test/test' (x86_64).
(lldb) r
Process 94625 launched: '/Users/camdennarzt/Developer/C/test/test' (x86_64)
Process 94625 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
frame #0: 0x00007fff20531c9e libdyld.dylib`stack_not_16_byte_aligned_error
libdyld.dylib`stack_not_16_byte_aligned_error:
-> 0x7fff20531c9e <+0>: movdqa %xmm0, (%rsp)
0x7fff20531ca3 <+5>: int3
0x7fff20531ca4 <+6>: nop
0x7fff20531ca5 <+7>: nop
Target 0: (test) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
* frame #0: 0x00007fff20531c9e libdyld.dylib`stack_not_16_byte_aligned_error
frame #1: 0x00007ffeefbfefc8
frame #2: 0x0000000100001941 test`mmk_mock_create_internal + 289
frame #3: 0x0000000100003315 test`mmkuser_exit_mock_create(tgt="\x80&", opts=(sentinel_ = 0, noabort = 0)) at test.c:4:1
frame #4: 0x00000001001ae010
frame #5: 0x00007fff20532f3d libdyld.dylib`start + 1
frame #6: 0x00007fff20532f3d libdyld.dylib`start + 1
Because I saw this I also tried compiling with -fno-stack-check -mmacosx-version-min=10.14 but it didn't help. I also tried with a homebrew installed clang but that didn't help either.
What am I doing wrong here? Or is there a bug in a library/compiler I'm using?
I do not know Criterion/Mimick so this may/may not make a difference.
If params[i].argv is to mimic main()'s argv, then .argv[argc] as NULL is missing.
argv[argc] shall be a null pointer.
C17 Spec.
params[0].argc = 1;
params[0].argv = cr_malloc(sizeof(char*) * 2 /* not 1 */);
params[0].argv[0] = cr_strdup("progname");
params[0].argv[1] = NULL; /* add */
Likewise for other params[i] and free_parse_args() needs update.

How to "unwind" a core dump in gdb

I'm not sure if this is the correct wording of the issue, but let's take the following example where I have a program that will crash/abort:
#include <assert.h>
int main(void)
{
int z=2;
assert (z>5);
}
And if I compile it with debugging and then run it:
$ gcc -ggdb3 a.c -o a.o && ./a.o
a.o: a.c:8: main: Assertion `z>5' failed.
Aborted (core dumped)
Now I'll open it up in gdb to see if I can inspect the program:
$ gdb a.o core
Core was generated by `./a.o'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
If I now "run" the program with r I will get something like this (from gdb-dashboard viewer):
My question is the stack is now pretty deep into the C runtime / linux:
─── Stack ──────────────────────────────────────────────────────────────────────────────────────────
[0] from 0x00007ffff7a22f47 in __GI_raise+199 at ../sysdeps/unix/sysv/linux/raise.c:51
[1] from 0x00007ffff7a248b1 in __GI_abort+321 at abort.c:79
[2] from 0x00007ffff7a1442a in __assert_fail_base+330 at assert.c:92
[3] from 0x00007ffff7a144a2 in __GI___assert_fail+66 at assert.c:101
[4] from 0x00005555555546ce in main+52 at a.c:8
Is it possible that I can unwind the stack to where the error was triggered:
[4] from 0x00005555555546ce in main+52 at a.c:8
So that I can see what the registers, variables, etc. were at that point? Another way to phrase the question is "How do I ignore things outside my code when inspecting a core dump / gdb" ?
Here's a stab at this:
To travel up or down the callstack, use up|down. In this case we do up 4 to get back to main:
>>> up 4
#4 0x000055555555467e in main () at a.c:7
7 assert (z>5);
info frame and info locals can tell us high-level information about the function:
>>> info locals
z = 2
__PRETTY_FUNCTION__ = "main"
>>> info frame
Stack level 4, frame at 0x7fffffffe0e0:
rip = 0x55555555467e in main (a.c:7); saved rip = 0x7ffff7a05b97
caller of frame at 0x7fffffffe0c0
source language c.
Arglist at 0x7fffffffe0d0, args:
Locals at 0x7fffffffe0d0, Previous frame's sp is 0x7fffffffe0e0
Saved registers:
rbp at 0x7fffffffe0d0, rip at 0x7fffffffe0d8
For example, given the above, we can make a guess that the un-optimized assembly would put z into %rbp-4 and we can examine its value there:
>>> x/d $rbp-4
0x7fffffffe0cc: 2
# or, in long-form to ensure our rbp address above from `info` is the same:
>>> x/d 0x7fffffffe0d0-4
0x7fffffffe0cc: 2

"target record-full" in gdb makes "n" command fail on printf with "Process record does not support instruction 0xc5 at address 0x7ffff7dee6e7"?

I was tring to use "reverse-step" and "reverse-next" command inside gdb. Stack overflow tells me that I should run "target record-full" in the execution context where I wish to "rn" and "rs". But some weird error happened:
1
2 #include<stdio.h>
3 int i=0;
4 void fa()
5 {
6 ++i;
7 printf("%d\n",i);
8 ++i;
9 }
10 int main(){
11 fa();
12 return 0;
13 }
I compile and run this program:
(gdb) b 4
Breakpoint 1 at 0x40052a: file test02.c, line 4.
(gdb) r
Starting program: /home/Troskyvs/a.out
Breakpoint 1, fa () at test02.c:6
6 ++i;
(gdb) target record-full
(gdb) n
7 printf("%d\n",i);
(gdb) n # Error happens here!
Process record does not support instruction 0xc5 at address 0x7ffff7dee6e7.
Process record: failed to record execution log.
Program stopped.
_dl_runtime_resolve_avx () at ../sysdeps/x86_64/dl-trampoline.h:81
81 ../sysdeps/x86_64/dl-trampoline.h: No such file or directory.
Well if I don't run "target record-full", then the 2nd "n" will be OK and run to next line. I don't quite get the error information here.
Is it related to "target record-full"? How can I conquor it?
I tried another approach:
(gdb) set exec-direction reverse
(gdb) n
No more reverse-execution history.
fa () at test02.c:7
7 printf("%d\n",i);
(gdb) n
No more reverse-execution history.
fa () at test02.c:7
7 printf("%d\n",i);
(gdb) n
Well, it doesn't work
AVX is not supported as of GDB 7.11.1
The underlying problem seems to be that AVX instructions are not currently supported, but glibc uses them on Ubuntu 16.04 64-bit:
gdb reverse debugging avx2
https://sourceware.org/ml/gdb/2016-08/msg00028.html
rr is an awesome working alternative: https://github.com/mozilla/rr Here is a minimal working example: Setting breakpoint in GDB where the function returns
Actually for the simple case you have, a record-full should work if you add the parameter "-static" to your gcc compilation command.

Segmentation fault while using MPI_Barrier in `libpmpi.12.dylib`

I install mpich using brew install mpich, but if I use MPI_Barrier, I will get segmentation fault. See the simple code below:
// A.c
#include "mpi.h"
#include <stdio.h>
int main(int argc, char *argv[])
{
int rank, nprocs;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&nprocs);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Barrier(MPI_COMM_WORLD);
printf("Hello, world. I am %d of %d\n", rank, nprocs);fflush(stdout);
MPI_Finalize();
return 0;
}
mpicc A.c -g -O0 -o A
After running mpirun -n 2 ./A, I got error below:
================================================================================== =
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 60914 RUNNING AT pivotal.lan
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault: 11 (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
The detailed stack from lldb -c /cores/core.60914:
(lldb) target create --core "core.60914"
warning: (x86_64) /cores/core.60914 load command 82 LC_SEGMENT_64 has a fileoff + filesize (0x27d3b000) that extends beyond the end of the file (0x27d3a000), the segment will be truncated to match
warning: (x86_64) /cores/core.60914 load command 83 LC_SEGMENT_64 has a fileoff (0x27d3b000) that extends beyond the end of the file (0x27d3a000), ignoring this section
bCore file '/cores/core.60914' (x86_64) was loaded.
(lldb) bt
* thread #1: tid = 0x0000, 0x000000010176f432 libpmpi.12.dylib`MPID_Request_create + 244, stop reason = signal SIGSTOP
* frame #0: 0x000000010176f432 libpmpi.12.dylib`MPID_Request_create + 244
frame #1: 0x000000010178d2fa libpmpi.12.dylib`MPID_Isend + 152
frame #2: 0x0000000101744d6f libpmpi.12.dylib`MPIC_Sendrecv + 351
frame #3: 0x00000001016861df libpmpi.12.dylib`MPIR_Barrier_intra + 401
frame #4: 0x00000001016866f2 libpmpi.12.dylib`MPIR_Barrier + 67
frame #5: 0x0000000101686789 libpmpi.12.dylib`MPIR_Barrier_impl + 90
frame #6: 0x00000001016860fb libpmpi.12.dylib`MPIR_Barrier_intra + 173
frame #7: 0x00000001016866f2 libpmpi.12.dylib`MPIR_Barrier + 67
frame #8: 0x0000000101686789 libpmpi.12.dylib`MPIR_Barrier_impl + 90
frame #9: 0x00000001015a8ed9 libmpi.12.dylib`MPI_Barrier + 820
frame #10: 0x0000000101590ed8 a.out`main(argc=1, argv=0x00007fff5e66fa40) + 88 at b.c:11
frame #11: 0x00007fff8f7805ad libdyld.dylib`start + 1
The usage is copied from official guide. What's the problem of MPI_Barrier function implementation in libmpi.12.dylib? Thanks.

How can I break on multiple clang/ubsan warnings in gdb?

Take the following test program (compiled with clang 3.4 and run under gdb 7.6.1):
#include <limits.h>
#include <stdio.h>
int main(void)
{
int a = INT_MAX + 1;
int b = INT_MAX + 2;
printf("Result: a = %d, b = %d\n", a, b);
}
I would like to be able to use gdb to automatically stop at the second occurrence of undefined behaviour here (int b = ...).
If I compile with:
clang -fsanitize=undefined -O0 -ggdb3 -o test test.c
...then running the program under gdb just results in it running to completion:
test.c:6:21: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
test.c:7:21: runtime error: signed integer overflow: 2147483647 + 2 cannot be represented in type 'int'
Result: a = -2147483648, b = -2147483647
[Inferior 1 (process 24185) exited normally]
But if I use:
clang -fsanitize=undefined-trap -fsanitize-undefined-trap-on-error -O0 -ggdb3 -o test test.c
...then I can't continue past the first occurrence:
Program received signal SIGILL, Illegal instruction.
0x0000000000400556 in main () at test.c:6
6 int a = INT_MAX + 1;
(gdb) c
Continuing.
Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
Is it possible to get gdb to break when (and only when) undefined behaviour is flagged, but to let the program continue otherwise? I'm after a technique that will work not just on this example, but in general, where the offending line might be inside a loop, the values may be determined at runtime, etc.
On x86-64 the instruction that causes SIGILL and stops a program is ud2 (http://asm.inightmare.org/opcodelst/index.php?op=UD2). In order to archive your goal you can change in gdb handling of SIGILL and use jumping (you need to add 2 to $pc on x86_64):
This is how the instruction ud2 is placed in the code of your test program on x86_64:
0x00000000004004f0 <+32>: 0f 85 02 00 00 00 jne 0x4004f8 <main+40>
=> 0x00000000004004f6 <+38>: 0f 0b ud2
0x00000000004004f8 <+40>: b8 ff ff ff 7f mov $0x7fffffff,%eax
These are gdb commands that is necessary to use:
handle SIGILL stop print nopass
set $pc = $pc + 2
This is an example for your test program:
$ gdb -q ./test
Reading symbols from /home/test...done.
(gdb) handle SIGILL stop print nopass
Signal Stop Print Pass to program Description
SIGILL Yes Yes No Illegal instruction
(gdb) r
Starting program: /home/test
Program received signal SIGILL, Illegal instruction.
0x00000000004004f6 in main () at test.c:6
6 int a = INT_MAX + 1;
(gdb) set $pc = $pc + 2
(gdb) c
Continuing.
Program received signal SIGILL, Illegal instruction.
0x000000000040051f in main () at test.c:7
7 int b = INT_MAX + 2;
(gdb) set $pc = $pc + 2
(gdb) c
Continuing.
Result: a = -2147483648, b = -2147483647
[Inferior 1 (process 7898) exited normally]
(gdb)
Useful links:
http://sourceware.org/gdb/current/onlinedocs/gdb/Jumping.html#Jumping
http://sourceware.org/gdb/current/onlinedocs/gdb/Signals.html#Signals

Resources