I am currently working on a huge program written by another version. Over the last days I implemented some new features, but today I realized that this does not work correctly with some old features.
I used the lldb debugger on the program and get the following error message:
MT(3934,0x7fff797e3310) malloc: *** error for object 0x307400650: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Process 3934 stopped
* thread #1: tid = 0x52b27, 0x00007fff92e24866 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
frame #0: 0x00007fff92e24866 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill + 10:
-> 0x7fff92e24866: jae 0x7fff92e24870 ; __pthread_kill + 20
0x7fff92e24868: movq %rax, %rdi
0x7fff92e2486b: jmpq 0x7fff92e21175 ; cerror_nocancel
0x7fff92e24870: ret
After moving up some levels I can see:
frame #6: 0x000000010000e602 MT`free_matrix(m=0x0000000307400520, nrl=1, nrh=7, ncl=1, nch=7) + 66 at nrutil.c:282
(lldb) up
frame #7: 0x000000010000b925 MT`equations + 1221 at equations.c:619
616
617 //Free memory for next round or final
618 free_vector(B,1,n);
-> 619 free_matrix(A,1,n,1,n);
620
If you want I could post the function for free_matrix(..) and the function which allocates the matrix but its basically malloc and free.
Now I definitely don't want anybody to give a solution. I am just asking how you would proceed in this case.
So my main question is: Is the error definitely tied to the matrix A or may there be other details I should search for?
Related
When running my application, I receive a segmentation fault then a window pops up letting me know that:
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0x0000000128108eb3 WP_ForcePowerRun(gentity_s*, forcePowers_t, usercmd_s*) + 4435
0x0000000128107c15 WP_ForcePowersUpdate(gentity_s*, usercmd_s*) + 501
0x0000000128033bc1 ClientThink_real(gentity_s*, usercmd_s*) + 1153
0x0000000128035843 ClientThink(int, usercmd_s*) + 995
0x00000001280bf68a NPC_Think(gentity_s*) + 1786
0x000000012805ef4a G_RunFrame(int) + 1418
0x000000010d28b4d0 SV_Frame(int, float) + 576
0x000000010d266c4c Com_Frame() + 588
0x000000010d2d7525 main + 389
0x00007fff2046af3d start + 1
So I have two questions:
What are the numbers on the right side of each function i.e +4435, +501, etc.?
Is there any way to determine the specific line number where the segmentation fault is happening?
Those are the offset, aka how many bytes after the beginning of the function the fault happened.
You can compile with debug symbols and use a debugger like GDB.
I want to programmatically convert backtrace stack addresses (eg obtained from backtrace_symbols/libunwind) to file:line:column. I'm on OSX but doubt this makes a difference.
All of these give wrong line number (line 11) for the call to fun1():
atos
addr2line
llvm-symbolizer
lldb image lookup --address using lldb's pc addresses in bt
lldb bt itself gives correct file:line:column, (line 7) as shown below.
How do I programmatically get the correct stack address such that, when using atos/addr2line/llvm-symbolizer/image lookup --address, it would resolve to the correct line number? lldb bt is doing it correctly, so there must be a way to do it. Note that if I use backtrace_symbols or libunwind (subtracted from info.dli_saddr after calling dladdr), I'd end up with the same address 0x0000000100000f74 as shown in lldb bt that points to the wrong line number 11
Note: in .lldbinit, if I add settings set frame-format frame start-addr:${line.start-addr}\n it will show the correct address (ie resolve to 0x0000000100000f6f instead of 0x0000000100000f74, which will resolve to the correct line 7). However, how do I programmatically generate start-addr from a c program without calling spawning a call to lldb -p $pid (calling lldb has other issues, eg overhead compared to llvm-symbolizer, and in fact can hang forever even with -batch).
clang -g -o /tmp/z04 test_D20191123T162239.c
test_D20191123T162239.c:
void fun1(){
}
void fun1_aux(){
int a = 0;
fun1(); // line 7
mylabel:
if(1){
a++; // line 11
}
}
int main(int argc, char *argv[]) {
fun1_aux();
return 0;
}
lldb /tmp/z04
(lldb) target create "/tmp/z04"
Current executable set to '/tmp/z04' (x86_64).
(lldb) b fun1
Breakpoint 1: where = z04`fun1 + 4 at test_D20191123T162239.c:2:1, address = 0x0000000100000f54
(lldb) r
Process 7258 launched: '/tmp/z04' (x86_64)
Process 7258 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x0000000100000f54 z04 fun1 + 4 at test_D20191123T162239.c:2:1
1 void fun1(){
-> 2 }
3
4 void fun1_aux(){
5 int a = 0;
6
7 fun1();
Target 0: (z04) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
* frame #0: 0x0000000100000f54 z04 fun1 + 4 at test_D20191123T162239.c:2:1
frame #1: 0x0000000100000f74 z04 fun1_aux + 20 at test_D20191123T162239.c:7:3
frame #2: 0x0000000100000fab z04 main(argc=1, argv=0x00007ffeefbfb748) + 27 at test_D20191123T162239.c:16:3
frame #3: 0x00007fff71c182e5 libdyld.dylib start + 1
frame #4: 0x00007fff71c182e5 libdyld.dylib start + 1
(lldb)
(lldb) image lookup --address 0x0000000100000f74
Address: z04[0x0000000100000f74] (z04.__TEXT.__text + 36)
Summary: z04`fun1_aux + 20 at test_D20191123T162239.c:11:8
echo 0x0000000100000f74 | llvm-symbolizer -obj=/tmp/z04
fun1_aux
test_D20191123T162239.c:11:8
atos -o /tmp/z04 0x0000000100000f74
fun1_aux (in z04) (test_D20191123T162239.c:11)
likewise with addr2line
It's easier to understand if you look at the disassembly for fun1_aux -- you'll see a CALLQ instruction to fun1, followed by something like a mov %rax, $rbp-16 or something like that, the first instruction of your a++ line. When you have called fun1, the return address is the instruction that will be executed when fun1 exits, the mov %rax, $rbp-16 or whatever.
This isn't intuitively how most people think of the computer working -- they expect to look at frame 1, fun1_aux, and see the "current pc value" be the CALLQ, because the call is executing. But of course, that's not correct, the call instruction has completed, and the saved pc is going to point to the next instruction.
In cases like this, the next instruction is part of the next source line, so it's a little extra confusing. Even better is if you have a function that calls a "noreturn" function like abort() -- the final instruction in the function will be a CALLQ, and if you look at the return address instruction, it may point to the next function.
So when lldb is symbolicating stack frames above frame #0, it knows to do a symbol lookup with saved_pc - 1 to move the address back into the CALLQ instruction. That's not a valid address, so it should never show you saved_pc - 1, but it should do symbol / file & line lookups based on it.
You can get the same effect for your manual symbolication by doing the same thing. The one caveat is if you have an asynchronous interrupt (_sigtramp on macOS), the frame above _sigtramp should not have its saved pc value decremented. You could be executing the first instruction of a function when the signal is received, and decrementing it would put you in the previous function which would be very confusing.
I am trying to do a buffer overflow for a project. The buffer needs to overflow to /bin/sh. I have found the correct return address, but I do not seem to be successfully getting an overflow.
Program received signal SIGSEGV, Segmentation fault.
0xb7fbc544 in msg (params) at myfile.c:167
167 msg_length = ctx->backend->send_msg_pre(msg, msg_length);
(gdb) backtrace
#0 0xb7fbc544 in msg (params) at myfile.c:167
#1 0xb7fbc869 in my_function(params) at myfile.c:912
#2 0xb7e4c190 in ?? () at ../sysdeps/unix/sysv/linux/system.c:76 from /lib/i386-linux-gnu/libc.so.6
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
To find the /bin/sh I followed this post:
(gdb) print &system
$15 = (<text variable, no debug info> *) 0xb7e4c190 <__libc_system>
(gdb) find &system,+9999999,"/bin/sh"
0xb7f6ca24
warning: Unable to access 16000 bytes of target memory at 0xb7fc292c, halting search.
1 pattern found.
(gdb)
I have found the return address and verified it is correct (if I replace it with a different function it calls that function). The original return address is bolded
0xbffff690: 0x90909090 0x90909090 0x90909090 0xb7e4c190
0xbffff6a0: 0xb7f6ca24 0x0804c050 0x0000004d 0x0804c158
The payload I sent was 0xb7e4c190 0xb7f6ca24 50
This overflow is a bit more tricky than others, because I need to do some padding to it in the front and the back. The way the item I am overflowing works is that it sets 6 bytes, so each set will take up a portion of one address and another.
I think the problem is that I have to overflow the last byte. I matched what it was originally, but that does not seem to work:
payload = payload + ['\x90'] + ['\xc1'] + ['\xe4'] + ['\xb7']
payload = payload + ['\x24'] + ['\xca'] + ['\xf6'] + ['\xb7'] + ['\x50']
Is there something I am missing here? Because the stack does not show the /bin/sh call in the overflow, I feel like that is not correct.
I'm new to reverse engeneering. I wrote the following C code to help me understand a bit more about stack frames.
#include <stdio.h>
int sum(int a, int b,int c)
{
return(a+b+c);
}
int media(int a, int b,int c)
{
int total;
total = a + b + c;
return (total/3);
}
int main ()
{
int num1,num2,num3;
char keypress[1];
num1 = 5;
num2 = 10;
num3 = 15;
printf ("\nCalling sum function\n");
sum(num1,num2,num3);
printf ("\nWaiting a keypress to call media function\n");
scanf ("%c",keypress);
media(num1,num2,num3);
printf ("\nWaiting a keypress to end\n");
scanf ("%c",keypress);
return(0);
}
As far as I know every time you call a function
a stack frame is created (see: ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_41.html). So, my goal with the above C code is to see, at least, three stack-frames.
1) main function - stack frame
2) sum function - stack frame
3) media function - stack frame
BTW: Those printfs are just to help me 'follow' the program in gdb. =)
So I guess if I compare the output of info frame after the program started with the output of info frame just after sum function is called I would get different output right? I did not got it as you can see:
Temporary breakpoint 1, main () at parastack.c:27
warning: Source file is more recent than executable.
27 num1 = 5;
(gdb) nexti
28 num2 = 10;
(gdb) info frame
Stack level 0, frame at 0x7fffffffdf00:
rip = 0x400605 in main (parastack.c:28); saved rip = 0x7ffff7a3c790
source language c.
Arglist at 0x7fffffffdef0, args:
Locals at 0x7fffffffdef0, Previous frame's sp is 0x7fffffffdf00
Saved registers:
rbp at 0x7fffffffdef0, rip at 0x7fffffffdef8
(gdb) nexti
29 num3 = 15;
(gdb) nexti
31 printf ("\nCalling sum function\n");
(gdb) nexti
0x0000000000400618 31 printf ("\nCalling sum function\n");
(gdb) nexti
Calling sum function
32 sum(num1,num2,num3);
(gdb) info frame
Stack level 0, frame at 0x7fffffffdf00:
rip = 0x40061d in main (parastack.c:32); saved rip = 0x7ffff7a3c790
source language c.
Arglist at 0x7fffffffdef0, args:
Locals at 0x7fffffffdef0, Previous frame's sp is 0x7fffffffdf00
Saved registers:
rbp at 0x7fffffffdef0, rip at 0x7fffffffdef8
(gdb) nexti
0x0000000000400620 32 sum(num1,num2,num3);
(gdb) info frame
Stack level 0, frame at 0x7fffffffdf00:
rip = 0x400620 in main (parastack.c:32); saved rip = 0x7ffff7a3c790
source language c.
Arglist at 0x7fffffffdef0, args:
Locals at 0x7fffffffdef0, Previous frame's sp is 0x7fffffffdf00
Saved registers:
rbp at 0x7fffffffdef0, rip at 0x7fffffffdef8
just after sum function is called
Your problem is that you never actually stopped inside of the sum function. You stopped after you printed that you are about to call it, and then you stepped a few instructions, but you never actually landed inside (it takes a few instructions to prepare arguments, one more to actually call, and few more inside the function to set up the frame).
You should start by setting breakpoints inside sum and media, and doing info frame when these breakpoints are hit. You'll notice that the breakpoint is set a few instructions after the beginning of the function (i.e. after function prolog). The skipped instructions are exactly the ones that set up the new frame.
After you understand how that works, you should progress to using step and next commands.
And after that you can graduate to using disas, stepi and nexti commands.
Based on my interpretation of your prose, your understanding of stack frames is slightly off. You are correct that when a function is called a stack frame is created, however, what you're missing is that when a function returns, the stack frame is popped. The stack is in the same state is was before the function began executing except that the program counter contains the address of the first instruction of the statement immediately following the function that just finished executing. So, you should not expect to see 3 stack frames after the two functions in main execute. You will only see one since you're only one frame deep into main().
As for the gdb session, as #Employed Russian points out, you never actually step into any function when printing the stack frame information.
Thanks for everyone that helped me. Below are the gdb session with shows that the stack-frame changed.
First I recompiled the C code: gcc -ggdb stack.c -o stack.bin
gdb stack.bin
(gdb) break sum
(gdb) start
(gdb) info frame
Stack level 0, frame at 0x7fffffffe1a0:
rip = 0x400653 in main (stack.c:26); saved rip 0x7ffff7a6fead
source language c.
Arglist at 0x7fffffffe190, args:
Locals at 0x7fffffffe190, Previous frame's sp is 0x7fffffffe1a0
Saved registers:
rbp at 0x7fffffffe190, rip at 0x7fffffffe198
(gdb) continue
Continuing.
Calling sum function
Breakpoint 1, sum (a=5, b=10, c=15) at stack.c:6
6 total = a + b + c;
(gdb) info frame
Stack level 0, frame at 0x7fffffffe180:
rip = 0x4005dd in sum (stack.c:6); saved rip 0x400684
called by frame at 0x7fffffffe1a0
source language c.
Arglist at 0x7fffffffe170, args: a=5, b=10, c=15
Locals at 0x7fffffffe170, Previous frame's sp is 0x7fffffffe180
Saved registers:
rbp at 0x7fffffffe170, rip at 0x7fffffffe178
Now I will search/learn more about the information in the output.
I have a very frustrating problem. My application runs on a few machines flawlessly for a month.
However, there is one machine on which my application crashes nearly every day because of segfault.
It always crashes at the same instruction address:
segfault at 7fec33ef36a8 ip 000000000041c16d sp 00007fec50a55c80 error 6 in myapp[400000+f8000]
This address points to memcpy call.
Below, there is an excerpt #1 from my app:
....
uint32_t size = messageSize - sizeof(uint64_t) + 1;
stack->trcData = (char*)Realloc(stack->trcData,(stack->trcSize + size + sizeof(uint32_t)));
char* buffer = stack->trcData + stack->trcSize;
uint32_t n_size = htonl(size);
memcpy(buffer,&n_size,sizeof(uint32_t)); /* ip 000000000041c16d points here*/
buffer += sizeof(uint32_t);
....
stack->trcSize += size + sizeof(uint32_t);
....
where stack is a structure:
struct Stack{
char* trcData;
uint32_t trcSize;
/* ... some other elements */
};
and Realloc is a realloc wrapper:
#define Realloc(x,y) _Realloc((x),(y),__LINE__)
void* _Realloc(void* ptr,size_t size,int line){
void *tmp = realloc(ptr,size);
if(tmp == NULL){
fprintf(stderr,"R%i: Out of memory: trying to allocate: %lu.\n",line,size);
exit(EXIT_FAILURE);
}
return tmp;
}
messageSize is of uint32_t type and its value is always greater than 44 bytes. The code #1 runs in a loop. stack->trcData is just a buffer which collects some data until some condition is fulfilled. stack->trcData is always initialized to NULL. The application is compiled with gcc with optimization -O3 enabled. When I run it in gdb, of course it did not crash, as I expected;)
I ran out of ideas why myapp crashes during memcpy call. Realloc returns with no error, so I guess it allocated enough space and I can write to this area. Valgrind
valgrind --leak-check=full --track-origins=yes --show-reachable=yes myapp
shows absolutely no invalid reads/writes.
Is it possible that on this particular machine the memory itself is corrupted and it causes these often crashes? Or maybe I corrupt memory somewhere else in myapp, but if this is the case, why it does not crash earlier, when the invalid write is made?
Thanks in advance for any help.
Assembly piece:
41c164: 00
41c165: 48 01 d0 add %rdx,%rax
41c168: 44 89 ea mov %r13d,%edx
41c16b: 0f ca bswap %edx
41c16d: 89 10 mov %edx,(%rax)
41c16f: 0f b6 94 24 47 10 00 movzbl 0x1047(%rsp),%edx
41c176: 00
I'm not sure whether this information is relevant but all the machines, my application runs on successfully, have Intel processors whilst the one causing the problem has AMD.
Here is the cause of my problem.
The point is that at some loop step stack->trcSize + size exceeds UINT32_MAX. That means Realloc in fact shrinks stc->trcData. Next, I define buffer which now is far behind the allocated area. Hence, when I write to buffer I get segfault.
I've checked it and it was indeed the cause.