Can anyone tell me why my program is aborting? I am compiling it with efence which aborts on malloc(0), as the GDB backtrace suggests, the regcomp is doing a malloc(0)
1218 void extractTime(int extractStartTime)
1219 {
1220 char * charPtr, * numberFormatErr;
1221 regex_t re;
1222
1223 ( extractStartTime == 1 ) ? ( charPtr = getenv("EF_ERRTRACK_START") ) :
1224 ( charPtr = getenv("EF_ERRTRACK_END") );
1225
1226 if ( charPtr == NULL )
1227 return;
1228
1229 double envVal = strtod(charPtr, &numberFormatErr);
1230
1231 if ( (numberFormatErr == charPtr) || (*numberFormatErr != '\0') ) {
1232 ( extractStartTime == 1 ) ? EF_Print("eFence exited: EF_ERRTRACK_START is not a number\n") :
1233 EF_Print("eFence exited: EF_ERRTRACK_END is not a number\n");
1234 exit(1);
1235 }
1236 else if ( envVal < 0 ) {
1237 ( extractStartTime == 1 ) ? EF_Print("eFence exited: EF_ERRTRACK_START a negative number\n") :
1238 EF_Print("eFence exited: EF_ERRTRACK_END is a negative number\n");
1239 exit(1);
1240 }
1241
1242 /* If we are here then it is a valid number, now lets check if it is exponential or not */
1243
1244 regcomp(&re, "^([0-9]+[.]?[0-9]*|[0-9]*[.][0-9]+)[eE][+-]?[0-9]+$", REG_EXTENDED);
1245
1246 if ( regexec(&re, charPtr, 0, 0, 0) == 0 )
1247 {
1248 /* It is an exponential number, then already parsed by strtod earlier*/
1249 sprintf(charPtr, "%lf", envVal);
1250 }
1251
Here is the GDB backtrace:
(gdb) r
Starting program: /tmp/efence/ikatrack1_dev
[Thread debugging using libthread_db enabled]
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
ElectricFence Aborting: Allocating 0 bytes, probably a bug.
[New Thread 0x4001e350 (LWP 1528)]
Program received signal SIGILL, Illegal instruction.
[Switching to Thread 0x4001e350 (LWP 1528)]
0x4008734c in kill () from /devel/lib/libc.so.6
(gdb) bt
#0 0x4008734c in kill () from /devel/lib/libc.so.6
#1 0x0000b86c in EF_Abort (pattern=0x1000 <Address 0x1000 out of bounds>)
at print.c:137
#2 0x00009564 in memalign (alignment=4, userSize=0) at efence.c:533
#3 0x0000a5bc in malloc (size=0) at efence.c:1027
#4 0x400fe5bc in re_node_set_alloc (set=0x4025cfd8, size=0)
at regex_internal.c:959
#5 0x400ff2ac in register_state (dfa=0x25, newstate=0x4025cfc8, hash=86528)
at regex_internal.c:1550
#6 0x40102d64 in re_acquire_state_context (err=0xbebd7b88, dfa=0x40196f74,
nodes=0xbebd7b74, context=0) at regex_internal.c:1706
#7 0x4010c060 in re_compile_internal (preg=0xbebd7bf0,
pattern=0xcb74 "^([0-9]+[.]?[0-9]*|[0-9]*[.][0-9]+)[eE][+-]?[0-9]+$",
length=<value optimized out>, syntax=242428) at regcomp.c:989
#8 0x4010d5f8 in __regcomp (preg=0xbebd7bf0,
pattern=0xcb74 "^([0-9]+[.]?[0-9]*|[0-9]*[.][0-9]+)[eE][+-]?[0-9]+$",
cflags=1) at regcomp.c:480
#9 0x0000ae1c in extractTime (extractStartTime=1) at efence.c:1244
#10 0x0000aa1c in efence_ctor () at efence.c:1144
#11 0x0000c528 in __libc_csu_init (argc=1, argv=0xbebd7de4, envp=0xbebd7dec)
at elf-init.c:83
#12 0x40070fe8 in __libc_start_main (main=0x8c54 <main>, argc=1,
ubp_av=0xbebd7de4, init=0xc4d0 <__libc_csu_init>,
---Type <return> to continue, or q <return> to quit---
fini=0xc4c0 <__libc_csu_fini>, rtld_fini=0x4000ea50 <_dl_fini>,
stack_end=0xbebd7de4) at libc-start.c:179
#13 0x00008bcc in _start ()
From GDB backtrace, i can see that the problem lies in frame #4:
#4 0x400fe5bc in re_node_set_alloc (set=0x4025cfd8, size=0) where it is doing a malloc(0) but how come and why is it doing?
Change your electric fence settings (EF_ALLOW_MALLOC_0) to ignore this.
malloc(0) is not illegal. Read the error message again, it says "probably a bug". It aborts so that you will investigate. Once you've determined that it isn't actually a bug, skip it and keep running.
Related
So this is a follow up to my previous Question
I am trying to build a legacy codebase with Address Sanitizer. As soon as I run the binary, I get a Segmentation Fault.
==2940167==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000002aa79a0 at pc 0x000000d6aaf4 bp 0x7fffe7244140 sp 0x7fffe7244130
READ of size 4 at 0x000002aa79a0 thread T3
Thread 1 "block_hk.xcg." received signal SIGSEGV, Segmentation fault.
Here is the trace with gdb:
#0 0x00007ffff6f0dc26 in __sanitizer::internal_strlen(char const*) () from /lib64/libasan.so.5
#1 0x00007ffff6e57208 in printf_common(void*, char const*, __va_list_tag*) () from /lib64/libasan.so.5
#2 0x00007ffff6e59566 in vsnprintf () from /lib64/libasan.so.5
#3 0x00007ffff540ed5c in ?? () from /lib64/libdus.so
#4 0x00007ffff540f017 in ?? () from /lib64/libdus.so
#5 0x00007ffff540f860 in ?? () from /lib64/libdus.so
#6 0x00007ffff541a443 in logPrint () from /lib64/libdus.so
#7 0x00007ffff4761734 in mcsCreateSem () from /lib64/libcwl.so
#8 0x00007ffff4764b14 in mcsInitialize () from /lib64/libcwl.so
#9 0x00007ffff47678fc in cwlAppRunCPP () from /lib64/libcwl.so
#10 0x000000000112d719 in main (argc=1, argv=0x7ffffffe7478) at ./src/main.c:755
I am trying to implement a recursive tree function and I am getting a segfault because the function calls itself over and I would like to see what state my program is at the end.
I would like to see what state my program is at the end.
Run the program under GDB, and it will stop (when your program exhausts its stack) and show you the state at the end.
Example:
void fn(int j)
{
char buf[1024]; // consume some stack space.
buf[0] = 'a';
fn(j - buf[0]); // bad recursion -- should have checked for j - 'a' > 0
}
int main()
{
fn(10000);
return 0;
}
gcc -g t.c && gdb -q ./a.out
(gdb) run
Starting program: /tmp/a.out
Program received signal SIGSEGV, Segmentation fault.
0x0000555555555130 in fn (j=<error reading variable: Cannot access memory at address 0x7fffff7fef8c>) at t.c:2
2 {
(gdb) bt 10
#0 0x0000555555555130 in fn (j=<error reading variable: Cannot access memory at address 0x7fffff7fef8c>) at t.c:2
#1 0x0000555555555158 in fn (j=-759889) at t.c:5
#2 0x0000555555555158 in fn (j=-759792) at t.c:5
#3 0x0000555555555158 in fn (j=-759695) at t.c:5
#4 0x0000555555555158 in fn (j=-759598) at t.c:5
#5 0x0000555555555158 in fn (j=-759501) at t.c:5
#6 0x0000555555555158 in fn (j=-759404) at t.c:5
#7 0x0000555555555158 in fn (j=-759307) at t.c:5
#8 0x0000555555555158 in fn (j=-759210) at t.c:5
#9 0x0000555555555158 in fn (j=-759113) at t.c:5
(More stack frames follow...)
What you see here is the state of the program at the end, i.e. exactly what you asked for in the body of your question, but not in the title (it's a bad form to ask different questions in the body and the title).
Is there a way to break a program after it reaches a certain number of frames?
There is not, but there are various approximations.
For the case of the test program above, the answer is trivial. Since fn keeps calling itself, you can stop on Nth call. E.g. if you want to stop when recursion reaches 100 frames, do this:
(gdb) b fn
Breakpoint 1 at 0x1136: file t.c, line 4.
(gdb) ign 1 100
Will ignore next 100 crossings of breakpoint 1.
(gdb) run
Starting program: /tmp/a.out
Breakpoint 1, fn (j=300) at t.c:4
4 buf[0] = 'a';
(gdb) bt
#0 fn (j=300) at t.c:4
#1 0x0000555555555158 in fn (j=397) at t.c:5
#2 0x0000555555555158 in fn (j=494) at t.c:5
#3 0x0000555555555158 in fn (j=591) at t.c:5
#4 0x0000555555555158 in fn (j=688) at t.c:5
...
#99 0x0000555555555158 in fn (j=9903) at t.c:5
#100 0x0000555555555158 in fn (j=10000) at t.c:5
#101 0x0000555555555169 in main () at t.c:10
Things get more complicated when fn calls itself indirectly (but if the cycle is of fixed length K, then just divide N by K).
If the cycle is not of fixed length, or if some of the calls return (i.e. not every call recurs), then getting to N levels deep is bit more complicated. The easiest way to do that is to modify the program to keep recursion count in a global. Then you can simply set a conditional watch point. Example:
int recursion_count;
void fn(int j)
{
char buf[1024]; // consume some stack space.
recursion_count += 1;
buf[0] = 'a';
fn(j - buf[0]); // bad recursion -- should have checked for j - 'a' > 0
recursion_count -= 1;
}
int main()
{
fn(10000);
return 0;
}
gcc -g t.c && gdb -q ./a.out
(gdb) start
Temporary breakpoint 1 at 0x55555555517d: file t.c, line 15.
Starting program: /tmp/a.out
Temporary breakpoint 1, main () at t.c:15
15 fn(10000);
(gdb) p recursion_count
$1 = 0
(gdb) watch recursion_count
Hardware watchpoint 2: recursion_count
(gdb) cond 2 recursion_count == 100
(gdb) c
Continuing.
Hardware watchpoint 2: recursion_count
Old value = 99
New value = 100
fn (j=397) at t.c:8
8 buf[0] = 'a';
(gdb) bt
#0 fn (j=397) at t.c:8
#1 0x0000555555555167 in fn (j=494) at t.c:9
#2 0x0000555555555167 in fn (j=591) at t.c:9
#3 0x0000555555555167 in fn (j=688) at t.c:9
...
#97 0x0000555555555167 in fn (j=9806) at t.c:9
#98 0x0000555555555167 in fn (j=9903) at t.c:9
#99 0x0000555555555167 in fn (j=10000) at t.c:9
#100 0x0000555555555187 in main () at t.c:15
Voila.
Currently learning heap exploitation but there is something weird I don't understand:
This is the next chunk to be allocated by a call to malloc(0x80):
0x602090: 0x0000000000000000 0x0000000000000091
0x6020a0: 0x00007ffff7dd1b78 0x0000000000601120
...
Calling another malloc(0x80) after this, my goal is to get back 0x601130. This works, but only if 0x601128 == 0x90:
0x601120: 0x0000000000602010 0x0000000000000090
0x601130: 0x0000000000602130 0x00000000006021c0
If I change 90 to any other value, I get memory corruption:
*** Error in `censored': malloc(): memory corruption: 0x00000000006021d0 **
...
#0 0x00007ffff7a42428 in __GI_raise (sig=sig#entry=0x6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff7a4402a in __GI_abort () at abort.c:89
#2 0x00007ffff7a847ea in __libc_message (do_abort=0x2, fmt=fmt#entry=0x7ffff7b9de98 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff7a8f13e in malloc_printerr (ar_ptr=0x7ffff7dd1b20 <main_arena>, ptr=0x6021d0,
str=0x7ffff7b9acff "malloc(): memory corruption", action=<optimized out>) at malloc.c:5006
#4 _int_malloc (av=av#entry=0x7ffff7dd1b20 <main_arena>, bytes=bytes#entry=0x80) at malloc.c:3474
Here is the code in GLIBC_2.2.5, at malloc:3474:
bck = victim->bk;
if (__builtin_expect (chunksize_nomask (victim) <= 2 * SIZE_SZ, 0)
|| __builtin_expect (chunksize_nomask (victim)
> av->system_mem, 0))
malloc_printerr (check_action, "malloc(): memory corruption",
chunk2mem (victim), av);
size = chunksize (victim);
Now, from what I've read and my understanding of this code, the victim chunksize should be bigger than 2 * SIZE_SZ (so bigger than 16 bytes in 64bits) and smaller than av->system_mem. Here av->system_mem is equal to:
gdb-peda$ p av->system_mem
$1 = 0x21000
So I would expect any value between 0x10 and 0x21000 to pass that checks. Why does it not?
I'm trying to run efence on my code, and it always cores here:
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
Program received signal SIGSEGV, Segmentation fault.
memalign (alignment=4, userSize=28) at ../utils/libefence/efence.c:492
492 ../utils/libefence/efence.c: No such file or directory.
in ../utils/libefence/efence.c
(gdb) bt
#0 memalign (alignment=4, userSize=28) at ../utils/libefence/efence.c:492
#1 0xf7ff928c in malloc (size=27) at ../utils/libefence/efence.c:816
#2 0x41c92c67 in operator new(unsigned int) () from /usr/lib/libstdc++.so.6
#3 0x41c78204 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_create(unsigned int, unsigned int, std::allocator<char> const&) () from /usr/lib/libstdc++.so.6
#4 0x41c7a468 in char* std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /usr/lib/libstdc++.so.6
#5 0x41c7a5d6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) ()
from /usr/lib/libstdc++.so.6
#6 0xefb12078 in __static_initialization_and_destruction_0 ()
at ../include/isan/objstoredefs/core/Parameters.h:125
#7 _GLOBAL__sub_I_RecurrWindowPBI.cc(void) ()
at ../dme/svc/common/src/gen/ifc/beh/./imp/trig/RecurrWindowPBI.cc:77
#8 0xefbc9dfd in __do_global_ctors_aux ()
from /isan/lib/libsvc_ifc_behcommon.so
#9 0xefaf59b5 in _init () from /isan/lib/libsvc_ifc_behcommon.so
#10 0x419fd486 in __ctype_init () from /lib/libc.so.6
#11 0x4100ed39 in ?? () from /lib/ld-linux.so.2
#12 0x4100ee8f in ?? () from /lib/ld-linux.so.2
#13 0x410011ef in ?? () from /lib/ld-linux.so.2
(gdb) frame 2
#2 0x41c92c67 in operator new(unsigned int) () from /usr/lib/libstdc++.so.6
I try to run the program using GDB with environment set to LD_PRELOAD for the efence lib.
The fault seems to be here:
491 for ( slot = allocationList, count = slotCount ; count > 0; count-- ) {
492 if ( slot->mode == FREE
493 && slot->internalSize >= internalSize ) {
494 if ( !fullSlot
495 ||slot->internalSize < fullSlot->internalSize){
496 fullSlot = slot;
497 if ( slot->internalSize == internalSize
498 && emptySlots[0] )
499 break; /* All done, */
500 }
501 }
502 else if ( slot->mode == NOT_IN_USE ) {
503 if ( !emptySlots[0] )
504 emptySlots[0] = slot;
505 else if ( !emptySlots[1] )
506 emptySlots[1] = slot;
507 else if ( fullSlot
508 && fullSlot->internalSize == internalSize )
509 break; /* All done. */
510 }
511 slot++;
512 }
But, in GDB, I'm able to dup the slot structure without any issues:
(gdb) p slot
$1 = (Slot *) 0xef846000
(gdb) p slot->mode
$2 = NOT_IN_USE
(gdb)
(gdb) x/10i $eip
=> 0xf7ff9590 <memalign+448>: mov 0x10(%edi),%edx
0xf7ff9593 <memalign+451>: cmp $0x1,%edx
0xf7ff9596 <memalign+454>: jne 0xf7ff95c0 <memalign+496>
0xf7ff9598 <memalign+456>: mov 0xc(%edi),%edx
0xf7ff959b <memalign+459>: cmp %edx,%esi
0xf7ff959d <memalign+461>: ja 0xf7ff95f8 <memalign+552>
0xf7ff959f <memalign+463>: test %ecx,%ecx
0xf7ff95a1 <memalign+465>: je 0xf7ff95a8 <memalign+472>
0xf7ff95a3 <memalign+467>: cmp 0xc(%ecx),%edx
0xf7ff95a6 <memalign+470>: jae 0xf7ff95f8 <memalign+552>
(gdb) p/x $edi
$5 = 0xef846000
(gdb) ptype (*slot)
type = struct _Slot {
void *userAddress;
void *internalAddress;
size_t userSize;
size_t internalSize;
Mode mode;
}
(gdb) p allocationList
$6 = (Slot *) 0xef846000
(gdb) p allocationList[0]
$7 = {userAddress = 0x0, internalAddress = 0x0, userSize = 0,
internalSize = 0, mode = NOT_IN_USE}
Any help?
The efence code is as in:
http://linux.softpedia.com/get/Programming/Debuggers/Electric-Fence-3305.shtml
Thanks!!
So the suspect slot at 0xef846000 SEGV's but is not NULL. The explanation is that with efence the neighboring segments are read/write protected, causing a SIGBUS. But I guess on linux this BUS turns out to be a SEGV instead.
Within the GDB repl those segment read protection bits are disabled/overruled.
So the remaining question is in what memory segment allocationList[0] is located? I guess it's in a fence without read bits. How could that happen? No idea.
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x18d1c070
0x96350250 in strlen ()
(gdb) bt
#0 0x96350250 in strlen ()
#1 0x963574d1 in strdup ()
#2 0x9636e420 in asl_set_query ()
#3 0x9636e2d9 in asl_set ()
#4 0x9636d5d2 in vsyslog ()
#5 0x9636d3e1 in syslog ()
#6 0x23405e78 in gp_log (level=4, fmt=0x23429b68 "%-s:%4d: size: %d\n") at ../../rpc/mac/gp_lib.c:49
#7 0x23404c61 in rpc_encap (out=0xbfffb23c, args=0xbfffd2d0) at ../../rpc/rpc.c:178
#8 0x23405269 in rpc_encap_args (pkt_out=0xbfffb20c, pd=0x2342e460, args=0xbfffd2d0) at ../../rpc/rpc.c:120
#9 0x23405549 in rpc_call_common (c=0x23053048, pd=0x2342e460, args=0xbfffd2d0, timeout=0, pkt_in=0xbfffd30c, size_in=0xbfffd308) at ../../rpc/rpc.c:227
#10 0x234057f9 in rpc_call_actor (pd=0x2342e460, args=0xbfffd2d0, timeout=0, pkt_in=0xbfffd30c, size_in=0xbfffd308) at ../../rpc/rpc.c:204
#11 0x23402839 in MobileIPAPIStopRoaming_w () at ../../MIPSDKv4/MobileIPSDK/MobileIP4_w.c:229
#12 0x23401420 in MobileIPAPIStopRoaming (mipHandle=1) at ../../MIPSDKv3/MobileIPSDKv3.c:300
#13 0x21c0e1a6 in GP_ICM_CONNMGR::CMobileIpSdk::MobileIPAPIStopRoaming ()
#14 0x21c0e6cf in GP_ICM_CONNMGR::CMobileIpSdk::Uninit ()
#15 0x1e749c19 in GP_ICM_RULEMGR::CRulesManager::Uninit ()
#16 0x0004df18 in CConnectionManager::UninitConnectionManager ()
#17 0x0013e6b4 in ICMApplication::UninitICMApplication ()
#18 0x001495b2 in main ()
(gdb) info registers
eax 0xffffffff -1
ecx 0x18d1c07c 416399484
edx 0x18d1c070 416399472
ebx 0x9636e2f0 -1774787856
esp 0xbfffaa4c 0xbfffaa4c
ebp 0xbfffaa78 0xbfffaa78
esi 0x964c6cc0 -1773376320
edi 0x18d1c07c 416399484
eip 0x96350250 0x96350250 <strlen+16>
eflags 0x10286 66182
cs 0x17 23
ss 0x1f 31
ds 0x1f 31
es 0x1f 31
fs 0x0 0
gs 0x37 55
(gdb) frame 1
#1 0x963574d1 in strdup ()
(gdb) frame 2
#2 0x9636e420 in asl_set_query ()
(gdb) frame 3
#3 0x9636e2d9 in asl_set ()
(gdb) frame 4
#4 0x9636d5d2 in vsyslog ()
(gdb) frame 5
#5 0x9636d3e1 in syslog ()
(gdb) frame 6
#6 0x23405e78 in gp_log (level=4, fmt=0x23429b68 "%-s:%4d: size: %d\n") at ../../rpc/mac/gp_lib.c:49
49 syslog(5, "%s", buf);
(gdb) p buf
$5 = "rpc_encap: 178: size: 48\n", '\0' <repeats 998 times>
(gdb) p level
$6 = 4
(gdb) p fmt
$7 = 0x23429b68 "%-s:%4d: size: %d\n"
The gp_log function is this:
#define LOG_DEBUG(format, ...) gp_log(DL_DEBUG, "%-s:%4d: " format, __FUNCTION__, __LINE__, ##__VA_ARGS__)
#define LOG_MAX_LENGTH 1024
void gp_log(int level, const char *fmt, ...)
{
va_list ap;
char buf[LOG_MAX_LENGTH] = {0};
if (level > log_level)
return;
va_start(ap, fmt);
vsnprintf(buf, LOG_MAX_LENGTH, fmt, ap);
va_end(ap);
syslog(level, "%s", buf);
//printf("%s", buf);
}
This statement in rpc_encap print out the contents of an integer, which lead to this crash.
LOG_DEBUG("size: %d\n", size);
The crash is 100% reproduceable. I examine the parameters to syslog() every time before
crash:
buf is always "rpc_encap: 178: size: 48\n".
level is always 4.
If I use printf instead of syslog, the crash disappeared.
In order to build a universal library for Mac 10.5/6/7(Intel arch), I add these flags when compiling on 10.6 or 10.7,
CFLAGS = -force_cpusubtype_ALL -mmacosx-version-min=10.5 -arch i386
and crash happened.
Now I remove these flags and build on 10.5, no more crash(also the library works on 10.5/6/7).