I have a program, compiled using xlc_r, that spawns off multiple threads and am trying to trace it to see if there's any memory leaks. I've gone through this article detailing how I can use the MALLOCDEBUG feature that's built in to AIX, but after running format_mallocdebug_op.sh, it shows memory leaks all over the place for random pthread and file methods such as pthread_attri_init, _pth_init, fopen, fwrite, etc.
I then made a smaller test program that purposefully doesn't free a char * and compiled it with xlc_r and almost the exact same output appeared. I then compiled the test program again but with xlc and it worked correctly, showing the one char * memory leak and that was it. It seems that the MALLOCDEBUG feature doesn't work well with multi-threaded compiled applications. Is there a setting to tell it to be aware of this?
Related
We have a C library that we want to distribute, alongside C example code.
The library is of course built in release mode.
The example code project is in cmake so that it can be easily run on both Linux and Windows.
On Linux (debug and release) as well as on Windows (release), we have no issue.
However, on Windows (debug), we have an issue when leaving the main : the program triggers an assertion:
Invalid address specified to RtlValidateHeap
Expression: _CrtIsValidHeapPointer(block)
Then when continuing the process, it raises the following exception:
Unhandled exception at [...] (ntdll.dll)
0xC000000D STATUS_INVALID_PARAMETER
As this seemed related to runtime library, we tryed changing it from MDd (multithreaded dll debug) to MD (multithreaded dll) [ more on these here ] and it solved the issue.
However, this seems like a work around rather than a fix: the release library (built with MD) should be usable in a debug program using MDd, right?
As we understand it, conflicts in runtime library only appears when allocation is made in the caller and deallocation is made in the callee or vice-versa.
So we tracked all allocations to check them and everything seems ok.
We ran leak detections on the example code in both Linux (Valgrind) and Windows (CrtDbg) but they didn't find any leak, everything seems fine.
Is it right to expect a release library built with MD to run in an MDd program?
If not, it seems strange: libraries are always distributed in release, yet used in debug solutions while developing...
If yes, what can cause the issue?
It sounds more like a heap corruption than a leak. It implies someone is overwriting the heap (passed its allocated memory). Finding it can be pain in the but.
First, check your example code. Strip it down to the bare minimum "hello world" and then build it up until it happens again. Then check the example code. If it is not the example code, check which library functions were called and code-review those.
As an aid, you can use the MS heap check functions. place them at function entry and function exit, or maintain global versions that you regularly check. The following is an example:
#include <crtdbg.h>
void example(void)
{
_CrtMemState memStateStart, memStateEnd, memStateDelta;
// Make a checkpoint of the heap's state so we can later check the heap is still OK
_CrtMemCheckpoint( &memStateStart );
//
// do your things
//
// Check the heap
_CrtMemCheckpoint( &memStateEnd );
_CrtSetReportMode( _CRT_WARN, _CRTDBG_MODE_WNDW );
_CrtSetReportMode( _CRT_ERROR, _CRTDBG_MODE_WNDW );
_CrtSetReportMode( _CRT_ASSERT, _CRTDBG_MODE_WNDW );
if (_CrtMemDifference( &memStateDelta, &memStateStart, &memStateEnd ))
_CrtMemDumpStatistics( &memStateDelta );
_CrtDumpMemoryLeaks();
}
Are there any safeguards built into GCC that check for memory leaks? If so how can I use them? When I compile with "gcc -Wall -o run run.c", the compiler does not seem to care if any allocated heap-space is being freed at the end of the code. I could not find any simple fixes for this on Google.
Thanks much for your time.
EDIT:
Google Searches did point to Valgrind among other tools. But I was curious as to why the compiler cant deal with this issue. As a newbie, it seemed a simple enough task to check if every "malloc" has a "free" associated with it.
There are two ways to analyze code for problems - static analysis and run-time analysis. Static analysis reads the code - this is what compilers do really well. Run-time analysis for code problems happens when the code is linked against another set of libraries that see what the code actually does as it runs under surveillance. Finding memory leaks is difficult for static analysis but not for a run-time analysis package.
Other run-time analyses are things like code coverage - does all parts of your code run? gcov does this, like valgrind and electric fence look for memory problems like leaks.
So, no, there are no really good compiler safeguards for testing memory leaks.
There is -fsanitize=leak GCC flag.
It overrides malloc/calloc/free to make them count allocated and freed blocks of memory.
If your program is compiled with this flag, it prints information about detected leaks to the terminal after execution.
You can read about it here and here.
Also, I have never used it, so this answer is completely based on GCC manual.
I have a quite intriguing issue with my program (compiled with gcc 4.6.4 on ubuntu 12.04). When I dynamically build the executable, the program runs flawlessly. But when I build it statically (with -static flag), it gives me a 'core dumped' after exiting (e.g. after 'return 0' in main). Unfortunately, the whole program is too big to pose in here. What are the possibilities?
In addition of the two possibilities in johnnycrash answer:
Some functions with __attribute__ ((destructor)) is called, and dump core.
The memory heap is corrupted (check with valgrind)
Some function registered with atexit(3) is crashing
Some library/function is linked "twice"
1) You have a thread still executing.
2) You are overwriting memory and you get lucky with the dynamic libraries.
For some time now I've been happily using dlmalloc for a cross-platform project (Windows, Mac OS X, Ubuntu). Recently, however, it seems that using dlmalloc leads to a crash-on-exit on Windows 7.
To make sure that it wasn't something goofy in my project, I created a super-minimal test program-- it doesn't do anything but return from main. One version ("malloctest") links to dlmalloc and the other ("regulartest") doesn't. On WinXP, both run fine. On Windows 7, malloctest crashes. You can see screencasts of the tests here.
My question is: why is this happening? Is it a bug in dlmalloc? Or has the loader in Windows 7 changed? Is there a workaround?
fyi, here is the test code (test.cpp):
#include <stdio.h>
int main() {
return 0;
}
and here is the nmake makefile:
all: regulartest.exe malloctest.exe
malloctest.exe: malloc.obj test.obj
link /out:$# $**
regulartest.exe: test.obj
link /out:$# $**
clean:
del *.exe *.obj
For brevity, I won't include the dlmalloc source in this post, but you can get it (v2.8.4) here.
Edit: See these other relavent SO posts:
Is there a way to redefine malloc at link time on Windows?
Globally override malloc in visual c++
Looks like a bug in the C runtime. Using Visual Studio 2008 on Windows 7, I reproduced the same problem. After some quick debugging by putting breakpoints in dlmalloc and dlfree, I saw that dlfree was getting called with an address that it never returned earlier from dlmalloc, and then it was hitting an access violation shortly thereafter.
Thankfully, the C runtime's source code is distributed along with VS, so I could see that this call to free was coming from the __endstdio function in _file.c. The corresponding allocation was in __initstdio, and it was calling _calloc_crt to allocate its memory. _calloc_crt calls _calloc_impl, which calls HeapAlloc to get memory. _malloc_crt (used elsewhere in the C runtime, such as to allocate memory for the environment and for argv), on the other hand, calls straight to malloc, and _free_crt calls straight to free.
So, for the memory that gets allocated with _malloc_crt and freed with _free_crt, everything is fine and dandy. But for the memory that gets allocated with _calloc_crt and freed with _free_crt, bad things happen.
I don't know if replacing malloc like this is supported -- if it is, then this is a bug with the CRT. If not, I'd suggest looking into a different C runtime (e.g. MinGW or Cygwin GCC).
Using dlmalloc in cross-platform code is an oxymoron. Replacing any standard C functions (especially malloc and family) results in undefined behavior. The closest thing to a portable way to replace malloc is using search-and-replace (not #define; that's also UB) on the source files to call (for example) my_malloc instead of malloc. Note that internal C library functions will still use their standard malloc, so if the two conflict, things will still blow up. Basically, trying to replace malloc is just really misguided. If your system really has a broken malloc implementation (too slow, too much fragmentation, etc.) then you need to do your replacement in an implementation-specific way, and disable the replacement on all systems except ones where you've carefully checked that your implementation-specific replacement works correctly.
Is it possible to tell valgrind to ignore some set of libraries?
Specifically glibc libraries..
Actual Problem:
I have some code that runs fine in normal execution. No leaks etc.
When I try to run it through valgrind, I get core dumps and program restarts/stops.
Core usually points to glibc functions (usually fseek, mutex etc).
I understand that there might be some issue with incompatible glibc / valgrind version.
I tried various valgrind releases and glibc versions but no luck.
Any suggestions?
This probably doesn't answer your question, but will provide you the specifics of how to suppress certain errors (which others have alluded to but have not described in detail):
First, run valgrind as follows:
valgrind --gen-suppressions=all --log-file=valgrind.out ./a.out
Now the output file valgrind.out will contain some automatically-generated suppression blocks like the following:
{
stupid sendmsg bug: http://sourceware.org/bugzilla/show_bug.cgi?id=14687
Memcheck:Param
sendmsg(mmsg[0].msg_hdr)
fun:sendmmsg
obj:/usr/lib/libresolv-2.17.so
fun:__libc_res_nquery
obj:/usr/lib/libresolv-2.17.so
fun:__libc_res_nsearch
fun:_nss_dns_gethostbyname4_r
fun:gaih_inet
fun:getaddrinfo
fun:get_socket_fd
fun:main
}
Where "stupid sendmsg bug" and the link are the name that I added to refer to this block. Now, save that block to sendmsg.supp and tell valgrind about that file on the next run:
valgrind --log-file=valgrind --suppressions=sendmsg.supp ./a.out
And valgrind will graciously ignore that stupid upstream bug.
As noted by unwind, valgrind has an elaborate mechanism for controlling which procedures are instrumented and how. But both valgrind and glibc are complicated beasts, and you really, really, really don't want to do this. The easy way to get a glibc and valgrind that are mutually compatible is to get both from the Linux distro of your choice. Things should "just work", and if they don't, you have somebody to complain to.
Yes, look into Valgrind's suppression system.
You probably want to ask about this on the Valgrind user's mailing list (which is extremely helpful). You can suppress output from certain calls, however, suppressing the noise is all you are doing. The calls are still going through Valgrind.
To accomplish what you need, you (ideally) match Valgrind appropriately with glibc or use the macros in valgrind/valgrind.h to work around them. Using those, yes, you can tell valgrind not to touch certain things. I'm not sure what calls are borking everything, however you can also (selectively) not run bits of code in your own program if its run under valgrind. See the RUNNING_ON_VALGRIND macro in valgrind/valgrind.h.
The other thing that comes to mind is to make sure that Valgrind was compiled correctly to deal with threads. Keep in mind that atomic operations under Valgrind could cause your program to crash during racey operations, where it otherwise might not, if not properly configured.
If you have been swapping versions of valgrind and glibc, there's a chance you found a match, but incorrectly configured valgrind at build time.