Assume I have a C program (running under Linux) which manipulates many data structures, some complex, several of which can grow and shrink but should not in general grow over time. The program is observed to have a gradually increasing RSS over time (more so than can be explained by memory fragmentation). I want to find what is leaking. Running under valgrind is the obvious suggestion here, but valgrind (with --leak-check=full and --show-reachables=yes) shows no leak. I believe this to be because the data structures themselves are correctly being freed on exit, but one of them is growing during the life of the program. For instance, there might be a linked list which is growing linearly over time, with someone forgetting to remove the resource on the list, but the exit cleanup correctly freeing all the items on the list at exit. There is a philosophical question as to whether these are in fact 'leaks' if they are freed, of course (hence the quote marks in the question).
Are there any useful tools to instrument this? What I'd love is the ability to run under valgrind and have it produce a report of current allocations just as it does on exit, but to have this happen on a signal and allow the program to continue. I could then look for what stack trace signatures had growing allocations against them.
I can reliably get a nice large 'core' file from gdb with generate-core-file; is there some way to analyse that off-line, if say I compiled with a handy malloc() debugging library that instrumented malloc()?
I have full access to the source, and can modify it, but I don't really want to instrument every data structure manually, and moreover I'm interested in a general solution to the problem (like valgrind provides) rather than how to address this particular issue.
I've looked for similar questions on here but they appear all to be:
Why does my program leak memory?
How do I detect memory leaks at exit? (no use for me)
How do I detect memory leaks from a core file? (great, but none has a satisfactory answer)
If I was running under Solaris I'm guessing the answer would be 'use this handy dtrace script'.
Valgrind includes a gdbserver. This basically means you can use gdb to connect to it, and e.g. issue a leak dump, or to show all reachable memory while running. Ofcourse, you have to judge whether there is a "memory leak" or not, as valgrind can't know if there's a bug in the application logic that fails to release memory, but still keep references to it.
Run valgrind with the --vgdb=yes flag and then run the commands:
valgrind --vgdb=yes --leak-check=full --show-reachable=yes ./yourprogram
gdb ./yourprogram
(gdb) target remote | vgdb
(gdb) monitor leak_check full reachable any
See the docs for more info, here and here
You can also do this programatically in your program
#include <valgrind/memcheck.h>
and at an appropriate place in the code do:
VALGRIND_DO_LEAK_CHECK;
(iirc that'll show reachable memory too, as long as valgrind is run with --show-reachable=yes
There's the Valgrind Massif tool which shows general memory usage of your application, not just for leaked memory. It breaks down malloc()s and free()s by calling functions and their backtraces, so you can see which functions keep allocating memory without releasing it. This can be an excellent tool for finding leaks of the type you mentioned.
Unfortunately the tooling around Massif is a bit weird... The ms_print tool provided with Valgrind is only useful for the most basic tasks; for real work you probably want something that displays a graph. There are several tools for this strewn around the net - see eg. Valgrind Massif tool output graphical interface? .
Related
I got a core from g_ascii_strdown() which said it got error while doing malloc() and it crashed. So if the process hit out-of-memory case, is there a way to find out, from core and gdb, which part of program (caller, stack or thread) allocated memory ?
In other words, If I have the core file. What kind of information exist in core file that I can use to find out memory allocations ?
Generally speaking, malloc does not save any book-keeping information that would allow you to figure out what parts of your program allocated the memory. Since that information isn't in the live process, it won't be in the core-dump either.
If you want to figure that out, you'll need to use some malloc debug library/program instead. I'd normally recommend Valgrind for this, since it contains an excellent malloc debugger, but Valgrind only works on live processes. There might be some alternative malloc library that you can link against that keeps such information around in such a way that it would also be saved in core-dumps, but if there is, I can't say I know about it.
Smells like some memory leak. Use valgrind on some reproducible run exhibiting the issue (you'll need to start the program again under valgrind). You might not be able to find the issue post-mortem using the core dump alone
BTW, the glib running documentation also suggests calling g_mem_set_vtable (glib_mem_profiler_table) very early at startup (at the beginning of your main)
See also the Boehm conservative garbage collector
At last, it is not important to know which part of the memory exhausted. The available memory is a whole process thing. When running your program of pid 1234, use cat /proc/1234/maps & see proc(5)
I have multithreading program(using ucontext routines) in which I have allocated the stacks for child threads using malloc().Actual problem arises, whenever I try to free the memory allocated for that thread(core dump).
I suspect that memory block for the particular thread is being corrupted by the stack growth of some other thread/ or on the other hand it could be the logical error in the code.
To verify that, I want to track the changes done to the memory block, so that I can be 100% sure about my hypothesis.
Using mtrace, I am not able to achieve what I wanted.
Any suggestions?
Use Valgrind ..
It's a professional tool for tracking running programs specializing in tracing heap allocations, threads variables and tracing child processes as well ..
If I understand your problem, then the --verbose, the --check-stack-vars and the --read-var-info options will be particularly helpful for you ..
I've written a debugger using ptrace(2) that is mainly for auditing system calls and redirecting standard IO of a child process. I would also like to detect memory leaks using this debugger.
I thought that it might be as easy as counting references to the system call brk(2), but it isn't. Unfortunately (or fortunately), Linux seems to call brk(2) at the end of the program regardless of whether or not the memory was properly freed.
I've seen this in a program that calls malloc(3) and free(3) and a program that just calls malloc(3) - they both have equal counts of brk(2) calls by the time the program has called exit_group(2), which is happens on return (perhaps I could be interpreting those results incorrectly?).
Or, perhaps exit_group(2) isn't equivalent to 'return' from main, and I should be setting a different break point for auditing the call count of brk(2).
I found a similar question here, but I still haven't found an answer.
I understand that Valgrind is a perfect tool for this, but it would cause considerable overhead.
Does anyone have helpful information on detecting memory leaks with ptrace(2)? Is possible with ptrace(2)? Is there a more practical way? Is there an API for memory debugging child processes?
Edit:
If there's other functions involved with allocate memory, I would count those too. On the page for malloc, it says that mmap(2) is also used in memory allocation. So, I would be counting that too.
Use gdb's heap extension. It will do what you want. IF you want to use it programatically, just pipe the results to your application to do post processing:
https://fedorahosted.org/gdb-heap/
I am using valgrind to find memory leaks on my program however it is taking a long time and its loading. When I run the program without valgrind it takes second, what is the problem and what should I look for in the code.
There is no problem as far as I can see unless you can verify an infinite loop or some other run-time error ... Valgrind basically acts like a virtual machine or virtual execution environment running the program, watching all variables, memory allocations, etc., etc. and therefore will run quite a bit slower than native code. You'll get the same effect if you ran your program inside a debugger like gdb and set it to watch every writable memory location.
I am porting a game built in c to use opengl for porting reasons. Everything was going fine until, for some reasons, the malloc functions of the game stopped working.
I has been searching for an answer to this. The only thing I found about it is that malloc is returning NULL. A very simple malloc that creates a simple structure of a few bytes.
I made a some tries and I wrote the command inside a loop. And my surprise was that it failed the first 10 times, but after that the command worked and the game started. The structure has been created several times after that with no problem.
I would say to leave it as it is, but I am sure there is something wrong behind and in other systems may not work correctly. That's why I want to ask what kind of problem I am having, why and how to avoid it. The environment is not very friendly and I lack of debugging tools, so I would appreciate as many details as possible.
thank you.
EDIT:
The code is simple.
ObjectBase* newcoin;
newcoin= (ObjectBase*)calloc(1,sizeof(ObjectBase));
while the ObjectBase structure is very small. No more than 200 bytes.
The hardware is a windows 7 based computer with 4 GB or memory. Even with this, the error is:
Visual C++ CRT: Not enough memory to complete call to strerror
It worked fine before.
More problems. I am having the same problem when loading files from disk. It happens about 10 to 15 times and then works correctly.
The problem happens always. Each time I restart the application, it crashes.
Use valgrind to debug possible memory leaks in your program.
If you checked that your program is not leaking or misbehaving w.r.t. malloc, it may happen that malloc returns NULL because no memory is available.
It could even happen that malloc returns NULL, then some third-party library releases some memory with free, then malloc could give you again some fresh memory.
You might also consider using Boehm's conservative garbage collector that is use GC_malloc instead of malloc and not bother much about free-ing memory.
If on Linux, use the /proc pseudo-file system to learn about your process (e.g. /proc/1234/maps for process of pid 1234, also /proc/self/maps from inside the process, and /proc/self/statm etc). See also the pmap command.