Is there any way how to find out where the person forgot to free() memory?
Valgrind only says how many allocs and frees there were but not where the person didnt free when he should have. Is there any way how to find out the piece of memory that should be freed?
I guess that what you would like is to have valgrind reporting an error
when you 'lose' the last pointer to a piece of memory.
valgrind does not have such a functionality. There used to be an experimental
tool doing that (--tool=omega), but it is not (anymore?) in the valgrind
repository, IIUC, because there was many false positive and/or false negative.
The closest you can do is to use valgrind + gdb (via vgdb), and put
some breakpoints at various places in your program, and do a leak search
at all these places. With a kind of 'dichotomic' search, you might find
the place where you lose the pointer.
See http://www.valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.gdbserver
and http://www.valgrind.org/docs/manual/mc-manual.html#mc-manual.monitor-commands
for more details about using gdb with valgrind and doing interactive
leak searches.
Related
I have a large body of legacy code that I inherited. It has worked fine until now. Suddenly at a customer trial that I cannot reproduce inhouse, it crashes in malloc. I think that I need to add instrumentation e.g on top of malloc I have my own malloc that stores some meta information about each malloc e.g. who has made the malloc call. When it crashes, I can then look up the meta information and see what was happening. I had done something similar years ago but cannot recall it now...I am sure people have come up with better ideas. Will be glad to have inputs.
Thanks
Is memory allocation broken?
Try valgrind.
Malloc is still crashing.
Okay, I'm going to have to assume that you mean SIGSEGV (segmentation fault) is firing in malloc. This is usually caused by heap corruption. Heap corruption, that itself does not cause a segmentation fault, is usually the result of an array access outside of the array's bounds. This is usually nowhere near the point where you call malloc.
malloc stores a small header of information "in front of" the memory block that it returns to you. This information usually contains the size of the block and a pointer to the next block. Needless to say, changing either of these will cause problems. Usually, the next-block pointer is changed to an invalid address, and the next time malloc is called, it eventually dereferences the bad pointer and segmentation faults. Or it doesn't and starts interpreting random memory as part of the heap. Eventually its luck runs out.
Note that free can have the same thing happen, if the block being released or the free block list is messed up.
How you catch this kind of error depends entirely on how you access the memory that malloc returns. A malloc of a single struct usually isn't a problem; it's malloc of arrays that usually gets you. Using a negative (-1 or -2) index will usually give you the block header for your current block, and indexing past the array end can give you the header of the next block. Both are valid memory locations, so there will be no segmentation fault.
So the first thing to try is range checking. You mention that this appeared at the customer's site; maybe it's because the data set they are working with is much larger, or that the input data is corrupt (e.g. it says to allocate 100 elements and then initializes 101), or they are performing things in a different order (which hides the bug in your in-house testing), or doing something you haven't tested. It's hard to say without more specifics. You should consider writing something to sanity check your input data.
Try Asan
AddressSanitizer (aka ASan) is a memory error detector for C/C++. It finds:
Use after free (dangling pointer dereference)
Heap buffer overflow
Stack buffer overflow
Global buffer overflow
Use after return
Use after scope
Initialization order bugs
Memory leaks
Please find the links to know more and how to use it
https://github.com/google/sanitizers/wiki/AddressSanitizer and
https://github.com/google/sanitizers/wiki/AddressSanitizerFlags
I know this is old, but issues like this will continue to exist as long as we have pointers. Although valgrind is the best tool for this purpose, it has a steep learning curve and often the results are too intimidating to understand.
Assuming you are working on some *nux, another tool I can suggest is electricfence. Quote:
Electric Fence helps you detect two common programming bugs:
software that overruns the boundaries of a malloc() memory allocation,
software that touches a memory allocation that has been released by free().
Unlike other malloc() debuggers, Electric Fence will detect read accesses
as well as writes, and it will pinpoint the exact instruction that causes
an error.
Usage is amazingly simple. Just link your code with an additional library lefence
When you run the application, a corefile will be generated when memory is corrupted, instead of when corrupted memory is used.
When using LibreSSL's libtls, is it possible to free the memory that is allocated by tls_init()?
I found using valgrind that a simple program that only calls tls_init() code leaves 2058 blocks of memory "still reachable". I am aware that "still reachable" memory is not strictly a memory leak and it will be cleaned up by the OS when the program exits... But it makes identifying my own memory issues much more obnoxious.
OpenSSL's SSL_library_init() doesn't seem to have the same problem.
I'm on Linux with a source build from the libressl-portable github page.
This question was also asked on the libressl mailing list. The answer is no, at least not with the current libtls API.
The current best alternative for memory testing with valgrind is to use the --suppressions option.
Actually, I know it's necessary to free the memory allocated with malloc(), but I wonder if I can know there is some memory left being not collected if I forget to collect it with free().
Valgrind would be your best bet
http://valgrind.org/
In addition to valgrind answers, you may link your executables against Boehm GC – C garbage collector that may run in leak detection mode.
https://en.wikipedia.org/wiki/Boehm_garbage_collector
http://www.hboehm.info/gc/
http://www.hboehm.info/gc/leak.html
You can use a tool like valgrind. Check out this video on how to use it, courtesy of Harvard's CS50 available on edx. It gives a very good explanation on how to use it, as well as some examples on both correct and incorrect code.
What you are trying to do is impossible. Just keep track of all the memory you allocated and erase it when needed
I have a question out of curiosity relating to checking for memory leaks.
Being someone who has used valgrind frequently to check for memory leaks in my code for the last year or two, I suddenly came to think that it only detects lost/unfreed memory after the life of the program.
So, in light of that, I was thinking that if you have a long-running program which malloc()'s intermittently and doesn't free() until the application exits, then the potential to eat memory (not necessarily through leaks) is huge and isn't observable using these tools because they only check after the programs lifetime. Are there GDB-like tools which can stop an application while running and check for memory which is and isn't referenced at an instance in the life of the application?
Are there GDB-like tools which can stop an application while running and check for memory which is and isn't referenced at an instance in the life of the application?
Yes: Valgrind.
Specifically, the SVN version of Valgrind has a gdbserver stub embedded into it.
This allows you to do all kinds of cool debugging, not possible before:
You can run program under valgrind and have GDB breakpoints at the same time
You can ask valgrind: is this memory allocated? was this variable initialized?
You can ask valgrind: what new leaks happened since last time I asked for leaks?
I think you can also ask it to list not-leaked new allocations.
What I have done, which was not a tool, for a long-running socket-based server, was to do an operation, but before that print out the amount of free memory, then print it out after my operation, and see if there was any difference.
I knew that in theory my server should have returned all memory used on each call to the server, so if I was the only one calling it, it shouldn't use much more memory than when it started.
You may find that some memory was needed on the first call, so you may want to make several calls, so everything is initialized, then you can do checks like this.
The other option is to create a list of all memory you are mallocing, then when you free it delete from the list that node, and at the end, see which ones still haven't been freed.
That is not generally possible in a language that supports pointer arithmetic, since for example - you could cast an pointer to an integer and back. See http://www.cs.umd.edu/class/sum2003/cmsc311/Notes/BitOp/pointer.html
Leaked memory is defined as memory that is not referenced by anything in the program.
If you malloced memory and somewhere in your data there is a pointer pointing to that memory it isn't "lost" as far as any automatic check can now.
However, if you allocated memory, never free'ed it but you don't have any pointer pointing to it you have most likely leaked that memory - as there is no way for you to reference it.
Programs like valgrind can find leaks of the kind described above (lost of reference). AFAIK nothing can find "logical" leaks where you still hold a reference to the memory.
What are some techniques in detecting/debugging memory leak if you don't have trace tools?
Intercept all functions that allocate and deallocate memory (depending on the platform, the list may look like: malloc, calloc, realloc, strdup, getcwd, free), and in addition to performing what these functions originally do, save information about the calls somewhere, in a dynamically growing global array probably, protected by synchronization primitives for multithreaded programs.
This information may include function name, amount of memory requested, address of the successfully allocated block, stack trace that lets you figure out what the caller was, and so on. In free(), remove corresponding element from the array (if there are none, a wrong pointer is passed to free which is also a error that's good to be detected early). When the program ends, dump the remaining elements of the array - they will be the blocks that leaked. Don't forget about global objects that allocate and deallocate resources before and after main(), respectively. To properly count those resources, you will need to dump the remaining resources after the last global object gets destroyed, so a small hack of your compiler runtime may be necessary
Check out your loops
Look at where you are allocating variables - do you ever de-allocate them?
Try and reproduce the leak with a small subset of suspected code.
MAKE trace tools - you can always log to a file.
One possibility could be to compile the code and execute it on a system where you can take advantage of built in tools (e.g. libumem on Solaris, or the libc capability on Linux)
Divide and conquer is the best approach. If you have written you code in a systematic way, it should be pretty easy to call subsets of you code. Your best bet is to execute each section of code over and over and see if your memory usage steadily climbs, if not move on to the next section of code.
Also, the wikipedia article on memory leaks has several great links in the references section on detecting memory leaks for different systems (window, macos, linux, etc)
Similar questions on SO:
Memory leak detectors for C
Strategies For Tracking Down Memory Leaks When You’ve Done Everything Wrong
In addition to the manual inspection techniques mentioned by others, you should consider a code analysis tool such as valgrind.
Introduction from their site:
Valgrind is an award-winning
instrumentation framework for building
dynamic analysis tools. There are
Valgrind tools that can automatically
detect many memory management and
threading bugs, and profile your
programs in detail. You can also use
Valgrind to build new tools.
The Valgrind distribution currently
includes six production-quality tools:
a memory error detector, two thread
error detectors, a cache and
branch-prediction profiler, a
call-graph generating cache profiler,
and a heap profiler. It also includes
two experimental tools: a
heap/stack/global array overrun
detector, and a SimPoint basic block
vector generator. It runs on the
following platforms: X86/Linux,
AMD64/Linux, PPC32/Linux, PPC64/Linux,
and X86/Darwin (Mac OS X).
I have used memtrace
http://www.fpx.de/fp/Software/MemTrace/
http://sourceforge.net/projects/memtrace/
You may need to call the statistics function to printout if there are any leaks. Best thing is to call this statistics function before and after a module or piece of code gets executed.
* Warning * Memtrace is kind enough to allow memory overwrite/double free. It detects these anomalies and gracefully avoids any crash.