I'm looking for a tool which can help me to generate a memory accesses graph. I'm trying to optimize a search algorithm (written in c) and it would be very useful to know how the memory accesses are performed in order to optimize the memory accesses pattern.
I heard that cachegrind from valgrind could help me, but I think is not exactly what I'm looking for as I think it doesn't generate a trace of memory accesses and I already know the other information that can show me using the PAPI library.
The graph I want to generate has in the x axis the memory access order and in the y axis the memory address space (absolute or relative memory directions)
If you're using gcc or the gnu std C library, the malloc and free hooks might help. https://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html#Hooks-for-Malloc
Some tools in this project do what you want:
http://www.inf.usi.ch/faculty/hauswirth/research/TraceVisualization.html
I don't know if you can download them anywhere or ask them to share a copy, I used them because I was a student of this professor!
Related
I'm working on an embedded program. I use the avr-gcc tool chain to compile the C source from my MacBook Pro. Until recently things have been going pretty well. In my latest development iteration though, I seem to have introduced some sort of intermittent bug that I'm suspecting is some sort of stack or other memory corruption error.
I've never used Valgrind, but it seems it gets rave reviews, but most of the references seem to refer to malloc/free types of errors. I don't do any malloc'ing. It's a smallish embedded program, no OS. Can Valgrind help me? Any pointers on how I would use it to help find static memory mismanagement errors in a cross-compiled scenario would be really helpful!
Or is there a different tool or technique I should look at to validate my code's memory management?
Yes, valgrind can definitely help you. In addition to a lot of heap-based analysis (illegal frees, memory leaks, etc.) its memcheck tool detects illegal reads and writes, i.e. situations when your program accesses memory that it should not access. This analysis does not differentiate between static and dynamic memory: it would report accesses outside of a stack frame, accesses beyond bounds of a static array, and so on. It also detects access to variables that have not been onitialized previously. Both situations are undefined behavior, and can lead to crash.
Frama-C is a static analysis framework (as opposed to Valgrind which provides dynamic analysis). It was originally designed with embedded, possibly low-level code in mind. Frama-C's “value analysis“ plug-in basically detects all C undefined behaviors that you may want to know about in embedded code (including accessing an invalid pointer).
Since it is a static analyzer, it does not execute the code(*) and is thus ideal in a cross-compiled context. Look for option -machdep. Values for this option include x86_64 x86_32 ppc_32 x86_16.
Disclaimer: I am one of the contributors of Frama-C's “value analysis” plug-in.
(*) though if you provide all inputs and set precision on maximum, it can interpret the source code as precisely as any cross-compilation+execution would.
I'm currently playing around with the CSR 1000 chip and I wanted to allocate memory. I tried using malloc but the compiler tells me:
undefined reference to `malloc'
I assume that is because gcc is run with -nostdlib parameter
So please could somebody with CSR uEnergy SDK experience, tell me why I can't allocate memory, and how I should do it instead??
If there is an SDK bundled with that chip that provides basic routines for memory allocation then use those, alternatively you can write your own allocator or use an existing one off of the web (with some fiddling).
As a quick solution you can probably mark a region in memory using a modified linker script or by using the gcc 'section' attribute (more here) and then use that as your heap arena in your malloc allocator.
A very simple allocator would not keep any accounting information such as headers/footers but rather allocate linearly one region after another (free-ing would essentially be a no-op in this case), this won't get you far but you will be able to run simple programs.
You probably want something more sophisticated, you could also look into implementing some kind of memory pool or any of the standard allocation algorithms.
The classic book The C Programming Language by Dennis Ritchie and Brian Kernighan provides a simple memory allocator if I re-call correctly. You may want to have a look at that.
I have three months of experience with this chip.
The malloc function is found in standard C library, which is typically available in desktop software development or embedded linux. But this is a small and resource-limited embedded chip. There is no standard C library.
If you browse the uEnergy SDK installation directory, something like this: C:\uEnergy_SDK-2.0.0\doc\reference\html\index.html. Click Modules tag on the top. You will find that under the section "C Standard Library APIs", CSR provides a few functions that mimic a subset of the standard C library. Unfortunately, there is no methods like malloc.
In general, when you work with small embedded systems, it is quite often that there is no dynamic memory allocation. However, for RF applications which are usually event-driven, there is typically a simple dynamic memory allocation function provided so incoming packets can be handed to you by the OS to your application. I used TI's CC2430 and its Zigbee stacks. They provide functions osal_mem_alloc and osal_mem_free, which mimic the malloc and free in the standard C library.
From my experience working with both chips, I found that CSR is much more protective than TI, in the same way as iOS vs. Android. You don't know what MCU they use except they tell you that it is a 16-bit RISC.
I suspect they have the dynamic memory allocation internally but your application just can't use those functions. RF packets are handed to you by the OS in the AppProcessLmEvent function, from there you get your data via the p_event_data pointer. You don't have to deallocate it as the OS will do it for you once you finish handling that event.
So back to your question, you can allocate memory so you just reserve a block of memory as global array and work on it.
Hope this helps.
add #include <malloc.h> to the head of your file
I'm developing C module for php under linux and I'm trying to find a way that could help me profile my code by maximum memory spike (usage).
Using valgrind I can get total memory allocation within the code. But as it is with allocated memory it comes and goes ;). What I need to get is the highest memory usage that appeared during C application run so I could get total overview on memory requirements and have some measurement point for optimization of the code.
Does anyone know any tool/trick/good practice that could help ?
Take a look at Massif: http://valgrind.org/docs/manual/ms-manual.html
Have you checked massif (one of Valgrind's tool)?
this is actually what you are looking for
another possibility would be memusage (one of glibc's utilities, glibc-utils)
Is there a libc function (or equivalent) to know the current size of the heap?
I have a memory problem in my application, and it seems being able to monitor the heap when I want to would help me find the problem. So is there a way to know the current size of the heap?
No.
As the functionality you want is for debugging, it would make a lot more sense for you to use your debugger or your operating system's resource accounting to monitor the process's memory usage, instead of trying to code that into your program.
If you really want your program to keep track of its own memory usage, the only portable way to do this is to avoid using malloc and free directly and instead call them through wrappers that increment/decrement a counter. This will not account for memory fragmentation, but if your interest is in the logical memory usage of your program and not the impact on physical resources, a counter implemented this way might actually be more informative than looking at the operating system's resource accounting.
If you only care about a particular target platform or family of platforms, there may also be functions above and beyond the C standard which do what you want. On POSIX, lookup getrusage.
What you need is http://valgrind.org/
Still wouldn't help.
Do you want to know :
The total address space.
The space available to user programs
The space unallocated, to this process, including swap or not
The biggest free chunk available
etc.
You can use tools like Purify to debug memory issues. This article from IBM contains a lot of details about the sources of such problems and pointers to solutions.
What are some techniques in detecting/debugging memory leak if you don't have trace tools?
Intercept all functions that allocate and deallocate memory (depending on the platform, the list may look like: malloc, calloc, realloc, strdup, getcwd, free), and in addition to performing what these functions originally do, save information about the calls somewhere, in a dynamically growing global array probably, protected by synchronization primitives for multithreaded programs.
This information may include function name, amount of memory requested, address of the successfully allocated block, stack trace that lets you figure out what the caller was, and so on. In free(), remove corresponding element from the array (if there are none, a wrong pointer is passed to free which is also a error that's good to be detected early). When the program ends, dump the remaining elements of the array - they will be the blocks that leaked. Don't forget about global objects that allocate and deallocate resources before and after main(), respectively. To properly count those resources, you will need to dump the remaining resources after the last global object gets destroyed, so a small hack of your compiler runtime may be necessary
Check out your loops
Look at where you are allocating variables - do you ever de-allocate them?
Try and reproduce the leak with a small subset of suspected code.
MAKE trace tools - you can always log to a file.
One possibility could be to compile the code and execute it on a system where you can take advantage of built in tools (e.g. libumem on Solaris, or the libc capability on Linux)
Divide and conquer is the best approach. If you have written you code in a systematic way, it should be pretty easy to call subsets of you code. Your best bet is to execute each section of code over and over and see if your memory usage steadily climbs, if not move on to the next section of code.
Also, the wikipedia article on memory leaks has several great links in the references section on detecting memory leaks for different systems (window, macos, linux, etc)
Similar questions on SO:
Memory leak detectors for C
Strategies For Tracking Down Memory Leaks When You’ve Done Everything Wrong
In addition to the manual inspection techniques mentioned by others, you should consider a code analysis tool such as valgrind.
Introduction from their site:
Valgrind is an award-winning
instrumentation framework for building
dynamic analysis tools. There are
Valgrind tools that can automatically
detect many memory management and
threading bugs, and profile your
programs in detail. You can also use
Valgrind to build new tools.
The Valgrind distribution currently
includes six production-quality tools:
a memory error detector, two thread
error detectors, a cache and
branch-prediction profiler, a
call-graph generating cache profiler,
and a heap profiler. It also includes
two experimental tools: a
heap/stack/global array overrun
detector, and a SimPoint basic block
vector generator. It runs on the
following platforms: X86/Linux,
AMD64/Linux, PPC32/Linux, PPC64/Linux,
and X86/Darwin (Mac OS X).
I have used memtrace
http://www.fpx.de/fp/Software/MemTrace/
http://sourceforge.net/projects/memtrace/
You may need to call the statistics function to printout if there are any leaks. Best thing is to call this statistics function before and after a module or piece of code gets executed.
* Warning * Memtrace is kind enough to allow memory overwrite/double free. It detects these anomalies and gracefully avoids any crash.