If we would like to check for memory leaks in a C++ program, we can overload the new and delete operators to keep track of the memory that was allocated. What if we would like to check for leaks in a C program? Since there is no operator overloading in C, can we over-write the malloc function pointer to intercept calls to malloc and track memory allocation? Is there an easier way without using any external utilities? Please provide some code as I am not familiar with over-writing method pointers.
Note: I would like to do this without any external utilities for practice.
As suggested, there already exist excellent tools like Valgrind to do this.
Further:
I would like to do this without any external utilities for practice
This is interesting and I am sure would be fulfilling,
You can use macro trick to detect such memory usage and leak errors, in fact write your own neat leak detector. You should be able to do this as long as you have a single allocation and deallocation function in your project.
#define malloc(X) my_malloc( X, __FILE__, __LINE__, __FUNCTION__)
void* my_malloc(size_t size, const char *file, int line, const char *func)
{
void *p = malloc(size);
printf ("Allocated = %s, %i, %s, %p[%li]\n", file, line, func, p, size);
/*Link List functionality goes in here*/
return p;
}
You maintain a Linked List of addresses being allocated with the file and line number from where there allocated. You update the link list with entries in your malloc.
Similar to above you can write an implementation for free, wherein you check the address entries being asked to be freed against your linked list. If there is no matching entry its a usage error and you can flag it so.
At the end of your program you print or write the contents of your linked list to an logfile. If there are no leaks your linked list should have no entries but if there are some leaks then the logfile gives you exact location of where the memory was allocated.
Note that in using this macro trick, you lose the type checking which functions offer but it's a neat little trick I use a lot of times.
Hope this helps and All the Best :)
Valgrind is what you need.
I remember reading first chapter of Algorithms in a Nutshell which talked about this although it didn't include code. Just added in case you find it interesting.
since there is no operator overloading in c can we over-write malloc
function point to intercept calls to malloc and track memory
allocation
Actually, you can. GIve LD_PRELOAD a read.
In addition to #Als's answer which will wrap calls in your source code, if you're using gnu ld, you can have the linker wrap all calls (presumably to malloc, realloc, calloc, and free) at link time, irrespective of where they come from. You then write __wrap_malloc etc and can call the original function with, for example, __real_malloc.
See --wrap=symbol in http://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_3.html
I don't know how this works with calls from shared libraries. I'm guessing it doesn't.
Use mallinfo function it worked for me on Xilinx Zynq baremetal using Xilinx SDK gcc. I tested with intentional memory leak - I have no idea why but google results are were incredibly terrible at finding this solution spread the word to help other devs!
Related
My question is regarding when using free() is appropriate in C. I'm using gcc 4.3.2.
Suppose, if had to deallocate a bunch of memory, in a linked list, the ideal way to go about it is(I'm guessing):
int freeLL(node *a)
{
if(a->next != NULL)
freeLL(a->next);
free(a);
return 0;
}
Now, suppose I were to do a similar thing on the following ADT:
A pointer "VertexNode" which has 2 more pointers: "Vertex" and "Edge"(say). Equivalent to saying:
struct vertexnode
{
vertex *v;
edge *e;
}
typedef struct vertexnode* VertexNode;
Later, when initializing an instance, I'll do something like -
VertexNode V = malloc(sizeof(struct vertexnode));
V->v = malloc(sizeof(vertex));
So, ultimately while freeing: I used the same analogy as I used for the linked list.
free(V->v);
free(V);
This gave a runtime error, and when I commented out "free(V->v)", the program worked fine.
My questions are:
a) Is simply doing free(V) sufficient? I mean, does free() work recursively on all pointers inside a given pointer?
b) If not, is there a memory leak in this case? And how do I ideally prevent that?
c) Lastly, is there a way I can keep track of how many bytes were allocated by malloc() and
how many of them were freed by free()?
I'm very sorry for the long question. Thank you in advance for your time and patience.
No, free does not work recursively, therefore you would indeed have memory leaks. The runtime error you are occuring is probably realated to a logic error (maybe V->v is NULL or you did not alloc it before freeing).
If you are working with linux, using valgrind can help you profiling your program and mention leak errors. Compile using cc *.c -ggdb and then run valgrind --leakcheck=full ./a.out will output leak errors.
To answer your question:
a) No. free() doesn't recursively free struct's member pointers.
b) Yes, there is memory leak in that case. You have to free all allocated memory with your code.
c) You may use tool to check memory leak, such as valgrind, that's easy. And I know some projects implements their own memory management, they wrap malloc and free in their own APIs, so that you can track memory usage in their APIs.
For a complete memory management I would to have a memory pool. This would take away all the pain of having to call a number of free's for each malloc that is called. I use Apache Portable runtime (APR) for doing memory pooling. Just allocate a chunk of memory at the start to initialize. The allocate as much as you want for each pointer. Then at the end just make one call to free all the memory. This is much more efficient then doing lots of mallocs and frees that leads to memory leaks.
As a side note. I suggest you use valgrind to test you application if you don't use the memory pool. In fact you should always use valgrind.
The thing is: you need to make sure that the pointer you're trying to free is actually initialized.
I'd also check whether V->v is not NULL before calling free on it (see my comment on your question).
free is not "recursive". You're leaking memory if you don't free e and v (in your example) as well.
A: free() does not work recursively.
- If you're on the ADT concept, then I suggest you perform the allocation inside the type by creating a createVertexNode() function, and do the checking and deallocation inside a freeVertexNode() function
B: If you wasn't able to free it, then it will be a memory leak
- you can avoid it by making sure your ADT functions check and free the memory it allocates, or ...
C: Use a memory leak checker. Visual Studio has a built-in one, or use other such as valgrind or Rational Purify. I'm sure there are more free open source libraries that, at the simplest, override the malloc() and free() calls
Basically the thing to remember is calls to malloc()/free() should be 1:1. If I call malloc() to get some memory, I need to call free() to return it when I'm done.
This answers question a), no it's not recursive. (or else one call to free() would be good enough).
b) You bet there'd be a memory leak! Think about some simple code:
typedef struct pointless {
struct pointless * next;
}pl;
pl * head = malloc(sizeof(pl)); // head gets 4 bytes
pl->next = malloc(sizeof(pl)); // head->next gets 4 bytes
free(head); // head's allocated 4 bytes are deleted
Obviously this is a pointless list, since it just points to the next empty element, but it illustrates the point. My total list has 8 bytes allocated to it (4 for head, 4 for head's next). When I call free() it works on head's 4 bytes, but that's it.
This is a good thing it works this way! Think about deleting a single item from the middle of a linked list, you want to free the memory for that node, but not the whole thing! However given my above example, you'd have a memory leak as head->next wasn't free'd; and once you call free(head) there's no guarantee that you can access head->next anymore. It's possible of course... but UB at this point.
c) memory management at that level is done by the OS. If you want to keep track of how much you allocated/freed you'd have to do something like:
int total_mem = 0;
head = malloc((total_mem += sizeof(pl)));
free(head);
total_mem -= sizeof(head);
Of course you could put wrappers around the malloc() and free() calls to do the math for you which would be much more manageable, but you get the idea.
As an assignment in operating systems we have to write our own code for malloc and free in C programming language, I know if i asked the code for it there is no point of me to study. i'm facing the problem of not knowing where to include initializing char array with 50000 bytes and making two lists free and used. in my function i can't trigger malloc or free to happen automatically. and a 3rd party main program will be used to test my functions.....
if my file is mymalloc.c or what ever
void* myalloc(size_t size)
{
//code for allocating memory
}
void myfree(void *ptr)
{
//code for free the memory
}
where do the code for initiating memory space and lists will go..
I will provide you with the basic concept which you can use to write your own code for malloc() and free() functions using C.
Assume that we have a contiguous block of memory of a certain size. It will be our abstract sense of memory which will carry all the requested memory allocations plus the data structures that are used to hold data about those allocated blocks.
We use a simple linked list to carry the data related to the allocated as well as free blocks of memory.
Its structure is as follows.
struct block{
size_t size; /*Specifies the size of the block to which it refers*/
int free; /*This is the flag used to identify whether a block is free
or not*/
struct block *next; /*This points to the next metadata block*/
};
You will need 2 source files for this purpose. One is mymalloc.h which is the header file which contains the initialization parts and the function prototypes of the rest of the functions that we are going to implement. The other is the mymalloc.c source file which contains all the necessary function implementations.
There needs to be a function to initialize the first free memory block.
And another function to split a block of memory which has more than enough space to give to the requested size. And another method to scan through the linked list and merge any consecutive blocks that are free, so that it prevents external fragmentation.
Note: We use the First-fit-algorithm to find a free block to allocate memory.
I think this will help anyone who is in search of a simple way to write their own malloc and free functions using C. Please follow the following link for a detailed explanation.
http://tharikasblogs.blogspot.com/p/how-to-write-your-own-malloc-and-free.html
I think you only have to implement a memory manager. So you don't have to use brk, sbrk, ...
Just put used memory in a simple array and fragment it somehow. Since it's homework you want to make it as simple as possible or else you run into problems due to complexity/time constraints of your assignment.
You only have to decide which tactic you want to use. I'd suggest to use the buddy system. Though it's a bit more complicated than the most simple ones.. maybe fixed sized fragmentation is simpler..
Maybe this is also a good read.
Don't do something low-level as suggested in the other answers..
The implementation greatly depends upon operating system and architecture, anyhow you may take a look at this: http://www.raspberryginger.com/jbailey/minix/html/lib_2ansi_2malloc_8c-source.html
(and study how it works!).
If you are on a unix system you can look the manual of brk and sbrk. Those system calls "push/set" the limit of the heap.
Using those you can manage your memory pages, allocating them as you need.
I would advise a chained-list to manage your different allocated spaces and building functions to split them or to merge them if they are free.
If you need to try your code with high-level applications, you can name your functions malloc/free, compile them to a shared-object (.so) and then use LD_PRELOAD and LD_LIBRARY_PATH environment variables to load your .so and replace system's malloc.
Every command you call then will use your shared object and thus your malloc, telling you if your malloc is stable or if it fails to comply with reality.
If you need a clear example of this i'd be happy to put some code here, but I do not want to make my answer too hard to read.
First, you could make a fake malloc which always fail
/* fake malloc */
void* myalloc(size_t sz)
{ return NULL; }
but that is "cheating". You want to make a malloc which is useful.
You probably want to make a system call which asks the kernel for memory. Of course, you'll need the symetrical syscall to release memory. On Linux and many Posix systems you'll often use mmap and munmap syscalls.
(You could also use sbrk, but using mmap with munmap is easier and more general)
The idea is that you get big chunks of memory (with mmap) and then you manage smaller memory zones inside. The interesting detail is how to manage these smaller zones. You may want to deal with large malloc differently than "small" allocations.
You really want to read wikipedia page on memory allocation
You could have a global static variable that is initialized to zero. Then check that variable at the start of your malloc and free function. In your malloc function, if the variable is zero then initialize whatever you need, and then set the variable to non-zero. In your free function, just return if the variable is zero.
More like that, is a simple malloc :
void* my_malloc(size_t size)
{
return (sbrk(size));
}
man sbrk will help you.
The problem now is to create a free and to create a efficient malloc :-)
if you want to test your malloc you can do like this :
$> LD_PRELOAD=/mypath/my_malloc.so /bin/ls
but you need to create a dynamic library before because malloc is a .so
I have a function that recieves a pointer to dynamic array of 100 ints. But instead of 100 I have just 50 allocated by malloc or calloc before that.
Is there a way that I could check if any ellement (like 79th for example) is allocated rather than wonder what this SIGSEGV actually means ?
My question is purely theoretic and I have no actual code to show.
No, the pointer does not store its size. You may be better off storing the size and the pointer in a struct and passing it instead:
typedef struct
{
size_t size;
int *ptr;
} my_data;
void myFunc(my_data *data)
{
size_t i;
for(i = 0; i < data->size; i++)
{
// data->ptr[i];
}
}
void myFunc2(my_data *data, size_t index)
{
if(index < data->size)
{
// memory location exists
}
}
Well, you could do such a thing according to your description, given an array and looking for an index (which is slightly different from "any raw pointer"). And with some more work, it is even possible to do such a thing for any pointer.
The malloc function necessarily stores information about how much was allocated. Unluckily, there is no standard how this must be done. Some compilers over-allocate and store the size immediately preceding the allocated data. Others may store addresses in a map, yet others may do something else, you don't know.
However, most (all?) C libraries and at least one linker that I know of have explicit support for overloading/hooking/replacing allocation functions.
For example in the GNU C library, you can set __malloc_hook. and GNU ld lets you do such a thing at linker level with __wrap_malloc.
You could thus overload/hook malloc and free with a function that simply calls the real malloc function and stores the information how much was allocated yourself somewhere (e.g. by over-allocating and using the first word, or whatever you like).
Then write a function which takes a base pointer and an index. That function looks at the allocation info (now you know where to find it!), and can trivially check whether the index is in range. This does not work for "just any pointer".
An alternative solution which works for "just any pointer" would be to write an allocator that satisfies allocations from separate arenas rather than simply wrapping the real malloc. All allocations coming from the same arena have the same allocation size. Given any pointer, you would then only need to iterate over all your arenas and look whether the address is within the arena's start and end address.
However, one should normally be quite sure how much one has allocated, this should not be guesswork, or random luck, or something to figure out at runtime.
Also, given the presence of ready-to-use memory debuggers, I doubt it is really worth investing time in doing such a thing application-side. Just use something like valgrind, no need to write any code at all.
No, there's no portable and reliable way to check this from within the code.
There exist tools -- such as valgrind -- that may help diagnose certain types of memory bugs.
No, there isn't.
This is when you break out your dynamic analysis tool (e.g. valgrind), or use a real container that keeps information about its size.
Some years ago i used one library, i forget its name. Using it, you can create try-catch block and try to access to unknown data e.g. x[79] in try-block, and, if memory is not allocated in it, exception was generated.
I am working in large application which is written in C. We allocate memory through malloc and release the memory free for different pointer. I have seen all time we have repeat check
for memory failure case than release the memory.
I am planning to write a utility in which the programmer will pass the pointer which they want to
create and we will take care all of allocation and release of memory.
Does anyone have idea any utility function which can be use in C for allocation and release memory.
The first answer came my mind was boost library but I have received answer from my organization that they do not want to use boost library.
How about Boehm Garbage Collector?
If you are just looking for a unified way (without code duplication) to check if allocation was successful or not, please see the last segment of this post.
First of all boost is mainly a library for C++, therefor all of boost can't be used in C.
Writing your own Garbage Collector is not easy, especially in a language without true OOP (such as C). There are many implementations of garbage collectors available online, so instead of reinventing the wheel you could check out some of them.
If you are not going to use any of them, well.. at least they will provide you with some information regarding how the problem can be solved.
A garbage collector for C and C++ (Boehm-Demers-Weiser) [recommended]
Using the C/C++ Garbage Collection Library, libgc
Depending on the project size you might be better of using valgrind looking for leaking data and then manage the memory allocation/release by yourself.
C has been around for many years, and a lot of developers have managed without a automatic garbage collector. Why shouldn't you be able to do the same?
Simple error checking on allocation (and abort on error)
#include <stdio.h>
#include <stdlib.h>
void*
safe_alloc_check (void *p, size_t size) {
if (p == NULL) {
printf ("ERROR: Unable to allocate memory for %lu bytes!", size);
exit (-1);
}
return p;
}
#define s_malloc(N) safe_alloc_check(malloc(N),N)
#define s_calloc(C,N) safe_alloc_check(calloc(C,N),N)
#define s_realloc(P,N) safe_alloc_check(realloc(P,N),N)
...
int *p = s_malloc (sizeof (int));
Wrap malloc in a function (often called xmalloc) which aborts when malloc fails to return a new memory chunk (by returning NULL).
If you are on a Unix system, using valgrind to catch memory leaks is helpful.
If you want a garbage collector for C, consider using Boehm's garbage collector
I'm not sure what you're after. An often used technique for handling memory allocation and deallocation in C is implementing a memory trace data structure which holds a number of pointers to memory that has been allocated using your own implementation of malloc, calloc, realloc, e.g. mymalloc, mycalloc, myrealloc (calling the original functions themselves) and then using a new function free_memtrace to free all the memory that has been recorded in a trace in one function call.
So e.g. you could do
MEMTRACE mt; // Possibly a module-scoped global variable
begin_memtrace( &mt );
ptr1 = mymalloc( size );
ptr2 = mycalloc( n, sizeof(type) );
end_memtrace( &mt );
and then at some point in your program
free_memtrace( &mt );
to clear the memory recorded in mt.
The begin_memtrace, end_memtrace functions are needed so that your allocation functions "know" where to record the pointers. That information could be stored locally in the module where you implement mymalloc, mycalloc, myfree, free_memtrace ....
A project I am working on involves a flight vehicle with GNC code written in a C library (.out). We must call this C code from LabVIEW (the primary avionics software) in the form of a .out library, and the nature of the software requires static pointers to store data between successive calls to the function. We call the GNC executive function at regular intervals throughout a flight. I'm now trying to call this function using a Matlab MEX wrapper in a DLL on Windows, and this has uncovered some memory management issues.
I am declaring the structures at the beginning of the function like this:
static Nav_str *Nav_IN_OUT_ptr;
static hguid_ref *Guid_IN_OUT_ptr;
static HopControl *Control_IN_OUT_ptr;
Nav_IN_OUT_ptr = (Nav_str *)malloc(sizeof(Nav_str));
Guid_IN_OUT_ptr = (hguid_ref *)malloc(sizeof(hguid_ref));
Control_IN_OUT_ptr = (HopControl *)malloc(sizeof(HopControl));
This happens during every run of the function. However, after this function is called iteratively several times, it always crashes with a memory segmentation fault after it tries to exit. My understanding was that this memory was supposed to clean itself up, is that incorrect?
In order to clean it up manually, I added these lines to the end, to be called only on a clean-up iteration:
free(Nav_IN_OUT_ptr);
free(Guid_IN_OUT_ptr);
free(Control_IN_OUT_ptr);
Is this the correct way to free this memory? Can I free this memory? Might there be another reason for the segmentation error other than C not giving up the memory properly after the last call, or Matlab not properly managing its memory? I've searched all over for someone with a similar problem (even contacting Mathworks) without much luck, so any comments or suggestions would be much appreciated.
Failing to free memory is not going to cause a segmentation fault. It's probably likely your problem lies somewhere else. The two likely conditions are:
Overflowing a buffer
Using a pointer to memory that has previously been free'd.
Using a bad pointer value, somehow set incorrectly.
Trying to free a pointer not returned by malloc'd (or already free'd)
My understanding was that this memory
was supposed to clean itself up, is
that incorrect?
Yes, you need to call free() to release the memory back to the heap. I would also suggest that you set the pointer value to null after the free, this may help you catch condition 2, from above.
Nav_IN_OUT_ptr = (Nav_str *)malloc(sizeof(Nav_str));
This code statement is questionable. What is Nav_str type? Are you sure you don't mean to use strlen(Nav_str)+1?
I also need to ask what is the purpose for making your pointers static? Static function variables are basically globals, and only to be used in rare cases.
Your code does have a memory leak - it is allocating that memory each time the function is called. Even your current method still has the memory leak - if you only call free() once, in the final iteration, then you have only freed the most recent allocation.
However, a memory leak will not generally cause a segmentation fault (unless your memory leak exhausts all available memory, causing subsequent malloc() calls to return NULL).
If you wish to have static structures that are only allocated once and re-used, you do not need to use malloc() at all - you can simply change your declarations to:
static Nav_str Nav_IN_OUT;
static hguid_ref Guid_IN_OUT;
static HopControl Control_IN_OUT;
... and use Nav_IN_OUT.field instead of Nav_IN_OUT_ptr->field, and &Nav_IN_OUT in place of Nav_IN_OUT_ptr (if you are directly passing the pointer value to other functions).
My understanding was that this memory was supposed to clean itself up, is that incorrect?
Sorry, but you were incorrect. :) Memory allocated with malloc() will persist until you manually remove it with free(). (You did get this right in the end. Hooray. :)
Is this the correct way to free this memory? Can I free this memory?
That is the correct way to free the memory, but it might not be in the correct place. In general, try to write your free() calls the same time you write your malloc() calls.
Maybe you allocate at the start of a function and then free at the end of the function. (In that case, on-stack memory use might be better, if the memory is only ever used by functions called by the original function.)
Maybe you have a foo_init() function that calls malloc() and creates associated contexts from an API, then you pass that context into other routines that operate on that data, and then you need to place the free() calls into a foo_destroy() or foo_free() or similar routine. All your callers then need to balance the foo_init() and foo_free() calls. This would be especially appropriate if you can't just write the foo_init() and foo_destroy() calls in one function; say, your objects might need to be removed at some random point in a larger event loop.
And maybe the data should just be allocated once and live forever. That would be correct for some application designs, and it's tough to tell just from the variable names whether or not these blocks of data should live forever.
Might there be another reason for the segmentation error other than C not giving
up the memory properly after the last call, or Matlab not properly managing its memory?
There sure could be; perhaps this memory is being returned too soon, perhaps some pointer is being free()ed two or more times, or you're overwriting your buffers (that malloc(sizeof(Nav_str)) call is a little worrying; it is probably just allocating four or eight bytes, based on the pointer size on your platform; and before you replace it with strlen(), note that strlen() won't leave space for a NUL byte at the end of the string; malloc(len+1); is the usual pattern for allocating memory for a string, and I get concerned any time I don't see that +1 in the call.)
Some time with valgrind would doubtless help find memory errors, and maybe some time with Electric Fence could help. valgrind is definitely newer, and can definitely handle 'large' programs better (since electric fence will allocate a new page for every malloc(), it can be expensive).