I'd like to free automatically multiple malloced memory at the end of a program in C.
For example :
str1 = malloc(sizeof(char) * 10);
str2 = malloc(sizeof(char) * 10);
str3 = malloc(sizeof(char) * 10);
I don't want a function like this :
void my_free()
{
free(str1);
free(str2);
free(str3);
}
but a function which free all the memory allocated during the program.
but a function which free all the memory allocated during the program.
There's no such function in C. C doesn't do memory management automatically. So it's your responsibility to track the memory allocations and free them appropriately.
Most modern operating systems (perhaps, not embedded systems) would reclaim the memory allocated during execution at process termination. So you can skip calling free(). However, if your application is a long running one then this will become a problem if it keeps allocating memory. Depending your application, you may devise a strategy to free memory appropriately.
As Blue Moon pointed out in his answer, one of the main features of C compared to other languages is the missing memory management. While this gives you a lot of freedom it can on the other hand lead to severe bugs in your code.
Technically the detection of memory leaks is not possible with a confidence level of 100%, but there are quite powerful static code analyzers out there to guide you.
In the last embedded project I worked on we used FlexeLint. It is costly for non-commercial products but the benefit is enormours. A lot of potential bugs and leaks could be detected with such a static analyzer without even executing the code.
There is another static analyzer, free of cost for open source projects called Coverity Scan. I did not try it myself but it is probably worth a shot.
After witnessing what a good analyzer like FlexeLint is able to detect beyond mere compilation errors, I personally would not launch another C Project without such analyzis tools in place.
While this is not a direct answer to your question, it can be an improvement to your workflow because such errors as forgetting a free call will be detected in most cases.
Related
What happen if I end the execution by passing return 0; after using a malloc and without freeing the part of memory allocated?
int * var;
var = (int *)malloc(sizeof(int)) ;
free(var) ;
return 0;
The program contains a memory leak, as explained here. To answer the question specifically, the effect of the memory leak depends on the environment; in the best case, nothing happens, in the worst case, the machine might crash sooner or later. The existence of the memory leak is to be regarded as a bug in any case.
It is implementation specific.
On most operating systems (notably desktop or server OSes, e.g. Linux, MacOSX, Windows...) once a process is terminated, every used resources is released, and that includes its virtual address space. Hence even unfreed heap memory is released.
In particular, if you code a quickly running program (e.g. you know that it always will run in less than few seconds), it might be easier to accept some memory leak (but if you do, please comment that in your code and/or documentation). And many real life programs are doing that (in particular the GCC compiler, and probably several Unix shells).
On the contrary, if you are coding a server (e.g. a database or compute server), you should avoid any memory leaks which would make the server's process RSS grow indefinitely (till some crash). You usually should take great care that every allocated heap memory to deal with one request gets free-d after replying to that request.
On some embedded operating systems, if you don't release all the resources explicitly (free all heap allocated memory, fclose all opened streams) and properly, you have a resource leak.
See also this related question. On many OSes (including Linux) you can use valgrind to hunt memory leak bugs. With a recent gcc you might use debugging options like -g -fsanitize=address
Read also (at least for the concepts and the terminology) something about garbage collection and about fragmentation. If programming in C, you might consider using Boehm's garbage collector.
There is however a very good practical reason to systematically free all previously malloc-ed memory: it is a good programming discipline, and it helps a lot using tools like valgrind which helps debugging genuine memory bugs. Also it makes your code more clean, and you could reuse it in some different contexts (e.g. as part of some library, usable in long-running processes).
If you don't free the memory, your application will have a memory leak. So if you don't free the memory and leave the application on for a couple of days, it will start to slow down, and eventually crash.
I am working in large application which is written in C. We allocate memory through malloc and release the memory free for different pointer. I have seen all time we have repeat check
for memory failure case than release the memory.
I am planning to write a utility in which the programmer will pass the pointer which they want to
create and we will take care all of allocation and release of memory.
Does anyone have idea any utility function which can be use in C for allocation and release memory.
The first answer came my mind was boost library but I have received answer from my organization that they do not want to use boost library.
How about Boehm Garbage Collector?
If you are just looking for a unified way (without code duplication) to check if allocation was successful or not, please see the last segment of this post.
First of all boost is mainly a library for C++, therefor all of boost can't be used in C.
Writing your own Garbage Collector is not easy, especially in a language without true OOP (such as C). There are many implementations of garbage collectors available online, so instead of reinventing the wheel you could check out some of them.
If you are not going to use any of them, well.. at least they will provide you with some information regarding how the problem can be solved.
A garbage collector for C and C++ (Boehm-Demers-Weiser) [recommended]
Using the C/C++ Garbage Collection Library, libgc
Depending on the project size you might be better of using valgrind looking for leaking data and then manage the memory allocation/release by yourself.
C has been around for many years, and a lot of developers have managed without a automatic garbage collector. Why shouldn't you be able to do the same?
Simple error checking on allocation (and abort on error)
#include <stdio.h>
#include <stdlib.h>
void*
safe_alloc_check (void *p, size_t size) {
if (p == NULL) {
printf ("ERROR: Unable to allocate memory for %lu bytes!", size);
exit (-1);
}
return p;
}
#define s_malloc(N) safe_alloc_check(malloc(N),N)
#define s_calloc(C,N) safe_alloc_check(calloc(C,N),N)
#define s_realloc(P,N) safe_alloc_check(realloc(P,N),N)
...
int *p = s_malloc (sizeof (int));
Wrap malloc in a function (often called xmalloc) which aborts when malloc fails to return a new memory chunk (by returning NULL).
If you are on a Unix system, using valgrind to catch memory leaks is helpful.
If you want a garbage collector for C, consider using Boehm's garbage collector
I'm not sure what you're after. An often used technique for handling memory allocation and deallocation in C is implementing a memory trace data structure which holds a number of pointers to memory that has been allocated using your own implementation of malloc, calloc, realloc, e.g. mymalloc, mycalloc, myrealloc (calling the original functions themselves) and then using a new function free_memtrace to free all the memory that has been recorded in a trace in one function call.
So e.g. you could do
MEMTRACE mt; // Possibly a module-scoped global variable
begin_memtrace( &mt );
ptr1 = mymalloc( size );
ptr2 = mycalloc( n, sizeof(type) );
end_memtrace( &mt );
and then at some point in your program
free_memtrace( &mt );
to clear the memory recorded in mt.
The begin_memtrace, end_memtrace functions are needed so that your allocation functions "know" where to record the pointers. That information could be stored locally in the module where you implement mymalloc, mycalloc, myfree, free_memtrace ....
I have to do a project in C where I have to constantly allocate memory for big data structures and then free it. Does there exista a library with a function that helps to keep track of the memory usage so I can be sure if I am doing things correctly? (I'm new to C)
For example, a function that returns:
A) The total of memory used by the program at the moment, OR
B) The total of memory left,
would do the job. I already googled for that and searched in other answers.
Thanks!
Try tcmalloc: you are looking for a heap profiler, although valgrind might be more useful initially.
If you're worried about memory leaks, valgrind is probably what you need. On the other hand, if you're more concerned just with whether you're data structures are using excessive memory, you might just use the common mallinfo function included as an extension to malloc in many unix standard libraries including glibc on Linux.
Although some people excoriate it, the book "Writing Solid Code" by Steve Maguire has a lot of reasonable ideas about how to track your memory usage without modifying the system memory allocation functions. Basically, instead of calling the raw malloc() etc functions directly, you call your own memory allocation API built on top of the standard one. Your API can track allocations and frees, detect double frees, frees of non-allocated memory, unreleased (leaked) memory, complete dumps of what is allocated, etc. You either need to crib the code from the book or write your own equivalent code. One interesting problem is providing a stack trace for each allocation; there isn't a standard way to determine the call stack. (The book is a bit dated now; it was written just a few years after the C89 standard was published and does not exploit const qualifiers.)
Some will argue that these services can be provided by the system malloc(); indeed, they can, and these days often are. You should look carefully at the manual provided for your version of malloc(), and decide whether it provides enough for you. If not, then the wrapper API mechanism is reasonable. Note that using your own API means you track what you explicitly allocate, while leaving library functions not written to use your API using the system services - as, indeed, does your code, under the covers.
You should also look into valgrind. It does a superb job tracking memory abuses, and in particular will report leaked memory (memory that was allocated but not freed). It also spots when you read or write outside the bounds of an allocated space, spotting buffer overflows.
Nevertheless, ultimately, you need to be disciplined in the way you write your code, ensuring that every time you allocate memory, you know when it will be released.
Every time you allocate/free memory, you could log how big your data structure is.
Today, I appeared for an interview and the interviewer asked me this,
Tell me the steps how will you design your own free( ) function for
deallocate the allocated memory.
How can it be more efficient than C's default free() function ? What can you conclude ?
I was confused, couldn't think of the way to design.
What do you think guys ?
EDIT : Since we need to know about how malloc() works, can you tell me the steps to write our own malloc() function
That's actually a pretty vague question, and that's probably why you got confused. Does he mean, given an existing malloc implementation, how would you go about trying to develop a more efficient way to free the underlying memory? Or was he expecting you to start discussing different kinds of malloc implementations and their benefits and problems? Did he expect you to know how virtual memory functions on the x86 architecture?
Also, by more efficient, does he mean more space efficient or more time efficient? Does free() have to be deterministic? Does it have to return as much memory to the OS as possible because it's in a low-memory, multi-tasking environment? What's our criteria here?
It's hard to say where to start with a vague question like that, other than to start asking your own questions to get clarification. After all, in order to design your own free function, you first have to know how malloc is implemented. So chances are, the question was really about whether or not you knew anything about how malloc can be implemented.
If you're not familiar with the internals of memory management, the easiest way to get started with understanding how malloc is implemented is to first write your own.
Check out this IBM DeveloperWorks article called "Inside Memory Management" for starters.
But before you can write your own malloc/free, you first need memory to allocate/free. Unfortunately, in a protected mode OS, you can't directly address the memory on the machine. So how do you get it?
You ask the OS for it. With the virtual memory features of the x86, any piece of RAM or swap memory can be mapped to a memory address by the OS. What your program sees as memory could be physically fragmented throughout the entire system, but thanks to the kernel's virtual memory manager, it all looks the same.
The kernel usually provides system calls that allow you to map in additional memory for your process. On older UNIX OS's this was usually brk/sbrk to grow heap memory onto the edge of your process or shrink it off, but a lot of systems also provide mmap/munmap to simply map a large block of heap memory in. It's only once you have access to a large, contiguous looking block of memory that you need malloc/free to manage it.
Once your process has some heap memory available to it, it's all about splitting it into chunks, with each chunk containing its own meta information about its size and position and whether or not it's allocated, and then managing those chunks. A simple list of structs, each containing some fields for meta information and a large array of bytes, could work, in which case malloc has to run through the list until if finds a large enough unallocated chunk (or chunks it can combine), and then map in more memory if it can't find a big enough chunk. Once you find a chunk, you just return a pointer to the data. free() can then use that pointer to reverse back a few bytes to the member fields that exist in the structure, which it can then modify (i.e. marking chunk.allocated = false;). If there's enough unallocated chunks at the end of your list, you can even remove them from the list and unmap or shrink that memory off your process's heap.
That's a real simple method of implementing malloc though. As you can imagine, there's a lot of possible ways of splitting your memory into chunks and then managing those chunks. There's as many ways as there are data structures and algorithms. They're all designed for different purposes too, like limiting fragmentation due to small, allocated chunks mixed with small, unallocated chunks, or ensuring that malloc and free run fast (or sometimes even more slowly, but predictably slowly). There's dlmalloc, ptmalloc, jemalloc, Hoard's malloc, and many more out there, and many of them are quite small and succinct, so don't be afraid to read them. If I remember correctly, "The C Programming Language" by Kernighan and Ritchie even uses a simple malloc implementation as one of their examples.
You can't blindly design free() without knowing how malloc() works under the hood because your implementation of free() would need to know how to manipulate the bookkeeping data and that's impossible without knowing how malloc() is implemented.
So an unswerable question could be how you would design malloc() and free() instead which is not a trivial question but you could answer it partially for example by proposing some very simple implementation of a memory pool that would not be equivalent to malloc() of course but would indicate your presence of knowledge.
One common approach when you only have access to user space (generally known as memory pool) is to get a large chunk of memory from the OS on application start-up. Your malloc needs to check which areas of the right size of that pool are still free (through some data structure) and hand out pointers to that memory. Your free needs to mark the memory as free again in the data structure and possibly needs to check for fragmentation of the pool.
The benefits are that you can do allocation in nearly constant time, the drawback is that your application consumes more memory than actually is needed.
Tell me the steps how will you design your own free( ) function for deallocate the allocated memory.
#include <stdlib.h>
#undef free
#define free(X) my_free(X)
inline void my_free(void *ptr) { }
How can it be more efficient than C's default free() function ?
It is extremely fast, requiring zero machine cycles. It also makes use-after-free bugs go away. It's a very useful free function for use in programs which are instantiated as short-lived batch processes; it can usefully be deployed in some production situations.
What can you conclude ?
I really want this job, but in another company.
Memory usage patterns could be a factor. A default implementation of free can't assume anything about how often you allocate/deallocate and what sizes you allocate when you do.
For example, if you frequently allocate and deallocate objects that are of similar size, you could gain speed, memory efficiency, and reduced fragmentation by using a memory pool.
EDIT: as sharptooth noted, only makes sense to design free and malloc together. So the first thing would be to figure out how malloc is implemented.
malloc and free only have a meaning if your app is to work on top of an OS. If you would like to write your own memory management functions you would have to know how to request the memory from that specific OS or you could reserve the heap memory right away using existing malloc and then use your own functions to distribute/redistribute the allocated memory through out your app
There is an architecture that malloc and free are supposed to adhere to -- essentially a class architecture permitting different strategies to coexist. Then the version of free that is executed corresponds to the version of malloc used.
However, I'm not sure how often this architecture is observed.
The knowledge of working of malloc() is necessary to implement free(). You can find a implementation of malloc() and free() using the sbrk() system call in K&R The C Programming Language Chapter 8, Section 8.7 "Example--A Storage Allocator" pp.185-189.
Are there any "good" C libraries for garbage collection?
I know about the Boehm GC, is it maintained nowadays?
What about http://tinygc.sourceforge.net?
What are your experiences with these libraries?
You could use Boehm's Garbage Collector. Many projects I have worked with use it.
Although I do not know of a good, easy, and effective GC for C, I would like to weigh in some thoughts.
For many years, I avoided heap allocations for fear of memory leaks. And I do not know of any way to have a memory leak without allocating from the heap.
But I do not know of any better way to return memory from a function (such as a string, or an array of structs) than by heap allocation. Heap allocation allows you to allocate precisely the amount of memory you need without knowing how much memory you need at compile time. If your program will load files to memory, you probably will not know at compile time how much memory you might need. If you go the static memory route, then you always have to allocate the maximum memory you might ever need. Then your program can become a memory hog when it does not need to be. Heap is better.
But then keeping track of heap allocations can be difficult in some kinds of programs, for example, a program that has many modules inserting and deleting elements from an in-memory database. And memory leaks can be so difficult to solve. A good GC seems to me a very good way to stop memory leaks because of the automated tracking and freeing of heap allocations.
So, this post responds to the comment that "If you have problems in C using malloc() and free() correctly, you should switch to another language." If the programming problem is complicated and the solution needs heap allocations, the even the best programmers will have to debug memory leaks.
In some programs, there could be an alternative to using garbage collection: You can assume that an operating system will free all the program's allocations when the program terminates. So, your main program might be able to call a second program that uses heap allocations then writes results to a file and terminates. I tested this on Windows. I observed that Windows deallocated heap memory that my test program allocated on the test program's termination. I ran the test program many times. There was no diminishing of available memory.
Of course, having a second program that runs a short while and terminates is often not a viable solution.