This question is similar to c malloc questions (mem corruption) but I ask it again because I want more specific information than what was provided.
So I have a program with one malloc, followed by some complex code, followed by one free. Somewhere in the complex code, memory gets corrupted by either a double-free or an out-of-bounds write (both in separate regions of memory from the original malloc). This causes the original free to fail. This behaviour is fairly deterministic.
My questions are several:
What are the minimal conditions for a memory corruption to affect a separate memory region like this.
Are there any proactive steps that can be taken to prevent this cross-corruption.
Is it defined behaviour wrt standards to use pointer arithmetic to jump back and forth between contiguously allocated memory.
/* 3 example */
void *m = malloc(sizeof(header_struct) + sizeof(body_struct));
body_struct *b = (body_struct*) (((header_struct*)m)+1);
header_struct *h = (header_struct*) (((header_struct*)b)-1);
Good questions.
Q1. The minimal conditions under the standard are anything that triggers Undefined Behaviour. Unfortunately, that's a rather long list and not actionable. In practice the list comes down to 4 common scenarios: object underflow, object overflow, dangling reference or wild store.
Object underflow happens when you write to bytes just before the allocated block. It is very common for those bytes to contain critical block links and the damage is usually severe.
Object overflow happens when you write to bytes just after the allocated block. There is usually a small a amount of padding at the end so a byte or two will usually do no serious damage. If you keep writing you will eventually hit something important, as per underflow.
Dangling reference means writing via a pointer that used to be valid. It could be a pointer to a local variable that has gone out of scope, or to a block that has been freed. These are nasty.
Wild store means writing to an address way outside an allocated block. That could be a small positive address (say pointer value of 0x20) and in a debug environment these areas can often be protected, or it can be random garbage because the pointer itself got damaged. These are less common, but very hard to find and fix.
Q2. A debug heap is your first level of protection. It will check the links and write special patterns into unused space and generally help to find and fix problems. if you are using a debug heap then free() usually triggers some diagnostic activities, but you can usually find some other call to do the same thing. On Windows that is HeapValidate().
You can do more by implementing your own heap with guards/sentinels (look that up) and your own heap checking functions. Beyond that (in C at least), you just have to get better at writing the code. You can add assertions so at least the code fails fast.
Then you can use external tools. One is valgrnd, but it is not always feasible. In one case we wrote a complete heap logging system to track every allocation, to find problems like these.
Q3. Your second example does not guarantee the correct alignment of body_struct in line 2. According to the C standard n1570 S7.22.3, memory returned by malloc() is suitable aligned to be used as a pointer to any object. The compiler will lay out the structs with this assumption.
However, that requirement does not extend to the members of an array of structs. It is implementation-defined whether the second member of an array of a struct like this is aligned or not.
struct s {
double d;
char c;
} ss[2];
With this in mind, your code is valid C, but may have implementation-defined or Undefined Behaviour, depending on alignment requirements. It is certainly not recommended.
(1) Any undefined behaviour can cause such memory corruption, including but not restricted to writing to any memory location that is not part of an object.
(2) Write your code carefully :-(
(3) The second assignment is not portable and can lead to all kinds of problems due to alignment problems. To make this correct and portable, you usually use a flexible array member. If you always allocate one header and one body, define a new struct
typedef struct {
header_struct header;
body_struct body;
} body_plus_header_struct;
If you allocate one header and a variable number of bodies, write
typedef struct {
header_struct header;
body_struct bodies [];
} body_plus_header_struct;
Here, body_plus_header_struct has a size that is guaranteed to be rounded up so that the address of the bodies array has the correct alignment. To allocate a struct for n bodies, allocate
body_plus_header_struct* p = malloc (sizeof (*p) + n * sizeof (p->bodies [0]));
Related
I program embedded systems. One golden rule is that we never call malloc(); all data must be statically allocated at compile time.
Hence, I am not really familiar with Variable Length Arrays, which were introduced with C99.
The concept seems clear enough, and I don't need it explaining. My question is what happens at run time if there isn't enough free memory for such an array?
I would imagine that it is o/s dependent, maybe compiler dependent, to what would GCC/Linux do, and MS visual Studio C on Windows? Any X99 or Posix definitions?
From the point of view of the Standard, an attempt to allocate a VLA with a size the implementation cannot accommodate invokes Undefined Behavior. Because the Standard provides no means of discovering what size array an implementation could safely create, and does not mandate that implementations allow any particular size, any attempt to create a VLA object with a size greater than 1 should be regarded as invoking Undefined Behavior except in cases where one happens to know enough about implementation's inner workings to determine the size of VLA it will be able to handle.
If malloc() is unavailable, one's best bet may be to define a large array of
whatever type has the coarsest alignment requirement, store its address into
a volatile-qualified pointer [the storage in which the pointer itself resides
should be thus qualified] read it back, and interpret that as the start of a
memory pool. No other use should be made of the original array object. While
the Standard wouldn't guarantee that a compiler wouldn't decide that it should generate code that checks whether the pointer still identifies the original object and, if it does, skipping any code that would use that pointer to access anything other than the original object's type, the use of volatile on the pointer should make that really unlikely.
Once a memory pool is created, you can write your own memory-management
functions to use it, though any time a pointer is returned to the pool it
may be necessary to use the volatile-pointer-laundering hack to prevent
compilers from using type-based aliasing to justify treating the last uses
of storage as its old type as unsequenced relative to the first uses of
storage as a new type.
Variable length arrays are typically allocated on the stack. Like any other variable allocated on the stack this is normally done by subtracting from the stack pointer (or adding to it for an upwards-growing stack). A frame pointer is likely to be used so that the function can keep track of it's stack frame in the face of dynamically determined changes to the stack pointer. As with other stack allocations there is typically no error checking in this process.
This brings a couple of dangers.
The space allocated to the stack may be overflowed. Depending on the platform this may result in some kind of memory error from the kernel, it may result in the platform dynamically allocating more stack space or it may result in overwriting of other memory.
On platforms that use protection pages beyond the stack for automatic stack growth and/or detecting stack overflows a sufficiently large stack allocation may "skip over" those pages. This may lead to memory protection errors or worse memory corruption in cases where the stack overflow would normally have been caught.
Neither of these risks is specific to variable length arrays but variable length arrays make them far more likely to lie hidden until the code is invoked with particular parameters.
If I define a structure...
struct LinkNode
{
int node_val;
struct LinkNode *next_node;
};
and then create a pointer to it...
struct LinkNode *mynode = malloc(sizeof(struct LinkNode));
...and then finally free() it...
free(mynode);
...I can still access the 'next_node' member of the structure.
mynode->next_node
My question is this: which piece of the underlying mechanics keeps track of the fact that this block of memory is supposed to represent a struct LinkNode? I'm a newbie to C, and I expected that after I used free() on the pointer to my LinkNode, that I would no longer be able to access the members of that struct. I expected some sort of 'no longer available' warning.
I would love to know more about how the underlying process works.
The compiled program no longer has any knowledge about struct LinkedNode or field named next_node, or anything like that. Any names are completely gone from the compiled program. The compiled program operates in terms of numerical values, which can play roles of memory addresses, offsets, indices and so on.
In your example, when you read mynode->next_node in the source code of your program, it is compiled into machine code that simply reads the 4-byte numerical value from some reserved memory location (known as variable mynode in your source code), adds 4 to it (which is offset of the next_node field) and reads the 4-byte value at the resultant address (which is mynode->next_node). This code, as you can see, operates in terms of integer values - addresses, sizes and offsets. It does not care about any names, like LinkedNode or next_node. It does not care whether the memory is allocated and/or freed. It does not care whether any of these accesses are legal or not.
(The constant 4 I repeatedly use in the above example is specific for 32-bit platforms. On 64-bit platforms it would be replaced by 8 in most (or all) instances.)
If an attempt is made to read memory that has been freed, these accesses might crash your program. Or they might not. It is a matter of pure luck. As far as the language is concerned, the behavior is undefined.
There isn't and you can't. This is a classic case of undefined behavior.
When you have undefined behavior, anything can happen. It may even appear to work, only to randomly crash a year later.
It works by pure luck, because the freed memory has not yet been overwritten by something else. Once you free the memory, it is your responsibility to avoid using it again.
No part of the underlying Memory keeps track of it. It's just the semantics the programming language gives to the chunk of memory. You could e.g. cast it to something completely different and can still access the same memory region. However the catch here is, that this is more likely to lead to errors. Especially type-safty will be gone. In your case just because you called free doesn't mean that the underlying memory canges at all. There is just a flag in your operating system that marks this region as free again.
Think about it this way: the free-function is something like a "minimal" memory management system. If your call would require more than setting a flag it would introduce unneccessary overhead. Also when you access the member you (i.e. your operating system) could check if the flag for this memory region is set to "free" or "in use". But that's overhead again.
Of course that doesn't mean it wouldn't make sense to do those kind of things. It would avoid a lot of security holes and is done for example in .Net and Java. But those runtimes are much younger than C and we have much more ressources these days.
When your compiler translates your C code into executable machine code, a lot of information is thrown away, including type information. Where you write:
int x = 42;
the generated code just copies a certain bit pattern into a certain chunk of memory (a chunk that might typically be 4 bytes). You can't tell by examining the machine code that the chunk of memory is an object of type int.
Similarly, when you write:
if (mynode->next_node == NULL) { /* ... */ }
the generated code will fetch a pointer sized chunk of memory by dereferencing another pointer-sized chunk of memory, and compare the result to the system's representation of a null pointer (typically all-bits-zero). The generated code doesn't directly reflect the fact that next_node is a member of a struct, or anything about how the struct was allocated or whether it still exists.
The compiler can check a lot of things at compile time, but it doesn't necessarily generate code to perform checks at execution time. It's up to you as a programmer to avoid making errors in the first place.
In this specific case, after the call to free, mynode has an indeterminate value. It doesn't point to any valid object, but there's no requirement for the implementation to do anything with that knowledge. Calling free doesn't destroy the allocated memory, it merely makes it available for allocation by future calls to malloc.
There are a number of ways that an implementation could perform checks like this, and trigger a run-time error if you dereference a pointer after freeing it. But such checks are not required by the C language, and they're generally not implemented because (a) they would be quite expensive, making your program run more slowly, and (b) checks can't catch all errors anyway.
C is defined so that memory allocation and pointer manipulation will work correctly if your program does everything right. If you make certain errors that can be detected at compile time, the compiler can diagnose them. For example, assigning a pointer value to an integer object requires at least a compile-time warning. But other errors, such as dereferencing a freed pointer, cause your program to have undefined behavior. It's up to you, as a programmer, to avoid making those errors in the first place. If you fail, you're on your own.
Of course there are tools that can help. Valgrind is one; clever optimizing compilers are another. (Enabling optimization causes the compiler to perform more analysis of your code, and that can often enable it to diagnose more errors.) But ultimately C is not a language that holds your hand. It's a sharp tool -- and one that can be used to build safer tools, such as interpreted languages that do more run-time checking.
You need to assign NULL to mynode->next_node:
mynode->next_node = NULL;
after freeing the memory so it will indicate that you are not using anymore the memory allocated.
Without assigning the NULL value, it is still pointing to the previously freed memory location.
I read somewhere that it is disastrous to use free to get rid of an object not created by calling malloc, is this true? why?
That's undefined behavior - never try it.
Let's see what happens when you try to free() an automatic variable. The heap manager will have to deduce how to take ownership of the memory block. To do so it will either have to use some separate structure that lists all allocated blocks and that is very slow an rarely used or hope that the necessary data is located near the beginning of the block.
The latter is used quite often and here's how i is supposed to work. When you call malloc() the heap manager allocates a slightly bigger block, stores service data at the beginning and returns an offset pointer. Smth like:
void* malloc( size_t size )
{
void* block = tryAlloc( size + sizeof( size_t) );
if( block == 0 ) {
return 0;
}
// the following is for illustration, more service data is usually written
*((size_t*)block) = size;
return (size_t*)block + 1;
}
then free() will try to access that data by offsetting the passed pointer but if the pointer is to an automatic variable whatever data will be located where it expects to find service data. Hence undefined behavior. Many times service data is modified by free() for heap manager to take ownership of the block - so if the pointer passed is to an automatic variable some unrelated memory will be modified and read from.
Implementations may vary but you should never make any specific assumptions. Only call free() on addresses returned by malloc() family functions.
By the standard, it's "undefined behavior" - i.e. "anything can happen". That's usually bad things, though.
In practice: free'ing a pointer means modifying the heap. C runtime does virtually never validate if the pointer passed comes from the heap - that would be to costly in either time or memory. Combine these two factoids, and you get "free(non-malloced-ptr) will write something somewhere" - the resutl may be some of "your" data modified behind your back, an access violation, or trashing vital runtime structures, such as a return address on the stack.
Example: A disastrous scenario:
Your heap is implemented as a simple list of free blocks. malloc means removing a suitable block from the list, free means adding it to the list again. (a typical if trivial implementation)
You free() a pointer to a local variable on the stack. You are "lucky" because the modification goes into irrelevant stack space. However, part of the stack is now on your free list.
Because of the allocator design and your allocation patterns, malloc is unlikely to return this block. Later, in an completely unrelated part of the program, you actually do get this block as malloc result, writing to it trashes some local variables up the stack, and when returning some vital pointer contains garbage and your app crashes. Symptoms, repro and location are completely unrelated to the actual cause.
Debug that.
It is undefined behaviour. And logically, if behaviour is undefined, you cannot be sure what has happened, and if the program is still operating properly.
Some people have pointed out here that this is "undefined behavior". I'm going to go farther and say that on some implementations, this will either crash your program or cause data corruption. It has to do with how "malloc" and "free" are implemented.
One possible way to implement malloc/free is to put a small header before each allocated region. On a malloc'd region, that header would contain the size of the region. When the region is freed, that header is checked and the region is added to the appropriate freelist. If this happens to you, this is bad news. For example, if you free an object allocated on the stack, suddenly part of the stack is in the freelist. Then malloc might return that region in response to a future call, and you'll scribble data all over your stack. Another possibility is that you free a string constant. If that string constant is in read-only memory (it often is), this hypothetical implementation would cause a segfault and crash either after a later malloc or when free adds the object to its freelist.
This is a hypothetical implementation I am talking about, but you can use your imagination to see how it could go very, very wrong. Some implementations are very robust and are not vulnerable to this precise type of user error. Some implementations even allow you to set environment variables to diagnose these types of errors. Valgrind and other tools will also detect these errors.
Strictly speaking, this is not true. calloc() and realloc() are valid object sources for free(), too. ;)
Please have a look at what undefined behavior means. malloc() and free() on a conforming hosted C implementation are built to standards. The standards say the behavior of calling free() on a heap block that was not returned by malloc() (or something wrapping it, e.g. calloc()) is undefined.
This means, it can do whatever you want it to do, provided that you make the necessary modifications to free() on your own. You won't break the standard by making the behavior of free() on blocks not allocated by malloc() consistent and even possibly useful.
In fact, there could be platforms that (themselves) define this behavior. I don't know of any, but there could be some. There are several garbage collecting / logging malloc() implementations that might let it fail more gracefully while logging the event. But thats implementation , not standards defined behavior.
Undefined simply means don't count on any kind of consistent behavior unless you implement it yourself without breaking any defined behavior. Finally, implementation defined does not always mean defined by the host system. Many programs link against (and ship) uclibc. In that case, the implementation is self contained, consistent and portable.
It would certainly be possible for an implementation of malloc/free to keep a list of the memory blocks thats been allocated and in the case the user tries to free a block that isn't in this list do nothing.
However since the standard says that this isn't a requirement most implementation will treat all pointers coming into free as valid.
I'm trying to build my own Hash Table in C from scratch as an exercise and I'm doing one little step at a time. But I'm having a little issue...
I'm declaring the Hash Table structure as pointer so I can initialize it with the size I want and increase it's size whenever the load factor is high.
The problem is that I'm creating a table with only 2 elements (it's just for testing purposes), I'm allocating memory for just those 2 elements but I'm still able to write to memory locations that I shouldn't. And I also can read memory locations that I haven't written to.
Here's my current code:
#include <stdio.h>
#include <stdlib.h>
#define HASHSIZE 2
typedef char *HashKey;
typedef int HashValue;
typedef struct sHashTable {
HashKey key;
HashValue value;
} HashEntry;
typedef HashEntry *HashTable;
void hashInsert(HashTable table, HashKey key, HashValue value) {
}
void hashInitialize(HashTable *table, int tabSize) {
*table = malloc(sizeof(HashEntry) * tabSize);
if(!*table) {
perror("malloc");
exit(1);
}
(*table)[0].key = "ABC";
(*table)[0].value = 45;
(*table)[1].key = "XYZ";
(*table)[1].value = 82;
(*table)[2].key = "JKL";
(*table)[2].value = 13;
}
int main(void) {
HashTable t1 = NULL;
hashInitialize(&t1, HASHSIZE);
printf("PAIR(%d): %s, %d\n", 0, t1[0].key, t1[0].value);
printf("PAIR(%d): %s, %d\n", 1, t1[1].key, t1[1].value);
printf("PAIR(%d): %s, %d\n", 3, t1[2].key, t1[2].value);
printf("PAIR(%d): %s, %d\n", 3, t1[3].key, t1[3].value);
return 0;
}
You can easily see that I haven't allocated space for (*table)[2].key = "JKL"; nor (*table)[2].value = 13;. I also shouldn't be able read the memory locations in the last 2 printfs in main().
Can someone please explain this to me and if I can/should do anything about it?
EDIT:
Ok, I've realized a few things about my code above, which is a mess... But I have a class right now and can't update my question. I'll update this when I have the time. Sorry about that.
EDIT 2:
I'm sorry, but I shouldn't have posted this question because I don't want my code like I posted above. I want to do things slightly different which makes this question a bit irrelevant. So, I'm just going to assume this was question that I needed an answer for and accept one of the correct answers below. I'll then post my proper questions...
Just don't do it, it's undefined behavior.
It might accidentially work because you write/read some memory the program doesn't actually use. Or it can lead to heap corruption because you overwrite metadata used by the heap manager for its purposes. Or you can overwrite some other unrelated variable and then have hard times debugging the program that goes nuts because of that. Or anything else harmful - either obvious or subtle yet severe - can happen.
Just don't do it - only read/write memory you legally allocated.
Generally speaking (different implementation for different platforms) when a malloc or similar heap based allocation call is made, the underlying library translates it into a system call. When the library does that, it generally allocates space in sets of regions - which would be equal or larger than the amount the program requested.
Such an arrangement is done so as to prevent frequent system calls to kernel for allocation, and satisfying program requests for Heap faster (This is certainly not the only reason!! - other reasons may exist as well).
Fall through of such an arrangement leads to the problem that you are observing. Once again, its not always necessary that your program would be able to write to a non-allocated zone without crashing/seg-faulting everytime - that depends on particular binary's memory arrangement. Try writing to even higher array offset - your program would eventually fault.
As for what you should/should-not do - people who have responded above have summarized fairly well. I have no better answer except that such issues should be prevented and that can only be done by being careful while allocating memory.
One way of understanding is through this crude example: When you request 1 byte in userspace, the kernel has to allocate a whole page atleast (which would be 4Kb on some Linux systems, for example - the most granular allocation at kernel level). To improve efficiency by reducing frequent calls, the kernel assigns this whole page to the calling Library - which the library can allocate as when more requests come in. Thus, writing or reading requests to such a region may not necessarily generate a fault. It would just mean garbage.
In C, you can read to any address that is mapped, you can also write to any address that is mapped to a page with read-write areas.
In practice, the OS gives a process memory in chunks (pages) of normally 8K (but this is OS-dependant). The C library then manages these pages and maintains lists of what is free and what is allocated, giving the user addresses of these blocks when asked to with malloc.
So when you get a pointer back from malloc(), you are pointing to an area within an 8k page that is read-writable. This area may contain garbage, or it contain other malloc'd memory, it may contain the memory used for stack variables, or it may even contain the memory used by the C library to manage the lists of free/allocated memory!
So you can imagine that writing to addresses beyond the range you have malloc'ed can really cause problems:
Corruption of other malloc'ed data
Corruption of stack variables, or the call stack itself, causing crashes when a function return's
Corruption of the C-library's malloc/free management memory, causing crashes when malloc() or free() are called
All of which are a real pain to debug, because the crash usually occurs much later than when the corruption occurred.
Only when you read or write from/to the address which does not correspond to a mapped page will you get a crash... eg reading from address 0x0 (NULL)
Malloc, Free and pointers are very fragile in C (and to a slightly lesser degree in C++), and it is very easy to shoot yourself in the foot accidentally
There are many 3rd party tools for memory checking which wrap each memory allocation/free/access with checking code. They do tend to slow your program down, depending on how much checking is applied..
Think of memory as being a great big blackboard divided into little squares. Writing to a memory location is equivalent to erasing a square and writing a new value there. The purpose of malloc generally isn't to bring memory (blackboard squares) into existence; rather, it's to identify an area of memory (group of squares) that's not being used for anything else, and take some action to ensure that it won't be used for anything else until further notice. Historically, it was pretty common for microprocessors to expose all of the system's memory to an application. An piece of code Foo could in theory pick an arbitrary address and store its data there, but with a couple of major caveats:
Some other code `Bar` might have previously stored something there with the expectation that it would remain. If `Bar` reads that location expecting to get back what it wrote, it will erroneously interpret the value written by `Foo` as its own. For example, if `Bar` had stored the number of widgets that were received (23), and `Foo` stored the value 57, the earlier code would then believe it had received 57 widgets.
If `Foo` expects the data it writes to remain for any significant length of time, its data might get overwritten by some other code (basically the flip-side of the above).
Newer systems include more monitoring to keep track of what processes own what areas of memory, and kill off processes that access memory that they don't own. In many such systems, each process will often start with a small blackboard and, if attempts are made to malloc more squares than are available, processes can be given new chunks of blackboard area as needed. Nonetheless, there will often be some blackboard area available to each process which hasn't yet been reserved for any particular purposes. Code could in theory use such areas to store information without bothering to allocate it first, and such code would work if nothing happened to use the memory for any other purpose, but there would be no guarantee that such memory areas wouldn't be used for some other purpose at some unexpected time.
Usually malloc will allocate more memory than you require to for alignment purpose. Also because the process really have read/write access to the heap memory region. So reading a few bytes outside of the allocated region seldom trigger any errors.
But still you should not do it. Since the memory you're writing to can be regarded as unoccupied or is in fact occupied by others, anything can happen e.g. the 2nd and 3rd key/value pair will become garbage later or an irrelevant vital function will crash due to some invalid data you've stomped onto its malloc-ed memory.
(Also, either use char[≥4] as the type of key or malloc the key, because if the key is unfortunately stored on the stack it will become invalid later.)
I was told by a rather smart person that you cannot implement garbage collection in C because of it's weakly typed. The basic idea seems to be that C gives you too much freedom. He mentioned casting pointers without type checking...
I don't really grok the idea. Can someone give me an explanation and possibly a code sample of why this wouldn't work.
NOTE: Obviously C is about speed and why would you want to add garbage collection? I'm just curious really.
He probably referred to the fact that you can cast a pointer to an int and back to the original pointer type. It's pretty much impossible for a GC to clean up correctly when you do that, consider:
char * p = (char *) malloc(16);
int i = (int) p;
p = 0;
// GC runs and finds that the memory is no longer referenced
p = (char *) i;
// p is now a dangling pointer
EDIT: The above will only produce a dangling pointer with a precise GC. As others have pointed out, a conservative collector can still correctly handle this scenario as it assumes that any bit pattern that could be a valid pointer actually is a pointer and will thus not free the memory allocated. However, this is of course no longer possible when i is further modified such that it no longer looks like a valid pointer to the collector, e.g. as follows:
char * p = (char *) malloc(16);
int i = ~((int) p);
p = 0;
// GC runs and finds that the memory is no longer referenced
p = (char *) ~i;
// p is now a dangling pointer
Moreover, (again as others have pointed out) it's only impossible to implement a GC for C if you want to retain the full functionality of the language. If you refrain from using tricks like the above (i.e. you confine yourself to a subset of the possible operations) then GC is indeed feasible.
It's perfectly possible to implement whatever memory manager you can think of in C. The catch is that you then have to use its allocation/deallocation functions exclusively and restrict your 'pointer magic' to things it can keep track of. Aditionally, the memory management might be restricted to certain supported types.
For example, Objective-C's retain/release system and autorelease pools are basically memory managers implemented in C. Many libraries also implement their own, simple form of memory management like reference counting.
Then, there's the Boehm garbage collector. To use it, just replace your malloc()/realloc() calls with the Boehm versions and you never have to call free() again. Read about the possible issues with this approach.
Also, check this wikipedia page for a quick overview on how conservative garbage collectors work.
If you read the right papers and you have a bachelor's degree in CS, it's actually pretty easy to implement a decent conservative garbage collector for C---I have a dozen students who have done it as a class exercise taking about four weeks. Then spend 20 years improving it and you get the Boehm collector (libgc).
The basic idea is simple: if there is a bit pattern anywhere in a register, on the stack, in a global variable, or in a live heap object, and that bit pattern happens to be an address that falls inside an object allocated with malloc, than that object is considered live. Any object that is not live cannot possibly be reached by following pointers, and so it can be reclaimed and used to satisfy future allocation requests. This technique operates on the hardware representation of pointers, and it is completely independent of the type of pointer---types are irrelevant here.
It is true there is a caveat: conservative garbage-collection techniques can be fooled by willfully hiding pointers. Compress pointer-containing structures, keep the only copy of a pointer on disk, obfuscate a pointer by XORing 0xdeadbeef, and all these techniques will break a conservative collector. But this kind of problem is extremely rare unless done deliberately. Authors of optimizing compilers are usually careful not to hide pointers from such a collector.
The most interesting part of your question is why do it. Three reasons:
It eliminates the possibility of many memory-manangement bugs.
It simplifies your APIs because it is no longer necessary to specify who allocates memory, who owns the allocated memory, whether it's necessary to copy memory, and who is responsible for freeing memory.
Believe it or not, it can be faster than using malloc and free.
It's not impossible to implement a garbage collector for C (and in fact, they do exist, as a simple google search reveals), it's just difficult, because it can be difficult to determine if a certain string of bits is a pointer into an allocated block or just looks like one.
The reason this is an issue is because C (and C++, for that matter) allows you to cast from a pointer type to an integral type, so an integer variable might hold an address within an allocated block, preventing the GC from freeing that block, even though that value wasn't intended to be a pointer.
For example, let's say I have a block of memory allocated. Let's suppose this block of memory is allocated starting at address 0x00100000 (1,048,576), and is 1 MB long, so extends to 0x001FFFFF (2,097,151).
Let's say I also am storing the size of an image file in a variable (let's call it fileSize). This image file happens to be 1.5 MB (1,572,864 bytes).
So when the garbage collector runs, it will come across my fileSize variable, find it containing a value that corresponds to an address within my allocated block, and decide that it cannot free this block, lest it invalidate my maybe pointer. That's because the GC doesn't know if I've done this:
int fileSize;
{
char *mem = (char*)malloc(1048576);
fileSize = (int)(mem + 524288);
}
// say GC runs here
or if I've just done this:
int fileSize;
{
char *mem = (char*)malloc(1048576);
fileSize = 1572864;
}
// say GC runs here;
In the latter case, it is safe to free the block at *mem, (if no other references exist), whereas in the former, it's not. It must be conservative and assume that it's not, so the memory "leaks" (at least until fileSize goes out of scope or is changed to a value outside the allocated block).
But garbage collectors for C (and C++) do exist. Whether or not they are valuable is a matter for a different discussion.
The problem is that there's no way for the runtime to know for certain if any piece of memory is referenced or not. Even if you wrap all memory allocation in code that registers the usage, you can still obtain pointers to used memory via regular pointer manipulation (or by mistake). Casts only make the problem harder for the runtime. So if the runtime frees a piece of memory, it will mess things up for any pointers still pointing to that area of memory. Obviously the situation only gets worse when you consider that garbage collection has to work for multi-threaded applications as well.
It's impossible to implement a precise garbage collector for C because of the freedoms afforded C's pointers and the fact that the length of a C array is anyone's guess. This means a lot of sophisticated garbage collection approaches can't be used. (Copying and compacting garbage collectors come to mind.)
It is, however, possible to implement a conservative garbage collector (boehm), which basically assumes everything that looks like it might be a pointer is a pointer. This isn't very efficient, but it works for a suitably lenient definition of "works".
C is not weakly typed, but this code illustrates the difficulty in building a garbage collector into the language:
#include <stdio.h>
#include <stdlib.h>
int GetSomeMemory() {
char* pointerToHeapMemory = malloc(10);
return (int)pointerToHeapMemory;
}
int main() {
int memoryAddress = GetSomeMemory();
/* at this point a garbage collector might decide to clear up the memory that
* was allocated in GetSomeMemory on the grounds that pointerToHeapMemory
* is no longer in scope. But the truth is we still know about that memory and
* we're about to use it again... */
char* anotherPointerToHeapMemory = (char*) memoryAddress;
sprintf(anotherPointerToHeapMemory, "123456789\0");
printf("%s\n", anotherPointerToHeapMemory);
}
Garbage collection can be done so long as everyone working on a project agrees to avoid this kind of thing and use a common set of functions for allocating and accessing memory. For example, this is a C garbage collector implementation