I'm reading a lot about malloc() and free() in Standard C. As I understand it, you malloc() for some memory exactly once and then you free() that same memory exactly once. It may be bad practice, but I understand that after you malloc() memory, you can define multiple pointers to it. And once you free() any of those pointers, the allocated memory is de-allocated?
Consider this toy example:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(){
char* p = (char*)malloc(10 * sizeof(char)); // allocate memory
int* q = (int*)p; // pointer to the same block of memory
*p = 'A'; // Input some data
printf("TEST:: %c %d\n", *p, *q); // Everything's ok so far...
free(p); // free() my allocated memory?
sleep(10); // wait
printf("%c\n", *q); // q now points to de-allocated memory
// shouldn't this segfault?
free(q); // *** SEGFAULTS HERE ***
return 0;
}
Output is:
[Linux]$ ./a.out
TEST:: A 65
*** Error in `./a.out': double free or corruption (fasttop): 0x0000000001ac4010 ***
======= Backtrace: =========
...lots of backtrack info...
So I assume that when I free() the first pointer, the memory is considered free()ed, but the data value(s) I wrote in this block of memory are still "there", which is why I can access them via the second pointer?
(I'm not proposing that this is a good idea, I'm trying to understand the logic of the system.)
When you malloc memory, you're given a pointer to some space, and when you free it, you're giving it back to the system. Often, you can still access this memory, but using memory after you have freed it is VERY BAD.
The exact behavior is undefined, but on most systems you can either continue to access the memory, or you get a segfault.
One interesting experiment you can try is to try and malloc more memory after you free'd that pointer. On most systems I've tried, you get the same block back (which is a problem, if you were relying on the data being there in the freed block). Your program would end up using both pointers, but since they point to the same physical data, you'll be overwriting your own data!
The reason for this is that when you malloc data (depending on the malloc implementation of course), malloc first requests a block of data from the operating system (typically much larger than the malloc request), and malloc will give you a segment of that memory. You'll be able to access any part of the memory malloc originally got from the operating system though, since to the operating system, it's all memory your program is internally using. When you make a free, you're telling the malloc system that the memory is free, and can be given back to the program later on.
Writing outside of the malloc area is very dangerous because
It can segfault, depending on your c implementation
You can overwrite metadata structures malloc is relying on, which causes VERY BAD PROBLEMS when you free/malloc more data later on
If you are interested in learning more, I would recommend running your program through valgrind, a leak detector, to get a better picture of what's freed/not freed.
PS: On systems without an OS, you most likely wont get a segfault at all, and you'll be able to wite anywhere willy nilly. The OS is responsible for triggering a segfault (when you write/read to memory you don't have access to, like kernel or protected memory)
If you are interested in learning more, you should try to write your own malloc, and/or read/learn about the memory management operating systems do.
The crash in your code is due to double free. Appendix J.2 of C11 says that behaviour is undefined for example when:
The pointer argument to the free or realloc function does not match a pointer earlier returned by a memory management function, or the space has been deallocated by a call to free or realloc (7.22.3.3, 7.22.3.5).
However it is possible to write code that will crash on Linux just by reading a value from memory that was just freed.
In glibc + Linux there are two different mechanisms of memory allocations. One uses the brk/sbrk to resize the data segment, and the other uses the mmap system call to ask the operating system to give large chunks of memory. The former is used for small allocations, like your 10 characters above, and mmap for large chunks. So you might get a crash by even accessing the memory just after free:
#include <stdio.h>
#include <stdlib.h>
int main(){
char* p = malloc(1024 * 1024);
printf("%d\n", *p);
free(p);
printf("%d\n", *p);
}
And finally, the C11 standard says that the behaviour is undefined even when
The value of a pointer that refers to space deallocated by a call to the free or realloc function is used (7.22.3).
This means that after not only that dereferencing the pointer (*p) has undefined behaviour, but also that it is not safe to use the pointer in any other way, even doing p == NULL has UB. This follows from C11 6.2.4p2 that says:
The value of a pointer becomes indeterminate when the object it points to (or just past) reaches the end of its lifetime.
Related
I'm starting to learn C programming and stumbled upon a situation that shows my lack of understanding of how memory allocation with malloc() works.
In a loop like the following:
// will only know `xlen` at runtime so I guess I have to use malloc here
char *x = malloc((xlen + 1) * sizeof(*x));
assert(x != NULL); // memory allocation failed
while (fgets(line, sizeof(line), fp) != NULL) {
strncpy(x, line, xlen); // copy first xlen characters of line
x[xlen] = '\0'; // ensure null terminating string
// do something with x
// x can now be "reused" for another line/loop iteration
// free(x) - errors out with "pointer being freed was not allocated"
}
fclose(fp)
free(x);
If the statement free(x) is called inside the loop then when I run this program I get an error message something like a.out(37575,0x7fff964ce3c0) malloc: *** error for object 0x7fab47c02630: pointer being freed was not allocated.
Why am I seeing this error message and what is causing it?
Will the memory address block of x be "re-used" at each iteration of the loop? (I'd say yes, and that would actually be what I wanted in this case) In this case, is it safe to only free the allocated memory of x outside the scope of the loop?
free(x); frees the memory allocated by char *x = malloc(.... It frees all of the memory, and you don't have to worry about how much memory that was, as it keeps track of that. You just have to call free(x); once as you correctly do. This is why you get the error if you free it inside the loop.
Does it mean that the memory address block of x will be "re-used" at
each iteration? (I'd say yes, and that would actually be what I wanted
in this case)
Yes, you use the same memory. It overwrites the memory every time.
In this case, is it safe to only free the allocated memory of x
outside the loop scope?
Yes, you have to free it outside of the loop. Because if you free it, then all of it is freed. If you did that inside the loop, then it would be undefined behavior to keep accessing that memory in following loop iterations.
The malloc() function provides a pointer to an area of memory that can then be used just like any pointer to a memory location.
The way to think of malloc() and free() is to consider it in terms of who owns a memory area. The malloc() function owns the memory that it is managing. When you call the malloc() function, or calloc() function, the function takes some of the memory it is managing and transfers ownership to you. When the function is called and the memory allocated there is a transfer of ownership from the memory management system of malloc() to you and at that point you own that memory and it is up to you to manage it. When the free() function is called to release the memory, you are transferring ownership of the memory area from you back to the memory management system of malloc().
Whenever a transfer of ownership is done, the entity that is giving up ownership is not supposed to do anything further with the memory area since it no longer owns it.
So when you do the malloc() you now own the memory and can use and reuse the memory area as much or as little as you want until such time that you call the free() function to give it back to malloc().
A pointer to a memory region given by malloc() is like any other pointer so far as using it is concerned.
Once you have transferred ownership of a memory area back to malloc() by using thee free() function, you can't call free() again with the same pointer without introducing an error. Once a memory area has been returned using free() you no longer have ownership of the memory so you shouldn't try to do anything else with it and that includes calling free() with the pointer again.
How malloc() manages memory is an implementation detail that will vary depending on the compiler and the C Runtime used. In some cases if you malloc() memory then free() the memory and then malloc() the same amount of memory again you may get lucky and get the same memory area again however it is not something you can count on or should expect.
malloc() provides you a pointer to a memory area that has a couple of guarantees. First of all the memory area provided will be at least as large as you requested. It may be larger but it won't be smaller than what you requested. Secondly the address provided will be on a memory boundary suitable for the machine architecture so that you can use the address provided for any variable type whether a built in such as int or double or a struct or an array of some type.
As the owner of the memory area provided by malloc() you are responsible for giving the memory area back to the malloc() functionality once you are done with it.
Your other primary responsibility is to respect the size of the memory area and to not write a larger data block into the area than the size requested when the malloc() was done. You are guaranteed only the amount of memory requested and writing a larger block of data into the memory will probably overwrite memory that you do not own but is owned by something else.
Unfortunately because the malloc() functionality, at least the optimized or non-debug version of the library, is designed with little as possible overhead, the functionality has few consistency and sanity checks in place. The malloc() functionality trusts that you will obey the rules and guidelines and only use what you own. And often when you break the rules and guidelines, you won't see the effect at the point where you have made a mistake but rather at some other point when some mysterious memory corruption is discovered and your program crashes.
You can use and reuse dynamically allocated memory as long as you have not called free() on it. If you free it and attempt to use it again, then you will have undefined behaviour.
You can continue to reusing memory (eg buf created by char *buf = calloc(sizeOfBuf); ) allocated to you on a single call to calloc(), until free(buf) is called, just as if buf was created statically, eg char buf[sizeOfBuf]; = 0. However, if the size of the buffer initially created by using calloc() needs to change, realloc() is available to do this, and is the preferable method. There are some caveats about using realloc however. Here is an example of using realloc, this one packaged into a function that resizes an existing dynamically allocated buffer, and takes care of some of the caveats:
realloc usage example:
// initial call to calloc
char *buf = calloc(sizeOfBuf);
if(buf)
{
while(someConditionIsTrue)
{
// read new content
//...
//new content needs to be added to buf
char *tmp = realloc(buf, newSizeOfBuffer);
if(!tmp)//if failed...
{
free(buf);//... free original buf to prevent memory loss
return NULL;// return null, caller must test for this
}
buf = tmp;//...else copy new mem location back to original pointer
//continue doing stuff with buf
}
//when done with buf, free it
free(buf);
}
One other suggestion, consider calloc() returns uninitialized memory, i.e. you own a space that contains whatever was occupying it when it was given to you. It is a good idea to follow that command with a method to clean up the buffer:
memset(buf, 0, sizeOfBuf);
Or use calloc().
If the return value of a function, say, myFunc, is an allocated space. Does the parent function still need to free that part of the memory?
For example:
char * myFunc() {
return malloc(5);
// free() not called
}
int main() {
char * str = myFunc();
// does str need to be freed?
return 0;
}
To free() or not to free()?
When a process terminates, all of its memory is returned to the system, including heap memory allocated by functions in the malloc package. In programs that allocate memory and continue using it until program termination, it is common to omit calls to free(), relying on this behavior to automatically free the memory. This can be especially useful in programs that allocate many blocks of memory, since adding multiple calls to free() could be expensive in terms of CPU time, as well as perhaps being complicated to code.
Although relying on process termination to automatically free memory is acceptable for many programs, there are a couple of reasons why it can be desirable to explicitly free all allocated memory:
1. Explicitly calling free() may make the program more readable and maintainable in the face of future modifications.
2. If we are using a malloc debugging library to find memory leaks in a program, then any memory that is not explicitly freed will be reported as a memory leak. This can complicate the task of finding real memory leaks.
3. To avoid memory leak.
It is recommended that str is to be freed: replace // does str need to be freed? with free(str). This is good programming practice to avoid potential memory leaks.
In your case, the OS will clean up the allocated memory taken by this short program.
I wish to free blocks of memory which I don't have pointers to. In my program, I call malloc sequentially, hoping that the memory created by malloc(1), malloc(4), malloc(5) is continuous. Then I free these memory when I only have the pointer to malloc(5). But I can't think of how this can be done; I cannot simply create a pointer that reference to the address of ptr[-5] and then free 5 bytes of memory? How can this be done?
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
int main(){
malloc(1);
malloc(4);
char* ptr = malloc(5);
free(ptr);
}
You cannot do what you want to do. You should not even try to do what you want to do.
Even if you work out exactly what malloc() is doing, your program would then be relying on undefined behavior. The behavior could change when a new version of the C library arrives, and your program would almost certainly fail if you compiled it using a different toolchain (switch from GNU C to Microsoft C or whatever).
Any time you allocate memory, you need to keep track of the pointer. If your program doesn't even know about the memory, there is no way to free it.
Keep track of your memory allocations. If you are designing data structures to be dynamically allocated, your design should include features to track them, such as keeping a list of addresses in a linked list or something.
If this seems like a lot of work, maybe consider using a managed language like C# or Java or Python or whatever.
free(void*)
[deallocate] A block of memory previously allocated by a call to malloc, calloc or realloc is deallocated, making it available again for further allocations.
If ptr does not point to a block of memory allocated with the above functions, it causes undefined behavior.
- http://www.cplusplus.com/reference/cstdlib/free/
There is no way.
But I can't think of how this can be done
That's because it is not possible. The blocks that you get back from malloc can come in truly arbitrary order. The only way to free a dynamically allocated block of memory is to keep a pointer to it accessible to your program. Anything else is undefined behavior.
Note: Implementations of malloc perform "bookkeeping" to figure out what kind of block you are releasing. While it is not impossible to hack into their implementation, there is no way of doing it in a standard-compliant, portable way.
You cannot create a [-5]...thing for a variety of reasons but the from a practical standpoint you have to remember that memory allocated with malloc() is coming off of the heap and not the stack so to "count" to it from somewhere else is difficult (since multiple calls to malloc are not guaranteed to be sequential).
What happens when a pointer loses its association to memory (or goes out of scope) without being freed is called a memory leak and without exhaustive techniques not readily available in C (Java's mark/sweep garbage collection for example, or mallocing the entire memory and scanning it or something) it is not possible to reclaim this memory.
So you cannot free memory in C when a pointer is not known.
First of all - as it seems you do not understand how malloc works - passing continuous numbers to malloc, won't make it allocate an array. malloc is defined as follows:
void* malloc (size_t size);
While an integer can be converted to size_t, it's still the number of bytes allocated, not the element number. If you want to allocate an array, do it as follows:
int* myDynamicArray = malloc(sizeof(int)*numberOfElements);
Then, you can access the elements by doing:
int i;
for(i=0;i<numberOfElements;i++)
printf("%d",myDynamicArray[i]);
Then, like others pointed out - you can deallocate the memory by calling the free function. free is defined as follows:
void free (void* ptr);
And you simply call it by doing:
free(myDynamicArray);
This is by no means an endorsement of what you have done, but it is possible assuming you know that the blocks were allocated continuously.
For example:
int main(){
char* ptr1=malloc(1);
char* ptr2=malloc(4);
char* ptr3=malloc(5);
// Verify that the memory is in fact continuous.
assert(ptr3==(ptr2+4));
assert(ptr3==(ptr1+5));
free(ptr3); // Frees 5 bytes at ptr3
free(ptr3-4); // Frees 4 bytes at ptr2
free(ptr3-5); // Frees 1 byte at ptr1
}
So, you if you have a pointer and know for a fact that you allocated a set of continuous bytes before it, you can simply offset the pointer with pointer arithmetic. It is highly dangerous and not recommended, but it is possible.
Edit:
I ran a test program and on my architecture, it allocated in 32 byte chunks, so ptr1+32==ptr2, and ptr2+32=ptr3. It did this for any chunks less than or equal to 24 bytes. So if I allocated 24 or less, then each ptr would be 32 bytes greater than the previous. If I allocated 25 or more, then it allocated an additional 16 bytes, making the total 48.
So, in my architecture, you'd need to be much more creative in how you generate your pointers using pointer arithmetic since it will not work as expected.
Here is an example program that works for all sizes of ptr1, ptr2, and ptr3 on my architecture.
#define ROUNDUP(number, multiple) (((number + multiple -1)/multiple)*multiple)
#define OFFSET(size) ((size < 24) ? 32 : ROUNDUP(size+8,16))
int main(int argc, char* argv[]){
char* ptr1, *ptr2, *ptr3;
int s1=atoi(argv[1]);
int s2=atoi(argv[2]);
int s3=atoi(argv[3]);
ptr1=(char*)malloc(s1);
ptr2=(char*)malloc(s2);
ptr3=(char*)malloc(s3);
fprintf(stdout, "%p %p %p\n", ptr1, ptr2, ptr3);
assert(ptr3==(ptr2+OFFSET(s2)));
assert(ptr2==(ptr1+OFFSET(s1)));
// Try to construct ptr2 from ptr3.
free(ptr3);
free(ptr3-OFFSET(s2));
free(ptr3-OFFSET(s2)-OFFSET(s1));
}
For the code below:
(1) "main" calls a function "f1".
(2) function "f1" does some number crunching; creates an array of "char" with malloc and then, returns the pointer of the array to the main (without de-allocating -freeing- the array).
I have 3 questions related to the case:
(1) I assume, although the function "f1" has terminated, the allocated char array still stays allocated until the main program terminates completely. That is, the allocated memory still belongs to the main and no other process can access (I mean, interfere with) it from outside. Am I right?
(2) Do I have to free the array (allocated in "f1") before the program terminates (or does it get freed as soon as the main program terminates) ?
(3) If the answer for the second question is "yes" then how do you free an array allocated in another function?
note: I want to stay within the boundaries of pure c and not to spill over to c++.
char *f1 (...) {
...
...
char *fTmp = malloc (length1 * sizeof (char));
char *fData = malloc (length2 * sizeof (char));
...
...
free (fTmp);
return (fData);
}
int main () {
char *fData = f1 (...);
...
return (0);
}
I assume, although the function "f1" has terminated, the allocated char array still stays allocated until the main program terminates completely.
True. Dynamically allocated memory has nothing to do with functions, it belongs to process.
That is, the allocated memory still belongs to the main and no other process can access it from outside. Am I right?
Memory doesn't belong to main() (intended as function) but to process itself (of which main() is just the entry point). In a system with memory protection (where each process is isolated from the others) it's not accessible from outside. You can, however, allocate it in a system specific way to share memory across processes.
Do I have to free the array (allocated in "f1") before the program terminates (or does it get freed as soon as the main program terminates) ?
Yes. Unallocated memory - in most systems - is automatically deallocated by Operating System when process terminates but this is system dependant. IMO even when OS does it you should always deallocate, using such automatic deallocation as a red flag (I forget that to deallocate, is it a bug? something I missed?). Moreover if f1 is invoked 1000 times it'll leak memory each time quickly eating all available memory. Think about a process in a server, it may (and should) be up and running for years.
If the answer for the second question is "yes" then how do you free an array allocated in another function?
It's nice when who allocates memory also frees it. If it's not possible then caller will become responsible for such memory. It's, for example, what strdup() does. In such case called function must return (somehow) a pointer to allocated memory (or an handle/token that can be used by another specialized function). For example:
char* pBuffer = f1();
// Use it
free(pBuffer);
Note that there are many many techniques if you want to hide such internal pointer. You may use a token (for example an integer, key in a dictionary), a typedef or an opaque type.
Yes, memory allocated with malloc() stays until it is freed. How else could a function ever return variable-sized data to its caller?
When a program exits, all the memory it allocated with malloc() is freed. However, it's generally not a good idea to keep lots of unneeded memory around until the program terminates, as it can impact performance, or the system can run out of virtual memory. This can be a particular concern for long-running programs, their memory use sometimes keeps growing until they use of all the available virtual memory.
You call free() on the pointer returned by the function. So in your case, main() can do free(fData) after it's done using the array.
This should all be covered in any C programming class or textbook.
malloc allocates memory on heap and therefore this memory remains allocated until it is freed by free function or program terminate successfully.
In your case you freed ftemp in f1 so it is no longer exist after function terminates. fdata is still on heap and it is accessible to main as you are returning pointer to that allocated location.
Once main terminates successfully, memory pointed by fdata get freed.
So, it's considered good to free memory as soon as you don't need it any more. There is no point in freeing blocks at the end of a program, because all of the program's space is given back to the system when the process terminates (considering modern operating systems).
Yes, it's still in the heap. However, you are confusing about the concept of process. Unless you create another process (using fork on *nix), it's still the same process.
It's a good habit to free the memory when it's not used. But if the program terminates normally, the allocated memory is freed by the system.
Like this:
int main () {
char *fData = f1 (...);
//...
free(fData);
//...
}
Using malloc will allocate memory on the heap until it you free it.
This means you need to assure every malloc has a corresponding free, also it is not implied that no other process can't access your data. It's just a value at an address.
In your main you must free(fData) to avoid a memory leak.
To sum up then:
1) Your first assumption is correct, the second and third is not. It will stay allocated, but it's not local to the main, and not bound to the process as it terminates.
2) Yes, you must free it
3) Use the pointer you get from the function. If you do not return a pointer to your allocated data from a function, make sure that function frees it.
There are two basic types of memory you can work with in C. The two types are the stack and the heap. In general, the variables you create within a function will be allocated on the stack and will be freed when the function returns. Memory allocated in the heap will persist and you are obligated to manage that allocation within your program. Memory in the heap will remain allocated until you free is up using the pointer (memory address) that refers to the data block.
A little reading on both will help you understand. I'd point out that you have two instances of fData, each with their own scope. Both pointers point to the memory you allocate with:
char *fData = malloc (length2 * sizeof (char));
.. even though they pass in and out of scope as your code executes.
If you don't free memory you're not using, eventually this would accumulate- if you done this with many other pointers- and your program could run out of memory. After freeing a memory block using the free function, I'd also suggest assigning NULL to the pointer as this protects against dangling pointers as even if you've freed a pointer, if you try accessing it, you could get undefined behaviour, whereas access and operations on NULL pointers result in a crash, so you'd easily be able to trace the problem
Hi I have following code
#include <stdio.h>
#include <conio.h>
typedef struct test
{
int a;
int b;
int c[10];
}tester;
typedef struct done
{
tester* t;
int nn;
}doner;
void main()
{
doner d;
d.t = (tester*)malloc(sizeof(d.t));
d.t->a = 10;
d.t->c[0] = 10;
printf("%d\n", d.t->a);
getch();
return;
}
I think the statement:
d.t = (tester*)malloc(sizeof(d.t));
is incorrect it should be:
d.t = (tester*)malloc(sizeof(tester));
but when I run this code it is not crashing please let me the why is this.
The fact that it is not crashing is because it has undefined behavior. The correct code is the second one, but no need for casting.
d.t = malloc(sizeof(tester));
Also, You need to free the malloc'ed pointer.
On many system, the heap is not checked when writing to the malloc'ed buffer, but only when freeing the allocated memory. In such case, you will probably get some kind of crash when you free the memory.
The fact that it's not crashing is a big reason why these sort of memory allocation bugs are so insidious and hard to detect. Your program only allocates the one structure, and doesn't fill it up, so the fact that it runs past the amount of memory allocated to it doesn't affect anything else.
If your program made more use of dynamically-allocated memory, then either the calls to malloc/free would trigger a crash because your structure overwrote the heap's linking metadata, or other parts of the program writing to their own malloc'ed data would overwrite your structure. Either way, not pretty.
Yes, you're right. It should be sizeof(tester), because d.t is just a pointer.
Now if you write sizeof(d.t) you invoke Undefined Behavior which is not a guarantee of crash. The program may run correctly, run incorrectly, crash, or order a pizza. There is no guarantee of what will happen with a program that has undefined behavior, even prior to the construct that leads to it.
As freeing the malloced memory - in this small sample program you don't need to worry about it, because the system will free the memory after your program exits, but in general you should try to free whatever you've allocated so as to avoid memory leaks in larger programs.
By default, the linker asks the OS to allocate 1MiB of (stack) memory for the program at the start-up. Your program doesn't crash because all of the references are still in the same memory (same address space) which was reserved by the OS for your program. Technically you haven't allocated that memory, but as the pointers are still in the valid memory range so your program can access it.
This is just like, in most cases, you can write to d.t->c[10] (although valid indexes are 0-9)
Crashes occur when pointers are used which correspond to memory locations outside the allocated memory. Google Page Fault for detailed understanding, if you are interested.