Related
I am learning the basic of C programming and find it quite strange and difficult. Especially with dynamic memory allocation, pointer and similar things. I came upon this function and don't quite understand what is wrong with it.
char *strdup(const char *p)
{
char *q;
strcpy(q, p);
return q;
}
I think I have to malloc and free q. But the function " return q". Doesn't that mean that it will store q value in its own memory. So the data still be saved after the function executed?
When it is appropriate to use malloc? As I understand so far is that I have to malloc a new variable every time I need that variable declared in a function to be used elsewhere. Is that true? Is there any other situation where malloc is needed?
The type of q is a pointer, and pointers hold addresses -- so what you are returning is the address that pointer holds.
Until you give that pointer a valid address, it points off to who-knows-where, memory that you may or may not own and have the right to access. So, the strdup call will copy a string from the address held in p into some location you probably don't own.
If you had done a malloc first, and given q the results of the malloc, then q would hold a valid address, and your strdup would put the copy into memory that you did own (assuming you malloc'd enough space for the string -- a strlen on p would tell you how much you needed).
Then, when you returned q, you would be giving the caller the address as well. Any code with that address can see the string you put there. If some future code were to free that address, then what it holds is up in the air -- it could be anything at all.
So, you don't want to free q before you return the address that it holds -- you need to let the caller free the address it gets from you, when it is ready to do so.
In terms of when you malloc, yes, if you want to return an address that will remain viable after your function completes, you need to malloc it -- giving the caller the address of a local variable, for example, would be bad: the memory is freed when the function returns, you don't own it anymore.
Another typical use of malloc is for building up dynamic data structures like trees and lists -- you can't know how much memory you need up front, so you build the list or tree up as you need to, malloc'ing more memory for each node in the structure.
My personal rules are: use malloc() when an object is too big to put on the stack or/and when it has to live outside the scope of the current block. In your case, I believe, you should do something like the following:
char *strdup(const char *p)
{
char *q;
q = malloc(strlen(p) + 1);
if(NULL != q)
strcpy(q, p);
return q;
}
malloc(X) creates space (of size X bytes) on the heap for you to play with. The data that you write to these X byes stays put when your function returns, as a result, you can read what your strdup() function wrote to that space on the heap.
free() is used for freeing space on the heap. You can pass a pointer to this function that you obtained only as a result of a malloc() or realloc() call. This function may not clear out the data, it just means that a subsequent call to malloc() may return the address of the same space that you just freed. I say "may" because these are all implementaion defined and should not be relied upon.
In the piece of code you wrote, the strcpy() function copies the bytes one by one from p to q until it finds a \0 at the location pointed to by q (and then it copies the \0 as well). In order to write data somewhere, you need to allocate space first for the data to be written, hence, one option is that you call malloc() to create some space and then write data there.
Well, calling free() is not mandatory as your OS will reclaim the space allocated by malloc() when you program ends, but as long as your program runs, you may be ocupying more space than you need to - and that's bad for your program, for other programs, for the OS and the universe as a whole.
I think I have to malloc and free q. But the function " return q". Doesn't that mean that it will store q value in its own memory. So the data still be saved after the function executed?
No, your data won't be saved. In fact your pointer q is being used without allocating it's size can cause problems. Also, once this function execution complete, variable char* q will be destroyed.
You need to allocate memory to pointer q before copying data as suggested by #Michael's answer.But once you finish using the data return by this function, you will need to manually free() the memory you allocated or else it will cause memory leak (a situation where the memory is allocated but there is no pointer refer to that chunk of memory you allocated and hence will be inaccessible throughout program execution)
char *strdup(const char *p) // From #Michael's answer
{
char *q;
q = malloc(strlen(p) + 1);
if(NULL != q)
strcpy(q, p);
return q;
}
void someFunction()
{
char* aDupString = strdup("Hello World!!!");
/*... somecode use aDupString */
free(aDupString); // If not freed, will cause memory leaks
}
When it is appropriate to use malloc? Is there any other situation where malloc is needed?
It is appropriate to use in following situations:
1> The usage size of array are unknown at compile time.
2> You need size flexibility. For example, your function need to work with small data size and large data size. (like Data structure such as Link list,Stacks, Queues, etc.)
As I understand so far is that I have to malloc a new variable every time I need that variable declared in a function to be used elsewhere. Is that true?
I think this one is partially true. depending on what you are trying to achive, there might be a way to get around using malloc though. For example, your strdup can also be rewrite in following way:
void strdup2(const char *p, char* strOut)
{
// malloc not require
strcpy(strOut, p);
}
void someFunction()
{
char aString[15] = "Hello World!!!";
char aDupStr[sizeof(aString)];
strdup2(aString, aDupStr);
// free() memory not required. but size is not dynamic.
}
I am in a bit of dilemma, thus, first of all I would like to apologize of if the next questions will be a bit more noobish or if they have been asked before (I couldn't find the answers for those though).
Anyway, I will explain it by giving a task as an example (it's not homework, it's just for the sake of my question). Here it goes:
Given a string from stdin index each word, then print each word on one line.
Example:
str[] = "Stack is awesome"
str_index {
[0] => "Stack"
[1] => "is"
[2] => "awesome"
}
I know that there are many ways to solve this, but, again, for the sake of my question
Bare this solution:
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
/* fgets adds an unwanted '\n' at the end, so I made
* a special function to read from stdin that removes
* that '\n'.
*/
int read(char *str, int size) {
// fgets adds an unwanted '\n' at the end, so we remove it
fgets(str, size, stdin);
int length = strlen(str);
str[length - 1] = '\0';
return length;
}
/* A function that breaks a string into words, indexes them
* and prints them out, all done dynamically with malloc.
*/
void str_index(char *str) {
char **index, *ptr;
int i = 0, number_of_words;
index = malloc(sizeof(char *));
ptr = strtok(str, " ");
for(i = 0; ptr != NULL; i++) {
index = realloc(index, (i + 1) * sizeof(char *));
index[i] = malloc(50 * sizeof(char));
strcpy(index[i], ptr);
ptr = strtok(NULL, " ");
}
number_of_words = i;
for(i = 0; i < number_of_words; i++) {
printf("%s\n", index[i]);
}
return;
}
int main() {
char str[250];
read(str, 250);
str_index(str);
return 0;
}
QUESTIONS
Where do I have to free the arrays that I have allocated dynamically
in str_index?
Do we have to free them within the function str_index? If so, why?
What I know is that when a function is done executing all local
variables are destroyed.
Why do we have to free them in main? Isn't main a function aswell,
thus upon finishing executing it all variables defined in that function are destroyed.
I'm guessing you are doing a university course. The problem (in my opinion) with university courses is that they start by teaching high level languages where everything is done magically, and then teach you a low level language. If I ruled the world, everyone would start with assembler, then C, then be allowed to 'progress' to Java etc.
To your question, the problem you have is the assumption that 'things might be done magically'. C doesn't do very much magically at all. In particular, if you malloc() or calloc() anything, or allocate anything using something that uses the heap allocator (for instance strdup()), it's your responsibility to free it. And you will need to do that explicitly. If you don't, you will have a memory leak. The first order problem is thus 'if I allocated it, I must ensure it is freed'. The second order problem is 'if I used a library that might have allocated stuff, I need to work out how to ensure it knows I've done, so it can free stuff'. If you bear this in mind, your C programming life will be happy, and valgrind will be your friend.
Let's now consider your questions:
You ask where you should free your dynamically allocated memory. Technically, in this example, you don't need to, because exiting your program will free all memory on the heap. However, let's suppose you want to use this function repeatedly. You want to free the allocation as soon as you no longer have a use for it. In the example presented, that would be immediately before the return. If you had other exits from the function, then make sure you free your allocation before every return. A useful error handling hint is to exit via the same code, and whenever you free() the allocation, also set the pointer to the allocation to NULL. On entry, also initialise the pointer to NULL. Then on exit (a valid use of goto), you can simply check the pointer against NULL, and if it is not null, free() it. (In fact once you get really cocky, you will know free() on NULL on most platforms is a no-op, so you can unconditionally free it). The setting the pointer to NULL bit is to avoid a double free.
This is the difference between the stack and the heap. Local variables are allocated on the stack. C destroys them automatically when a function returns. This is one of the few bits of magic C does. Note that I said it destroys the variables, not the things they point to. So if you have a pointer to allocated (heap) memory in a local variable, and the function returns, it will 'free' the variable (in the sense it will no longer be on the stack), but the allocated (heap) memory will not be freed. This is why you must free heap allocated memory only referenced in a function before the pointers to it are destroyed by exiting the function - see the answer to 1 above.
You don't need to free anything in main() in your example. If you had coded your function to return a pointer to memory on the heap (for instance if you'd coded the equivalent of strdup()) then your main() function would need to free() that. That brings up the important point that what the caller needs to free() depends on how the called function is designed. It's thus important that the called function makes this obvious in documentation.
Where do I have to free the arrays that I have allocated dynamically in str_index?
Just before the return; statement of your function str_index.
Do we have to free them within the function str_index?
Not necessary. Depends on the program requirement.
What I know is that when a function is done executing all local variables are destroyed.
Yes, it is true for space allocated on stack (if variable is not static), but not for the space allocated on heap.
Why do we have to free them in main?
Not necessary that it have to free in main. Depends on the program requirement.
May be similar question found on SO. But, I didn't found that, here is the scenario
Case 1
void main()
{
char g[10];
char a[10];
scanf("%[^\n] %[^\n]",a,g);
swap(a,g);
printf("%s %s",a,g);
}
Case 2
void main()
{
char *g=malloc(sizeof(char)*10);
char *a=malloc(sizeof(char)*10);
scanf("%[^\n] %[^\n]",a,g);
swap(a,g);
printf("%s %s",a,g);
}
I'm getting same output in both case. So, my question is when should I prefer malloc() instead of array or vice-verse and why ?? I found common definition, malloc() provides dynamic allocation. So, it is the only difference between them ?? Please any one explain with example, what is the meaning of dynamic although we are specifying the size in malloc().
The principle difference relates to when and how you decide the array length. Using fixed length arrays forces you to decide your array length at compile time. In contrast using malloc allows you to decide the array length at runtime.
In particular, deciding at runtime allows you to base the decision on user input, on information not known at the time you compile. For example, you may allocate the array to be a size big enough to fit the actual data input by the user. If you use fixed length arrays, you have to decide at compile time an upper bound, and then force that limitation onto the user.
Another more subtle issue is that allocating very large fixed length arrays as local variables can lead to stack overflow runtime errors. And for that reason, you sometimes prefer to allocate such arrays dynamically using malloc.
Please any one explain with example, what is the meaning of dynamic although we are specifying the size.
I suspect this was significant before C99. Before C99, you couldn't have dynamically-sized auto arrays:
void somefunc(size_t sz)
{
char buf[sz];
}
is valid C99 but invalid C89. However, using malloc(), you can specify any value, you don't have to call malloc() with a constant as its argument.
Also, to clear up what other purpose malloc() has: you can't return stack-allocated memory from a function, so if your function needs to return allocated memory, you typically use malloc() (or some other member of the malloc familiy, including realloc() and calloc()) to obtain a block of memory. To understand this, consider the following code:
char *foo()
{
char buf[13] = "Hello world!";
return buf;
}
Since buf is a local variable, it's invalidated at the end of its enclosing function - returning it results in undefined behavior. The function above is erroneous. However, a pointer obtained using malloc() remains valid through function calls (until you don't call free() on it):
char *bar()
{
char *buf = malloc(13);
strcpy(buf, "Hello World!");
return buf;
}
This is absolutely valid.
I would add that in this particular example, malloc() is very wasteful, as there is more memory allocated for the array than what would appear [due to overhead in malloc] as well as the time it takes to call malloc() and later free() - and there's overhead for the programmer to remember to free it - memory leaks can be quite hard to debug.
Edit: Case in point, your code is missing the free() at the end of main() - may not matter here, but it shows my point quite well.
So small structures (less than 100 bytes) should typically be allocated on the stack. If you have large data structures, it's better to allocate them with malloc (or, if it's the right thing to do, use globals - but this is a sensitive subject).
Clearly, if you don't know the size of something beforehand, and it MAY be very large (kilobytes in size), it is definitely a case of "consider using malloc".
On the other hand, stacks are pretty big these days (for "real computers" at least), so allocating a couple of kilobytes of stack is not a big deal.
This question is a bit long due the source code, which I tried to simplify as much as possible. Please bear with me and thanks for reading along.
I have an application with a loop that runs potentially millions of times. Instead of several thousands to millions of malloc/free calls within that loop, I would like to do one malloc up front and then several thousands to millions of realloc calls.
But I'm running into a problem where my application consumes several GB of memory and kills itself, when I am using realloc. If I use malloc, my memory usage is fine.
If I run on smaller test data sets with valgrind's memtest, it reports no memory leaks with either malloc or realloc.
I have verified that I am matching every malloc-ed (and then realloc-ed) object with a corresponding free.
So, in theory, I am not leaking memory, it is just that using realloc seems to consume all of my available RAM, and I'd like to know why and what I can do to fix this.
What I have initially is something like this, which uses malloc and works properly:
Malloc code
void A () {
do {
B();
} while (someConditionThatIsTrueForMillionInstances);
}
void B () {
char *firstString = NULL;
char *secondString = NULL;
char *someOtherString;
/* populate someOtherString with data from stream, for example */
C((const char *)someOtherString, &firstString, &secondString);
fprintf(stderr, "first: [%s] | second: [%s]\n", firstString, secondString);
if (firstString)
free(firstString);
if (secondString)
free(secondString);
}
void C (const char *someOtherString, char **firstString, char **secondString) {
char firstBuffer[BUFLENGTH];
char secondBuffer[BUFLENGTH];
/* populate buffers with some data from tokenizing someOtherString in a special way */
*firstString = malloc(strlen(firstBuffer)+1);
strncpy(*firstString, firstBuffer, strlen(firstBuffer)+1);
*secondString = malloc(strlen(secondBuffer)+1);
strncpy(*secondString, secondBuffer, strlen(secondBuffer)+1);
}
This works fine. But I want something faster.
Now I test a realloc arrangement, which malloc-s only once:
Realloc code
void A () {
char *firstString = NULL;
char *secondString = NULL;
do {
B(&firstString, &secondString);
} while (someConditionThatIsTrueForMillionInstances);
if (firstString)
free(firstString);
if (secondString)
free(secondString);
}
void B (char **firstString, char **secondString) {
char *someOtherString;
/* populate someOtherString with data from stream, for example */
C((const char *)someOtherString, &(*firstString), &(*secondString));
fprintf(stderr, "first: [%s] | second: [%s]\n", *firstString, *secondString);
}
void C (const char *someOtherString, char **firstString, char **secondString) {
char firstBuffer[BUFLENGTH];
char secondBuffer[BUFLENGTH];
/* populate buffers with some data from tokenizing someOtherString in a special way */
/* realloc should act as malloc on first pass through */
*firstString = realloc(*firstString, strlen(firstBuffer)+1);
strncpy(*firstString, firstBuffer, strlen(firstBuffer)+1);
*secondString = realloc(*secondString, strlen(secondBuffer)+1);
strncpy(*secondString, secondBuffer, strlen(secondBuffer)+1);
}
If I look at the output of free -m on the command-line while I run this realloc-based test with a large data set that causes the million-loop condition, my memory goes from 4 GB down to 0 and the app crashes.
What am I missing about using realloc that is causing this? Sorry if this is a dumb question, and thanks in advance for your advice.
realloc has to copy the contents from the old buffer to the new buffer if the resizing operation cannot be done in place. A malloc/free pair can be better than a realloc if you don't need to keep around the original memory.
That's why realloc can temporarily require more memory than a malloc/free pair. You are also encouraging fragmentation by continuously interleaving reallocs. I.e., you are basically doing:
malloc(A);
malloc(B);
while (...)
{
malloc(A_temp);
free(A);
A= A_temp;
malloc(B_temp);
free(B);
B= B_temp;
}
Whereas the original code does:
while (...)
{
malloc(A);
malloc(B);
free(A);
free(B);
}
At the end of each of the second loop you have cleaned up all the memory you used; that's more likely to return the global memory heap to a clean state than by interleaving memory allocations without completely freeing all of them.
Using realloc when you don't want to preserve the existing contents of the memory block is a very very bad idea. If nothing else, you'll waste lots of time duplicating data you're about to overwrite. In practice, the way you're using it, the resized blocks will not fit in the old space, so they get located at progressively higher and higher addresses on the heap, causing the heap to grow ridiculously.
Memory management is not easy. Bad allocation strategies lead to fragmentation, atrocious performance, etc. The best you can do is avoid introducing any more constraints than you absolutely have to (like using realloc when it's not needed), free as much memory as possible when you're done with it, and allocate large blocks of associated data together in a single allocation rather than in small pieces.
You are expecting &(*firstString) to be the same as firstString, but in fact it is taking the address of the argument to your function rather than passing through the address of the pointers in A. Thus every time you call you make a copy of NULL, realloc new memory, lose the pointer to the new memory, and repeat. You can easily verify this by seeing that at the end of A the original pointers are still null.
EDIT: Well, it's an awesome theory, but I seem to be wrong on the compilers I have available to me to test.
I am working with my first straight C project, and it has been a while since I worked on C++ for that matter. So the whole memory management is a bit fuzzy.
I have a function that I created that will validate some input. In the simple sample below, it just ignores spaces:
int validate_input(const char *input_line, char** out_value){
int ret_val = 0; /*false*/
int length = strlen(input_line);
out_value =(char*) malloc(sizeof(char) * length + 1);
if (0 != length){
int number_found = 0;
for (int x = 0; x < length; x++){
if (input_line[x] != ' '){ /*ignore space*/
/*get the character*/
out_value[number_found] = input_line[x];
number_found++; /*increment counter*/
}
}
out_value[number_found + 1] = '\0';
ret_val = 1;
}
return ret_val;
}
Instead of allocating memory inside the function for out_value, should I do it before I call the function and always expect the caller to allocate memory before passing into the function? As a rule of thumb, should any memory allocated inside of a function be always freed before the function returns?
I follow two very simple rules which make my life easier.
1/ Allocate memory when you need it, as soon as you know what you need. This will allow you to capture out-of-memory errors before doing too much work.
2/ Every allocated block of memory has a responsibility property. It should be clear when responsibility passes through function interfaces, at which point responsibility for freeing that memory passes with the memory. This will guarantee that someone has a clearly specified requirement to free that memory.
In your particular case, you need to pass in a double char pointer if you want the value given back to the caller:
int validate_input (const char *input_line, char **out_value_ptr) {
: :
*out_value_ptr =(char*) malloc(length + 1); // sizeof(char) is always 1
: :
(*out_value_ptr)[number_found] = input_line[x];
: :
As long as you clearly state what's expected by the function, you could either allocate the memory in the caller or the function itself. I would prefer outside of the function since you know the size required.
But keep in mind you can allow for both options. In other words, if the function is passed a char** that points to NULL, have it allocate the memory. Otherwise it can assume the caller has done so:
if (*out_value_ptr == NULL)
*out_value_ptr =(char*) malloc(length + 1);
You should free that memory before the function returns in your above example. As a rule of thumb you free/delete allocated memory before the scope that the variable was defined in ends. In your case the scope is your function so you need to free it before your function ends. Failure to do this will result in leaked memory.
As for your other question I think it should be allocated going in to the function since we want to be able to use it outside of the function. You allocate some memory, you call your function, and then you free your memory. If you try and mix it up where allocation is done in the function, and freeing is done outside it gets confusing.
The idea of whether the function/module/object that allocates memory should free it is somewhat of a design decision. In your example, I (personal opinion here) think it is valid for the function to allocate it and leave it up to the caller to free. It makes it more usable.
If you do this, you need to declare the output parameter differently (either as a reference in C++ style or as char** in C style. As defined, the pointer will exist only locally and will be leaked.
A typical practice is to allocate memory outside for out_value and pass in the length of the block in octets to the function with the pointer. This allows the user to decide how they want to allocate that memory.
One example of this pattern is the recv function used in sockets:
ssize_t recv(int socket, void *buffer, size_t length, int flags);
Here are some guidelines for allocating memory:
Allocate only if necessary.
Huge objects should be dynamically
allocated. Most implementations
don't have enough local storage
(stack, global / program memory).
Set up ownership rules for the
allocated object. Owner should be
responsible for deleting.
Guidelines for deallocating memory:
Delete if allocated, don't delete
objects or variables that were not
dynamically allocated.
Delete when not in use any more.
See your object ownership rules.
Delete before program exits.
In this example you should be neither freeing or allocating memory for out_value. It is typed as a char*. Hence you cannot "return" the new memory to the caller of the function. In order to do that you need to take in a char**
In this particular scenario the buffer length is unknown before the caller makes the call. Additionally making the same call twice will produce different values since you are processing user input. So you can't take the approach of call once get the length and call the second time with the allocated buffer. Hence the best approach is for the function to allocate the memory and pass the responsibility of freeing onto the caller.
First, this code example you give is not ANSI C. It looks more like C++. There is not "<<" operator in C that works as an output stream to something called "cout."
The next issue is that if you do not free() within this function, you will leak memory. You passed in a char * but once you assign that value to the return value of malloc() (avoid casting the return value of malloc() in the C programming language) the variable no longer points to whatever memory address you passed in to the function. If you want to achieve that functionality, pass a pointer to a char pointer char **, you can think of this as passing the pointer by reference in C++ (if you want to use that sort of language in C, which I wouldn't).
Next, as to whether you should allocate/free before or after a function call depends on the role of the function. You might have a function whose job it is to allocate and initialize some data and then return it to the caller, in which case it should malloc() and the caller should free(). However, if you are just doing some processing with a couple of buffers like, you may tend to prefer the caller to allocate and deallocate. But for your case, since your "validate_input" function looks to be doing nothing more than copying a string without the space, you could just malloc() in the function and leave it to the caller. Although, since in this function, you simply allocate the same size as the whole input string, it almost seems as if you might as well have the caller to all of it. It all really depends on your usage.
Just make sure you do not lose pointers as you are doing in this example
Some rough guidelines to consider:
Prefer letting the caller allocate the memory. This lets it control how/where that memory is allocated. Calling malloc() directly in your code means your function is dictating a memory policy.
If there's no way to tell how much memory may be needed in advance, your function may need to handle the allocation.
In cases where your function does need to allocate, consider letting the caller pass in an allocator callback that it uses instead of calling malloc directly. This lets your function allocate when it needs and as much as it needs, but lets the caller control how and where that memory is allocated.