I'm sorry to ask what I believe is a silly question but I feel like I have a concept very wrong and need to solve it.
I have this code for listed links, that I've played with a bit to get my head arround it:
http://www.thegeekstuff.com/2012/08/c-linked-list-example/
And now I need that same way of operating with lists, but instead of the struct being int val, I want it to be char * val, so the struct would be like:
struct nodoTemas{
char * nombreTema;
struct nodoSuscriptor * nodoCerodeSuscriptores;
struct nodoTemas * next;
};
I get the char * from reading from a file like this:
while ((read = getline(&line, &len, fp)) != -1) {
add_to_list(line,true);
}
I can not fin the difference at a conceptual level, at all. Yeah getline adds a \n to the char * so it kind of messes my fprintf's and what not, but the result I get from executing the code is:
creating list with headnode as [tema1
]
Adding node to end of list with value [tema2
]
-------Printing list Start-------
[tema2
]
[tema2
]
When it should be
-------Printing list Start-------
[tema1
]
[tema2
]
Since that's what it got first!!! I just don't see the difference, I've re-read info about char*, char and char[], I've re-read getline man, and I simply don't understand why if the code in the link works perfectly fine with int's, if I use char*, it adds and iterates properly and then kind of... breaks down and overwrites?? Is that what it's happening? No clue.
Also if I should be posting this somewhere else please tell me.
I also found this post but I think the problem comes from a different matter in my case, not completly sure though
Adding Char Value to a Linked List using Looping [C]
As I guessed in my original comment, you have problem because you are not managing the space allocated by getline() and used by add_to_list() correctly.
You say you are using:
while ((read = getline(&line, &len, fp)) != -1) {
add_to_list(line, true);
}
You haven't shown the code for add_to_list(), but the symptoms are that you simply assign the value of line (the pointer) in the list, rather than making a copy for the space in the function. You can probably fix the code superficially by:
char *line = 0;
size_t len = 0;
ssize_t nbytes;
while ((nbytes = getline(&line, &len, fp)) != -1)
{
add_to_list(line, true);
line = 0;
len = 0;
}
free(line);
This will allocate new memory for each line read. The add_to_list() function can then use it.
However, you have to be sure that you do end up freeing the memory eventually — when you free the list. Note that the free() shown is not the only one you need. It releases any memory pre-emptively allocated by getline() before it detected EOF.
An alternative design is:
char *line = 0;
size_t len = 0;
ssize_t nbytes;
while ((nbytes = getline(&line, &len, fp)) != -1)
add_to_list(line, true);
free(line);
Now you have to modify add_to_list() to make a copy of the data from line. You still have to free the line, as shown, and the code that frees the list still has to release the copied data.
I already had a free(line), but you mention that's not the only free I need. Mind expanding a bit on that one? Would it be necesary to free every node of the list? The list is supposed to hold topics in a publisher-subscriber model so unless the process ends, I believe the memory should hold the list?
There is no need to free anything until you've finished with it; premature freeing is as bad as never freeing. When you allocate memory, you always have to know when the memory will be freed. For each allocation, make sure you can identify how the memory will be freed, and when it will be freed.
It is sometimes a legitimate decision that the memory will never be released (or not until the process exits), but for long-running processes, that is something to decide carefully. And even then, it is only legitimate if there is a pointer (or chain of pointers) that allows you to get at the memory. If there is no way to access the memory, then it is leaked and wasted, and that is not a good idea in big programs (especially not in long-running programs).
If you allocated memory for each item in the list, then you will need to free the memory for each item in the list when you free the item. That may occur when you demolish the whole list, or it may happen piecemeal as you decide that an item is no longer needed. What's crucial is that you know that memory was allocated, and that it will be freed when it is no longer needed. If you don't ensure this, you will leak memory, and your program may grow until it runs out of memory.
If possible, use a tool like valgrind to help you keep track of memory misuse — it spots both memory leaks and some memory abuse (reading and writing out of bounds).
getline is reusing the pointer of line for storing the char data. use strdup to store it in your link node.
Related
This question already has answers here:
Should I free allocated memory on abnormal termination?
(7 answers)
Closed 9 years ago.
Sometimes things like library errors will not allow my program to continue further, like a call to SDL_Init going bad. Should I attempt to free as much memory as possible, or just quit? I haven't seen any small examples where people don't just quit, but I'm not smart enough to read DOOM-3 code or anything of that sort.
I wouldn't. If your program crashes, because of some exotic, unforeseen things that happen in your program, it might even be pointless to try and free any allocated heap memory.
I think it'd be best if you just call exit (EXIT_FAILURE), if you can, and leave the OS to reclaim the memory you allocated as good as it possibly can.
However, I would try to clean up any other resources that you've used/claimed/opened that may also cause leak. Close as many opened file pointers as you can, or flush any buffers lying around.
Other than that, I'd say: leave it to the OS. Your program has crashed, or is crashing: trying to clean up after yourself in an unforeseen situation might be pointless, or -who knows- eventually do more harm than good.
Of course, if by "a library errors" you mean something like:
MYSQL *connection = mysql_init();
if (connection == NULL)
{//connection could not be initiated
//what to do here?
}
Or, no lib:
char **arr_of_strings = malloc(200*sizeof(char *));
//some code
arr_of_strings[0] = calloc(150, sizeof(char));
//some more
arr_of_strings[120] = calloc(350, sizeof(char));
if (arr_of_strings == NULL)
{//ran out of heap memory
//what do I do here?
}
So, basically: it's a matter of: What does your program have to do, and can you easily find a way around the problems you're being faced with.
If, for example, you're writing a mysql client, and the mysql_init call fails, I think it pretty evident you cannot continue. You could try to provide a fallback for every reason why this could happen, or you could just exit. I'd opt for the latter.
In the second case, it's pretty clear that you've depleted the heap memory. If you're going to write these strings to a file anyway, you could prevent this kind of error like so:
int main()
{
char **arr_str = malloc(20*sizeof(char *));
const char *fileName = "output.txt";
int i, j;
int alloc_str(char ***str, int offset, int size);
void write_to_file(const char *string, const char *fileName);
for(i=0;i<10;++i)
{
if (alloc_str(&arr_str, i, 100+i) == -1)
{
if (i == 0) exit(EXIT_FAILURE);//this is a bigger problem
for (j=0;i<i;++j)
{//write what we have to file, and free the memory
if (arr_str[j] != NULL)
{
write_to_file(arr_str[j], fileName);
free(arr_str[j]);
arr_str[j] = NULL;
}
if (alloc_str(&arr_str, i, 100+i) != -1) break;//enough memory freed!
}
}
//assign value to arr_str[i]
}
for(i=0;i<10;++i) free(arr_str[i]);
free(arr_str);
return 0;
}
void write_to_file(const char *string, const char *fileName)
{//woefully inefficient, but you get the idea
FILE* outFile = fopen(fileName, "a");
if (outFile == NULL) exit (EXIT_FAILURE);
fprintf(outFile, "%s\n", string);
fclose(outFile);
}
int alloc_str(char ***str, int offset, int size)
{
(*str)[offset] = calloc(size, sizeof(char));
if ((*str)[offset] == NULL) return -1;
return 0;
}
Here, I'm attempting to create an array of strings, but when I run out of memory, I'll just write some of the strings to a file, deallocate the memory they take up, and carry on. I could then refer back to the file to which I wrote the strings I had to clear from memory. In this case, I can ensure, though it does cause some additional overhead, my program will run just fine.
In the second case, freeing memory is a must, though. I have to free up the memory required for my program to continue running, but all things considered, it's an easy fixed.
It depends. These days operating systems cleanup the mess you made, but on embedded systems you may not be that lucky. But even then there is a question that, "So what, my system busted anyway. I'll just reboot/restart/try again"
Personally I like to arrange my code in a way that when exiting, it checks which resources are in use and free those. It doesn't matter if it's normal exit or error.
Free as much memory as possible and do other necessary work(etc. log, backup) instead of just quit. It is the program's duty to free the memory that it allocated. Do not depend on OS, thought it will free the memory after the program ends.
I wrote a memory leak detect module before long, it require the program free the allocated memory. If it does not free the memory, the module can not work, it can not figure out whether the memory block left is leaked or not.
I'll ask you for some help with a really really simple program which implements a search planning algorithm. Well, the problem I got is a little weird for me: after allocating memory for a simple array of characters declared locally in a function, I work over this array, it has all the expected behaviors and everything goes ok, but when I call the free() function to this array before the function ends, the program stops and abort. If someone with some experience about this problem (or not...) could help me, i'd be really thankful. Well, here follows some lines of a "simulated" code to show what I'm talking about (it's not exactly what's written, but the :
char* v_AllTheWorld
char* v_ReadALineOfTheWorld;
v_AllTheWorld = malloc(COLUMNS * LINES * sizeof(char)); /*LINES and COLUMNS are defined constants*/
if(v_AllTheWorld == NULL)
{
fprintf(stderr, "..."); //etc, etc.
exit(1);
}
v_ReadALineOfTheWorld = malloc(COLUMNS * sizeof(char)); /*the "sizeof(char)" looks useless, but there's no point in talking about that here, i guess.*/
if(v_ReadALineOfTheWorld == NULL)
{
fprintf(stderr, "..."); //etc, etc.
exit(1);
}
while(/*some_condition (number of lines to be read)*/)
{
//read data string from stream and stores in v_ReadALineOfTheWorld (fscanf);
//appends the read data to v_AllTheWorld (strncat);
}
free(v_ReadALineOfTheWorld);
/*The program is stopping right after this call in the debug.*/
return v_AllTheWorld;
I didn't put the head of the function, the declaration of it, and I didn't represent the stream or how the data is manipulated in details, but no other calls of "malloc" or "free" are made and all the code that is written is executed unconditionally (out of any "if" or similar). Course, the last bahavior doesn't include the allocation tests, but you got what I'm saying.
So, well, I hope I did it right asking this way, I hope I detailed the problem the right way and I hope you may help me.
Oh, I almost forgot that: as you probably noticed, the program is in C.
This could happen if you are writing outside the bounds of v_ReadALineOfTheWorld. Some malloc libraries store info about the mallocd region in a wrapper around the region, and if you corrupt that info free could crash.
Okay I've read through a massive amount of of the answers here on SO, and many other places but I just can't seem to grasp this simple function. Please forgive me for something so simple I haven't done c/c++ code in over 8 years and I'm very much trying to re-learn, so please have patience...
I've tried many different ways to do this from assigning a string through a function param by shifting in the value to just straight returning it, but nothing seems to work within the while. I also get no errors during compile time, but I do get segfaults at runtime. I would very much like to find out why the following function does not work... I just don't understand why the else returns fine as type char *content, but strcat(content, line); does not. Even though the man pages for strcat shows that strcat's definition should be (char *DEST, const char *SRC). As I currently understand it trying to do a cast to a const char on the line variable within the while would just return an integer to the pointer. So I'm stumped here and would like to be educated by those who have some time!
char * getPage(char *filename) {
FILE *pFile;
char *content;
pFile = fopen(filename, "r");
if (pFile != NULL) {
syslog(LOG_INFO,"Reading from:%s",filename);
char line [256];
while (fgets(line, sizeof line, pFile) != NULL) {
syslog(LOG_INFO,">>>>>>>Fail Here<<<<<<<");
strcat(content, line);
}
fclose(pFile);
} else {
content = "<!DOCTYPE html><html lang=\"en-US\"><head><title>Test</title></head><body><h1>Does Work</h1></body></html>";
syslog(LOG_INFO,"Reading from:%s failed, serving static response",filename);
}
return content;
}
Very much appreciate all the great answers in this post. I would give everyone in the discussion a checkmark but unfortunately I can't...
This is pretty simple, but very surprising if you're used to a higher-level language. C does not manage memory for you, and C doesn't really have strings. That content variable is a pointer, not a string. You have to manually allocate the space you need for the string before calling strcat. The correct way to write this code is something like this:
FILE *fp = fopen(filename, "r");
if (!fp) {
syslog(LOG_INFO, "failed to open %s: %s", filename, strerror(errno));
return xstrdup("<!DOCTYPE html><html lang=\"en-US\"><head><title>Test</title>"
"</head><body><h1>Does Work</h1></body></html>");
} else {
size_t capacity = 4096, offset = 0, n;
char *content = xmalloc(capacity);
size_t n;
while ((n = fread(content + offset, 1, capacity - offset, fp)) > 0) {
offset += n;
if (offset == capacity) {
capacity *= 2;
content = xrealloc(content, capacity);
}
}
if (n < 0)
syslog(LOG_INFO, "read error from %s: %s", filename, strerror(errno));
content[offset] = '\0';
fclose(fp);
return content;
}
Notes:
Error messages triggered by I/O failures should ALWAYS include strerror(errno).
xmalloc, xrealloc, and xstrdup are wrapper functions around their counterparts with no leading x; they crash the program rather than return NULL. This is almost always less grief than trying to recover from out-of-memory by hand in every single place where it can happen.
I return xstrdup("...") rather than "..." in the failed-to-open case so that the caller can always call free(content). Calling free on a string literal will crash your program.
Gosh, that was a lot of work, wasn't it? This is why people tend to prefer to write web apps in a higher-level language. ;-)
You need to allocate memory for content. It has to be big enough for the entire file the way you are doing it. You can either allocate a huge buffer up front and hope for the best, or allocate a smaller one and realloc it as needed.
Even better would be rearranging the code to avoid the need for storing the whole file all at once, although if your caller needs a whole web page as a string, that may be hard.
Note also that you need to return the same type of memory from both your code paths. You can't return a static string sometimes and a heap-allocated string other times. That's guaranteed to call headaches and/or memory leaks. So if you are copying the file contents into a block of memory, you should also copy the static string into the same type of block.
content is just a pointer to a string not an actual string - it has 0 bytes of space reserved for your string. You need to allocate memory large enough to hold hour string. Note that after you will have to free it
char *content=malloc(256);
And your code should be ok - oh and I suggest using strncat
The 2nd assignment to content worked ok before - because you are setting the pointer to point to your const string. If you change content to a malloc'ed region of memory - then you would also want to strncpy your fixed string into content.
Ideally if you can use C++ std::string.
char *foo is only a pointer to some piece of memory holding the characters that form the string. So you cannot use strcat because you don't have any memory to copy to. Inside the if statement you are allocating local memory on the stack with char line[256] that holds the line, but since that memory is local for the function is will disappear once it returns, so you cannot return line;.
So what you really want is to allocate some persistent memory, e.g. with strdup or malloc, so that you can return it from the function. Note that you cannot mix constants and allocated memory (because the user of your function must free the memory - which is only possible if it is not a constant).
So you could use something like this:
char * getPage(const char *filename) {
FILE *pFile;
char *content;
pFile = fopen(filename, "r");
if (pFile != NULL) {
syslog(LOG_INFO,"Reading from:%s",filename);
/* check the size and allocate memory */
fseek(pFile, 0, SEEK_END);
if (!(content = malloc(ftell(pfile) + 1))) { /* out of memory ... */ }
rewind(pFile);
/* set the content to be empty */
*content = 0;
char line [256];
while (fgets(line, sizeof line, pFile) != NULL) {
syslog(LOG_INFO,">>>>>>>Fail Here<<<<<<<");
strcat(content, line);
}
fclose(pFile);
} else {
content = strdup("<!DOCTYPE html><html lang=\"en-US\"><head><title>Test</title></head><body><h1>Does Work</h1></body></html>");
syslog(LOG_INFO,"Reading from:%s failed, serving static response",filename);
}
return content;
}
It is not the most efficient way of doing this (because strcat has to find the end every time), but the least modification of your code.
An earlier answer suggested the solution:
char content[256];
This buffer will not be large enough to hold anything but the smallest files and the pointer content goes out of scope when return content; is executed. (Your earlier line, content = "static.."; is fine, because the string is placed in the .rodata data segment and its pointer will always point to the same data, for the entire lifetime of the program.)
If you allocate the memory for content with malloc(3), you can "grow" the space required with realloc(3), but this introduces the potential for a horrible error -- whatever you handed the pointer to must clean up after the memory allocation when it is done with the data (or else you leak memory), and it cannot simply call free(3) because the content pointer might be to statically allocated memory.
So, you have two easy choices:
use strdup(3) to duplicate the static string each time you need it, and use content = malloc(size); for the non-static path
make your caller responsible for providing the memory; every call needs to provide sufficient memory to handle either the contents of the file or the static string.
I would probably prefer the first approach, if only because the size needed for the second approach cannot be known prior to the call.
content is a wild pointer; the variable contains garbage, so it's pointing somewhere into left field. When you copy data to it using strcat, the data goes to some random, probably bad, location. The cure for this is to make content point somewhere good. Since you want it to outlive your function call, it needs to be allocated someplace besides the function's call stack. You need to use malloc() to allocate some space on the heap. Then the caller will own the memory, and should call free() to delete it when it's no longer needed.
You'll need to change the else part that directly assigns to content, as well, to use strcpy, so that the free() will always be valid. You can't free something that you didn't allocate!
Through all of this code, make sure you remember how much space you allocated with malloc(), and don't write more data than you have space, or you'll get more crashes.
I've written a function to test if a given path is a valid Maildir directory (standard Maildir has the three subfolders "cur" "new" and "tmp" ). Function takes in the supposed directory, checks for those subfolders, and returns appropriately.
I'm getting a segfault at the second free statement with the current code, and I similarly got an "invalid next size" error with code of slightly different organization. Even more confusing, it only segfaults on some directories, while successfully completing on others, with no discernible reason (though it is consistent on which ones it will segfault on). With the second free() commented out, all accurately-formatted directories complete successfully.
Obviously I'm double-freeing. My question is, why and how? If the first free is inside the conditional statement and we return immediately after freeing, we never get to the second free. If we get to the second free, that means we skipped the first one... right?
I realize in this context it's perfectly fine because the system will reclaim the memory at the end of the program, but I'm more interested in the reason this is happening than in just making the code work. What if I were looking at a different situation, functions called by functions called by functions etc. and memory could possibly be a concern? Don't I need that 2nd free to reclaim memory?
int is_valid_folder(char* maildir)
{
struct stat *buf;
buf = (struct stat *) malloc(sizeof(struct stat));
char* new = strdup(maildir);
char* cur = strdup(maildir);
char* tmp = strdup(maildir);
strcat (cur, "/cur"); strcat (new, "/new"); strcat (tmp, "/tmp");
if(stat(cur, buf) || stat(tmp, buf) || stat(new, buf))
{
printf("Problem stat-ing one of the cur/new/tmp folders\n");
printf("Error number %d\n", errno);
free(buf);
return 1;
}
free(buf);
return 0; //a valid folder path for this function
}
You have several buffer overflows: strdup() probably allocates a char array that is just large enough to hold the maildir string, and the calls to strcat() will then overflow the arrays. (strcat(), as opposed to strdup(), does not create a new char array, so you must ensure yourself that the array you give it is large enough to hold the resulting string.)
By the way, valgrind is your friend when it comes to tracking down memory management bugs.
There's not enough space in the duplicate strings for the concatenation.
try:
char* new = (char*)calloc(strlen(maildir) + 5);
etc
I know you got it, but just as a tip... (too big for a comment)
Check the return value of strdup() for NULL and free() those pointers when you are done with them. If you don't memory will leak (it is leaking in your current code).
The strdup() function shall return a pointer to a new string, which is a duplicate of the string pointed to by s1. The returned pointer can be passed to free(). A null pointer is returned if the new string cannot be created.
This question is a bit long due the source code, which I tried to simplify as much as possible. Please bear with me and thanks for reading along.
I have an application with a loop that runs potentially millions of times. Instead of several thousands to millions of malloc/free calls within that loop, I would like to do one malloc up front and then several thousands to millions of realloc calls.
But I'm running into a problem where my application consumes several GB of memory and kills itself, when I am using realloc. If I use malloc, my memory usage is fine.
If I run on smaller test data sets with valgrind's memtest, it reports no memory leaks with either malloc or realloc.
I have verified that I am matching every malloc-ed (and then realloc-ed) object with a corresponding free.
So, in theory, I am not leaking memory, it is just that using realloc seems to consume all of my available RAM, and I'd like to know why and what I can do to fix this.
What I have initially is something like this, which uses malloc and works properly:
Malloc code
void A () {
do {
B();
} while (someConditionThatIsTrueForMillionInstances);
}
void B () {
char *firstString = NULL;
char *secondString = NULL;
char *someOtherString;
/* populate someOtherString with data from stream, for example */
C((const char *)someOtherString, &firstString, &secondString);
fprintf(stderr, "first: [%s] | second: [%s]\n", firstString, secondString);
if (firstString)
free(firstString);
if (secondString)
free(secondString);
}
void C (const char *someOtherString, char **firstString, char **secondString) {
char firstBuffer[BUFLENGTH];
char secondBuffer[BUFLENGTH];
/* populate buffers with some data from tokenizing someOtherString in a special way */
*firstString = malloc(strlen(firstBuffer)+1);
strncpy(*firstString, firstBuffer, strlen(firstBuffer)+1);
*secondString = malloc(strlen(secondBuffer)+1);
strncpy(*secondString, secondBuffer, strlen(secondBuffer)+1);
}
This works fine. But I want something faster.
Now I test a realloc arrangement, which malloc-s only once:
Realloc code
void A () {
char *firstString = NULL;
char *secondString = NULL;
do {
B(&firstString, &secondString);
} while (someConditionThatIsTrueForMillionInstances);
if (firstString)
free(firstString);
if (secondString)
free(secondString);
}
void B (char **firstString, char **secondString) {
char *someOtherString;
/* populate someOtherString with data from stream, for example */
C((const char *)someOtherString, &(*firstString), &(*secondString));
fprintf(stderr, "first: [%s] | second: [%s]\n", *firstString, *secondString);
}
void C (const char *someOtherString, char **firstString, char **secondString) {
char firstBuffer[BUFLENGTH];
char secondBuffer[BUFLENGTH];
/* populate buffers with some data from tokenizing someOtherString in a special way */
/* realloc should act as malloc on first pass through */
*firstString = realloc(*firstString, strlen(firstBuffer)+1);
strncpy(*firstString, firstBuffer, strlen(firstBuffer)+1);
*secondString = realloc(*secondString, strlen(secondBuffer)+1);
strncpy(*secondString, secondBuffer, strlen(secondBuffer)+1);
}
If I look at the output of free -m on the command-line while I run this realloc-based test with a large data set that causes the million-loop condition, my memory goes from 4 GB down to 0 and the app crashes.
What am I missing about using realloc that is causing this? Sorry if this is a dumb question, and thanks in advance for your advice.
realloc has to copy the contents from the old buffer to the new buffer if the resizing operation cannot be done in place. A malloc/free pair can be better than a realloc if you don't need to keep around the original memory.
That's why realloc can temporarily require more memory than a malloc/free pair. You are also encouraging fragmentation by continuously interleaving reallocs. I.e., you are basically doing:
malloc(A);
malloc(B);
while (...)
{
malloc(A_temp);
free(A);
A= A_temp;
malloc(B_temp);
free(B);
B= B_temp;
}
Whereas the original code does:
while (...)
{
malloc(A);
malloc(B);
free(A);
free(B);
}
At the end of each of the second loop you have cleaned up all the memory you used; that's more likely to return the global memory heap to a clean state than by interleaving memory allocations without completely freeing all of them.
Using realloc when you don't want to preserve the existing contents of the memory block is a very very bad idea. If nothing else, you'll waste lots of time duplicating data you're about to overwrite. In practice, the way you're using it, the resized blocks will not fit in the old space, so they get located at progressively higher and higher addresses on the heap, causing the heap to grow ridiculously.
Memory management is not easy. Bad allocation strategies lead to fragmentation, atrocious performance, etc. The best you can do is avoid introducing any more constraints than you absolutely have to (like using realloc when it's not needed), free as much memory as possible when you're done with it, and allocate large blocks of associated data together in a single allocation rather than in small pieces.
You are expecting &(*firstString) to be the same as firstString, but in fact it is taking the address of the argument to your function rather than passing through the address of the pointers in A. Thus every time you call you make a copy of NULL, realloc new memory, lose the pointer to the new memory, and repeat. You can easily verify this by seeing that at the end of A the original pointers are still null.
EDIT: Well, it's an awesome theory, but I seem to be wrong on the compilers I have available to me to test.