Why libcurl still leaves reachable blocks after cleanup calls? - c

I have been using libcurl to handle get/post requests and valgrind always shows me the same amount of reachable blocks (21). What I missing in the standard cURL's configuration to avoid that?
This is the simplest code with the "error":
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
int main(int argc, char const *argv[]) {
CURL *curl;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
·
·
·
curl_easy_cleanup(curl);
curl_global_cleanup();
return 0;
}
I compile with
$ gcc -o test cURL.c -lcurl
Valgrind check
$ valgrind --leak-check=full --track-origins=yes ./test
==4295== Memcheck, a memory error detector
==4295== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4295== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==4295== Command: ./test
==4295==
==4295==
==4295== HEAP SUMMARY:
==4295== in use at exit: 858 bytes in 21 blocks
==4295== total heap usage: 4,265 allocs, 4,244 frees, 185,353 bytes allocated
==4295==
==4295== LEAK SUMMARY:
==4295== definitely lost: 0 bytes in 0 blocks
==4295== indirectly lost: 0 bytes in 0 blocks
==4295== possibly lost: 0 bytes in 0 blocks
==4295== still reachable: 858 bytes in 21 blocks
==4295== suppressed: 0 bytes in 0 blocks
==4295== Reachable blocks (those to which a pointer was found) are not shown.
==4295== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==4295==
==4295== For counts of detected and suppressed errors, rerun with: -v
==4295== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

libcurl links against many libraries, and some of them do not have a function like curl_global_cleanup which reverts initialization and frees all memory. This happens when libcurl is linked against NSS for TLS support, and also with libssh2 and its use of libgcrypt. GNUTLS as the TLS implementation is somewhat cleaner in this regard.
In general, this is not a problem because these secondary libraries are only used on operating systems where memory is freed on process termination, so an explicit cleanup is not needed (and would even slow down process termination). Only with certain memory debuggers, the effect of missing cleanup routines is visible, and valgrind deals with this situation by differentiating between actual leaks (memory to which no pointers are left) and memory which is still reachable at process termination (so that it could have been used again if the process had not terminated).

Related

Custom allocator: Valgrind shows 7 allocs, 0 frees, no leaks

I am working on a clone of the malloc (3) functions (malloc, realloc and free for now).
I would like to add support for Valgrind. I'm using these docs. However, after adding calls to the VALGRIND_MEMPOOL_FREE, VALGRIND_MEMPOOL_ALLOC and VALGRIND_CREATE_MEMPOOL macros, I get the following from Valgrind:
==22303== HEAP SUMMARY:
==22303== in use at exit: 0 bytes in 0 blocks
==22303== total heap usage: 7 allocs, 0 frees, 2,039 bytes allocated
==22303==
==22303== All heap blocks were freed -- no leaks are possible
This is despite my realloc calling VALGRIND_MEMPOOL_FREE and my free calling VALGRIND_MEMPOOL_FREE.
What could be the cause of this ?
What could be the cause of this ?
This is due to a bug in valgrind. See the link to the valgrind bug tracker in my comment to your answer.
From the other link in my comment:
A cursory search through the source code indicates that MEMPOOL_ALLOC
calls new_block, which increments cmalloc_n_mallocs, but there is no
corresponding change to cmalloc_n_frees in MEMPOOL_FREE.
/* valgrind.c */
#include <valgrind/valgrind.h>
int main(int argc, char *argv[]) {
char pool[100];
VALGRIND_CREATE_MEMPOOL(pool, 0, 0);
VALGRIND_MEMPOOL_ALLOC(pool, pool, 8);
VALGRIND_MEMPOOL_FREE(pool, pool);
VALGRIND_DESTROY_MEMPOOL(pool);
return 0;
}
$ gcc valgrind.c -g
$ valgrind --leak-check=full --show-leak-kinds=all ./a.out
==10186== Memcheck, a memory error detector
==10186== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==10186== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==10186== Command: ./a.out
==10186==
==10186==
==10186== HEAP SUMMARY:
==10186== in use at exit: 0 bytes in 0 blocks
==10186== total heap usage: 1 allocs, 0 frees, 8 bytes allocated
==10186==
==10186== All heap blocks were freed -- no leaks are possible
==10186==
==10186== For counts of detected and suppressed errors, rerun with: -v
==10186== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
$
Taken from here: http://valgrind.10908.n7.nabble.com/VALGRIND-MEMPOOL-FREE-not-reflected-in-heap-summary-td42789.html
A cursory search through the source code indicates that MEMPOOL_ALLOC
calls new_block, which increments cmalloc_n_mallocs, but there is no
corresponding change to cmalloc_n_frees in MEMPOOL_FREE. Here's a
patch that increments it at the very end of MEMPOOL_FREE. This gives
me the behavior I expect.

Should I free the pointer returned by setlocale?

int main(int argc, char *argv[])
{
char *ret = setlocale(LC_ALL, NULL);
// should I free 'ret' ???
// free(ret);
return 0;
}
I've tried both on Linux and OS X 10.10, on Linux, I must not call 'free', but on OS X, if I do not call 'free', valgrind complains a memory leak.
==62032== Memcheck, a memory error detector
==62032== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==62032== Using Valgrind-3.11.0.SVN and LibVEX; rerun with -h for copyright info
==62032== Command: ./a.out
==62032==
--62032-- ./a.out:
--62032-- dSYM directory is missing; consider using --dsymutil=yes
==62032==
==62032== HEAP SUMMARY:
==62032== in use at exit: 129,789 bytes in 436 blocks
==62032== total heap usage: 519 allocs, 83 frees, 147,421 bytes allocated
==62032==
==62032== 231 bytes in 1 blocks are definitely lost in loss record 63 of 91
==62032== at 0x10000859B: malloc (in /usr/local/Cellar/valgrind/HEAD/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==62032== by 0x1001E68C8: currentlocale (in /usr/lib/system/libsystem_c.dylib)
==62032== by 0x100000F6B: main (in ./a.out)
==62032==
==62032== LEAK SUMMARY:
==62032== definitely lost: 231 bytes in 1 blocks
==62032== indirectly lost: 0 bytes in 0 blocks
==62032== possibly lost: 0 bytes in 0 blocks
==62032== still reachable: 94,869 bytes in 10 blocks
==62032== suppressed: 34,689 bytes in 425 blocks
==62032== Reachable blocks (those to which a pointer was found) are not shown.
==62032== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==62032==
==62032== For counts of detected and suppressed errors, rerun with: -v
==62032== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 17 from 17)
So,
in Linux, if I call 'free', it will crash.
in OS X, if I do not call 'free', it has a memory leak.
You should not free the string that you get. According to the C11 standard:
7.11.1.1 The setlocale function
The pointer to string returned by the
setlocale
function is such that a subsequent call
with that string value and its associated category will restore that part of the program’s
locale. The string pointed to shall not be modified by the program, but may be
overwritten by a subsequent call to the
setlocale
function
Additionally, the Linux man pages say:
This string may be allocated in
static storage.
which would result in your program crashing if you tried to free it.
It looks like the Linux implementation uses static storage, but the OSX one uses malloc. Regardless of what's going on under the hood, you should not modify it because the standard disallows you from doing so --- the fact that it would be safe on OSX is an implementation quirk you should ignore. Valgrind is essentially giving you a false positive here.

Definitely Lost Memory Leak in C Program

Would someone be able to shed light on why Valgrind classifies this program as a "Definitely Lost: 2 bytes in 1 block" memory leak? I understand that the commented line resolves the issue, but I don't understand the classification. According to Valgrind docs it appears that the memory leak should be classified as "Indirectly Reachable". I am also curious as to why this is even considered a memory leak and would appreciate an explanation. Is it good practice to manually free everything even though the program is terminating at the end of the main function?
#include <stdlib.h>
struct wrapper {
char *data;
};
char *strdup(const char *);
struct wrapper *walloc(struct wrapper *root)
{
if (root == NULL){
root = (struct wrapper *) malloc(sizeof(struct wrapper));
root->data = strdup("H");
}
return root;
}
int main(){
struct wrapper *root;
root = NULL;
root = walloc(root);
//free(root->data);
return 0;
}
Here is the Valgrind output:
$ valgrind --leak-check=full ./leak
==26489== Memcheck, a memory error detector
==26489== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==26489== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==26489== Command: ./leak
==26489==
==26489==
==26489== HEAP SUMMARY:
==26489== in use at exit: 2 bytes in 1 blocks
==26489== total heap usage: 2 allocs, 1 frees, 1,790 bytes allocated
==26489==
==26489== 2 bytes in 1 blocks are definitely lost in loss record 1 of 1
==26489== at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26489== by 0x4EB79C9: strdup (in /usr/lib/libc-2.20.so)
==26489== by 0x400542: walloc (leak.c:13)
==26489== by 0x400542: main (leak.c:23)
==26489==
==26489== LEAK SUMMARY:
==26489== definitely lost: 2 bytes in 1 blocks
==26489== indirectly lost: 0 bytes in 0 blocks
==26489== possibly lost: 0 bytes in 0 blocks
==26489== still reachable: 0 bytes in 0 blocks
==26489== suppressed: 0 bytes in 0 blocks
==26489==
==26489== For counts of detected and suppressed errors, rerun with: -v
==26489== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
strdup allocates memory on the heap (using malloc) and therefore you need to free this memory when it's no longer in use like for anyone of your own direct call to malloc.
You must do that even when the program finish because this is the only way to detect a memory leak. Of course, the notion of checking for any memory leak might seem a little overkill when a program finish because all of its allocated memory is then automatically released by the OS but don't forget that your small program is an exception here. Usually, most programs will take a very large amount of memory when running and might run out of memory or run much slower if there are multiple memory leaks inside them.
Even a small program should be well written; otherwise it will become impossible for you to write any big programs later because your bad habits will translate into a tons of coding errors.
Thanks to #sharth for pointing me in the right direction. The Direct Loss was actually properly detected by Valgrind, but was confusing due to the -O3 compilation which removed root entirely. Compiling without -O3 shows the proper direct loss of 8 bytes and indirect loss of 2 bytes.
Also, thanks to #SylvainL and #Lundin for their best practices comments.
FYI: The corrected Valgrind output looks like this:
==30492== Memcheck, a memory error detector
==30492== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==30492== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==30492== Command: ./leak
==30492==
==30492==
==30492== HEAP SUMMARY:
==30492== in use at exit: 10 bytes in 2 blocks
==30492== total heap usage: 3 allocs, 1 frees, 1,830 bytes allocated
==30492==
==30492== 10 (8 direct, 2 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
==30492== at 0x4C29F90: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30492== by 0x400687: walloc (leak.c:12)
==30492== by 0x4006C6: main (leak.c:23)
==30492==
==30492== LEAK SUMMARY:
==30492== definitely lost: 8 bytes in 1 blocks
==30492== indirectly lost: 2 bytes in 1 blocks
==30492== possibly lost: 0 bytes in 0 blocks
==30492== still reachable: 0 bytes in 0 blocks
==30492== suppressed: 0 bytes in 0 blocks
==30492==
==30492== For counts of detected and suppressed errors, rerun with: -v
==30492== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Don't see line numbers using Valgrind inside Ubuntu (Vagrant+Virtualbox)

I am currently reading and fallowing along the "Learn C The Hard Way"-book. On exercise 4 I have to install Valgrind. I first did this locally on my Macbook running Maverick, but I received a warning that Valgrind might not work 100%.
So now I tried it with Vagrant (using VirtualBox) with an Ubuntu 12.04 box. You can check out the exact Vagrantfile (setup) and the exercise files here on my github repo.
The problem:
I don't see the line numbers and instead I get something like 0x40052B.
I compiled the files by doing the fallowing:
$ make clean # Just to be sure
$ make
$ valgrind --track-origins=yes ./ex4
I pasted the result to pastebin here.
I found the fallowing 3 questions on SO that (partly) describes the same problem, but the answer's and there solutions didn't work for me:
Valgrind not showing line numbers in spite of -g flag (on Ubuntu 11.10/VirtualBox)
How do you get Valgrind to show line errors?
Valgrind does not show line-numbers
What I have tried sofar:
Added libc6-dbg
installed gcc and tried compiling with that instead of cc.
added --track-origins=yes to the valgrind-command
Added (and later removed) compiling with -static and -oO flags
So I am not sure where to go from here? I could try and install the latest (instead of v3.7) off gcc manually although that looked rather difficult.
edit:
#abligh answer seems to be right. I made this with kaleidoscope:
On the left side you see the result of: valgrind --track-origins=yes ./ex4 and on the right side the result of valgrind ./ex4.
I guess I still need to learn allot about c and it's tools.
I think you are just missing them in the output.
Here is some of your output (copied from Pastebin):
==16314== by 0x40052B: main (ex4.c:9)
^^--- LINE NUMBER
==16314== Uninitialised value was created by a stack allocation
==16314== at 0x4004F4: main (ex4.c:4)
^^--- LINE NUMBER
Though I think your invocation is wrong to check memory leaks. I wrote a very simple program to leak one item:
#include <stdio.h>
#include <stdlib.h>
int
main (int argc, char **argv)
{
void *leak;
leak = malloc (1);
printf ("Leaked %p\n", leak);
exit (0);
}
and compiled it using your Makefile:
gcc -Wall -g test.c -o test
Running your command:
$ valgrind --track-origins=yes ./test
==26506== Memcheck, a memory error detector
==26506== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==26506== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==26506== Command: ./test
==26506==
Leaked 0x51f2040
==26506==
==26506== HEAP SUMMARY:
==26506== in use at exit: 1 bytes in 1 blocks
==26506== total heap usage: 1 allocs, 0 frees, 1 bytes allocated
==26506==
==26506== LEAK SUMMARY:
==26506== definitely lost: 0 bytes in 0 blocks
==26506== indirectly lost: 0 bytes in 0 blocks
==26506== possibly lost: 0 bytes in 0 blocks
==26506== still reachable: 1 bytes in 1 blocks
==26506== suppressed: 0 bytes in 0 blocks
==26506== Rerun with --leak-check=full to see details of leaked memory
==26506==
==26506== For counts of detected and suppressed errors, rerun with: -v
==26506== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
But invoking the way I normally invoke it:
$ valgrind --leak-check=full --show-reachable=yes ./test
==26524== Memcheck, a memory error detector
==26524== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==26524== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==26524== Command: ./test
==26524==
Leaked 0x51f2040
==26524==
==26524== HEAP SUMMARY:
==26524== in use at exit: 1 bytes in 1 blocks
==26524== total heap usage: 1 allocs, 0 frees, 1 bytes allocated
==26524==
==26524== 1 bytes in 1 blocks are still reachable in loss record 1 of 1
==26524== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26524== by 0x40059C: main (test.c:8)
==26524==
==26524== LEAK SUMMARY:
==26524== definitely lost: 0 bytes in 0 blocks
==26524== indirectly lost: 0 bytes in 0 blocks
==26524== possibly lost: 0 bytes in 0 blocks
==26524== still reachable: 1 bytes in 1 blocks
==26524== suppressed: 0 bytes in 0 blocks
==26524==
==26524== For counts of detected and suppressed errors, rerun with: -v
==26524== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
Note the line numbers are again in brackets, e.g.
==26524== by 0x40059C: main (test.c:8)
^^^^^^^^ <- FILENAME AND LINE NUMBER

Just a loop, and 33 leaks

Look something strange on my mac :
$> cat main.c
#include <stdio.h>
int main(int ac, char **av) {
for (int i = 0; i < ac; i++)
printf("%s\n", av[i]);
return 0;
}
$> gcc main.c -std=c99
$> valgrind ./a.out hello my friends
And here is the result :
==725== Memcheck, a memory error detector
==725== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==725== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==725== Command: ./a.out hello my friends
==725==
--725-- ./a.out:
--725-- dSYM directory is missing; consider using --dsymutil=yes
./a.out
hello
my
friends
==725==
==725== HEAP SUMMARY:
==725== in use at exit: 6,146 bytes in 33 blocks
==725== total heap usage: 33 allocs, 0 frees, 6,146 bytes allocated
==725==
==725== LEAK SUMMARY:
==725== definitely lost: 0 bytes in 0 blocks
==725== indirectly lost: 0 bytes in 0 blocks
==725== possibly lost: 0 bytes in 0 blocks
==725== still reachable: 6,146 bytes in 33 blocks
==725== suppressed: 0 bytes in 0 blocks
==725== Rerun with --leak-check=full to see details of leaked memory
==725==
==725== For counts of detected and suppressed errors, rerun with: -v
==725== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)
If someone knows why, and could explain me where does theses leaks come from, I'd be thankful !!
Have a good day :-)
These aren't leaks. Objects listed as still reachable shouldn't trouble you. If you'd have a non-zero value in the rows above then it should ring an alarm bell though!
Those 33 blocks listed as still reachable are most probably some blocks allocated inside printf calls by your standard library. Nothing to be worried about.
Consider also taking a look at this answer to a similar question.
"still reachable" when a program has terminated is really nothing to worry about.
"still reachable" means that there are allocated memory which hasn't been released, but there are still pointers pointing towards this memory. Therefor valgrind doesn't flag it as a true memory "leak".
It's often a waste of time to spend time free:ing the allocated memory before a application ends, the allocated memory will be returned to the OS anyhow since the application is terminating.

Resources