I have a multi-threaded application which uses joinable threads.
In order to create the threads (which are in total nthreads workers+1 additional thread) I use a regular call to pthread_create.
However, when I try to debug memory leaks, I get a x bytes in y blocks are possibly lost in loss record z of w error, caused by the pthread_create calls.
Here is the part of the code of the program relevant to the issue :
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <errno.h>
int main(){
int i=0;
int nthreads=8;
pthread_t tid_w[nthreads+1];
if( pthread_create(&tid_w[0], NULL, dispatcher, (void*) (size_t) i)<0){
perror("Creating dispatcher thread");
return -1;
}
for(i=1; i<nthreads+1; i++){
if( pthread_create(&tid_w[i], NULL, worker, (void*) (size_t) i)<0){
perror("Creating worker thread");
return -1;
}
}
// thread joining
for(i=0; i<nthreads+1; i++){
if( pthread_join(tid_w[i], NULL)<0){
perror("Joining thread");
return -1;
}
}
return 0;
}
This is the output of valgrind (3.11)
==14418==
==14418== HEAP SUMMARY:
==14418== in use at exit: 14,496 bytes in 21 blocks
==14418== total heap usage: 25 allocs, 4 frees, 15,724 bytes allocated
==14418==
==14418== 560 bytes in 1 blocks are possibly lost in loss record 12 of 14
==14418== at 0x4C2DB95: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14418== by 0x40134C4: allocate_dtv (dl-tls.c:322)
==14418== by 0x40134C4: _dl_allocate_tls (dl-tls.c:544)
==14418== by 0x4E400D2: allocate_stack (allocatestack.c:588)
==14418== by 0x4E400D2: pthread_create##GLIBC_2.2.5 (pthread_create.c:537)
==14418== by 0x4018AE: main (chatty.c:132)
==14418==
==14418== 4,480 bytes in 8 blocks are possibly lost in loss record 13 of 14
==14418== at 0x4C2DB95: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14418== by 0x40134C4: allocate_dtv (dl-tls.c:322)
==14418== by 0x40134C4: _dl_allocate_tls (dl-tls.c:544)
==14418== by 0x4E400D2: allocate_stack (allocatestack.c:588)
==14418== by 0x4E400D2: pthread_create##GLIBC_2.2.5 (pthread_create.c:537)
==14418== by 0x401909: main (chatty.c:138)
==14418==
==14418== LEAK SUMMARY:
==14418== definitely lost: 0 bytes in 0 blocks
==14418== indirectly lost: 0 bytes in 0 blocks
==14418== possibly lost: 5,040 bytes in 9 blocks
==14418== still reachable: 9,456 bytes in 12 blocks
==14418== suppressed: 0 bytes in 0 blocks
==14418== Reachable blocks (those to which a pointer was found) are not shown.
==14418== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==14418==
==14418== For counts of detected and suppressed errors, rerun with: -v
==14418== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Line numbers might not match because I have removed a few socket-related parts from the code.
I also have read a lot about pthread_create causing memory leaks but it seems like the issue presents itself when in use of detached threads, which is not my case.
Any idea on how to solve the issue?
Related
Is there a way to let valgrind to report "where" a "definitely lost" happens?
What I want is not "where it's allocated", but "where that poor piece of memory got leaked".
For example, this piece of code has a "definitely lost" leak when f() returns:
#include <stdlib.h>
void f () {
void *ptr = malloc(42);
}
int main () {
f();
return 0;
}
But valgrind only reports the origin of allocation:
==9772== HEAP SUMMARY:
==9772== in use at exit: 42 bytes in 1 blocks
==9772== total heap usage: 1 allocs, 0 frees, 42 bytes allocated
==9772==
==9772== 42 bytes in 1 blocks are definitely lost in loss record 1 of 1
==9772== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9772== by 0x40053E: f (test.c:4)
==9772== by 0x400552: main (test.c:8)
==9772==
==9772== LEAK SUMMARY:
==9772== definitely lost: 42 bytes in 1 blocks
==9772== indirectly lost: 0 bytes in 0 blocks
==9772== possibly lost: 0 bytes in 0 blocks
==9772== still reachable: 0 bytes in 0 blocks
==9772== suppressed: 0 bytes in 0 blocks
==9772==
==9772== For counts of detected and suppressed errors, rerun with: -v
==9772== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
I'm using --leak-check=full and --track-origins=yes.
What I want is not "where it's allocated", but "where that poor piece
of memory got leaked".
No, Valgrind only reports where the memory was allocated, but not when how or why it was leaked. This behavior is documented in Memcheck manual:
If --leak-check=full is specified, Memcheck will give details for each
definitely lost or possibly lost block, including where it was
allocated. (Actually, it merges results for all blocks that have the
same leak kind and sufficiently similar stack traces into a single
"loss record". The --leak-resolution lets you control the meaning of
"sufficiently similar".) It cannot tell you when or how or why the
pointer to a leaked block was lost; you have to work that out for
yourself.
Also --track-origins has nothing to do with memory leaks. It is used for tracking the origin of uninitialised values.
I am using Valgrind tool to understand the different types of memory leaks:
Directly lost,Indirectly lost Still reachable and possibly lost.
Ex1:
#include<stdio.h>
#include <stdlib.h>
main() {
int *p, i;
p = malloc(10*sizeof(int));
for(i = 0;i < 10;i++)
p[i] = i;
//free(p);
printf("Freed\n ");
}
LEAK SUMMARY:
==31770== definitely lost: 0 bytes in 0 blocks
==31770== indirectly lost: 0 bytes in 0 blocks
==31770== possibly lost: 20 bytes in 1 blocks
==31770== still reachable: 0 bytes in 0 blocks
==31770== suppressed: 0 bytes in 0 blocks
eg2:
main() {
int *p, i;
p = malloc(10*sizeof(int));
// printf("freed");
}
LEAK SUMMARY:
==14950== definitely lost: 0 bytes in 0 blocks
==14950== indirectly lost: 0 bytes in 0 blocks
==14950== possibly lost: 0 bytes in 0 blocks
==14950== still reachable: 40 bytes in 1 blocks
==14950== suppressed: 0 bytes in 0 blocks
If a uncomment the printf statement:
LEAK SUMMARY:
==15889== definitely lost: 40 bytes in 1 blocks
==15889== indirectly lost: 0 bytes in 0 blocks
==15889== possibly lost: 0 bytes in 0 blocks
==15889== still reachable: 0 bytes in 0 blocks
==15889== suppressed: 0 bytes in 0 blocks
Please explain me exactly about those different memory leaks.
A quick internet search leads to the following site:
http://valgrind.org/docs/manual/faq.html#faq.deflost
The details are in the Memcheck section of the user manual.
In short:
"definitely lost" means your program is leaking memory -- fix
those leaks!
"indirectly lost" means your program is leaking memory in a
pointer-based structure. (E.g. if the root node of a binary tree is
"definitely lost", all the children will be "indirectly lost".) If you
fix the "definitely lost" leaks, the "indirectly lost" leaks should go
away.
"possibly lost" means your program is leaking memory, unless
you're doing unusual things with pointers that could cause them to
point into the middle of an allocated block; see the user manual for
some possible causes. Use --show-possibly-lost=no if you don't want to
see these reports.
"still reachable" means your program is probably ok -- it didn't
free some memory it could have. This is quite common and often
reasonable. Don't use --show-reachable=yes if you don't want to see
these reports.
"suppressed" means that a leak error has been suppressed. There
are some suppressions in the default suppression files. You can ignore
suppressed errors.
Update:
The difference between "definitely lost" and "still reachable" is as follows:
Your memory is definitely lost if all references to it are gone and you did not free it before - the classical case of a memory leak.
It is still reachable when, at the end of your program's lifetime, you did not free all memory that was dynamically allocated but there are still valid references to it, so that it could have been freed.
Since all of the program's memory is freed at program termination anyway, this isn't usually a problem. However for a long running program and large memory allocations, it might be worth manually freeing those if you don't need them anymore.
Let me show you a simple example:
#include <stdio.h>
#include <stdlib.h>
char *still_reachable;
char *definitely_lost_global;
int main()
{
char *definitely_lost_local;
// allocate 10 bytes of memory -> will get lost
definitely_lost_local = malloc(10 * sizeof *definitely_lost_local);
// allocate 20 bytes of memory -> will get lost
definitely_lost_global = malloc(20 * sizeof *definitely_lost_global);
definitely_lost_global = NULL;
// Now there aren't any references to those 20 bytes anymore, so memory is lost.
// allocate 40 bytes of memory. The global pointer has static storage duration.
// We do not change the pointer, so the respective memory will
// be still reachable by the end of program lifetime
still_reachable = malloc(40 * sizeof *still_reachable);
} // scope of definitely_lost_local ends here --> 10 bytes are lost!
We can guess that 10+20 = 30 bytes will be lost in this example, while 40 bytes will be still reachable. So let's check with valgrind:
==19474== Memcheck, a memory error detector
==19474== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==19474== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==19474== Command: ./prog
==19474==
==19474==
==19474== HEAP SUMMARY:
==19474== in use at exit: 70 bytes in 3 blocks
==19474== total heap usage: 3 allocs, 0 frees, 70 bytes allocated
==19474==
==19474== LEAK SUMMARY:
==19474== definitely lost: 30 bytes in 2 blocks
==19474== indirectly lost: 0 bytes in 0 blocks
==19474== possibly lost: 0 bytes in 0 blocks
==19474== still reachable: 40 bytes in 1 blocks
==19474== suppressed: 0 bytes in 0 blocks
==19474== Rerun with --leak-check=full to see details of leaked memory
==19474==
==19474== For counts of detected and suppressed errors, rerun with: -v
==19474== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Voilá, as expected.
I'm running valgrind and it's telling me that I need to free something and I'm not sure where I would do that. It specifically points to this line "toInsert->value = linkedlist->copy(element);
Where would I run free and on what data?
"
void linkedlist_append(LinkedList *linkedlist, void *element) {
ListNode *toInsert = (ListNode*)malloc(sizeof(ListNode));
toInsert->value = linkedlist->copy(element);
toInsert->next = NULL;
ListNode *last = linkedlist->head;
if (last==NULL)
linkedlist->head = toInsert;
else{
while(last-> next !=NULL){
last = last->next;
}
last->next = toInsert;
}
linkedlist->size++;
}
Output from Valgrind:
> ^C==30515==
==30515== HEAP SUMMARY:
==30515== in use at exit: 287 bytes in 47 blocks
==30515== total heap usage: 95 allocs, 1,038 frees, 2,159 bytes allocated
==30515==
==30515== 247 bytes in 46 blocks are definitely lost in loss record 2 of 2
==30515== at 0x4C28C20: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30515== by 0x4011AC: string_copy (string_fns.c:27)
==30515== by 0x4012CE: linkedlist_append (linkedlist.c:87)
==30515== by 0x4017BD: add_file (tester.c:194)
==30515== by 0x400FE1: main (tester.c:136)
==30515==
==30515== LEAK SUMMARY:
==30515== definitely lost: 247 bytes in 46 blocks
==30515== indirectly lost: 0 bytes in 0 blocks
==30515== possibly lost: 0 bytes in 0 blocks
==30515== still reachable: 40 bytes in 1 blocks
==30515== suppressed: 0 bytes in 0 blocks
==30515== Reachable blocks (those to which a pointer was found) are not shown.
==30515== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30515==
==30515== For counts of detected and suppressed errors, rerun with: -v
==30515== ERROR SUMMARY: 2200 errors from 10 contexts (suppressed: 0 from 0)
Valgrind is pointing you to the location of a malloc that is never freed. Before you terminate your program, you will need to iterate through the list an free all its elements.
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[]){
char *str = malloc(sizeof(char)*5);
str = strcpy(str, "test");
printf("%s\n", str);
free(str);
return 0;
}
When I use Valgrind on my Mac (OS X, 10.9.5) I get the following message:
==77215== HEAP SUMMARY:
==77215== in use at exit: 29,211 bytes in 374 blocks
==77215== total heap usage: 451 allocs, 77 frees, 35,160 bytes allocated
==77215==
==77215== 4,096 bytes in 1 blocks are still reachable in loss record 76 of 76
==77215== at 0x66CB: malloc (in /usr/local/Cellar/valgrind/3.10.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==77215== by 0x182855: __smakebuf (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x197217: __swsetup (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x1B0158: __v2printf (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x1B06AF: __xvprintf (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x187B29: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x18596F: printf (in /usr/lib/system/libsystem_c.dylib)
==77215== by 0x100000F2B: main (test.c:8)
==77215==
==77215== LEAK SUMMARY:
==77215== definitely lost: 0 bytes in 0 blocks
==77215== indirectly lost: 0 bytes in 0 blocks
==77215== possibly lost: 0 bytes in 0 blocks
==77215== still reachable: 4,096 bytes in 1 blocks
==77215== suppressed: 25,115 bytes in 373 blocks
==77215==
==77215== For counts of detected and suppressed errors, rerun with: -v
==77215== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 15 from 15)
Does printf allocate memory by itself? If I remove the printf I only get the following:
==77237== HEAP SUMMARY:
==77237== in use at exit: 25,115 bytes in 373 blocks
==77237== total heap usage: 450 allocs, 77 frees, 31,064 bytes allocated
==77237==
==77237== LEAK SUMMARY:
==77237== definitely lost: 0 bytes in 0 blocks
==77237== indirectly lost: 0 bytes in 0 blocks
==77237== possibly lost: 0 bytes in 0 blocks
==77237== still reachable: 0 bytes in 0 blocks
==77237== suppressed: 25,115 bytes in 373 blocks
==77237==
==77237== For counts of detected and suppressed errors, rerun with: -v
==77237== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 15 from 15)
Where does the 373 blocks come from?
Until the Valgrind team prioritizes OS X, you can safely assume that it will not give correct results on Apple systems running OS X versions newer than 10.7.
On my Mavericks (10.9.5) machine, I still get the following warning from Valgrind (3.9.0)
WARNING: Support on MacOS 10.8/10.9 is experimental and mostly broken.
WARNING: Expect incorrect results, assertions and crashes.
WARNING: In particular, Memcheck on 32-bit programs will fail to
WARNING: detect any errors associated with heap-allocated data.
For what it's worth, Valgrind 3.10.0 shows no leaks on my Debian Jessie installation.
It isn't telling you that there is a leak:
==77215== LEAK SUMMARY:
==77215== definitely lost: 0 bytes in 0 blocks
==77215== indirectly lost: 0 bytes in 0 blocks
==77215== possibly lost: 0 bytes in 0 blocks
==77215== still reachable: 4,096 bytes in 1 blocks
==77215== suppressed: 25,115 bytes in 373 blocks
it's telling you that there is a block which is still reachable; whether or not there is a leak is dependent on your definition of 'leak'. What it does mean is that there is a pointer to the block somewhere.
Refer to Still Reachable Leak detected by Valgrind for more information.
So I have a php, ruby and python background and got started with C. I'm using Valgrind to check if my programs are not doing silly things but I get this kind of output quite frequently:
14072== <...VALGRIND HEADER & COPYRIGHT...>
14158== HEAP SUMMARY:
14158== in use at exit: 137,084 bytes in 196 blocks
14158== total heap usage: 247 allocs, 51 frees, 149,496 bytes allocated
14158==
14158== 7 bytes in 1 blocks are definitely lost in loss record 3 of 74 at
14158== 0x4C2745D: malloc in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so
14158== by 0x50F3369: strdup in /usr/lib64/libc-2.18.so
14158== by 0x4E51B34: readline in /usr/lib64/libedit.so.0.0.43
14158== by 0x40083C: main in /home/<program location>/
14158==
14158== LEAK SUMMARY:
14158== definitely lost: 7 bytes in 1 blocks
14158== indirectly lost: 0 bytes in 0 blocks
14158== possibly lost: 0 bytes in 0 blocks
14158== still reachable: 137,077 bytes in 195 blocks
14158== suppressed: 0 bytes in 0 blocks
14158== Reachable blocks (those to which a pointer was found) are not shown.
14158== To see them, rerun with: --leak-check=full --show-leak-kinds=all
14158==
14158== For counts of detected and suppressed errors, rerun with: -v
14158== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
This valgrind debug output is from this quick REPL that simply shouts back user input on the screen or quits in case input equals to :exit:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <editline/readline.h>
static char prompt[5] = "repl>";
static char* cmd_exit = ":exit";
int main(int argc, char** argv) {
puts("Press Ctrl+c to Exit\n");
while(1) {
char* input = readline(prompt);
add_history(input);
if(strcmp(input,cmd_exit) == 0) {
puts("Bye!");
return 0;
}
printf("%s\n", input);
free(input);
}
return 0;
}
I've already tried to free both prompt and cmd_exit variables before function main returns a value but leak is still there according to Valgrind.
...
free(prompt);
free(cmd_exit);
return 0;
}
Some questions:
How to get rid of this reported memory leak?
Should I be using static variables in this situation?
Should I free those static variables too?
Any good advice or material is welcome.
You only need to free things you malloced, so you don't need to free the static variables.
As your readline function seems to use strdup and that uses malloc/calloc, you need to free input. You do this in the loop which is good, but you missed it when you exit the loop:
if(strcmp(input,cmd_exit) == 0) {
puts("Bye!");
free(input);
return 0;
}
By the way:
static char prompt[6] = "repl>";
The text "repl>" has 6 chars, r, e, p, l, > and a nul byte to mark the end of the string.