How do I use valgrind to find the memory leaks in a program?
Please someone help me and describe the steps to carryout the procedure?
I am using Ubuntu 10.04 and I have a program a.c, please help me out.
How to Run Valgrind
Not to insult the OP, but for those who come to this question and are still new to Linux—you might have to install Valgrind on your system.
sudo apt install valgrind # Ubuntu, Debian, etc.
sudo yum install valgrind # RHEL, CentOS, Fedora, etc.
sudo pacman -Syu valgrind # Arch, Manjaro, Garuda, etc
Valgrind is readily usable for C/C++ code, but can even be used for other
languages when configured properly (see this for Python).
To run Valgrind, pass the executable as an argument (along with any
parameters to the program).
valgrind --leak-check=full \
--show-leak-kinds=all \
--track-origins=yes \
--verbose \
--log-file=valgrind-out.txt \
./executable exampleParam1
The flags are, in short:
--leak-check=full: "each individual leak will be shown in detail"
--show-leak-kinds=all: Show all of "definite, indirect, possible, reachable" leak kinds in the "full" report.
--track-origins=yes: Favor useful output over speed. This tracks the origins of uninitialized values, which could be very useful for memory errors. Consider turning off if Valgrind is unacceptably slow.
--verbose: Can tell you about unusual behavior of your program. Repeat for more verbosity.
--log-file: Write to a file. Useful when output exceeds terminal space.
Finally, you would like to see a Valgrind report that looks like this:
HEAP SUMMARY:
in use at exit: 0 bytes in 0 blocks
total heap usage: 636 allocs, 636 frees, 25,393 bytes allocated
All heap blocks were freed -- no leaks are possible
ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
I have a leak, but WHERE?
So, you have a memory leak, and Valgrind isn't saying anything meaningful.
Perhaps, something like this:
5 bytes in 1 blocks are definitely lost in loss record 1 of 1
at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
by 0x40053E: main (in /home/Peri461/Documents/executable)
Let's take a look at the C code I wrote too:
#include <stdlib.h>
int main() {
char* string = malloc(5 * sizeof(char)); //LEAK: not freed!
return 0;
}
Well, there were 5 bytes lost. How did it happen? The error report just says
main and malloc. In a larger program, that would be seriously troublesome to
hunt down. This is because of how the executable was compiled. We can
actually get line-by-line details on what went wrong. Recompile your program
with a debug flag (I'm using gcc here):
gcc -o executable -std=c11 -Wall main.c # suppose it was this at first
gcc -o executable -std=c11 -Wall -ggdb3 main.c # add -ggdb3 to it
Now with this debug build, Valgrind points to the exact line of code
allocating the memory that got leaked! (The wording is important: it might not
be exactly where your leak is, but what got leaked. The trace helps you find
where.)
5 bytes in 1 blocks are definitely lost in loss record 1 of 1
at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
by 0x40053E: main (main.c:4)
Techniques for Debugging Memory Leaks & Errors
Make use of www.cplusplus.com! It has great documentation on C/C++ functions.
General advice for memory leaks:
Make sure your dynamically allocated memory does in fact get freed.
Don't allocate memory and forget to assign the pointer.
Don't overwrite a pointer with a new one unless the old memory is freed.
General advice for memory errors:
Access and write to addresses and indices you're sure belong to you. Memory
errors are different from leaks; they're often just IndexOutOfBoundsException
type problems.
Don't access or write to memory after freeing it.
Sometimes your leaks/errors can be linked to one another, much like an IDE discovering that you haven't typed a closing bracket yet. Resolving one issue can resolve others, so look for one that looks a good culprit and apply some of these ideas:
List out the functions in your code that depend on/are dependent on the
"offending" code that has the memory error. Follow the program's execution
(maybe even in gdb perhaps), and look for precondition/postcondition errors. The idea is to trace your program's execution while focusing on the lifetime of allocated memory.
Try commenting out the "offending" block of code (within reason, so your code
still compiles). If the Valgrind error goes away, you've found where it is.
If all else fails, try looking it up. Valgrind has documentation too!
A Look at Common Leaks and Errors
Watch your pointers
60 bytes in 1 blocks are definitely lost in loss record 1 of 1
at 0x4C2BB78: realloc (vg_replace_malloc.c:785)
by 0x4005E4: resizeArray (main.c:12)
by 0x40062E: main (main.c:19)
And the code:
#include <stdlib.h>
#include <stdint.h>
struct _List {
int32_t* data;
int32_t length;
};
typedef struct _List List;
List* resizeArray(List* array) {
int32_t* dPtr = array->data;
dPtr = realloc(dPtr, 15 * sizeof(int32_t)); //doesn't update array->data
return array;
}
int main() {
List* array = calloc(1, sizeof(List));
array->data = calloc(10, sizeof(int32_t));
array = resizeArray(array);
free(array->data);
free(array);
return 0;
}
As a teaching assistant, I've seen this mistake often. The student makes use of
a local variable and forgets to update the original pointer. The error here is
noticing that realloc can actually move the allocated memory somewhere else
and change the pointer's location. We then leave resizeArray without telling
array->data where the array was moved to.
Invalid write
1 errors in context 1 of 1:
Invalid write of size 1
at 0x4005CA: main (main.c:10)
Address 0x51f905a is 0 bytes after a block of size 26 alloc'd
at 0x4C2B975: calloc (vg_replace_malloc.c:711)
by 0x400593: main (main.c:5)
And the code:
#include <stdlib.h>
#include <stdint.h>
int main() {
char* alphabet = calloc(26, sizeof(char));
for(uint8_t i = 0; i < 26; i++) {
*(alphabet + i) = 'A' + i;
}
*(alphabet + 26) = '\0'; //null-terminate the string?
free(alphabet);
return 0;
}
Notice that Valgrind points us to the commented line of code above. The array
of size 26 is indexed [0,25] which is why *(alphabet + 26) is an invalid
write—it's out of bounds. An invalid write is a common result of
off-by-one errors. Look at the left side of your assignment operation.
Invalid read
1 errors in context 1 of 1:
Invalid read of size 1
at 0x400602: main (main.c:9)
Address 0x51f90ba is 0 bytes after a block of size 26 alloc'd
at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
by 0x4005E1: main (main.c:6)
And the code:
#include <stdlib.h>
#include <stdint.h>
int main() {
char* destination = calloc(27, sizeof(char));
char* source = malloc(26 * sizeof(char));
for(uint8_t i = 0; i < 27; i++) {
*(destination + i) = *(source + i); //Look at the last iteration.
}
free(destination);
free(source);
return 0;
}
Valgrind points us to the commented line above. Look at the last iteration here,
which is *(destination + 26) = *(source + 26);. However, *(source + 26) is
out of bounds again, similarly to the invalid write. Invalid reads are also a
common result of off-by-one errors. Look at the right side of your assignment
operation.
The Open Source (U/Dys)topia
How do I know when the leak is mine? How do I find my leak when I'm using
someone else's code? I found a leak that isn't mine; should I do something? All
are legitimate questions. First, 2 real-world examples that show 2 classes of
common encounters.
Jansson: a JSON library
#include <jansson.h>
#include <stdio.h>
int main() {
char* string = "{ \"key\": \"value\" }";
json_error_t error;
json_t* root = json_loads(string, 0, &error); //obtaining a pointer
json_t* value = json_object_get(root, "key"); //obtaining a pointer
printf("\"%s\" is the value field.\n", json_string_value(value)); //use value
json_decref(value); //Do I free this pointer?
json_decref(root); //What about this one? Does the order matter?
return 0;
}
This is a simple program: it reads a JSON string and parses it. In the making,
we use library calls to do the parsing for us. Jansson makes the necessary
allocations dynamically since JSON can contain nested structures of itself.
However, this doesn't mean we decref or "free" the memory given to us from
every function. In fact, this code I wrote above throws both an "Invalid read"
and an "Invalid write". Those errors go away when you take out the decref line
for value.
Why? The variable value is considered a "borrowed reference" in the Jansson
API. Jansson keeps track of its memory for you, and you simply have to decref
JSON structures independent of each other. The lesson here:
read the documentation. Really. It's sometimes hard to understand, but
they're telling you why these things happen. Instead, we have
existing questions about this memory error.
SDL: a graphics and gaming library
#include "SDL2/SDL.h"
int main(int argc, char* argv[]) {
if (SDL_Init(SDL_INIT_VIDEO|SDL_INIT_AUDIO) != 0) {
SDL_Log("Unable to initialize SDL: %s", SDL_GetError());
return 1;
}
SDL_Quit();
return 0;
}
What's wrong with this code? It consistently leaks ~212 KiB of memory for me. Take a moment to think about it. We turn SDL on and then off. Answer? There is nothing wrong.
That might sound bizarre at first. Truth be told, graphics are messy and sometimes you have to accept some leaks as being part of the standard library. The lesson here: you need not quell every memory leak. Sometimes you just need to suppress the leaks because they're known issues you can't do anything about. (This is not my permission to ignore your own leaks!)
Answers unto the void
How do I know when the leak is mine?
It is. (99% sure, anyway)
How do I find my leak when I'm using someone else's code?
Chances are someone else already found it. Try Google! If that fails, use the skills I gave you above. If that fails and you mostly see API calls and little of your own stack trace, see the next question.
I found a leak that isn't mine; should I do something?
Yes! Most APIs have ways to report bugs and issues. Use them! Help give back to the tools you're using in your project!
Further Reading
Thanks for staying with me this long. I hope you've learned something, as I tried to tend to the broad spectrum of people arriving at this answer. Some things I hope you've asked along the way: How does C's memory allocator work? What actually is a memory leak and a memory error? How are they different from segfaults? How does Valgrind work? If you had any of these, please do feed your curiousity:
More about malloc, C's memory allocator
Definition of a segmentation fault
Definition of a memory leak
Definition of a memory access error
How does Valgrind work?
Try this:
valgrind --leak-check=full -v ./your_program
As long as valgrind is installed it will go through your program and tell you what's wrong. It can give you pointers and approximate places where your leaks may be found. If you're segfault'ing, try running it through gdb.
You can run:
valgrind --leak-check=full --log-file="logfile.out" -v [your_program(and its arguments)]
You can create an alias in .bashrc file as follows
alias vg='valgrind --leak-check=full -v --track-origins=yes --log-file=vg_logfile.out'
So whenever you want to check memory leaks, just do simply
vg ./<name of your executable> <command line parameters to your executable>
This will generate a Valgrind log file in the current directory.
Related
I have a very interesting problem.
I'd like to use PCRE2, and its JIT function. The task is simple: read lines from a file, and find patterns.
Here is the sample code:
#include <stdio.h>
#include <string.h>
#define PCRE2_CODE_UNIT_WIDTH 8
#include <pcre2.h>
int search(pcre2_code *re, unsigned char * subject) {
pcre2_match_data *match_data_real = pcre2_match_data_create_from_pattern(re, NULL);
size_t len_subject = strlen((const char *)subject);
int rc = pcre2_match(
re,
(PCRE2_SPTR)subject,
len_subject,
0,
0,
match_data_real,
NULL
);
pcre2_match_data_free(match_data_real);
return rc;
}
int main(int argc, char ** argv) {
unsigned char subject[][100] = {
"this is a foobar",
"this is a barfoo",
"this is a barbar",
"this is a foofoo"
};
pcre2_code *re;
PCRE2_SPTR pattern = (unsigned char *)"foo";
int errornumber;
PCRE2_SIZE erroroffset;
re = pcre2_compile(
pattern,
PCRE2_ZERO_TERMINATED,
0,
&errornumber,
&erroroffset,
NULL
);
pcre2_jit_compile(re, PCRE2_JIT_COMPLETE);
FILE *fp;
int s = 0;
while(s < 2) {
search(re, subject[s++]);
}
if (argc >= 2) {
fp = fopen(argv[1], "r");
if (fp != NULL) {
char tline[2048];
while(fgets(tline, 2048, fp) != NULL) {
search(re, (unsigned char *)tline);
}
fclose(fp);
}
}
pcre2_code_free(re);
return 0;
}
Compile the code:
gcc -Wall -O2 -g pcretest.c -o pcretest -lpcre2-8
As you can see, in line 58 I check if there is an argument given, the code tries to open it as a file.
Also as you can see in line 49, I'd like to use PCRE2's JIT.
The code works as well, but I checked it with Valgrind, and found an interesting behavior:
if I add a file as argument, then Valgrind reports Conditional jump or move depends on uninitialised value(s) and Uninitialised value was created by a stack allocation, but it points to the main(). The command:
valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes -s ./pcretest myfile.txt
Without argument, there is no any Valgrind report. Command:
valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes -s ./pcretest
if I comment out the pcre2_jit_compile((*re), PCRE2_JIT_COMPLETE); in line 55, then everything works as well, no any Valgrind reports. Command:
valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes -s ./pcretest myfile.txt
The Valgrind's relevant output:
==31385== Conditional jump or move depends on uninitialised value(s)
==31385== at 0x4EECD1A: ???
==31385== by 0x1FFEFFFC1F: ???
==31385== Uninitialised value was created by a stack allocation
==31385== at 0x1090FA: main (pcretest.c:27)
...
==31385== HEAP SUMMARY:
==31385== in use at exit: 0 bytes in 0 blocks
==31385== total heap usage: 12 allocs, 12 frees, 13,486 bytes allocated
==31385==
==31385== All heap blocks were freed -- no leaks are possible
==31385==
==31385== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
==31385==
==31385== 1 errors in context 1 of 1:
==31385== Conditional jump or move depends on uninitialised value(s)
==31385== at 0x4EECD1A: ???
==31385== by 0x1FFEFFFC1F: ???
==31385== Uninitialised value was created by a stack allocation
==31385== at 0x1090FA: main (pcretest.c:27)
==31385==
==31385== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
In line 27 there is the int main(...).
What do I miss?
Observations:
The Valgrind report is telling you that the uninitialized data being accessed are in the stack frame of the initial call to main(). However,
even though you're compiling with debug information, the Valgrind report does not implicate a specific variable. Also,
the report's stack trace for the error does not present function names, and does not trace back to main(). And of course,
the error is not reported when you disable JIT compilation of the pattern.
Apparently, then, the error is associated with the machine code generated by PCRE2's JIT compiler. If you don't perform JIT compilation then you get correct operation via the ordinary matching path. If you do perform JIT compilation then the JIT-generated code is engaged, and that code triggers the Valgrind error. You might nevertheless get correct matching, but I would not rely on that for code that triggers the Valgrind error observed.
I played around with variations on your code, and discovered that the error is specifically associated with the calls to pcre2_match_data_create_from_pattern() and pcre2_match() in function search(). Either one will cause Valgrind to report the error. But why does the error occur only in some calls to search()?
It seems likely to be because the JIT compilation sets up data structures in main()'s stack frame that are clobbered by executing the body of the if (argc > 2) statement. This is consistent with the fact that I was able to avoid the error by adding an initializer for variable tline in that block:
char tline[2048] = {0};
I can imagine a variety of scenarios for why that might make a difference, all having to do with how the JIT-generated code and the compiler-generated code manipulate the stack pointer.
Personally, discovering such an issue would likely persuade me to stay far away from PCRE's JIT compiler. Definitely I would do that at least until I had evidence of pattern matching being a performance hotspot for my program. If you must engage the JIT, however, then here are some recommendations that might (or might not) help you avoid trouble:
Take "just in time" to heart: perform JIT as close as possible to when you actually use the pattern.
Do not assume that the JIT code is long-term viable. In particular, it probably is unsafe to use after the function that calls the JIT compiler returns, but it might not be good even that long.
Use the JIT-compiled regex (only) in the same function that runs the JIT compiler.
Make that function as simple as possible.
Declare all local variables of that function at the beginning, with initializers.
Test thoroughly.
That's more than seems to have been necessary to resolve the issue for your particular example code, but it's aimed more generally at reducing the cross section for the compiled program violating assumptions made by the JIT.
This is indeed caused by efficient use of SSE2. CPU-s use 1K or bigger pages to map memory, so a 16 byte aligned 16 byte read (SSE2 registers are 16 byte long) which intersects with a valid buffer is always valid. However, bytes before the start or after the end of the buffer might never be initialized. The algorithm ignores these bytes, so the random data (regardless it is initialized or not) have no effect on any computation.
I havent used valgrind before but I think it should detect some memory errors.
My code:
#include <stdio.h>
unsigned int a[2];
int main()
{
a[-1] = 21;
printf("%d,", a[-1]);
return 1;
}
As you can see, I am accessing a[-1] which I should not.
How am I using valgrind?
I am compiling with gcc -g -O0 codeFile.c
And executing: valgrind -s ./a.out
Result is:
==239== Memcheck, a memory error detector
==239== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==239== Using Valgrind-3.16.0.GIT and LibVEX; rerun with -h for copyright info
==239== Command: ./a.out
==239== 21,==239==
==239== HEAP SUMMARY:
==239== in use at exit: 0 bytes in 0 blocks
==239== total heap usage: 1 allocs, 1 frees, 1,024 bytes allocated
==239==
==239== All heap blocks were freed -- no leaks are possible
==239==
==239== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Shouldnt valgrind find these error, or am I using it wrong?
EDIT:
It seems that valgrind memcheck does not do anything for global variables and as suggested in the answers/comments that it should work with indexes further from the pointer, therefore:
I removed the global declaration and added it insude main, and accessed a[-10] instead of a[1]. Same behaviour.
int main()
{
unsigned int a[2];
a[-10] = 21;
printf("%d,", a[-10]);
return 1;
}
It actually throws error if I use a[-100] though. Whats the deal?
EDIT 2
Furthermore, why this has no errors
while (i <= 2)
{
j = a[i];
i++;
}
But this does
while (i <= 2)
{
printf("%d,", a[i]);
i++;
}
Valgrind usually can't find memory errors where the memory being modified is at a negative offset from the current stack pointer or memory that coincides with another variable in memory.
For example, if a was on the stack, a[3] would trigger memcheck. a[-1] would not, because that, for all Valgrind knows, could easily be valid memory.
To expand on that, here's a quote from the documentation with my emphasis added:
In this example, Memcheck can't identify the address. Actually the address is on the stack, but, for some reason, this is not a valid stack address -- it is below the stack pointer and that isn't allowed.
This quote is actually partially incorrect; when it says "below the stack pointer" it really means at a positive offset from the stack pointer, or interfering with another function's stack memory.
I should also note that (from your second edit) Valgrind doesn't actually complain until the value is used in some meaningful way. Assignment is, in Valgrind's eyes, not using the value in a meaningful way. Here's another quote to back that up with my emphasis added:
It is important to understand that your program can copy around junk (uninitialised) data as much as it likes. Memcheck observes this and keeps track of the data, but does not complain. A complaint is issued only when your program attempts to make use of uninitialised data in a way that might affect your program's externally-visible behaviour.
Because a is a global variable, you'll have a hard time trying to check the memory of it. One Valgrind tool I've used before that deals with this is exp-sgcheck (experimental static and global variable check), although I've found it to be unreliable (most likely due to it being experimental).
An easier and better way to detect these would be to enable compiler warnings or use a static analyzer (my favorite is LLVM's scan-build).
You declared a as an global array, so use --tool=exp-sgcheck to check for stack and global array overruns. Keep in mind that --tool=exp-sgcheck is an experimental implementation so it doesn't show up whenever enabling -s or --show-error-list=yes, you can read more about it here.
I am writing a pet project (a sort of single-threaded Lisp-like language interpreter in C), and I came upon the following issue: a pointer is overwritten while malloc() is run. Showing all the code would be too long, but I can share it if necessary. I would like to have some insight regarding how the problem can be debugged.
The bug happens during the subroutine that runs a user-defined function:
/* Determine the amount of arguments to the called function */
int argc = ast_len(fn_args)
printf("%s:%d %p\n", __FILE__, __LINE__, (void*) scope->vars->tail);
/* Allocate the memory to store the array of pointers to each of the arguments */
struct YL_Var** argv = malloc(sizeof(struct YL_Var*)*argc);
printf("%s:%d %p\n", __FILE__, __LINE__, (void*) scope->vars->tail);
You will get the following output:
interpreter.c:549 0x5558371c9480
interpreter.c:551 0x411
Segmentation fault (core dumped)
The pointer scope->vars->tail is overwritten during the call to malloc()!
Using gdb and hardware break points clearly showed that the value was overwritten inside malloc.c.
As the pointer is overwritten, the program segfaults soon after, both within gdb and in normal run. However it does not segfault when run inside valgrind, it even ends successfully.
So here is my question. How would you start to debug this mess? I am asking for advice, and not for an answer.
I am far from being a C expert :)
I guess I am at fault and this is of course no bug in glibc-2.26 or gcc 7.2.0.
I have no warnings on gcc with -Wall -Wextra -Wpedantic
valgrind shows some unfreed memory issue. Will fix them before doing anything else.
Thanks to everyone's comment, I found the issue.
scope->vars was not appropriately allocated (as some said).
When using valgrind, I found the following message:
==23054== Invalid write of size 8
==23054== at 0x10A380: varlist_prepend (interpreter.c:277)
==23054== by 0x109548: main (yl.c:39)
==23054== Address 0x5572a98 is 0 bytes after a block of size 8 alloc'd
==23054== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==23054== by 0x10A304: varlist_prepend (interpreter.c:274)
==23054== by 0x109548: main (yl.c:39)
==23054==
My code looked like this:
struct YL_VarList* vars = malloc(sizeof(vars));
As you can see, the * was missing.
This is the corrected version:
struct YL_VarList* vars = malloc(sizeof(*vars));
sizeof(vars) would return the size of struct YL_VarList*, while I want to allocate the size of struct YL_VarList.
Learning Valgrind here, and also learning how to write better C.
I am trying to parse the command line of an example program using GLib's command line parsing; in fact, took almost verbatim the provided example. The only difference is that I "pop" the first element of argv and use it as a command for the rest of the program; in order to do so, I skip the first argument and copy the rest to an array char **arguments:
// file: main.c
int main(int argc, char **argv)
{
const char *allowed_cmds[] = {"greet", "teerg"};
char cmd[24];
g_stpcpy(cmd, argv[1]);
char **arguments= (char**)calloc((argc - 1), sizeof(char*));
if (check_string_in_array(cmd, allowed_cmds, 2)) {
skip_elements_from_array(argv, argc, 1, arguments);
}
char saluted[24];
read_saluted_from_command_line(argc, arguments, saluted);
free(arguments);
// ... skipped ...
return 0;
}
// file: hello.c
int read_saluted_from_command_line(int argc, char **argv, char *result)
{
gchar *saluted = "world";
GError *error = NULL;
GOptionContext *context;
GOptionEntry entries[] =
{
{ "saluted", 's', G_OPTION_FLAG_NONE, G_OPTION_ARG_STRING, &saluted, "person or thing to salute", "WORLD" },
{ NULL }
};
context = g_option_context_new("- Say hello to a person or thing");
g_option_context_add_main_entries(context, entries, NULL);
if (!g_option_context_parse_strv(context, &argv, &error))
{
g_error("option parsing failed: %s\n", error->message);
exit(1);
}
g_option_context_free(context);
if (error != NULL)
g_error_free(error);
g_stpcpy(result, saluted);
return 0;
}
This code compiles and runs fine, but checking with Valgrind leads to:
$ valgrind --read-var-info=yes --track-origins=yes --leak-check=full ./hello greet
==7779== Memcheck, a memory error detector
==7779== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==7779== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==7779== Command: ./hello greet
==7779==
==7779== Invalid read of size 8
==7779== at 0x4E9F303: g_strv_length (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4200.1)
==7779== by 0x4E8B0AC: g_option_context_parse_strv (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4200.1)
==7779== by 0x40128C: read_saluted_from_command_line (hello.c:54)
==7779== by 0x401753: main (main.c:68)
==7779== Address 0x597a298 is 0 bytes after a block of size 8 alloc'd
==7779== at 0x4C2AD10: calloc (vg_replace_malloc.c:623)
==7779== by 0x4016EE: main (main.c:58)
The code uses function g_option_context_parse_strv because according to documentation this function does not "assum[e] that the passed-in array is the argv of the main function". Using g_option_context_parse leads to the same message.
I am quite sure that the offending variable is arguments because it is precisely alloc'd in main:68, but I don't understand why Valgrind thinks that "your program reads or writes memory at a place which Memcheck reckons it shouldn't". Even more puzzling to me is the fact that the error disappears if I move the code from a separate function in a different file and paste it directly into main.c. Is this an error in passing the char ** to the function?
(I have found several threads on Stack Overflow that discuss Valgrind and invalid reads, but all of them deal with structs that are defined by the OP, and none have anything to do with GLib).
Thanks for any help!
Before I get to (I think) the answer: when asking for help, you should always post a complete snippet which people can compile and run themselves (i.e., a SSCCE). Also, when looking at valgrind logs, it's important to make sure to post a complete example so people can see exactly where the warnings are coming from.
Based on what you've posted, the problem is that g_option_context_parse_strv expects a NULL-terminated array. Since you're not also passing a length, that is the only way for glib to know what is the array. As it is, since it doesn't encounter a NULL element glib will continue reading past the end of the array into uninitialized memory, which is where valgrind (rightfully) complains. You need to allocate room for an extra element in arguments and set it to NULL.
As for David's comment about glib and valgrind not getting along, it's very important to keep in mind that this is only the case for leaks, and even then only for certain types of leaks. Warnings about accessing uninitialized and/or invalid memory are every bit as "real" in glib-based programs as anywhere else. It's dangerous to simply disregard valgrind's output (or AddressSanitizer, or other similar tools) without understanding that.
The limitation in valgrind with leaks is that GLib allocates a small amount of memory for type information which is shared by every instance of a type. This information is never freed, though it is still accessible (which is why valgrind lists it as possibly lost, not definitely lost). Basically, you can usually disregard warnings about allocations coming from a g_gtype_* function being possibly lost, but that's it.
The reason information is never freed because doing so would simply waste performance. Obviously you would need to know when to free it, which means keeping track of whether or not it is still in use. That means either a tracing garbage collector (which isn't really an option for a C library), or reference counting. Reference counting requires keeping a counter synchronized across multiple cores and cache levels (and possibly other CPUs), which is a huge performance drain and very much not worth it just to avoid some easy-to-identify false positives in a couple tools (like valgrind and AddressSanitizer).
I use this code snippet:
// stackoverflow.c
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char** argv)
{
int i;
int a[10];
// init
a[-1] = -1;
a[11] = 11;
printf(" a[-1]= = %d, a[11] = %d\n", a[-1], a[11]);
printf("I am finished.\n");
return a[-1];
}
The compiler is GCC for linux x86. It works well without any run-time error. I also test this code in Valgrind, which don't trigger any memory error either.
$ gcc -O0 -g -o stack_overflow stack_overflow.c
$ ./stack_overflow
a[-1]= = -1, a[11] = 11
I am finished.
$ valgrind ./stack_overflow
==3705== Memcheck, a memory error detector
==3705== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==3705== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==3705== Command: ./stack_overflow
==3705==
a[-1]= = -1, a[11] = 11
I am finished.
==3705==
==3705== HEAP SUMMARY:
==3705== in use at exit: 0 bytes in 0 blocks
==3705== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==3705==
==3705== All heap blocks were freed -- no leaks are possible
==3705==
==3705== For counts of detected and suppressed errors, rerun with: -v
==3705== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
From my understanding, heap and stack is the same kind of memory. The only difference is that they grow in the opposite direction.
So my question is:
Why heap overflow/underflow will trigger an rum-time error, while stack overflow/underflow will not?
why C language designer didn't take this into account just like heap, other than leave it Undefined Behaviour
valgrind does not detect stack buffer overflows. Use AddressSanitizer. At least gcc 4.8 is required and libasan must be installed.
gcc -g -fsanitize=address stackbufferoverflow.c
==1955==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7fffff438d4c at pc 0x000000400a1d bp 0x7fffff438d10 sp 0x7fffff438d00
WRITE of size 4 at 0x7fffff438d4c thread T0
#0 0x400a1c in main /home/m/stackbufferoverflow.c:9
#1 0x7fe7e24e178f in __libc_start_main (/lib64/libc.so.6+0x2078f)
#2 0x400888 in _start (/home/m/a.out+0x400888)
Address 0x7fffff438d4c is located in stack of thread T0 at offset 28 in frame
#0 0x400965 in main /home/m/stackbufferoverflow.c:5
This frame has 1 object(s):
[32, 72) 'a' <== Memory access at offset 28 underflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow /home/m/stackbufferoverflow.c:9 main
why C language designer didn't take this into account just like heap, other than leave it Undefined Behaviour
The original C langauge designers wrote a kind of more comfortable and portable assembler for themselves. The original language has not been designed to be bullet-proof against programmers' errors.
If you are interested in an opposite example then look at Ada (http://en.wikipedia.org/wiki/Ada_%28programming_language%29).
C doesn't check things like out-of-bounds array indexing. It just does what you told it to, in this case to change element number 11 in an array of 10 elements. Typically, this means that your program writes to the location in memory where this item should have been stored, if it had existed. This may or may not cause some sort of visible error, such a crash. It might have no effect, or it could make your program do something strange. It depends on what, if anything, happened to be stored at that place in memory, and how it is used.
Some other programming languages do perform checks such as these, and guarantee that an error will be reported. The C standard gives no such guarantees, and just says that it will cause "undefined behaviour". One reason for this is that it should be possible to write very efficient programs in C, where checks would cause a small, but in some cases perhaps unacceptable, delay. Also, back when C was designed, computers were slower, and the delay would have been a worse problem.
There is also no guarantee in C that heap errors will be detected or reported. Valgrind is not part of the C language, but a different tool, and it does its best to find errors using other and more effective mechanisms than C would, but there is no guarantee that it will find all errors.
EDIT
Here's an interesting tuto:
http://gribblelab.org/CBootcamp/7_Memory_Stack_vs_Heap.html
BTW Clang (OSX) detects it, but it's just and extra feature, good old gcc would let you do it.
ctest.c:6:5: warning: array index 42 is past the end of the array (which contains 1 element) [-Warray-bounds]
a[42] = 42;
^ ~~
cpp.cpp:4:5: note: array 'a' declared here
int a[1];
^
1 warning generated.
Old
a[11] = 11;
Would trigger a Segmentation fault (but here it's only one byte it's just overriding the value of another variable, most likely), if you want a stack overflow try something that does an infinite recursion.
Also if you want to make your code segfault proof (for malloc only) I suggest you compile it with electric fence for your tests. It will prevent your program to go above its allocated memory (starting from the first byte)
http://linux.die.net/man/3/efence
As suggested in the comments Valgrind is also a useful tool.
http://valgrind.org/
Why does not stack overflow/underflow trigger an run-time error?
C is not limited to "heap" and "stack" implementations. Example: Variables in main() need not be in a "stack". Even GCC may optimize in way that defy a simple understanding. Many memory architectures are possible. Since C does not specify the underlying memory architecture, the following is simply undefined behavior. #Karoly Horvath
// Undefined behavior: accessing memory outside array's range.
int a[10];
a[-1] = -1;
a[11] = 11;
Any analysis may make sense with a given memory model on a given day of the week, but that behavior is just one of many possibilities.
Allocating heap storage always includes a test for insufficient memory; for stack space this is less critical due to the way stack space is reused over and over again. If they share the same block of storage, then they could collide.
GCC won't do this because heap space and stack space are separate; I don't know about Valgrind.
In at least one old language (Turbo C), an alloc() will fail if less than 256 bytes of storage remain between top-of-heap and bottom-of-stack. It is assumed 256 bytes is enough to accommodate stack growth. If it's not, you get some very weird run-time errors.
Turbo C has a compile-time option, -N, to check for stack overflow more thoroughly. Other languages may have a similar option.