I haven't programmed C for quite some time and my pointer-fu had degraded. I made a very elementary mistake and it took me well over an hour this morning to find what I'd done. The bug is minimally reproduced here: https://godbolt.org/z/3MdzarP67 (I am aware the program is absurd memory-management wise, just showing what happens).
The first call to realloc() breaks because of course, the pointer it's given points to stack memory, valgrind made this quite obvious.
I have a rule with myself where any time I track down a bug, if there is a warning that could have caught it I enable it on my projects. Often times this is not the case, since many bugs come from logic errors the compiler can't be expected to check.
However here I am a bit surprised. We malloc() and then immediately reassign that pointer which leaves the allocated memory inaccessible. It's obvious the returned pointer does not live outside the scope of that if block, and is never free()'d. Maybe it's too much to expect the compiler to analyze the calls and realize we're attempting to realloc() stack memory but I am surprised that I can't find anything to yell at me about the leaking of the malloc() returned pointer. Even clang's static analyzer scan-build doesn't pick up on it, I've tried various relevant options.
The best I could find was -fsanitize=address which at least prints out some cluing information during the crash instead of:
mremap_chunk(): invalid pointer
on Godbolt, or
realloc(): invalid old size
Aborted (core dumped)
on my machine, both of which are somewhat cryptic (although yes they do show plainly that there is some memory issue occurring). Still, this compiles without issues.
Since Godbolt links don't live forever here is the critical section of the code:
void add_foo_to_bar(struct Bar** b, Foo* f) {
if ((*b)->foos == NULL) {
(*b)->foos = (Foo*)malloc(sizeof(Foo));
// uncomment for correction
//(*b)->foos[(*b)->n_foos] = *f;
// obvious bug here, we leak memory by losing the returned pointer from malloc
// and assign the pointer to a stack address (&f1)
// comment below line for correction
(*b)->foos = f; // completely wrong
(*b)->n_foos++;
} else {
(*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
(*b)->foos[(*b)->n_foos] = *f;
(*b)->n_foos++;
}
}
the error occurs because f is a pointer to stack memory (intentional) but we obviously can't assign something that was supposed to have been malloc()'d to that.
Try -fanalyzer if your compiler is recent enough. When running it I get:
../main.c:30:28: warning: ‘realloc’ of ‘&f1’ which points to memory not on the heap [CWE-590] [-Wanalyzer-free-of-non-heap]
30 | (*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
‘main’: events 1-2
|
| 37 | int main() {
| | ^~~~
| | |
| | (1) entry to ‘main’
|......
| 45 | add_foo_to_bar(&b, &f1);
| | ~~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (2) calling ‘add_foo_to_bar’ from ‘main’
|
+--> ‘add_foo_to_bar’: events 3-5
|
| 19 | void add_foo_to_bar(struct Bar** b, Foo* f) {
| | ^~~~~~~~~~~~~~
| | |
| | (3) entry to ‘add_foo_to_bar’
| 20 | if ((*b)->foos == NULL) {
| | ~
| | |
| | (4) following ‘true’ branch...
| 21 | (*b)->foos = (Foo*)malloc(sizeof(Foo));
| | ~~~~
| | |
| | (5) ...to here
|
<------+
|
‘main’: events 6-7
|
| 45 | add_foo_to_bar(&b, &f1);
| | ^~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (6) returning to ‘main’ from ‘add_foo_to_bar’
| 46 | add_foo_to_bar(&b, &f2);
| | ~~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (7) calling ‘add_foo_to_bar’ from ‘main’
|
+--> ‘add_foo_to_bar’: events 8-11
|
| 19 | void add_foo_to_bar(struct Bar** b, Foo* f) {
| | ^~~~~~~~~~~~~~
| | |
| | (8) entry to ‘add_foo_to_bar’
| 20 | if ((*b)->foos == NULL) {
| | ~
| | |
| | (9) following ‘false’ branch...
|......
| 30 | (*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
| | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| | | |
| | | (10) ...to here
| | (11) call to ‘realloc’ here
|
No, but, runtime testing can save you.
If you can spare the execution overhead, I have seen many applications add an extra layer to memory allocation to track the allocations made and find leaks/errors.
Usually they replace malloc() and free() with a macros that include FILE and LINE
One example can be seen here (check the Heap.c and Heap.h files)
https://github.com/eclipse/paho.mqtt.c/tree/master/src
Googling "memory heap debugger" will probably turn up other examples. Or you could roll your own.
Related
When seeing:
void my_object_release(struct kobject *kobj)
{
struct my_object *mine = container_of(kobj, struct my_object, kobj);
/* Perform any additional cleanup on this object, then... */
kfree (mine);
}
in LWN’s The zen of kobjects, it seems incorrect in the third parameter kobj. I think it should be kobject.
The given code is correct: the third argument is the name of the container structure member to which the pointer points, not its type, so kobj is right. The example is somewhat confusing since the first kobj doesn’t correspond to the same thing as the second kobj: the first is the pointer in the caller’s scope.
Here’s a diagram to hopefully clarify the parameters of container_of:
container_of(kobj, struct my_object, kobj)
| | |
| | |
\------------+----------+--------------------------------\
| | |
| | |
/-----------------/ | |
| | |
V /-------------/ |
+------------------+ | |
| struct my_object | { | |
+------------------+ V V
+------+ +------+
struct kobject | kobj |; <-- You have a pointer to this, called | kobj |
+------+ +------+
...
};
container_of allows you to pass around a kobject pointer and find the containing object (as long as you know what the containing object is) — it allows you to use your knowledge of “what” to answer “where”.
It’s worth pointing out that container_of is a macro, which is how it can do seemingly impossible things (for developers not used to meta-programming).
I've got this code in C language:
char *options[100000];
int k[100000];
char *param[100000];
int n;
int i,j;
...
scanf("%d",&n);
for (i=0;i<n;i++)
{
scanf("%s%d",&options[i],&k[i]);
param[i]="On";
}
...
just as the programm reaches this point:
scanf("%s%d",&options[i],&k[i]);
I get the runtime error (stack overflow).
The input here should be like this:
word1 number1
word2 number2
and so on. I've got no idea why is this happening. What's the problem?
Ok... so I thought someone would provide an answer to your stack overflow problem but so far everybody only mentioned a problem you actually have (more on this later) that is unrelated to the stack overflow (it'll be problematic but only once you fix this first).
-----------------
| Your stack |
| (grows down) |
| |
-----------------
| |
| |
| |
| |
| |
| | -- max stack size is here
| |
| |
| |
| |
| |
-----------------
| Your heap |
| (grows up) |
| |
-----------------
And then you try to allocate a bunch of really big arrays and run out of space
-----------------
| Your stack |
| (grows down) |
| |
| |
| |
| |
| |
| |
| |
| | -- max stack size is here
| |
----------------- -- what you actually need
| |
| |
| |
| |
-----------------
| Your heap |
| (grows up) |
| |
-----------------
So you get a run-time error (stack overflow) because you've tried to use more stack space than what you have available.
The trick here is to use heap allocation (because on most platforms, at least all the ones I've heard of) the heap is massively bigger than the stack.
To allocate memory on the heap you use malloc (also, when you're done with it don't forget to release the memory using free, or else you'll leak the memory).
EDIT:
Bonus: The other problem you have. Other answers seem to indicate you're access/dereferencing/using memory that's not allocated. You're partially actually fine on this point.
You scanf call point to a char array (here's the problem) and an int in the array k (no problem. So right now all the entries in the options array point to nowhere/anywhere. You need to allocate memory for them (again using malloc).
As for strdup it allocates the memory itself and returns the pointer, again no problem here. Just don't forget to free it after you're done using it because again this would be a memory leak.
char *options[100000] allocates 100000 string pointers, not strings.
scanf is being passed gibberish.
First off, I've already found a few references which might answer my question. While I plan on reading them soon (i.e. after work), I'm still asking here in case the answer is trivial and does not require too much supplementary knowledge.
Here is the situation: I am writing a shared library (let's call it libA.so) which needs to maintain a coherent internal (as in, static variables declared in the .c file) state within the same process.
This library will be used by program P (i.e. P is compiled with -lA). If I understand everything so far, the address space for P will look something like this:
______________
| Program P |
| < |
| variables, |
| functions |
| from P |
| > |
| |
| < |
| libA: |
| variables, |
| functions |
| loaded (ie |
| *copied*) |
| from shared |
| object |
| > |
| < |
| stuff from |
| other |
| libraries |
| > |
|______________|
Now P will sometimes call dlopen("libQ.so", ...). libQ.so also uses libA.so (i.e. was compiled with -lA). Since everything happens within the same process, I need libA to somehow hold the same state whether the calls come from P or Q.
What I do not know is how this will translate in memory. Will it look like this:
______________
| Program P |
| < |
| P stuff |
| > |
| |
| < |
| libA stuff, |
| loaded by P |
| > | => A's code and variables are duplicated
| |
| < |
| libQ stuff |
| < |
| libA stuff,|
| loaded by Q|
| > |
| > |
|______________|
... or like this?
______________
| Program P |
| < |
| P stuff |
| *libA |
| *libQ |
| > |
| |
| < |
| libA stuff, |
| loaded by P |
| > | => A's code is loaded once, Q holds some sort of pointer to it
| |
| < |
| libQ stuff |
| *libA |
| > |
|______________|
In the second case, keeping a consistent state for a single process is trivial; in the first case, it will require some more effort (e.g. some shared memory segment, using the process id as the second argument to ftok()).
Of course, since I have a limited knowledge on how linking and loading works, the diagrams above may be completely wrong. For all I know, the shared libraries could be at a fixed space in memory, and every process accesses the same data and code. The behaviour could also depends on how A and/or P and/or Q were compiled. And this behaviour is probably not platform independent.
The code segment of a shared library exists in memory in a single instance per system. Yet, it can be mapped to different virtual addresses for different processes, so different processes see the same function at different addresses (that's why the code that goes to a shared library must be compiled as PIC).
Data segment of a shared library is created in one copy for each process, and initialized by whatever initial values where specified in the library.
This means that the callers of a library do not need to know if it is shared or not: all callers in one process see the same copy of the functions and the same copy of external variables defined in the library.
Different processes execute the same code, but have their individual copies of data, one copy per process.
I am learning c, and have come across an issue storing and managing structs; let me explain.
I am quite familiar with Java, and I am starting to understand how to properly use pointers and references in c. My goal is to make and store hundreds of stuct:
struct creature{
int id;
char name[12];
}
In java, I am able to create an Array like:
creature c [];
and then store and manipulate objects inside that array. I want the same effect in c using a struct "creature".
My first attempt was:
creature *creatures = malloc(sizeof(creature));
I am not entirely sure what that even does. My goal was to create a block of memory that strictly held struct pointers, like the Java array.
from here, I want to create new creature structs, and store their pointers in the creatures variable, that has memory allocated from using malloc.
I hope this was enough information to explain my problem.
Thanks!
Right, so what you want is an array of pointers. Note that in Java, whenever you have something like:
String s = ...;
This translates to a pointer in c:
String *s = ...;
Let's take the easy case first of a fixed-size array of pointers. This is simply:
creature *creatures[10];
for (var i=0; i < 10; i++) {
creatures[i] = malloc(sizeof(creature));
}
Now you have an array of pointers to creatures, each malloced separately. But, if you don't know how big your array of creatures should be, then you have to malloc the array of creatures. Note that if you have, say, a malloced int array, you'd do the following:
int *ints = malloc(sizeof(int) * 10);
ints[0]; ints[1]; //etc...
That is, an array is represented as a pointer to the first element. What you want now is an array of pointers, which ends up being a pointer to a pointer:
var numCritters = 10;
creature **creatures = malloc(sizeof(creature *) * numCritters);
for (var i=0; i < numCritters; numCritters++) {
creatures[i] = malloc(sizeof(creature));
}
The first malloc creates a pointer to an array of creature * pointers. Note we use sizeof(creature *) here, not sizeof(creature), because the array is of pointers, not of creatures.
Each of the next mallocs then creates a pointer to a creature, same as in the case of a fixed-size array.
Be sure to have a free for each malloc - that is, one for each creature * and one for the creature **.
Note that this is different from what Pankrates suggested. Which solution you want to use depends on what you want to do. His creates a big block of creatures:
creature *creatures = +--------------------+
| |
| |
| Creature 0 |
| |
| |
+--------------------+
| |
| |
| Creature 1 |
| |
| |
+--------------------+
| |
| |
| Creature 2 |
| |
| |
+--------------------+
| ... |
Whereas mine creates an array of pointers, each one pointing to a creature:
+--------------------+
| |
creature *creatures = +------+ | |
| ptr0 | ----> | Creature 0 |
+--------------------+ +------+ | |
| | +-- | ptr1 | | |
| | | +------+ +--------------------+
| Creature 1 | <-+ | ptr2 | --+
| | +------+ | +--------------------+
| | | .... | | | |
+--------------------+ | | |
+-> | Creature 2 |
| |
| |
+--------------------+
You are almost there. malloc takes the number of bytes that you want to allocate as an argument, therefore you need to multiply the size of the struct times the number of structs that you want to allocate:
int number_of_creatures = 10;
creature *creatures = malloc(sizeof(creature) * number_of_creatures);
You should also always check the returned pointer from malloc() to check if the allocation was successful
if (creatures == NULL)
printf("Allocating memory failed!\n");
I've got this code in C language:
char *options[100000];
int k[100000];
char *param[100000];
int n;
int i,j;
...
scanf("%d",&n);
for (i=0;i<n;i++)
{
scanf("%s%d",&options[i],&k[i]);
param[i]="On";
}
...
just as the programm reaches this point:
scanf("%s%d",&options[i],&k[i]);
I get the runtime error (stack overflow).
The input here should be like this:
word1 number1
word2 number2
and so on. I've got no idea why is this happening. What's the problem?
Ok... so I thought someone would provide an answer to your stack overflow problem but so far everybody only mentioned a problem you actually have (more on this later) that is unrelated to the stack overflow (it'll be problematic but only once you fix this first).
-----------------
| Your stack |
| (grows down) |
| |
-----------------
| |
| |
| |
| |
| |
| | -- max stack size is here
| |
| |
| |
| |
| |
-----------------
| Your heap |
| (grows up) |
| |
-----------------
And then you try to allocate a bunch of really big arrays and run out of space
-----------------
| Your stack |
| (grows down) |
| |
| |
| |
| |
| |
| |
| |
| | -- max stack size is here
| |
----------------- -- what you actually need
| |
| |
| |
| |
-----------------
| Your heap |
| (grows up) |
| |
-----------------
So you get a run-time error (stack overflow) because you've tried to use more stack space than what you have available.
The trick here is to use heap allocation (because on most platforms, at least all the ones I've heard of) the heap is massively bigger than the stack.
To allocate memory on the heap you use malloc (also, when you're done with it don't forget to release the memory using free, or else you'll leak the memory).
EDIT:
Bonus: The other problem you have. Other answers seem to indicate you're access/dereferencing/using memory that's not allocated. You're partially actually fine on this point.
You scanf call point to a char array (here's the problem) and an int in the array k (no problem. So right now all the entries in the options array point to nowhere/anywhere. You need to allocate memory for them (again using malloc).
As for strdup it allocates the memory itself and returns the pointer, again no problem here. Just don't forget to free it after you're done using it because again this would be a memory leak.
char *options[100000] allocates 100000 string pointers, not strings.
scanf is being passed gibberish.