I've been inspecting the heap memory when executing the following code:
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <sys/types.h>
struct data {
char name[64];
};
struct fp {
int (*fp)();
};
void winner()
{
printf("level passed\n");
}
void nowinner()
{
printf("level has not been passed\n");
}
int main(int argc, char **argv)
{
struct data *d;
struct fp *f;
d = malloc(sizeof(struct data));
f = malloc(sizeof(struct fp));
f->fp = nowinner;
printf("data is at %p, fp is at %p\n", d, f);
strcpy(d->name, argv[1]);
f->fp();
}
The code is compiled like this:
gcc winner.c -w -g -fno-stack-protector -z norelro -z execstack -o winner
(More code will be added later on so tags like fno-stack-protector are there)
I executed the project with argument "HELLO" and I set a break point on f->fp() and inspect the heap memory:
Everything after the first malloc() makes sense but I'm kinda puzzled about what happened after the second malloc(). The second chunk should only request 4 bytes of memory to store the function pointer but instead it took 12 bytes, which reflects on what is stored on the address 0x804a04c (4 bytes of metadata + 12 bytes of requested memory + status bit 1 = 17 => 0x00000011).
And as you can see, the function pointer did only take up four bytes on 0x804a050 with the address of nowinner (0x080484a1).
I read up on this SO post and this article but it seems it still can't explain why.
Your initial question can be answered very easily by printing sizeof of your pointer. You will not see a 12 here.
The answer to your question "Why is function pointer 12 bytes long?" is simply: "It's not!"
But your question describes a different underlying question:
"Why does allocating 4 bytes take 12 bytes on the heap?"
You are under the wrong impression that memory allocation only takes exactly what is needed to store the user data.
This is wrong.
Memory management also needs to store some management data for each allocation.
When you call free the runtime library needs to know the size of the allocated block.
Therefore you can take it as granted that every allocation consumes more memory than the requested amount.
Depending on the implementation of the heap this can be within the heap itself or in a separate area.
You can also not rely on taking the same amount of overhead for each allocation. There are weird implementation out there.
Some implementations take the requested amount and add fixed length of management data.
Some implementations use a buddy system and follow a sequence of fibonacci numbers to determine smallest suitable block size.
Related
This question already has answers here:
Getting a stack overflow exception when declaring a large array
(8 answers)
Closed 3 years ago.
I have the following program:
#include <stdio.h>
#include <sys/resource.h>
int main()
{
// Anything over ~8MB fails
short int big[4000000];
printf("%lu\n", sizeof(big));
}
ulimit shows that I have unlimited memory available to the program. However, if I try and allocate more memory I get an error:
short int big[6000000];
$ gcc main.c -o main.out && ./main.out
Segmentation fault: 11
Is there anything I need to change within the C program so that I can allocate, for example a 1GB array?
You're statically allocating an array on the stack, this means that the compiler will write code to reserve that space, and when your main() is called, it will try to move the stack pointer way out of the available mapped stack area for your program. Touching the stack would then cause a segmentation fault, which is what you see.
You could increase the stack size, but it's not that simple nor portable, and in general allocating such a large array on the stack is bad practice and should be avoided. To handle such a big array, you should dynamically allocate it, using for example malloc().
Here's a working example:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
short int *big;
big = malloc(6000000 * sizeof(short int));
if (big == NULL) {
fputs("Failed to allocate memory!\n", stderr);
return 1;
}
// Do whatever...
free(big);
return 0;
}
Also, remember that you cannot use sizeof() in this case since big is a dynamically allocated array (sizeof(big) would yield the size of the pointer, not the real size of the array). This is because sizeof() is a compile-time operator and can only help you if the size is known at compile time. In this case, it is not, since the space is allocated at runtime.
If you want to know the size of that array, you can simply calculate it with a multiplication:
short int *big;
const size_t big_size = 6000000ULL * sizeof(short int);
printf("Size: %zu\n", big_size);
big = malloc(big_size);
// ...
You can't statistically declare any array with that big size. It causes the program stack to overflow. What you need is to declare memory dynamically. And here linked list can serve your purpose.
#include <stdio.h>
#include <sys/resource.h>
int main()
{
// allocate the memory you need
short int* big = (short int*)malloc(6000000 * sizeof(short));
if(big)
{
printf("alloc all good\n");
// to free memory
free(big);
}
else
{
printf("alloc failed\n");
}
}
You should use dynamic memory allocation instead of statically defining an array for allocating 1GB of data.This link will help you learn the difference
This question already has answers here:
Getting a stack overflow exception when declaring a large array
(8 answers)
Closed 5 years ago.
Hi all I am really new to C (just started this week), and want to make sure that I am not looking down the wrong rabbit home and hoping to perhaps get pointed to the right rabbit hole.
I create a struct:
#define MAX 64
#define ARRAY_SIZE 2048
struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char arr[ARRAY_SIZE][MAX]; // Creates an array of 2048 cells with string of length 63 max
};
int main(){
...
...
int variable = 30;
struct object l[variable]; //This is where the crash happens. But only when either variable is too large (for instance works on 15 just fine, but anything over 20 it crashes), or when Array_SIZE is too larger, for instance works fine with 1024 but 2048 crashes.
...
...
}
The error I get on crash is the following: Process returned -1073741571 (0xC00000FD) in the cmd window. And the following in the debugger in the IDE:
Program received signal SIGSEGV, Segmentation fault.
[Inferior 1 (process 12120) exited with code 030000000375]
Am I doing something obviously wrong with how I declare an array of structs? Why would large numbers not work but lower numebrs work?
Does the above error indicate I am accessing something out of bounds somewhere? Ive been up and down the code and cant seem to find any reason why larger numbers dont work and lower ones do. My memory footprint doesnt seem to be the issue, just a few megs of memory.
I need help with what to look for (I cant find any instances of accessing anything out of bounds, so I get the feeling Im not chasing the right rabbit and need to look for something else)? Or maybe Im doing something illegal for C without knowing it?
I think your program crashes because you statically allocate too much memory on the stack.
Try using the malloc or calloc function. It dynamically allocates memory on the heap instead e.g. :
struct object *l = malloc(variable*sizeof(struct object));
Don't forget to free it afterwards using the free function.
free(l);
You have a memory size problem, try to increase the memory size for your program.
And if you want to use big array size and so allocate a lot of memory, you shouldn't allocate statically but dynamically.
So you should use malloc
typedef struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char arr[ARRAY_SIZE][MAX];
} myObject ;
int variable = 30;
myObject *l = malloc(sizeof(myObject) * variable);
I do not advice you to declare an array of 2048 statically, so you should initiate your struct with a function.
typedef struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char *arr[MAX];
} myObject ;
myObject *createNewObject() {
myObject *toReturn = malloc(sizeof(myObject) * variable);
if (toReturn == NULL)
return NULL;
toReturn->arr = malloc(sizeof(char) * ARRAY_SIZE);
return (toReturn);
}
void freeMyObject(myObject *objectToFree)
{
if (objectToFree && objectToFree->arr =! NULL)
free(objectToFree->arr)
if (objectToFree)
free(objectToFree)
}
void main()
{
myObject *myNewObj = createNewObject()
// You can do some stuff with myNewObj but always verify the pointers
freeMyObject(myNewObj);
}
You should also debug with valgrind when you works with malloc, so you don't have memory loss or problems.
Hope I helped
Well the problem you had is - you have used automatic memory allocation. Due to constraint of size of automatic storage your program crashed - you asked for more than you should.
So what is the solution?
Static memory allocation:
Solution being
static struct object l[variable];
Dynamic memory allocation
struct object *ptr = malloc( sizeof *ptr * variable);
The storage of these allocation is different from automatic variables - so free from the size constraint . In this case you have to free the dynamically allocated memory. That's why you will get around the problem you have.
Statically allocate is not a confusion free term. All these types of variable will have different scope - lifetime. Standard never mentions about stack or heap . It is the implementation that follow these to store automatically allocated memory and dynamically allocated memory.
This question already has answers here:
Using sizeof with a dynamically allocated array
(5 answers)
Closed 5 years ago.
I´m working on my program, that loads data from external .txt file - it is structure of 3 integers. Now Im struggling with function that allocates memory. I want to allocate memory for structure of 9 members (108 bytes).
My main program was too large to mess with so I created a smaller program to help me more easily figure out what is going on. Previous questions I went through seemed too complicated and unclear to me, that´s why I made my own.
Id like to know why the third printf, prints 4 and how to correctly allocate required memory (108 bytes)
Glad for any advice
#include <stdio.h>
#include <math.h>
int main(){
int object_counter = 9;
typedef struct myStruct {
int name;
int age;
int height;
} myStruct;
myStruct* dynamicStruct;
dynamicStruct = malloc(object_counter*sizeof(myStruct));
if (dynamicStruct == NULL) {
printf("ERROR\n");
return (-1);
} else {
printf("SUCCESS\n");
}
printf("Size is: %lu\n", sizeof(myStruct)); // prints 12, thats 3 times int, OK
printf("Size is: %lu\n", object_counter*sizeof(struct myStruct)); // prints 108, obviously 9 times 4 bytes
printf("Size is: %lu\n", sizeof(dynamicStruct)); // prints 4 ?? (expected 108)
return 0;
}
The sizeof any pointer is constant (8 bytes on my Linux/x86-64 system, which is also the sizeof(uintptr_t), etc....). It is unrelated to the runtime size of the pointed memory zone. In other words, the sizeof operator always gives a compile-time constant (except for VLAs).
You practically need to keep elsewhere the allocated size of a malloc-ed pointer. For example, you could keep the pointer value in one variable, and its allocated size in another variable.
You could consider an abstract data type (using flexible array members) approach like here.
I am trying to make a program that allocates memory and then fill the allocated memory with random data, in this case A and does this a number of times and then print the time it takes to fill 100 mb in milli seconds.
The program is supposed to allocate a multiple of 100 mb each time. The program is supposed to do this up to 9000 mb. The reason I do this is to demonstrate how the OS behaves when it runs out of free memory and use swap space. However I have issues doing this.
When I run the program it behaves like it's supposed to do until I reach 2100 mb then it stop allocating memory and after a while the error handler kicks in and quit the program because malloc() returns NULL.
This is the code:
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <sys/time.h>
#include <assert.h>
#include <unistd.h>
#define KILO 1024
#define MEGA (KILO*KILO)
#define INCREMENT 100
int diffTime(struct timeval * startTime, struct timeval * endTime) {
return ((endTime->tv_sec - startTime->tv_sec)*1000 + (endTime->tv_usec- startTime->tv_usec)/1000);
}
int createBigDatablock(long int storlek) {
char *arr = (char*) malloc(storlek*MEGA);
if (arr==NULL){
printf("Error: not allocating memory");
exit (-1);
}
for(int i = 0;i<storlek*MEGA;i++){
arr[i] = 'A';
}
fflush(stdin);
getchar();
free(arr);
return 0;
}
int main(void) {
long int i;
struct timeval startT, endT;
// struct timezone tzp;
for(i=INCREMENT;i<=9000;i=i+INCREMENT){
gettimeofday(&startT, NULL); //&tzp); /* hämta starttid */
createBigDatablock(i);
gettimeofday(&endT, NULL);//&tzp); /* hämta sluttid */
printf("Datablock %ld MB took %d msec\n",i, diffTime(&startT,&endT));
}
return 0;
}
I have tried to run this on a virtual machine with debian 8 with 4 gb memory and 570 mb swap. The memory is never completely filled and the swap space is never touched. If someone can help me figure this out I will be greatful
You probably have a 32bit system where you cannot allocate more than 2 gigabytes of memory per process.
If malloc returns NULL, that means that it couldn't allocate memory.
So the behaviour of your program is normal.
Another possibility is that it simply cannot find a large enough free contiguous memory block.
The reason why the swap space is never touched may be because there is always enough free memory available because your virtual machine has 4gb of memory.
If you want to use the swap space, you could try to allocate lots of smaller chunks of memory, for example 40 times 100Mb; then the overall quantity of memorty you are able to allocate may also be higher than 2100Mb.
I stumbled on this code and compiled it on a Fedora 34 X86_64 system. It would segfault at Datablock 2100 on the loop that writes an A to a memory location. I changed the i to long int i and my system ground to almost a halt but a watched free in another terminal showed it was filling swap. I have not fully tested it but it seems to be working. There are two i variables in this and perhaps they're considered separate.
for(int i = 0;i<storlek*MEGA;i++){
arr[i] = 'A';
for(long int i = 0;i<storlek*MEGA;i++){
arr[i] = 'A';
I am learning the basics of C, and now working with malloc(). Say I have a function that asks the user for input, populates a structure with said data, and that structure in saved in an array (all passed by reference from a main function).
I ran the program through Valgrind, I get the "still reachable" bytes (which should be ok right? not considered a leak), but any data saved I get lost blocks.
How would I go freeing that memory after the program has finished running? Also I have some (2) questions within the code just to clarify some things and would appreciate if someone could explain them to me.
Here is some code similar to what I am trying to do:
I have the following struct declared:
struct Person {
char name[MAX_INPUT];
int age;
};
I am writing a function that goes like this:
int function2(struct Person *list, int *index) {
struct Person *prsn = malloc(sizeof(struct Person));
// !Why do we sometimes cast the malloc or not?
// I sometimes get errors when I do, sometimes when I don't,
// while the surrounding code is pretty much the same.
assert(prsn != NULL);
// User input code goes here ...
// Now to save the Person created
strcpy(prsn->name, nameInput);
prsn->age = ageInput;
list[(*index)++] = *prsn;
// !Why use the dereferencing *prsn here?
// why not directly prsn? Or is that saving the memory address and not very useful.
return 0;
}
And this is my main function:
int main(int argc, char *argv[]) {
struct Person personList[MAX_SIZE];
int index;
function2(personList, &index);
// Before closing, I want to free any mallocs I have done here. free()
return 0;
}
Valgrind report:
LEAK SUMMARY:
==1766== definitely lost: 44 bytes in 1 blocks
==1766== indirectly lost: 0 bytes in 0 blocks
==1766== possibly lost: 0 bytes in 0 blocks
==1766== still reachable: 10,355 bytes in 34 blocks
==1766== suppressed: 0 bytes in 0 blocks
Thank you in advance.
Edit: Fixed function2 parameters, return and other things. I apologize, was writing it quickly to illustrate my main question about freeing memory. Thanks for the corrections tips, but the real code is actually compiling correctly and everything.
Edit2: After adding a simple loop at the end of main like suggested to use free(), I get the following errors.
==2216== LEAK SUMMARY:
==2216== definitely lost: 44 bytes in 1 blocks
==2216== indirectly lost: 0 bytes in 0 blocks
==2216== possibly lost: 0 bytes in 0 blocks
==2216== still reachable: 10,355 bytes in 34 blocks
==2216== suppressed: 0 bytes in 0 blocks
==2216==
==2216== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
==2216==
==2216== 1 errors in context 1 of 2:
==2216== Invalid free() / delete / delete[] / realloc()
==2216== at 0x563A: free (in /usr/local/Cellar/valgrind/3.8.1/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==2216== by 0x10000194E: main (in ./test.out)
==2216== Address 0x7fff5fbf9dd0 is on thread 1's stack
==2216==
==2216== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Code Analysis
Let's dissect this a bit at a time:
int function2(struct Person *list) {
As Iserni noted in his answer, this definition is inconsistent with your call to the function. I agree in general with his correction for the existing code (but I'm going to recommend modifying it shortly):
int function2(struct Person *list, int *index)
struct Person *prsn = malloc(sizeof(struct Person));
// !Why do we sometimes cast the malloc or not?
// I sometimes get errors when I do, sometimes when I don't,
// while the surrounding code is pretty much the same.
It depends on whether you're in C or C++; in C++, you must cast the return value from malloc() (if you use malloc() at all; you usually shouldn't in C++). In C, the cast is optional. There is a school of thought that omitting the cast reveals errors that inserting the cast can hide. I don't subscribe to that; I believe that malloc() should have been declared via <stdlib.h> and the compiler should be warning if there is no declaration in scope, and if there's a declaration in scope, the cast can't cover up sins. The other possible problem is that you assign a pointer to a non-pointer; that would be a mistake that the compiler should be complaining about too.
assert(prsn != NULL);
This is not normally considered a sensible long-term method of handling memory allocation errors.
// User input code goes here ...
// Now to save the Person created
strcpy(prsn->name, nameInput);
prsn->age = ageInput;
list[(*index)++] = *prsn;
// !Why use the dereferencing *prsn here?
Because:
list is a struct Person *.
Therefore list[i] is a struct Person (albeit you spelled i as (*index)++).
Therefore you must assign a struct Person to it.
prsn is a struct Person *.
Therefore *prsn is also a struct Person.
Therefore the assignment is 'correct'.
It also gives you the leak.
You've overwritten the content of list[i] with the content of *prsn.
You've not save the pointer to prsn anywhere.
So you leak memory as you return from the function.
The surgery required to fix this is non-negligible:
int function2(struct Person **item)
...
*item = prsn;
and you have to revise the call; I'll come back to that when dissecting main().
// why not directly prsn? Or is that saving the memory address and not very useful.
}
Your function is declared to return an int but you show no return. If you aren't going to return a value, declare the function as void, especially if you're going to ignore the return value as your code in main() does.
Your last comment is mostly covered by the discussion above; saving the memory address is crucial to stopping the leak so it is very useful.
And this is my main function:
int main(int argc, char *argv[]) {
struct Person personList[MAX_SIZE];
int index;
Using an uninitialized variable is bad news. It is at best only accidentally zeroed. You can't afford to have random values in use as array indexes; initialize it explicitly. Also, we're going to need an array of pointers, not an array of structures:
struct Person *personList[MAX_SIZE];
int index = 0;
...other code...
function2(personList, &index);
With the revised function:
function2(&personList[index++]);
This is preferable; it means that function2() doesn't need to know about the array; you just pass it the address of the pointer to which the allocated memory pointer should be assigned. This reduces the coupling between your main() function and function2(), which makes the code simpler all around.
// Before closing, I want to free any mallocs I have done here. free()
So you write:
for (int i = 0; i < index; i++)
free(personList[i]);
This releases all the memory allocated.
return 0;
}
I like to see an explicit return at the end of main(), even though C99 says it isn't 100% necessary.
Make sure you are compiling with enough warnings enabled. If you're using GCC, then gcc -Wall should be the minimum level of compilation warning you run with (and you should have no warnings in your code when you do so). I run with more stringent warnings: gcc -std=c99 -Wall -Wextra -Wmissing-prototypes -Wold-style-definition -Wstrict-prototypes -Wshadow usually. You need to include -O3 (or some level of optimization) to get some warnings. The stuff about prototypes reflects paranoia about an old code-base I work with which still has K&R function definitions around.
Answering comments
First question while I go over your details and try things out: is there a memory impact between an array of structs and an array of pointers?
Yes, but it may not be what you're thinking of. If you use an array of structures, there's no need to allocate memory dynamic for the structures since the compiler's already allocated them for you. Since the purpose of the exercise is to use pointers and malloc(), it is better, therefore, to use pointers. In terms of space, there will be slightly more total memory used with the array of pointers (but there will be less memory leaked).
I am trying to change my code to use an array of pointers. But using function2(personList, &index); to call function2 now gives me the following warning: incompatible pointer types passing 'struct Person *[512]' to parameter of type 'struct Person *'. Is it OK if I write extra code in my main question to go into details? As a note, I'm trying to reference variables as much as possible, so as to not temporarily have the program copy data from function to function.
The compiler is correct if you've not made all the changes. Your code using two arguments copies more data between functions than my code using one argument.
Version 1
The following program uses the proposed single-argument function2() to reduce the coupling between the main() function and function2(), simplifying both.
This code compiles with no warnings under GCC 4.7.1 on Mac OS X 10.7.5 using the command line:
gcc -O3 -g -std=c99 -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes \
-Wold-style-definition mem.c -o mem
When run under valgrind, it comes up with no memory leaked.
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
enum { MAX_INPUT = 28 };
enum { MAX_SIZE = 3 };
struct Person
{
char name[MAX_INPUT];
int age;
};
static void function2(struct Person **list)
{
struct Person *prsn = malloc(sizeof(struct Person));
assert(prsn != NULL);
char *nameInput = "This is my name";
int ageInput = 29; // Again!
strcpy(prsn->name, nameInput);
prsn->age = ageInput;
*list = prsn;
}
int main(void)
{
struct Person *personList[MAX_SIZE];
int index = 0;
function2(&personList[index++]);
function2(&personList[index++]);
function2(&personList[index++]);
for (int i = 0; i < index; i++)
free(personList[i]);
return 0;
}
Version 2
This keeps the two-argument version of function2() and has it do the counting that main() should be doing on its own. This program also compiles cleanly and runs cleanly under valgrind.
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
enum { MAX_INPUT = 28 };
enum { MAX_SIZE = 3 };
struct Person
{
char name[MAX_INPUT];
int age;
};
static void function2(struct Person **list, int *index)
{
struct Person *prsn = malloc(sizeof(struct Person));
assert(prsn != NULL);
char *nameInput = "This is my name";
int ageInput = 29; // Again!
strcpy(prsn->name, nameInput);
prsn->age = ageInput;
list[(*index)++] = prsn;
}
int main(void)
{
struct Person *personList[MAX_SIZE];
int index = 0;
function2(personList, &index);
function2(personList, &index);
function2(personList, &index);
for (int i = 0; i < index; i++)
free(personList[i]);
return 0;
}
I assume that your function2 is really
int function2(struct Person *list, int *index)
You would save your malloc'ed pointer there with
list[(*index)++] = prsn;
Then in main you would free the list with
while(index)
free(list[--index]);
In object oriented programming languages, an array of objects is really just an array of pointers to the objects. So if a pointer takes 4 bytes, and an object takes 5, an array of 10 objects will actually be 4*10 bytes long (plus overhead).
MyClass[] my_variable = new MyClass[10]; // this will allocate 10*pointersize bytes, plus overhead, you still need to allocate more space for the objects themselves
In C, an array of structures is an array of structures. If a structure takes 8 bytes, an array of 10 of them takes 80 bytes. You don't need to allocate even more space. It's already there.
struct data_t my_variable[10]; // this will allocate 10*sizeof(data_t) bytes
So the easy answer is:
int function2(struct Person *list, int *index) {
char* nameInput;
// User input code goes here ...
// IMPORTANT: the user input code must allocate the space for nameInput
// if it doesn't, then a copy of the buffer must be made
// instead of the direct assignment below
// the memory for this instance of the person was already created when the
// array was created, so just save the values
list[*index].name = nameInput;
list[*index].age = ageInput;
*index += 1;
return 0;
}
When your code executes this line:
list[(*index)++] = *prsn;
It is quite literally copying 8 bytes (on a 32 bit machine, sizeof(int) + sizeof(char*)), as I think you noticed something was up with your commment "// !Why use the dereferencing *prsn here?"
The address to the memory allocated with malloc isn't being copied in that line, but the contents of the memory are. That's why it is a memory leak, the address is lost when the function exits because it was never put into the array.