Allocating a large array on the stack in C [duplicate] - c

This question already has answers here:
Getting a stack overflow exception when declaring a large array
(8 answers)
Closed 3 years ago.
I have the following program:
#include <stdio.h>
#include <sys/resource.h>
int main()
{
// Anything over ~8MB fails
short int big[4000000];
printf("%lu\n", sizeof(big));
}
ulimit shows that I have unlimited memory available to the program. However, if I try and allocate more memory I get an error:
short int big[6000000];
$ gcc main.c -o main.out && ./main.out
Segmentation fault: 11
Is there anything I need to change within the C program so that I can allocate, for example a 1GB array?

You're statically allocating an array on the stack, this means that the compiler will write code to reserve that space, and when your main() is called, it will try to move the stack pointer way out of the available mapped stack area for your program. Touching the stack would then cause a segmentation fault, which is what you see.
You could increase the stack size, but it's not that simple nor portable, and in general allocating such a large array on the stack is bad practice and should be avoided. To handle such a big array, you should dynamically allocate it, using for example malloc().
Here's a working example:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
short int *big;
big = malloc(6000000 * sizeof(short int));
if (big == NULL) {
fputs("Failed to allocate memory!\n", stderr);
return 1;
}
// Do whatever...
free(big);
return 0;
}
Also, remember that you cannot use sizeof() in this case since big is a dynamically allocated array (sizeof(big) would yield the size of the pointer, not the real size of the array). This is because sizeof() is a compile-time operator and can only help you if the size is known at compile time. In this case, it is not, since the space is allocated at runtime.
If you want to know the size of that array, you can simply calculate it with a multiplication:
short int *big;
const size_t big_size = 6000000ULL * sizeof(short int);
printf("Size: %zu\n", big_size);
big = malloc(big_size);
// ...

You can't statistically declare any array with that big size. It causes the program stack to overflow. What you need is to declare memory dynamically. And here linked list can serve your purpose.

#include <stdio.h>
#include <sys/resource.h>
int main()
{
// allocate the memory you need
short int* big = (short int*)malloc(6000000 * sizeof(short));
if(big)
{
printf("alloc all good\n");
// to free memory
free(big);
}
else
{
printf("alloc failed\n");
}
}

You should use dynamic memory allocation instead of statically defining an array for allocating 1GB of data.This link will help you learn the difference

Related

Why is function pointer 12 bytes long?

I've been inspecting the heap memory when executing the following code:
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <sys/types.h>
struct data {
char name[64];
};
struct fp {
int (*fp)();
};
void winner()
{
printf("level passed\n");
}
void nowinner()
{
printf("level has not been passed\n");
}
int main(int argc, char **argv)
{
struct data *d;
struct fp *f;
d = malloc(sizeof(struct data));
f = malloc(sizeof(struct fp));
f->fp = nowinner;
printf("data is at %p, fp is at %p\n", d, f);
strcpy(d->name, argv[1]);
f->fp();
}
The code is compiled like this:
gcc winner.c -w -g -fno-stack-protector -z norelro -z execstack -o winner
(More code will be added later on so tags like fno-stack-protector are there)
I executed the project with argument "HELLO" and I set a break point on f->fp() and inspect the heap memory:
Everything after the first malloc() makes sense but I'm kinda puzzled about what happened after the second malloc(). The second chunk should only request 4 bytes of memory to store the function pointer but instead it took 12 bytes, which reflects on what is stored on the address 0x804a04c (4 bytes of metadata + 12 bytes of requested memory + status bit 1 = 17 => 0x00000011).
And as you can see, the function pointer did only take up four bytes on 0x804a050 with the address of nowinner (0x080484a1).
I read up on this SO post and this article but it seems it still can't explain why.
Your initial question can be answered very easily by printing sizeof of your pointer. You will not see a 12 here.
The answer to your question "Why is function pointer 12 bytes long?" is simply: "It's not!"
But your question describes a different underlying question:
"Why does allocating 4 bytes take 12 bytes on the heap?"
You are under the wrong impression that memory allocation only takes exactly what is needed to store the user data.
This is wrong.
Memory management also needs to store some management data for each allocation.
When you call free the runtime library needs to know the size of the allocated block.
Therefore you can take it as granted that every allocation consumes more memory than the requested amount.
Depending on the implementation of the heap this can be within the heap itself or in a separate area.
You can also not rely on taking the same amount of overhead for each allocation. There are weird implementation out there.
Some implementations take the requested amount and add fixed length of management data.
Some implementations use a buddy system and follow a sequence of fibonacci numbers to determine smallest suitable block size.

C creating a struct array beyond certain size causes a crash [duplicate]

This question already has answers here:
Getting a stack overflow exception when declaring a large array
(8 answers)
Closed 5 years ago.
Hi all I am really new to C (just started this week), and want to make sure that I am not looking down the wrong rabbit home and hoping to perhaps get pointed to the right rabbit hole.
I create a struct:
#define MAX 64
#define ARRAY_SIZE 2048
struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char arr[ARRAY_SIZE][MAX]; // Creates an array of 2048 cells with string of length 63 max
};
int main(){
...
...
int variable = 30;
struct object l[variable]; //This is where the crash happens. But only when either variable is too large (for instance works on 15 just fine, but anything over 20 it crashes), or when Array_SIZE is too larger, for instance works fine with 1024 but 2048 crashes.
...
...
}
The error I get on crash is the following: Process returned -1073741571 (0xC00000FD) in the cmd window. And the following in the debugger in the IDE:
Program received signal SIGSEGV, Segmentation fault.
[Inferior 1 (process 12120) exited with code 030000000375]
Am I doing something obviously wrong with how I declare an array of structs? Why would large numbers not work but lower numebrs work?
Does the above error indicate I am accessing something out of bounds somewhere? Ive been up and down the code and cant seem to find any reason why larger numbers dont work and lower ones do. My memory footprint doesnt seem to be the issue, just a few megs of memory.
I need help with what to look for (I cant find any instances of accessing anything out of bounds, so I get the feeling Im not chasing the right rabbit and need to look for something else)? Or maybe Im doing something illegal for C without knowing it?
I think your program crashes because you statically allocate too much memory on the stack.
Try using the malloc or calloc function. It dynamically allocates memory on the heap instead e.g. :
struct object *l = malloc(variable*sizeof(struct object));
Don't forget to free it afterwards using the free function.
free(l);
You have a memory size problem, try to increase the memory size for your program.
And if you want to use big array size and so allocate a lot of memory, you shouldn't allocate statically but dynamically.
So you should use malloc
typedef struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char arr[ARRAY_SIZE][MAX];
} myObject ;
int variable = 30;
myObject *l = malloc(sizeof(myObject) * variable);
I do not advice you to declare an array of 2048 statically, so you should initiate your struct with a function.
typedef struct object {
int q, c, p;
char name[MAX]; //Stores string up to 63 characters
char *arr[MAX];
} myObject ;
myObject *createNewObject() {
myObject *toReturn = malloc(sizeof(myObject) * variable);
if (toReturn == NULL)
return NULL;
toReturn->arr = malloc(sizeof(char) * ARRAY_SIZE);
return (toReturn);
}
void freeMyObject(myObject *objectToFree)
{
if (objectToFree && objectToFree->arr =! NULL)
free(objectToFree->arr)
if (objectToFree)
free(objectToFree)
}
void main()
{
myObject *myNewObj = createNewObject()
// You can do some stuff with myNewObj but always verify the pointers
freeMyObject(myNewObj);
}
You should also debug with valgrind when you works with malloc, so you don't have memory loss or problems.
Hope I helped
Well the problem you had is - you have used automatic memory allocation. Due to constraint of size of automatic storage your program crashed - you asked for more than you should.
So what is the solution?
Static memory allocation:
Solution being
static struct object l[variable];
Dynamic memory allocation
struct object *ptr = malloc( sizeof *ptr * variable);
The storage of these allocation is different from automatic variables - so free from the size constraint . In this case you have to free the dynamically allocated memory. That's why you will get around the problem you have.
Statically allocate is not a confusion free term. All these types of variable will have different scope - lifetime. Standard never mentions about stack or heap . It is the implementation that follow these to store automatically allocated memory and dynamically allocated memory.

Malloc for struct [duplicate]

This question already has answers here:
Using sizeof with a dynamically allocated array
(5 answers)
Closed 5 years ago.
I´m working on my program, that loads data from external .txt file - it is structure of 3 integers. Now Im struggling with function that allocates memory. I want to allocate memory for structure of 9 members (108 bytes).
My main program was too large to mess with so I created a smaller program to help me more easily figure out what is going on. Previous questions I went through seemed too complicated and unclear to me, that´s why I made my own.
Id like to know why the third printf, prints 4 and how to correctly allocate required memory (108 bytes)
Glad for any advice
#include <stdio.h>
#include <math.h>
int main(){
int object_counter = 9;
typedef struct myStruct {
int name;
int age;
int height;
} myStruct;
myStruct* dynamicStruct;
dynamicStruct = malloc(object_counter*sizeof(myStruct));
if (dynamicStruct == NULL) {
printf("ERROR\n");
return (-1);
} else {
printf("SUCCESS\n");
}
printf("Size is: %lu\n", sizeof(myStruct)); // prints 12, thats 3 times int, OK
printf("Size is: %lu\n", object_counter*sizeof(struct myStruct)); // prints 108, obviously 9 times 4 bytes
printf("Size is: %lu\n", sizeof(dynamicStruct)); // prints 4 ?? (expected 108)
return 0;
}
The sizeof any pointer is constant (8 bytes on my Linux/x86-64 system, which is also the sizeof(uintptr_t), etc....). It is unrelated to the runtime size of the pointed memory zone. In other words, the sizeof operator always gives a compile-time constant (except for VLAs).
You practically need to keep elsewhere the allocated size of a malloc-ed pointer. For example, you could keep the pointer value in one variable, and its allocated size in another variable.
You could consider an abstract data type (using flexible array members) approach like here.

Formal malloc to use realloc later [duplicate]

This question already has answers here:
Initially mallocate 0 elements to later reallocate and measure size
(2 answers)
Closed 6 years ago.
Below code is just an example which I use to come to the point later:
/* Extract digits from an integer and store in an array in reverse order*/
#include<stdio.h>
#include<stdlib.h>
int main(void)
{
int oct=316;
int* array=malloc(0*sizeof(int*)); //Step 1
int* temp;
size_t size=0;
while(oct/10>0 || oct%10>0){
temp=realloc(array,++size*sizeof(int)); // Step2 requires Step1
if(temp==NULL){
printf("Can't reallocate memory");
exit(-1);
}
else{
array=temp;
}
array[size-1]=oct%10;
oct/=10;
}
for(int i=0;i<size;i++)
printf("%d\n",array[i]);
return 0;
}
The realloc reference [1] states :
Reallocates the given area of memory. It must be previously allocated
by malloc(), calloc(), realloc()...
Initially I compiled the code without step1 and on running it I got a segmentation fault. Then I included step1, now the code compiles fine. I don't want to allocate some space without finding an integer to store so I have used a size of zero with malloc. However malloc reference [2] states :
If size is zero, the return value depends on the particular library
implementation (it may or may not be a null pointer), but the returned
pointer shall not be dereferenced.
Now, I doubt my implementation is portable. How can I work around this issue?
I tried doing int* array=NULL; but I got a segmentation fault.
You can initiate array to NULL. In the man page for realloc(void *ptr, size_t size) you can read:
If ptr is NULL, then the call is equivalent to malloc(size)
Moreover realloc does not change its parameter. It returns pointer to newly allocated/reallocated space. So, for reallocating you shall use a code like:
array = realloc(array,(++size)*sizeof(int));
if (array == NULL) { some error; }

Knowing the size of the array using pointer

How can i know the size of the array using a pointer that is allocated using malloc?
#include <stdio.h>
int main(){
int *ptr = (int *)malloc(sizeof(int * 10));
printf("Size:%d",sizeof(ptr));
free(ptr_one);
return 0;
}
I get only the size of the pointer in this case which is 8.How to modify the code to get the size of array which will be 40.
You cannot.
You will need to do the bookkeeping and keep track of it yourself. With new you allocate dynamic memory and while deallocating the memory you just call delete, which knows how much memory it has deallocate because the language takes care of it internally so that users do not need to bother about the bookkeeping. If you still need it explicitly then you need to track it through separate variable.
if your machine is 32 bit you will get pointer size always 4 bytes of any data type
if your machine is 64 bit you will get pointer size always 8 bytes of any data type
if you declare static array you will get size by using sizeof
int a[10];
printf("Size:%lu",sizeof(a));
But you did not get the size of array which is blocked by pointer. where the memory to the block is allocated dynamically using malloc .
see this below code:
#include <stdio.h>
#include<stdlib.h>
int main()
{
int i;
int *ptr = (int *)malloc(sizeof(int) * 10);
// printf("Size:%lu",sizeof(ptr));
// In the above case sizeof operater returns size of pointer only.
for(i=1;ptr && i<13 ;i++,ptr++)
{
printf("Size:%d %p\n",((int)sizeof(i))*i,ptr);
}
return 0;
}
output:
Size:4 0x8ec010
Size:8 0x8ec014
Size:12 0x8ec018
Size:16 0x8ec01c
Size:20 0x8ec020
Size:24 0x8ec024
Size:28 0x8ec028
Size:32 0x8ec02c
Size:36 0x8ec030
Size:40 0x8ec034 //after this there is no indication that block ends.
Size:44 0x8ec038
Size:48 0x8ec03c

Resources