What am I doing wrong?
While using Eclipse on a Mac (2GB RAM) I have encountered the following problem:
Whenever I try to create an array which exceeds 8384896 bytes, I get segmentation faults. The following program would execute:
#include <stdio.h>
main()
{
double x[1048112];
printf("sizeof(x) = %li", sizeof(x));
}
and the output would be (as expected):
sizeof(x) = 8384896
But increasing the number of elements in x or creating additional variables in main() would result in an unexecutable program and segfaults. It looks like I'm hitting some memory limit and I don't understand why this is happening.
I'd be really grateful if anyone could explain this to me, or maybe provide some sort of solution to my problem.
This is a stack overflow due to excessively large stack variables.
If you really need to allocate something that large, you can do so on the heap using malloc:
double *x = malloc(1048112 * sizeof(double));
Note that with this change, sizeof(x) no longer returns the size of the array, it returns the size of double *. If you need to know how large your array is, you'll need to keep track of that on your own.
And, just for completeness, when you are done with the data, you will need to call free, otherwise you'll have one heck of a memory leak:
free(x);
A process on OS X is limited, by default to 8MB stack (try running ulimit -s from the command line).
One option is to try and increase the stack size by using something like ulimit -s 65536. This should affect all new processes ran from the current shell session.
A better option is to allocate the array on the heap:
double *x = (double*)malloc(9999999)
And when you are finished with the array, don't forget to deallocate it using: free(x)
EDIT: try this reference for information on how to use the linker to increase maximum stack size on OS X. Again, the preferred option is just to allocate large arrays on the heap. Easier and more portable.
Other solutions to the problem include using a static array:
static double x[1234567];
in your function, or using a global variable outside the function. If the global array is declared static it won't be visible outside the file it is compiled from.
Either way, the array will not be renewed each time you call the routine, so you won't get a "fresh start" each time you call it but the same old data as before.
Yup, you are hitting a memory limit ... specifically, you are Stack Overflowing. So what should you do ? Allocate memory on the heap ... as so:
double *x=malloc(1048112*sizeof(double));
The malloc based solution is right, but this one will save you the trouble of having to track memory yourself:
#include <stdio.h>
static double x[1048112];
main()
{
printf("sizeof(x) = %li", sizeof(x));
}
Variables that are declared static outside of the function body are not allocated from the stack, and their visibility is limited to the file they're defined in.
You can either increase the amount of allowed space on the stack through linker settings, or preferably start using dynamic memory.
As a side note: In my experience, in some cases the questions of that nature ("why does my huge automatic array cause a crash") are rooted in the beginners' misunderstanding of the physical nature of an array object. More than once I encountered people that firmly believed that an array physically consists of a "pointer" that points to an additional block of memory for the actual elements. They also believed that an automatic array occupies only a tiny amount of stack memory (for the alleged "pointer"), while the large element block is allocated elsewhere, not in the stack. This might be one of the possible reasons why people (even those who are perfectly aware of the "limitedness" of stack space) see no issues with defining large automatic arrays.
Just to you to know this limit in windows is 1 mb
this code work
void myfunction()
{
static char yes[1100000]//allocated in the heap
}
this code don´t work
void myfunction()
{
char yes[1100000]//allocated in the stack
}
Related
The following code is generating a stack overflow error for me
int main(int argc, char* argv[])
{
int sieve[2000000];
return 0;
}
How do I get around this? I am using Turbo C++ but would like to keep my code in C
EDIT:
Thanks for the advice. The code above was only for example, I actually declare the array in a function and not in sub main. Also, I needed the array to be initialized to zeros, so when I googled malloc, I discovered that calloc was perfect for my purposes.
Malloc/calloc also has the advantage over allocating on the stack of allowing me to declare the size using a variable.
Your array is way too big to fit into the stack, consider using the heap:
int *sieve = malloc(2000000 * sizeof(*sieve));
If you really want to change the stack size, take a look at this document.
Tip: - Don't forget to free your dynamically allocated memory when it's no-longer needed.
There are 3 ways:
Allocate array on heap - use malloc(), as other posters suggested. Do not forget to free() it (although for main() it is not that important - OS will clean up memory for you on program termination).
Declare the array on unit level - it will be allocated in data segment and visible for everybody (adding static to declaration will limit the visibility to unit).
Declare your array as static - in this case it will be allocated in data segment, but visible only in main().
That's about 7MB of stack space. In visual studio you would use /STACK:###,### to reflect the size you want. If you truely want a huge stack (could be a good reason, using LISP or something :), even the heap is limited to small'sh allocations before forcing you to use VirtualAlloc), you may also want to set your PE to build with /LARGEADDRESSAAWARE (Visual Studio's linker again), but this configure's your PE header to allow your compiled binary to address the full 4GB of 32'bit address space (if running in a WOW64). If building truely massive binaries, you would also typically need to configure /bigobj as an additional linker paramerter.
And if you still need more space, you can radically violate convention by using something simular to (again MSVC's link) /merge:, which will allow you to pack one section into another, so you can use every single byte for a single shared code/data section. Naturally you would also need to configure the SECTIONS permissions in a def file or with #pgrama.
Use malloc. All check the return type is not null, if it is null then your system simply doesn't have enought memory to fit that many values.
You would be better off allocating it on the heap, not the stack. something like
int main(int argc, char* argv[])
{
int * sieve;
sieve = malloc(20000);
return 0;
}
Your array is huge.
It's possible that your machine or OS don't have or want to allocate so much memory.
If you absolutely need an enormous array, you can try to allocate it dynamically (using malloc(...)), but then you're at risk of leaking memory. Don't forget to free the memory.
The advantage of malloc is that it tries to allocate memory on the heap instead of the stack (therefore you won't get a stack overflow).
You can check the value that malloc returns to see if the allocation succeeded or failed.
If it fails, just try to malloc a smaller array.
Another option would be to use a different data structure that can be resized on the fly (like a linked list). Wether this option is good depends on what you are going to do with the data.
Yet another option would be to store things in a file, streaming data on the fly. This approach is the slowest.
If you go for storage on the hard drive, you might as well use an existing library (for databases)
As Turbo C/C++ is 16 bit compiler int datatype consumes about 2 bytes.
2bytes*2000000=40,00,000 bytes=3.8147MB space.
The auto variables of a function is stored in stack and it caused the overflow of the stack memory. Instead use the data memory [using static or global variable] or the dynamic heap memory [using the malloc/calloc] for creating the required memory as per the availability of the processor memory mapping.
Is there some reason why you can't use alloca() to allocate the space you need on the stack frame based on how big the object really needs to be?
If you do that, and still bust the stack, put it in allocated heap. I highly recommend NOT declaring it as static in main() and putting it in the data segment.
If it really has to be that big and your program can't allocate it on the heap, your program really has no business running on that type of machine to begin with.
What (exactly) are you trying to accomplish?
The following code is generating a stack overflow error for me
int main(int argc, char* argv[])
{
int sieve[2000000];
return 0;
}
How do I get around this? I am using Turbo C++ but would like to keep my code in C
EDIT:
Thanks for the advice. The code above was only for example, I actually declare the array in a function and not in sub main. Also, I needed the array to be initialized to zeros, so when I googled malloc, I discovered that calloc was perfect for my purposes.
Malloc/calloc also has the advantage over allocating on the stack of allowing me to declare the size using a variable.
Your array is way too big to fit into the stack, consider using the heap:
int *sieve = malloc(2000000 * sizeof(*sieve));
If you really want to change the stack size, take a look at this document.
Tip: - Don't forget to free your dynamically allocated memory when it's no-longer needed.
There are 3 ways:
Allocate array on heap - use malloc(), as other posters suggested. Do not forget to free() it (although for main() it is not that important - OS will clean up memory for you on program termination).
Declare the array on unit level - it will be allocated in data segment and visible for everybody (adding static to declaration will limit the visibility to unit).
Declare your array as static - in this case it will be allocated in data segment, but visible only in main().
That's about 7MB of stack space. In visual studio you would use /STACK:###,### to reflect the size you want. If you truely want a huge stack (could be a good reason, using LISP or something :), even the heap is limited to small'sh allocations before forcing you to use VirtualAlloc), you may also want to set your PE to build with /LARGEADDRESSAAWARE (Visual Studio's linker again), but this configure's your PE header to allow your compiled binary to address the full 4GB of 32'bit address space (if running in a WOW64). If building truely massive binaries, you would also typically need to configure /bigobj as an additional linker paramerter.
And if you still need more space, you can radically violate convention by using something simular to (again MSVC's link) /merge:, which will allow you to pack one section into another, so you can use every single byte for a single shared code/data section. Naturally you would also need to configure the SECTIONS permissions in a def file or with #pgrama.
Use malloc. All check the return type is not null, if it is null then your system simply doesn't have enought memory to fit that many values.
You would be better off allocating it on the heap, not the stack. something like
int main(int argc, char* argv[])
{
int * sieve;
sieve = malloc(20000);
return 0;
}
Your array is huge.
It's possible that your machine or OS don't have or want to allocate so much memory.
If you absolutely need an enormous array, you can try to allocate it dynamically (using malloc(...)), but then you're at risk of leaking memory. Don't forget to free the memory.
The advantage of malloc is that it tries to allocate memory on the heap instead of the stack (therefore you won't get a stack overflow).
You can check the value that malloc returns to see if the allocation succeeded or failed.
If it fails, just try to malloc a smaller array.
Another option would be to use a different data structure that can be resized on the fly (like a linked list). Wether this option is good depends on what you are going to do with the data.
Yet another option would be to store things in a file, streaming data on the fly. This approach is the slowest.
If you go for storage on the hard drive, you might as well use an existing library (for databases)
As Turbo C/C++ is 16 bit compiler int datatype consumes about 2 bytes.
2bytes*2000000=40,00,000 bytes=3.8147MB space.
The auto variables of a function is stored in stack and it caused the overflow of the stack memory. Instead use the data memory [using static or global variable] or the dynamic heap memory [using the malloc/calloc] for creating the required memory as per the availability of the processor memory mapping.
Is there some reason why you can't use alloca() to allocate the space you need on the stack frame based on how big the object really needs to be?
If you do that, and still bust the stack, put it in allocated heap. I highly recommend NOT declaring it as static in main() and putting it in the data segment.
If it really has to be that big and your program can't allocate it on the heap, your program really has no business running on that type of machine to begin with.
What (exactly) are you trying to accomplish?
The following code is generating a stack overflow error for me
int main(int argc, char* argv[])
{
int sieve[2000000];
return 0;
}
How do I get around this? I am using Turbo C++ but would like to keep my code in C
EDIT:
Thanks for the advice. The code above was only for example, I actually declare the array in a function and not in sub main. Also, I needed the array to be initialized to zeros, so when I googled malloc, I discovered that calloc was perfect for my purposes.
Malloc/calloc also has the advantage over allocating on the stack of allowing me to declare the size using a variable.
Your array is way too big to fit into the stack, consider using the heap:
int *sieve = malloc(2000000 * sizeof(*sieve));
If you really want to change the stack size, take a look at this document.
Tip: - Don't forget to free your dynamically allocated memory when it's no-longer needed.
There are 3 ways:
Allocate array on heap - use malloc(), as other posters suggested. Do not forget to free() it (although for main() it is not that important - OS will clean up memory for you on program termination).
Declare the array on unit level - it will be allocated in data segment and visible for everybody (adding static to declaration will limit the visibility to unit).
Declare your array as static - in this case it will be allocated in data segment, but visible only in main().
That's about 7MB of stack space. In visual studio you would use /STACK:###,### to reflect the size you want. If you truely want a huge stack (could be a good reason, using LISP or something :), even the heap is limited to small'sh allocations before forcing you to use VirtualAlloc), you may also want to set your PE to build with /LARGEADDRESSAAWARE (Visual Studio's linker again), but this configure's your PE header to allow your compiled binary to address the full 4GB of 32'bit address space (if running in a WOW64). If building truely massive binaries, you would also typically need to configure /bigobj as an additional linker paramerter.
And if you still need more space, you can radically violate convention by using something simular to (again MSVC's link) /merge:, which will allow you to pack one section into another, so you can use every single byte for a single shared code/data section. Naturally you would also need to configure the SECTIONS permissions in a def file or with #pgrama.
Use malloc. All check the return type is not null, if it is null then your system simply doesn't have enought memory to fit that many values.
You would be better off allocating it on the heap, not the stack. something like
int main(int argc, char* argv[])
{
int * sieve;
sieve = malloc(20000);
return 0;
}
Your array is huge.
It's possible that your machine or OS don't have or want to allocate so much memory.
If you absolutely need an enormous array, you can try to allocate it dynamically (using malloc(...)), but then you're at risk of leaking memory. Don't forget to free the memory.
The advantage of malloc is that it tries to allocate memory on the heap instead of the stack (therefore you won't get a stack overflow).
You can check the value that malloc returns to see if the allocation succeeded or failed.
If it fails, just try to malloc a smaller array.
Another option would be to use a different data structure that can be resized on the fly (like a linked list). Wether this option is good depends on what you are going to do with the data.
Yet another option would be to store things in a file, streaming data on the fly. This approach is the slowest.
If you go for storage on the hard drive, you might as well use an existing library (for databases)
As Turbo C/C++ is 16 bit compiler int datatype consumes about 2 bytes.
2bytes*2000000=40,00,000 bytes=3.8147MB space.
The auto variables of a function is stored in stack and it caused the overflow of the stack memory. Instead use the data memory [using static or global variable] or the dynamic heap memory [using the malloc/calloc] for creating the required memory as per the availability of the processor memory mapping.
Is there some reason why you can't use alloca() to allocate the space you need on the stack frame based on how big the object really needs to be?
If you do that, and still bust the stack, put it in allocated heap. I highly recommend NOT declaring it as static in main() and putting it in the data segment.
If it really has to be that big and your program can't allocate it on the heap, your program really has no business running on that type of machine to begin with.
What (exactly) are you trying to accomplish?
so, i was making this program that let people know the number of contiguous subarray which sum is equal to a certain value.
i have written the code , but when i try to run this code in vcexpress 2010, it says these error
Unhandled exception at 0x010018e7 in test 9.exe: 0xC00000FD: Stack overflow.
i have tried to search for the solution in this website and other webisites, but i can't seem to find any solution which could help me fix the error in this code(they are using recursion while i'm not).
i would be really grateful if you would kindly explain what cause this error in my code, and how to fix this error. any help would be appreciated. Thank you.
here is my code :
#include <stdio.h>
int main ()
{
int n,k,a=0,t=0;
unsigned long int i[1000000];
int v1,v2=0,v3;
scanf("%d %d",&n,&k);
for(v3=0;v3<n;v3++)
{
scanf("%d",&i[v3]);
}
do
{
for(v1=v2;v1<n;v1++)
{
t=i[v1]+t;
if(t==k)
{
a++;
break;
}
}
t=0;
v2++;
}while(v2!=n);
printf("%lu",a);
return 0;
}
Either move
unsigned long int i[1000000];
outside of main, thus making it a global variable (not an automatic one), or better yet, use some C dynamic heap allocation:
// inside main
unsigned long int *i = calloc(1000000, sizeof(unsigned long int));
if (!i) { perror("calloc"); exit(EXIT_FAILURE); };
BTW, for such a pointer, I would use (for readability reasons) some other name than i. And near the end of main you'll better free(i); to avoid memory leaks.
Also, you could move these 2 lines after the read of n and use calloc(n, sizeof(unsigned long int)) instead of calloc(1000000, sizeof(unsigned long int)) ; then you can handle arrays bigger than a million elements if your computer and system provides enough resources for that.
Your initial code is declaring an automatic variable which goes into the call frame of main on your call stack (which has a limited size, typically a megabyte or a few of them). On some operating systems there is a way to increase the size of that call stack (in an OS-specific way). BTW each thread has its own call stack.
As a rule of thumb, your C functions (including main) should avoid having call frames bigger than a few kilobytes. With the GCC compiler, you could invoke it with gcc -Wall -Wextra -Wframe-larger-than=1024 -g to get useful warnings and debug information.
Read the virtual address space wikipage. It has a nice picture worth many words. Later, find the way to query, on your operating system, the virtual address space of your process (on Linux, use proc(5) like cat /proc/$$/maps etc...). In practice, your virtual address space is likely to contain many segments (perhaps a dozen, sometimes thousands). Often, the dynamic linker or some other part of your program (or of your C standard library) uses memory-mapped files. The standard C heap (managed by malloc etc) may be organized in several segments.
If you want to understand more about virtual address space, take time to read a good book, like: Operating systems, three easy pieces (freely downloadable).
If you want to query the organization of the virtual address space in some process, you need to find an operating-system specific way to do that (on Linux, for a process of pid 1234, use /proc/1234/maps or /proc/self/maps from inside the process).
Memory is laid out much more differently than simply 4 segments(which was done long ago). The answer to the question can be generalized this way - the global or dynamically allocated memory space is handled differently than that of local variables by the system, where as the memory for local variable is limited in size, memory for dynamic allocation or global variables doesn't put a lower constraint like this.
In modern system the concept of virtual address space is there. The process from your program gets a chunk of it. That portion of memory is now responsible for holding the required memory.
Now for dynamic allocation and so on, the process can request more memory and depending on the other processes and so on, new memory request is serviced. For dynamic or global array there is no limit process wise (of course system wise there is- it cant haverequest all memory). That's why dynamic allocation or using global variable won't cause the process to run out of it's allocated memory unlike the automatic lifetime memory that it originally had for local variables.
Basically you can check your stack size
for example in Linux : ulimit -s (Kbytes)
and then decide how you manipulate your code regarding that.
As a concept I would never allocate big piece of memory on the stack because unless you know exactly the depth of your function call and the stack use, it's hard to control the precised allocated memory on stack during run time
While taking my first steps with C, I quickly noticed that int array[big number] causes my programs to crash when called inside a function. Not quite as quickly, I discovered that I can prevent this from happening by defining the array with global scope (outside the functions) or using malloc.
My question is:
Starting at which size is it necessary to use one of the above methods to make sure my programs won't crash?
I mean, is it safe to use just, e.g., int i; for counters and int chars[256]; for small arrays or should I just use malloc for all local variables?
You should understand what the difference is between int chars[256] in a function and using malloc().
In short, the former places the entire array on the stack. The latter allocates the memory you requested from the heap. Generally speaking, the heap is much larger than the stack, but the size of each can be adjusted.
Another key difference is that a variable allocated on the stack will technically be gone after you return from the method. (Oh, your program may function as though it's not gone for a bit if you continue to access that array, but ho ho ho danger lurks.) A hunk of memory allocated with malloc will remain allocated until you explicitly free it or the program exits.
You should use malloc for your dynamic memory allocation. For statically sized arrays (or any other object) within functions, if the memory required is to big you will quickly get a segmentation fault. I don't think a 'safe limit' can be defined, its probably implementation specific and other factors come in play too, like the current stack and objects within it created by callers to the current function. I would be tempted to say that anything below the page size (usually 4kb) should be safe as long as there is no recursion involved, but I do not think there are such guarantees.
It depends. If you have some guarantee that a line will never be longer than 100 ... 1000 characters you can get away with a fixed size buffer. If you don't: you don't. There is a difference between reading in a neat x KB config file and a x GB XML file (with no CR/LF). It depends.
The next choice is: do you want your program to die gracefully? It is only a design choice.