I was going through one of the threads.
A program crashed because
it had declared an array of 10^6 locally inside a function.
Reason being given was memory allocation failure on stack leads to crash.
when same array was declared globally, it worked well.(memory on heap saved it).
Now for the moment, let us suppose,
stack grows downward and heap upwards.
We have:
---STACK---
-------------------
---HEAP----
Now , I believe that if there is failure in allocation on stack,
it must fail on heap too.
So my question is: is there any limit on stack size?
(crossing the limit caused the program to crash).
Or am I missing something?
Yes, stack is always limited. In several languages/compilers you can set the requested size.
Usually default values (if not set manually) are about 1MB for current languages, which is enough unless you do something that usually isn't recommended (like you allocating huge arrays on the stack)
Contrary to all answers so far, on Linux with GCC (and I guess it is true for all modern POSIX operating systems), maximum stack size is a safety limit enforced by the operating system, that can be easily lifted.
I crafted a small program that calls recursively a function until at least 10 GB is allocated on stack, waits for input on the terminal, and then safely returns from all recursive calls up to main.
#include <stdio.h>
#include <string.h>
#include <sys/time.h>
#include <sys/resource.h>
void grow(unsigned cur_size)
{
if(cur_size * sizeof(int) < 10ul*1024ul*1024ul*1024ul) {
unsigned v[1000];
v[0] = cur_size;
for(unsigned i = 1; i < 1000; ++i) {
v[i] = v[i-1] + 1;
}
grow(cur_size + 1000);
for(unsigned i = 0; i < 1000; ++i) {
if(v[i] != cur_size + i)
puts("Error!");
}
} else {
putchar('#');
getchar();
}
}
int main()
{
struct rlimit l;
l.rlim_max = RLIM_INFINITY;
l.rlim_cur = RLIM_INFINITY;
setrlimit(RLIMIT_STACK, &l);
grow(0);
putchar('#');
getchar();
}
This all depends on what language and compiler you use. But programs compiled with for instance C or C++ allocate a fixed size stack at program startup. The size of the stack can usually be specified at compile time (on my particular compiler it default to 1 MB).
You don't mention which programming language, but in Delphi the compile options include maximum and minimum stack size, and I believe similar parameters will exist for all compiled languages.
I've certainly had to increase the maximum myself occasionally.
Yes, there is a limit on stack size in most languages. For example, in C/C++, if you have an improperly written recursive function (e.g. incorrect base case), you will overflow the stack. This is because, ignoring tail recursion, each call to a function creates a new stack frame that takes up space on the stack. Do this enough, and you will run out of space.
Running this C program on Windows (VS2008)...
void main()
{
main();
}
...results in a stack overflow:
Unhandled exception at 0x004113a9 in Stack.exe: 0xC00000FD: Stack overflow.
Maybe not a really good answer, but gives you a little more in depth look on how windows in general manages the memory: Pushing the Limits of Windows
Related
I'm trying to calculate stack memory used in my program.
should i add 4 for each integer i have defined?
what about something like char str[128], shoudld i add 128 or 129?
#define ARRAY1_LIMIT 200
#define ARRAY2_LIMIT 100
char* array1[ARRAY1_LIMIT];
char* array2[ARRAY2_LIMIT];
int i = 0;
int j = 0
array1[i] = (char *)malloc(sizeof(char)*5);
array2[j] = (char *)malloc(sizeof(char)*10);
I know that the heap memory is 5+15 = 15, but i don't know how to calculate the stack memory? is it 200 + 100?
If you're doing anything more than simply trying to learn about stack usage and how variable allocations affect it, you'll want to use a stack depth analysis tool. Such a tool can help you determine if your program could possibly overflow its stack under any possible sequence of events (excepting unexpected or unbounded recursion). You can write your own (I have, in C#, for embedded programs compiled in C for M16C and MIPS targets using GCC and IAR compilers), but it's really complex and not something for beginners to attempt.
Look for a "stack usage analyzer" or "stack usage analysis tool" for your particular processor and toolchain (e.g. x86/x64/ARM/etc and GCC/VisualStudio/IAR/etc).
If you're using GCC, you may be able to use the -fstack-usage option, but that only gives you the maximum stack usage on a per-function basis. By itself that's not terribly helpful, since to verify that a program won't blow its stack, you have to recursively walk the calltree to see what the maximum stack depth could be at any level of the call tree. If you also use the -Wstack-usage option, you can get a warning if any subprogram's stack usage can possibly exceed a specified stack depth, which is more useful than the information you get with merely the -fstack-usage option.
If you're trying to work out the amount of 'space allocated to those variables in a function, use the sizeof() operator.
Notice that the arrays are arrays of char * which are pointers to characters not characters.
#include <stdio.h>
int main(void) {
size_t total=0;
size_t first_array_size=sizeof(char *[200]);
printf("first array: %zu\n",first_array_size);
total+=first_array_size;
size_t second_array_size=sizeof(char *[100]);
printf("second array: %zu\n",second_array_size);
total+=second_array_size;
size_t int_size=sizeof(int);
printf("int size: %zu * 2 = %zu\n",int_size,int_size*2);
total+=int_size*2;
printf("total=%zu\n",total);
return 0;
}
Typical output (on a 64-bit architecture):
first array: 1600
second array: 800
int size: 4 * 2 = 8
total=2408
Results may vary.
Footnote: It's also worth understanding that the amount of space allocated to the stack frame may be greater.
For example arguments passed in are typically copied to the stack as is the return value along with a pointer representing the execution point to return to after the function is called.
There's also the complexity of alignment. For example on many modern machines space may be left between variables to make sure they're aligned. However optimizations may take some of that space back depending on how it re-orders the variables. It's also possible that values (particularly integer values) may be allocated to registers and not take up space on the stack.
Finally an implementation could in principle allocate the arrays on the heap. That's certainly a possible implementation for variable length arrays but I'm not aware of it being strictly disallowed for fixed size arrays.
I'm just curious if there is any correlation between the length of the address of a variable (pointer) on stack and heap. On many occasions I have seen that those regarding stack variables are usually longer when compared to heap. For example consider the following simple test:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int i = 0;
int *j = malloc(sizeof(int)); *j = 0;
printf("&i = %p\n j = %p\n", &i, j);
free(j);
return 0;
}
output:
&i = 0x7fffe9c7fa5c
j = 0x100e010
These results are obtained in linux using gcc; could this be OS/compiler dependent?
The results depend on positions of the heap(s) and stack(s) in the address space of the program. These are determined by linker and processor architecture.
Due to ASLR, the exact numbers should be random on modern systems.
Nevertheless, heaps will usually grow upwards, and stacks downwards. Additionally, for performance and memory management reasons, both heaps and stacks will always start on page boundaries.
I believe it's because of the physical parts of the memory which we decide that they're called stack and heap. Since they start at opposite ends and grow towards the middle, it makes sense that one is lower and the other higher. It would be interesting to see what happens if you allocate 2 consecutive vars on the stack and 2 consecutive ones on the heap. This would help see which way the stack and heap grow. Actually I think for this to work you need to make a new stack frame (a new method) and allocate the second vars there, otherwise you remain in the same stack frame.
I am just about finished reading K&R, and that is all the C that I know. All my compilation is done from Windows command line using MinGW, and I have no knowledge of advanced debugging methods (hence the "ghetto debug" comment in my 2nd program below).
I am trying to make a few small test programs to help me better understand how memory allocation works. These first couple programs do not use malloc or free, I just wanted to see how memory is allocated and de-allocated for standard arrays local to a function. The idea is that I watch my running processes RAM usage to see if it corresponds with what I understand. For this first program below, it does work as I expected. The alloc_one_meg() function allocates and initializes 250,000 4-byte integers, but that MB is de-allocated as soon as the function returns. So if I call that function 1000000 times in a row, I should never see my RAM usage go much above 1MB. And, it works.
#include <stdio.h>
#include <stdlib.h>
void alloc_one_meg() {
int megabyte[250000];
int i;
for (i=0; i<250000; i++) {
megabyte[i] = rand();
}
}
main()
{
int i;
for (i=0; i<1000000; i++) {
alloc_one_meg();
}
}
For this second program below, the idea was to not allow the function to exit, to have 1000 copies of the same function running at once, which I accomplished with recursion. My theory was that the program would consume 1GB of RAM before it de-allocated it all after the recursion finished. However, it doesn't get past the 2nd loop through the recursion (see my ghetto debug comment). The program crashes with a pretty non-informative (to me) message (a Windows pop-up saying ____.exe has encountered a problem). Usually I can always get to the bottom of things with my ghetto debug method... but it's not working here. I'm stumped. What is the problem with this code? Thanks!
#include <stdio.h>
#include <stdlib.h>
int j=0;
void alloc_one_meg() {
int megabyte[250000];
int i;
for (i=0; i<250000; i++) {
megabyte[i] = rand();
}
j++;
printf("Loop %d\n", j); // ghetto debug
if (j<1000) {
alloc_one_meg();
}
}
main()
{
alloc_one_meg();
}
Followup question posted here.
You're running into a stack overflow.
Local automatic storage variables (such as megabyte) are allocated on the stack, which has limited amount of space. malloc allocates on the heap, which allows much larger allocations.
You can read more here:
http://en.wikipedia.org/wiki/Stack_overflow
(I should note that the C language does not specify where memory is allocated - stack and heap are implementation details)
The size of the stack in a Windows program is usually around 1 MB, so on the second recursion, you're overflowing the stack. You shouldn't be allocating such large arrays on the stack, use malloc and free to allocate and deallocate the memory on the heap (there's no way to get around malloc for such sizes of arrays):
void alloc_one_meg() {
int *megabyte = malloc(sizeof(int) * 250000); // allocate space for 250000
// ints on the heap
int i;
for (i=0; i<250000; i++) {
megabyte[i] = rand();
}
j++;
printf("Loop %d\n", j); // ghetto debug
if (j<1000) {
alloc_one_meg();
}
free(megabyte); // DO NOT FORGET THIS
}
That said, you can actually change the stack size of a program and make it bigger (though I'd only do so as an educational exercise, not in production code). For Visual Studio you can use the /F compiler option, and on linux you can use setrlimit(3). I'm not sure what to use with MinGW though.
The memory you are allocating via the recursive functional calls is allocated from the stack. All of the stack memory must be contiguous. When your process starts a thread, Windows will reserve a range of virtual memory address space for that thread's stack. The amount of memory to be reserved is specified in your EXE file's "PE header." PE stands for "Portable Executable."
Using the dumpbin utility included with Visual Studio, with itself (dumpbin.exe) as the input file:
dumpbin /headers dumpbin.exe
... there is some output, and then:
100000 size of stack reserve
2000 size of stack commit
The "100000" is a hexadecimal number equal to 1,048,576, so this represents around 1MB.
In other words, the operating system will only reserve a 1MB address range for the stack. When that address range is used up, Windows may or may not be able to allocate further consecutive memory ranges to increase the stack. The result depends on whether further contiguous address range is available. It is very unlikely to be available, due to the other allocations Windows made when the thread began.
To allocate a maximum amount of virtual memory under Windows, use the VirtualAlloc family of functions.
StackOverflow. Is this a trick question?
I'm learning about my system's ability to calculate Ackermann's algorithm both the two and three parameter version. For very small values of m and n, my system will calculate and print results returning from A0 and A1 method calls. However anything higher than 3 or 4 does not return and freezes the terminal I'm using atm. My problem is that I do determine for what values of m and n my machine can compute.
I have tried a few things to catch a stack overflow, for all i know c++ doesn't have a stackoverflowexception I can catch. try-catch blocks don't work. In the below code, I use getrlimit() to find the stack limit, create a address location in main gStackRef. I call checkStack recursively checking the local variable pointer to gStackLimit.
Is there a better way of checking my stack usage in relation to recursive methods? Also I do i check for segment faults? I'll let you know I'm running on a unix terminal.
#include <cstdlib>
#include <iostream>
#define _XOPEN_SOURCE_EXTENDED 1
#include <sys/resource.h>
int getrlimit(int resource, struct rlimit *rlp);
using namespace std;
int * gStackRef;
int gStackLimit;
void checkStack(void);
int main(int argc, char *argv[])
{
int temp = 0;
gStackRef = &temp;
rlimit myl;
getrlimit(RLIMIT_STACK, &myl);
gStackLimit = (myl.rlim_cur / 3 * 8 / 10) ;/* modified for segment fault */
cout << gStackLimit << "\n";
checkStack();
}
void checkStack()
{
int temp = 0;
int* pVariableHere = &temp;
size_t stackUsage = gStackRef - pVariableHere;
printf("Stack usage: %d / %d \n", stackUsage, gStackLimit);
if(stackUsage > gStackLimit) return;
else checkStack();
}
However anything higher than 3 or 4 does not return and freezes the terminal I'm using atm.
That's kind of the point of the Ackermann function. It grows extremely rapidly. For m >= 4 and n >= 3, if you're calculating A(m, n) recursively, I doubt your function will return before you're dead.
I have tried a few things to catch a stack overflow, for all i know c++ doesn't have a stackoverflowexception I can catch.
Of course not. The process is out of stack space. It should be torn down immediately.
Is there a better way of checking my stack usage in relation to recursive methods?
If you have to use recursion, do it manually by creating your own stack data structure that is allocated on the heap instead of in the stack space. Use that to keep track of where you are in the recursion. Push and pop and as you recurse, instead of recursing by nested method calls.
But at the end, you shouldn't be using recursion to calculate Ackermann anyway.
I have tried a few things to catch a stack overflow, for all i know c++ doesn't have a stackoverflowexception I can catch. try-catch blocks don't work. In the below code, I use getrlimit() to find the stack limit, create a address location in main gStackRef. I call checkStack recursively checking the local variable pointer to gStackLimit.
POSIX does not have a "safe" way of detecting a stack overflow. Stack Overflows result in SIGSEGV signals, which you (generally) should not catch because they also are indicative of general segmentation faults, which should crash your program. Windows environments can deal with stack overflows safely, using EXCEPTION_STACK_OVERFLOW -- but in such cases what Windows is doing is merely putting a guard page at the end of the stack and notifying with SEH. If you use up the guard page (after ignoring the SEH exception), then your program gets terminated (just as it would in POSIX-land).
Is there a better way of checking my stack usage in relation to recursive methods? Also I do i check for segment faults? I'll let you know I'm running on a unix terminal.
No. Even what you're doing has undefined behavior. On some machines the stack grows up. On some machines the stack grows down. The compiler may insert any amount of slop space in between two methods. Technically, the compiler could implement things such that there were two separate stacks, located in two completely different memory segments, and still be conformant.
If you want to calculate Ackermann in a stack safe manner, either use an explicit stack structure allocated from the heap, or use dynamic programming.
stack is increasing or decreasing using C program ?
Right, in C usually variables in function scope are realized by means of a stack. But this model is not imposed by the C standard, a compiler could realize this any way it pleases. The word "stack" isn't even mentioned in the standard, and even less if it is in- or decreasing. You should never try to work with assumptions about that.
False dichotomy. There are plenty of options other than increasing or decreasing, one of which is that each function call performs the equivalent of malloc to obtain memory for the callee's automatic storage, calls the callee, and performs the equivalent of free after it returns. A more sophisticated version of this would allocate large runs of "stack" at a time and only allocate more when it's about to be exhausted.
I would call both of those very bad designs on modern machines with virtual memory, but they might make sense when implementing a multiprocess operating system on MMU-less microprocessors where reserving a range of memory for the stack in each process would waste a lot of address space.
How about:
int stack_direction(void *pointer_to_local)
{
int other_local;
return (&other_local > pointer_to_local) ? 1 : -1;
}
...
int local;
printf("direction: %i", stack_direction(&local);
So you're comparing the address of a variable at one location on the call stack with one at an outer location.
If you only like to know if the stack has been changed you can keep the last inserted object to the stack, peek at the top of it and compare the two.
EDIT
Read the comments. It doesn't seem to be possible to determine the stack direction using my method.
END EDIT
Declare an array variable on the stack and compare the addresses of consecutive elements.
#include <stdio.h>
#include <stdlib.h>
int
main(void)
{
char buf[16];
printf("&buf[0]: %x\n&buf[1]: %x\n", &buf[0], &buf[1]);
return 0;
}
The output is:
misha#misha-K42Jr:~/Desktop/stackoverflow$ ./a.out
&buf[0]: d1149980
&buf[1]: d1149981
So the stack is growing down, as expected.
You can also monitor ESP register with inline assembly. ESP register holds address to unallocated stack. So if something is pushed to stack - ESP decreases and if pop'ed - ESP increases. (There are other commands which modifies stack, for example function call/return).
For example what is going on with stack when we try to compute recursive function such as Fibonacci Number (Visual Studio):
#include <stdio.h>
int FibonacciNumber(int n) {
int stackpointer = 0;
__asm {
mov stackpointer, esp
}
printf("stack pointer: %i\n", stackpointer);
if (n < 2)
return n;
else
return FibonacciNumber(n-1) + FibonacciNumber(n-2);
}
int main () {
FibonacciNumber(10);
return 0;
}