In accordance to many sources, stack and heap are memory regions devided by empty space. As far as I got this, the stack and the heap grow towards each other as the program is running.
Does that mean that the stack and the heap are both of variable size? If they are, can one of them occupy the same addresses the other occupied during the previous run (suppose the two runs took place at the same memory addresses)?
If they are of variable size, I guess stack overflow occurs when the stack tries to take what belongs to the heap. If not, a stack bound is just what it is and placed somewhere in the middle of the initially empty space between the two regions of memory.
A bit of clarification: by the variability of size here I mean changing of the reserved space. That is, here I imply that if stack size is variable, the reserved for it space changes (like when you change size of an array using realloc, but that is just for comparison); stack overflow is then occurs by the stack hitting some bound, be it another region like heap or something else. If it is not variable, the reserved space stays the same, and stack overflow is merely caused by running out of the reserved space. And I also wonder if heap is variably sized too.
The stack must be contiguous in address space. The heap has no such constraint. Anything beyond that depends how a given platform lays out the memory.
If your address space cannot have holes, a simple layout for an embedded device can be: from address n and below is stack (stacks tends to grow down) and from n and above can be used for heap. Static is then some high address and code is in ROM.
Linux and Windows run on platforms with an MMU so have a lot more options with regards memory arrangement.
Related
I'm trying to understand the CPU's role in tracking a programs stack/heap allocation.
Reading some material, I've come across this:
The stack area traditionally adjoined the heap area and grew the opposite direction; when the stack pointer met the heap pointer, free memory was exhausted.
Are the stack and heap pointers stored in program specific registers?
If the stack pointer is pointing to the top of the stack, and (I'm assuming) the heap pointer is pointing to the end of the heap, how would these pointers ever meet without overwriting memory (overflow)?
How does this work in modern systems?
Are the stack and heap pointers stored in program specific registers?
CPUs of stack-based architectures (which represent the overwhelming majority of the CPUs in use today) have a special register for the stack pointer. This is possible because stack, by its very nature, does not get fragmented. Hence, a single pointer is sufficient.
There is no such thing as "heap pointer" because heap is potentially a fragmented data structure. Heap allocators keep a special table of memory fragments available for allocation, and adjust it when the program allocates and releases memory. Memory manager also keeps a pointer to the highest address that has been allocated from the heap.
If the stack pointer is pointing to the top of the stack, and (I'm assuming) the heap pointer is pointing to the end of the heap, how would these pointers ever meet without overwriting memory (overflow)?
Since stack pointer cannot cross without causing an error, many systems limit the size of the stack to a certain number, and make sure that the memory allocator would not let the high point of the heap to cross the upper limit of the stack.
Note: On systems that support concurrency there may be more than one stack active at a time. In this case the stacks are set up next to each other, with the upper limit monitored to detect stack overflows. Here is an article that describes techniques for detecting stack overflows.
I have some questions related to memory structure.
Are stack and heap determined by OS? or physically are they separated?
If they are determined by OS, which OS has the stack and heap as component of memory structure except Windows?
As I know default stack size is 1MB and I can expand the size manually, but why default size is so small?
And if stack size is 1MB, I can not hold data that exceed 1MB in a local variable?
And my last question is, is there any reason that programmer needs to be more aware of memory structure if he writes unmanaged code (e.g. native C++) rather than managed code (e.g. C#)?
The default stack size depends on the operating system. Some are small, some are large. The stack and the heap are "physically separated" in the sense that their addresses are different.
The reasons the differences are significant to programmers are many:
allocations on the stack are automatically deallocated when the function/method returns
large allocations on the stack may run out of memory even though the machine has free memory (due to stack size limitations)
in some environments (eg C#, Java, Python, JavaScript) while all local variables are allocated on the stack (as in C, C++) the variables allocated on the stack are all references (similar to pointers) and so always use a constant small amount of stack memory
recursion or deep function/method calling can run out of stack space
allocation and deallocation on the stack is faster than on the heap (the compiler computes the size and includes it in a single pointer move operation when the function is entered, no free lists to scan or anything -- this applies to C and C++ but not to C#, Java, etc.)
never return a pointer to data allocated on the stack; if you do then sometime later you will at best crash your program and at worst not know what data corruption has occurred
allocations on the stack can never cause memory leaks; allocations on the heap can be leaked
data on the stack is local to the function (unless it passes a pointer to it to a function it calls) and thus is thread-safe; data on the heap might be accessible from more than one thread and may need proper synchronization controls
why do structure's data members memory allocated from lower to higher address in stack growing higher to lower? What is the mechanism in this process?
One advantage is that Operating System is able to easily check whether overflow occurs in a process(program). Heap segment is usually growing from lower to higher when stack is growing from higher to lower. if two segment is gradually grow to be overlapped, OS can detect overflow in the process.
Here is a diagram for memory layout of C programs. Hope this help.
Also, such choice depends on OS designer. If you are a beginner, just don't care about it. Think easy.
The stack is usually the only thing that grows downwards, while everything else does grow upwards. A structure on the stack, however, is the same as the same structure in the heap: an area of memory identified by the address of its first byte in memory, and the filling mechanism is the same as well.
So, the interesting question is not "why is a structure on the stack filled upwards" but "why does the stack grow downwards?"
The answer is simple: Because the stack grows downwards and the objects on it are identified by the address of their first byte, the stack pointer always points to the first element on the stack.
This might seem trivial, but it simplifies things quite a bit on the machine level. On the PowerPC, for instance, there was an instruction that saves a value at an offset of a pointer and updates this same pointer to the calculated address. This instruction can atomically allocate an entire stack frame and link it to the previous stack frame, and it would not work the other way around. (I hope that wasn't too much technical detail...)
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
What and where are the stack and heap?
With regard to the basic concepts of memory layout in a c program, I understand the that:
The language uses two primary data structures stack and heap.
Stack is created to store the local variables and book keeping data of subroutines
Heap is created to store the dynamically allocated variables of the program
Heap is of variable length in nature.(Not very sure on stack)
Normally it is the responsibility of the compiler/Language to request the OS to create these data structures before the execution.
Questions
What is the initial size with which a stack/heap is created? and who decides it?
Wherein physical memory are they are created? I see a general description as "Stack is created in the top-level-address and the heap at the low-level-address" Please elobarate this
"Stack is created in the top-level-address and the heap at the
low-level-address" Please elobarate this
This is a myth. It may have a basis in historical truth. It might sometimes resonate with things you see in real life. But it is not literally true.
It's easy enough to explore, though:
#include <stdlib.h>
#include <stdio.h>
void check(int depth) {
char c;
char *ptr = malloc(1);
printf("stack at %p, heap at %p\n", &c, ptr);
if (depth <= 0) return;
check(depth-1);
}
int main() {
check(10);
return 0;
}
On my machine I see:
stack at 0x22ac3b, heap at 0x20010240
stack at 0x22ac0b, heap at 0x200485b0
stack at 0x22abdb, heap at 0x200485c0
stack at 0x22abab, heap at 0x200485d0
stack at 0x22ab7b, heap at 0x200485e0
stack at 0x22ab4b, heap at 0x200485f0
stack at 0x22ab1b, heap at 0x20048600
stack at 0x22aaeb, heap at 0x20048610
stack at 0x22aabb, heap at 0x20048620
stack at 0x22aa8b, heap at 0x20048630
stack at 0x22aa5b, heap at 0x20048640
So, the stack is going downwards and the heap is going upwards (as you might expect based on the myth), but the stack has the smaller address, and they are not growing toward each other (myth busted).
Btw, my check function is tail-recursive, and on some implementations with some compiler options you might see the stack not moving at all. Which tells you something about why the standard doesn't mandate how all this works -- if it did it might inadvertently forbid useful optimizations.
As mentioned already, sizes are OS specific. For e.g. on windows using Visual Studio, default stack size is 1MB
msdn
On Linux the following command can show show your current one.
ulimit -s or -a
On my Linux mint 64 bit it shows 8192 KB.
Every program when loaded in memory has several segments. In assembly one can indicate each of those using .data, .code etc prefix (intelx86).
It is data segment which has several sub sections. Both stack and heap are part of it in addition to several others.
Stack can also grow implicitly i.e. when you make another function call, an activation record is pushed on to stack, there by utilizing more memory of stack. That is why infinite recursion results in a crash when a program runs out of allocated stack.
When a function call returns, that activation record is popped and stack shrinks.
In contrast heap grows from the opposite direction and contains all dynamically allocated memory.
The reason these two segments grow in opposite direction is to maximize the utilization of their combined memory. Note that as mentioned in comments this is not a c standard, but most common OS's have this implemented.
------ stack starts ----------- stack grows downward
-------- Unless they cross each other a program is okay to run.
------- heap starts ------------heap grows upwards
If your program uses no heap, your stack can utilize maximum memory including that of heap too. If program makes few recursive calls and uses minimum local variables (i.e. uses less memory for stack), it can utilize heap to the most.
Other parts of data segment are BSS etc. which might contain fields such as uninitialized static variables
What is the initial size with which a stack/heap is created? and who decides it?
This is compiler- and OS-specific.
Wherein physical memory are they are created? I see a general description as "Heap is created in the top-level-address and stack at the low-level-address".
This is compiler- and OS-specific.
Really. The language standard does not mandate the minimum stack size nor specifies the location of either the stack or the heap in memory. And the reason for that is to make C programs less dependent on these details and therefore more portable to different platforms (read: different OSes, different CPUs, different compilers).
First of all, the C standard doesn't impose any requirements on how the stack/heap is implemented by
the platform.
What is the initial size with which a stack/heap is created? and who decides it?
Typically a fixed size of stack is allocated for every process by the OS which is platform-specific.
There's no limit on heap size, program usually have all of the available virtual address space.
Wherein physical memory are they are created?
This is platform specific. Typically stack grows downwards and heap grows upwards.
I've been told not to use stack allocated arrays because the stack is a precious resource.
Other people have suggested to me that in fact it is perfectly fine to use stack allocated arrays so long as the array is relatively small.
I would like to have a general rule of thumb: when should I use a stack allocated array?
And when should I use a heap allocated array?
While all of your memory is limited, even today with enormous amounts of RAM and virtual memory, there is still a limit. However, it's rather large, especially compared with the stack which can be anything from a couple of kb on small embedded systems to a couple of megabytes on a PC.
Besides that, there is also the question about how you are using it, and for what. For example, if you want to return an "array" from a function, it should never be on the stack.
In general, I would say that try to keep arrays on the stack small if you can. If you are creating an array with thousands of entries on the stack you should stop and think about what you want it for.
It depends on your platform.
Nowadays, if working on the popular x64 platform, you don't really have to worry about it.
Depending on the Operating System you use, you can check how much stack space and how much heap space a userland process is allowed to use.
For example, UNIX-like systems have soft and hard limits. Some you can crank up, some you can not.
Bottom line is that you don't usually need to worry about such things. And when you need to know, you are usually tied so closely to the platform you'll be developing for that you know all these details.
Hope I answered your question. If you want specific values please specify your exact hardware, operating system and user privileges.
The answer to this question is context dependent. When you write for an operating system kernel, for example, the stack might be quite limited, and allocating more than a thousand bytes in a stack frame could cause a problem.
In modern consumer systems, the space available for the stack is typically quite large. One problem systems used to have was that address space was limited and, once the stack was assigned an address, it could not grow any further than the next object in the address space in the direction of stack growth, regardless of the availability of physical memory or of virtual memory elsewhere in the address space. This is less of a problem with today’s address spaces.
Commonly, megabytes of space can be allocated in a stack frame, and doing so is cheap and easy. However, if many routines that allocate large amounts of space are called, or one or a few routines that allocate large amounts of space are called recursively, then problems can occur because too much space is used, running into some limit (such as address space or physical memory).
Of course, running into a physical memory limit will not be alleviated by allocating space from the heap. So only the issue of consuming the address space available for the stack is relevant to the question of whether to use stack or heap.
A simple test for whether this is a problem is to insert use of a great deal of stack space in your main routine. If you use additional stack space and your application still functions under a load that uses large amounts of stack space normally, then, when you remove this artificial reservation in main, you will have plenty of margin.
A better way would be to calculate the maximum your program could use and compare that to the stack space available from the system. But that is rarely easy with today’s software.
If you are running into stack space limits, your linker or your operating system may have options to make more available.
Scope of Global and static variables will be through out the life of a process. Memory for these variable will be allocated when a process is started and it will be freed only process exits.
But local variable(stack variable) has scope only to a function on which it is defined. Memory will be allocated when a function is invoked and it will be freed once control exits from the function.
Main intention of dynamic memory is to create a variable of user defined scope. If you want to control a scope of variable means, you can allocate memory for a variable x at one function and then pass the reference(address) to as many function you want and then finally you can free it.
So with the help of dynamic allocated memory, we can create a variable which has scope higher than a local variable and lesser than global or static variable.
Apart from this if the size is very very high its better to go for dynamic memroy, if the architecture contains memory constraint.
The good reason to use heap allocated memory is passing its ownership to some other function/struct. From the other hand, stack gives you memory management for free, you can not forget to deallocate memory from stack, while there is risk of leak if you use heap.
If you create an array just for local usage, the criteria of size of the one to use, however it is hard to give exact size, above which memory should be allocated on heap. One could say that a few hundreds bytes is enough to move to heap, for some others it will be less or more than that.