Higher level languages such as javascript don't give the programmer a
choice as to where variables are stored. But C does. My question is:
are there any guidelines as to where to store variables, eg dependent
on size, usage, etc.
As far as I understand, there are three possible locations to store
data (excluding code segment used for actual code):
DATA segment
Stack
Heap
So transient small data items should be stored on the stack?
What about data items which must be shared between functions. These
items could be stored on the heap or in the data segment. How do you
decide which to choose?
You're looking through the wrong end of the telescope. You don't specify particular memory segments in which to store a variable (particularly since the very concept of a "memory segment" is highly platform-dependent).
In C code, you decide a variable's lifetime, visibility, and modifiability based on what makes sense for the code, and based on that the compiler will generate the machine code to store the object in the appropriate segment (if applicable)
For example, any variables declared at file scope (outside of any function) or with the keyword static will have static storage duration, meaning they are allocated at program startup and held until the program terminates; these objects may be allocated in a data segment or bss segment. Variables declared within a function or block without the static keyword have automatic storage duration, and are (typically) allocated on the stack.
String literals and other compile-time constant objects are often (but not always!) allocated in a readonly segment. Numeric literals like 3.14159 and character constants like 'A' are not objects, and do not (typically) have memory allocated for them; rather, those values are embedded directly in the machine code instructions.
The heap is reserved for dynamic storage, and variables as such are not stored there; instead, you use a library call like malloc to grab a chunk of the heap at runtime, and assign the resulting pointer value to a variable allocated as described above. The variable will live in either the stack or a data segment, while the memory it points to lives on the heap.
Ideally, functions should communicate solely through parameters, return values, and exceptions (where applicable); functions should not share data through an external variable (i.e., a global). Function parameters are usually allocated on the stack, although some platforms may pass parameters via registers.
You should prefer local/stack variables to global or heap variables when those variables are small, used often and in a relatively small/limited scope. That will give the compiler more opportunities to optimize the code using them as it'll know they aren't going to change between function calls unless you pass around pointers to them.
Also, the stack is usually relatively small and allocating large structures or arrays on it may lead to stack overflows, especially so in recursive code.
Another thing to consider is the use of global variables in multithreaded programs. You want to minimize chances of race conditions and one strategy for that is maiking functions thread-safe and re-enterant by not using any global resources in them directly (if malloc() is thread-safe, if errno is per-thread, etc you can use them, of course).
Btw, using local variables instead of global variables also improves code readability as the variables are located close to the place where they're used and you can quickly find out their type and where and how they're used.
Other than that, if your code is correct, there shouldn't be much practical difference between making variables local or global or in the heap (of course, malloc() can fail and you should remember about it:).
C only allows you to specify where data is stored indirectly... via the scope of the variable and/or allocation. i.e., a local variable to a function is typically a stack variable unless it is declared static in which case it will likely be DATA/BSS. Variables created dynamically via new/malloc will typically be heap.
However, there's no guarantee of any of that... only the implication of it.
That said, the one thing that is guaranteed to be a bad idea is to declare large local variables in functions... common source of strange errors and stack overflows. Very large arrays and structures are best suited to dynamic allocation and keep the pointers in local/global as required.
Related
To my understandings, dynamic allocation is preferred to declaring local variables when
you need to allocate a large amount of data compared to the stack size, and/or
you want to control the duration of it dynamically.
But AFAIK global variables are allocated on the data segment, which is also fairly large compared to the stack.
Suppose I have a large variable whose lifetime will be throughout the entire program and the concerns are only its size. Will there be still any disadvantages in declaring it globally instead of calling malloc?
People often warn that global variables are evil because they make the code complex and unclear, but I don't think I should care for this because in my case the file scope is small.
The term "global variable" isn't very helpful, because it actually means something in the global namespace accessible everywhere. Such variables are definitely bad and should always be avoided.
Instead we can declare a variable at file scope without making it global, by adding static (internal linkage instead of external linkage). Once that is done, the whole "globals are bad" discussion is irrelevant - the variable is now only accessible from the translation unit where it was declared.
Another valid concern against having a static int array[n]; at file scope is thread-safety. Such an array might have to be protected against race conditions if used by more than one thread. But turning it dynamic won't solve that problem: either you need to access the whole array from multiple threads, in which case you need protection anyway. Or you only need to access different parts of the array from multiple threads, in case you won't need protection. Regardless of how it was allocated.
With these things in mind, no there is nothing wrong of having a static int array[n]; declared at file scope and allocated at .data/.bss (if it will fit). In fact there are many big advantages of doing so compared to dynamic allocation: no fragmentation, no allocation overhead, no memory leaks.
I'm currently using C/C++. But what is the main reason of using malloc/new instead of just declare some var on the stack. Like: int a;
Or another example:
int *a; // then use this to track an array
int a = (int)malloc(sizeof(int));
Automatic variables have a lifetime that ends when the program leaves the block of code that declares them. Sometimes you want them to live longer than that; dynamic allocation gives you full control over their lifetime. Sometimes they are too big for the stack; dynamic memory is (typically) less restricted.
With that flexibility comes responsibility: you need to delete them when you've finished with them, but not before. This is difficult to get right if you try to hold onto a raw pointer and do it yourself; so (in C++) learn about RAII, and use ready-made management types like smart pointers and containers to do the work for you.
TL;DR: stack and heap are two different memory areas, they serve different purposes, they have their own access pattern (with everything that implies) and policies (yes, stack memory is way more controlled than heap memory on hw-enforced systems).
Stack memory is a precious, limited resource that needs to be allocated contiguously (you can't chunk it as you would for a heap allocation).
As a consequence of this, its default dimension on many x86 platforms is usually around 1 MB (although this can be increased). Definitely small with regard to how much memory you might allocate with a heap allocation. Your question isn't an exact duplicate of this question but I believe you should take a look at it if you're interested in other reasons for the stack being a limited resource.
Other reasons include visibility scopes (stack stuff is collected/destroyed at the end of the scope while heap allocated memory can be used in other parts of your application until you free it).
The C standard function malloc() with its friends (free(), calloc() and realloc()) allows to dynamically allocate arbitrary large (allowed by OS though) chunks of memory in runtime. This has informal name of dynamic storage duration (formal one from N1570 is allocated storage duration) with following characteristics:
lifetime is cotrolled by malloc() and free() calls. This is unlikely to automatic storage duration variables, where their lifetime is restricted by scope (e.g. block, more precisely by { and }, with exception to VLAs) and static storage duration, which have lifetime of whole execution of program
scope is determined by availability of pointer to allocated data
In other words you use malloc() when you need maximum flexibility of data allocation in runtime. Automatic variables are practically limited by stack size and static variables require compile-time reservation for exact (i.e. static) amount of memory.
C does not have garbage collection, hence whenever we allocate memory using malloc/calloc/realloc, we need to manually free it after its use is over. How are variables of other data types like int, char etc. handled by C? How is the memory allocated to those variables released?
That depends. If you allocate any of those data types with malloc/calloc/realloc you will still need to free them.
On the other side, if a variable is declared inside a function, they are called automatic variables and whenever that function ends they'll be automatically collected.
The point here is not the data type per se, is the storage location. malloc/calloc/realloc allocate memory in the heap whereas automatic variables (variables declared inside functions) are allocated in the stack.
The heap is completely managed by the programmer, while the stack works in a way that when a function ends, the stack frame is shrink and every variable occupying that frame will be automatically overwritten when another function is called.
To grasp a better feeling of these, take a look at the memory layout of a C program. Other useful references might be free(3) man page and Wikipedia page for Automatic variables.
Hope this helps!
Resources (such as memory) have nothing to do with variables. You never have to think about variables. You only have to think about the resource itself, and you need to manage the lifetime of the resource. There are function calls that acquire a resource (such as malloc) and give you a handle for the resource (such as a void pointer), and you have to call another function (such as free) later on with that handle to release the resource.
Memory is only one example, C standard I/O-files work the same way, as do mutexes, sockets, window handles, etc. (In C++, add "dynamically allocated object" to the list.) But the central concept is that of the resource, the thing that needs acquiring and releasing. Variables have nothing to do with it except for the trivial fact that you can use variables to store the resource handles.
Yesterday I had an interview where the interviewer asked me about the storage classes where variables are stored.
My answer war:
Local Variables are stored in Stack.
Register variables are stored in Register
Global & static variables are stored in data segment.
The memory created dynamically are stored in Heap.
The next question he asked me was: why are they getting stored in those specific memory area? Why is the Local variable not getting stored in register (though I need an auto variable getting used very frequently in my program)? Or why global or static variables are not getting stored in stack?
Then I was clueless. Please help me.
Because the storage area determines the scope and the lifetime of the variables.
You choose a storage specification depending on your requirement, i.e:
Lifetime: The duration you expect the particular variable needs to be alive and valid.
Scope: The scope(areas) where you expect the variable to be accessible.
In short, each storage area provides a different functionality and you need various functionality hence different storage areas.
The C language does not define where any variables are stored, actually. It does, however, define three storage classes: static, automatic, and dynamic.
Static variables are created during program initialization (prior to main()) and remain in existence until program termination. File-scope ('global') and static variables fall under the category. While these commonly are stored in the data segment, the C standard does not require this to be the case, and in some cases (eg, C interpreters) they may be stored in other locations, such as the heap.
Automatic variables are local variables declared in a function body. They are created when or before program flow reaches their declaration, and destroyed when they go out of scope; new instances of these variables are created for recursive function invocations. A stack is a convenient way to implement these variables, but again, it is not required. You could implement automatics in the heap as well, if you chose, and they're commonly placed in registers as well. In many cases, an automatic variable will move between the stack and heap during its lifetime.
Note that the register annotation for automatic variables is a hint - the compiler is not obligated to do anything with it, and indeed many modern compilers ignore it completely.
Finally, dynamic objects (there is no such thing as a dynamic variable in C) refer to values created explicitly using malloc, calloc or other similar allocation functions. They come into existence when explicitly created, and are destroyed when explicitly freed. A heap is a convenient place to put these - or rather, one defines a heap based on the ability to do this style of allocation. But again, the compiler implementation is free to do whatever it wants. If the compiler can perform static analysis to determine the lifetime of a dynamic object, it might be able to move it to the data segment or stack (however, few C compilers do this sort of 'escape analysis').
The key takeaway here is that the C language standard only defines how long a given value is in existence for. And a minimum bound for this lifetime at that - it may remain longer than is required. Exactly how to place this in memory is a subject in which the language and library implementation is given significant freedom.
It is actually just an implementation detail that is convenient.
The compiler could, if he wanted to, generate local variables on the heap if he wishes.
It is just easier to create them on the stack since when leaving a function you can adjust the frame pointer with a simple add/subtract depending on the growth direction of the stack and so automatically free the used space for the next function. Creating locals on the heap however would mean more house-keeping work.
Another point is local variables must not be created on the stack, they can be stored and used just in a register if the compiler thinks that's more appropriate and has enough registers to do so.
Local variables are stored in registers in most cases, because registers are pushed and poped from stack when you make function calls It looks like they are on stack.
There is actually no such tings as register variables because it is just some rarely used keyword in C that tells compiler to try to put this in registers. I think that most compilers just ignore this keyword.
That why asked you more, because he was not sure if you deeply understand topic. Fact is that register variables are virtually on stack.
in embedded systems we have different types of memories(read only non volatile(ROM), read write non volatile(EEPROM, PROM, SRAM, NVRAM, flash), volatile(RAM)) to use and also we have different requirements(cannot change and also persist after power cycling, can change and also persist after power cycling, can change any time) on data we have. we have different sections because we have to map our requirements of data to different types of available memories optimistically.
I came across the following definition:
A static object is one that exists from the time it is constructed and created until the end of the program. Stack- and Heap- based objects are thus excluded. Static objects are destroyed when the program exits, i.e. their destructors are called when main finishes executing.
Why are stack- and heap- based objects excluded???
Here is what I know about stacks and heaps: The stack is the part of the system memory where all the variables are stored before run-time. The heap is the part of the system memory where all the variables are stored during run-time, e.g. dynamically allocated memory. This means that if I declare an integer variable i in my code and assign the value of say 123 to it, then that will be stored in my stack, because the compiler knows the value during the compile time (before run-time). But if I define a pointer variable and want to initialize it somewhere else, then that will be stored in my heap, since it is unknown to the compiler at the compile time.
There are several storage durations:
Static → whole program lifetime
Automatic (stack) → until the end of the current function
Dynamic (heap) → until it gets explicitly ended (via delete)
"A static object is one that exists from the time it is constructed and created until the end of the program. Stack- and Heap- based objects are thus excluded."
Why are stack- and heap- based objects excluded???
They are "excluded" because they do not exist from the time it is constructed and created until the end of the program.
None of this contradicts what you wrote / understand in your 2nd paragraph, though there may be nuances depending on the programming language that you are talking about.
What you've found is a poorly worded definition of static. Nothing more, nothing less.
In general, a static object is "created" by the compiler at compile time. Its behavior as to program exit is likely to be different across languages. For example, in C, there is no special handling at all (and AFAIK that's also true for Objective-C). Often these objects "live" in a read-only memory area that the compiler created and "attached" to the program. When the program is loaded into memory this read-only area is mapped into the program's memory which is a very fast operation. For example, all the static strings (as in printf("I'm a static string.");) in C are treated that way.
Then there's the stack, aka call stack, and a stack. A stack in general is just a data structure, aka LIFO (last-in-first-out). The call stack is indeed created by the OS and is normally limited in size. It stores all the information that are necessary for function call. That mean for each function call, its arguments and other info is "pushed" to the stack (put on top of the stack) and a little space for the function variables is reserved. Once the function returns, all this stuff is removed and only the return value is left (though even this is not always true, often the return value is passed in a CPU register).
You can store values to the stack, and languages like C++ even allow you to store objects on the stack. They "automatically" get cleaned once its enclosing function returns.
You can store also store a pointer to such an object living in the stack in another variable as well. But what you probably mean is that normally you create an object in the heap (e.g. via new in Java, C++, C#, etc. or alloc in Objective-C) and you get a pointer to that object in return.
Back to the start: static objects are known to the compiler at compile time, but everything that has to do with heap and stack is by definition only known at run time.