If I create a variable within a new set of curly braces, is that variable popped off the stack on the closing brace, or does it hang out until the end of the function? For example:
void foo() {
int c[100];
{
int d[200];
}
//code that takes a while
return;
}
Will d be taking up memory during the code that takes a while section?
No, braces do not act as a stack frame. In C, braces only denote a naming scope, but nothing gets destroyed nor is anything popped off the stack when control passes out of it.
As a programmer writing code, you can often think of it as if it is a stack frame. The identifiers declared within the braces are only accessible within the braces, so from a programmer's point of view, it is like they are pushed onto the stack as they are declared and then popped when the scope is exited. However, compilers don't have to generate code that pushes/pops anything on entry/exit (and generally, they don't).
Also note that local variables may not use any stack space at all: they could be held in CPU registers or in some other auxiliary storage location, or be optimized away entirely.
So, the d array, in theory, could consume memory for the entire function. However, the compiler may optimize it away, or share its memory with other local variables whose usage lifetimes do not overlap.
The time during which the variable is actually taking up memory is obviously compiler-dependent (and many compilers don't adjust the stack pointer when inner blocks are entered and exited within functions).
However, a closely related but possibly more interesting question is whether the program is allowed to access that inner object outside the inner scope (but within the containing function), ie:
void foo() {
int c[100];
int *p;
{
int d[200];
p = d;
}
/* Can I access p[0] here? */
return;
}
(In other words: is the compiler allowed to deallocate d, even if in practice most don't?).
The answer is that the compiler is allowed to deallocate d, and accessing p[0] where the comment indicates is undefined behaviour (the program is not allowed to access the inner object outside of the inner scope). The relevant part of the C standard is 6.2.4p5:
For such an object [one that has
automatic storage duration] that does
not have a variable length array type,
its lifetime extends from entry into the block with which it is associated
until execution of that block ends in
any way. (Entering an enclosed block
or calling a function suspends, but
does not end, execution of the current
block.) If the block is entered
recursively, a new instance of the
object is created each time. The
initial value of the object is
indeterminate. If an initialization is
specified for the object, it is
performed each time the declaration is
reached in the execution of the block;
otherwise, the value becomes
indeterminate each time the
declaration is reached.
Your question is not clear enough to be answered unambiguously.
On the one hand, compilers don't normally do any local memory allocation-deallocation for nested block scopes. The local memory is normally allocated only once at function entry and released at function exit.
On the other hand, when the lifetime of a local object ends, the memory occupied by that object can be reused for another local object later. For example, in this code
void foo()
{
{
int d[100];
}
{
double e[20];
}
}
both arrays will usually occupy the same memory area, meaning that the total amount of the local storage needed by function foo is whatever is necessary for the largest of two arrays, not for both of them at the same time.
Whether the latter qualifies as d continuing to occupy memory till the end of function in the context of your question is for you to decide.
It's implementation dependent. I wrote a short program to test what gcc 4.3.4 does, and it allocates all of the stack space at once at the start of the function. You can examine the assembly that gcc produces using the -S flag.
No, d[] will not be on the stack for the remainder of routine. But alloca() is different.
Edit: Kristopher Johnson (and simon and Daniel) are right, and my initial response was wrong. With gcc 4.3.4.on CYGWIN, the code:
void foo(int[]);
void bar(void);
void foobar(int);
void foobar(int flag) {
if (flag) {
int big[100000000];
foo(big);
}
bar();
}
gives:
_foobar:
pushl %ebp
movl %esp, %ebp
movl $400000008, %eax
call __alloca
cmpl $0, 8(%ebp)
je L2
leal -400000000(%ebp), %eax
movl %eax, (%esp)
call _foo
L2:
call _bar
leave
ret
Live and learn! And a quick test seems to show that AndreyT is also correct about multiple allocations.
Added much later: The above test shows the gcc documentation is not quite right. For years it has said (emphasis added):
"The space for a variable-length array is deallocated as soon as the array name's scope ends."
They might. They might not. The answer I think you really need is: Don't ever assume anything. Modern compilers do all kinds of architecture and implementation-specific magic. Write your code simply and legibly to humans and let the compiler do the good stuff. If you try to code around the compiler you're asking for trouble - and the trouble you usually get in these situations is usually horribly subtle and difficult to diagnose.
Your variable d is typically not popped off the stack. Curly braces do not denote a stack frame. Otherwise, you would not be able to do something like this:
char var = getch();
{
char next_var = var + 1;
use_variable(next_char);
}
If curly braces caused a true stack push/pop (like a function call would), then the above code would not compile because the code inside the braces would not be able to access the variable var that lives outside the braces (just like a sub-function cannot directly access variables in the calling function). We know that this is not the case.
Curly braces are simply used for scoping. The compiler will treat any access to the "inner" variable from outside the enclosing braces as invalid, and it may re-use that memory for something else (this is implementation-dependent). However, it may not be popped off of the stack until the enclosing function returns.
Update: Here's what the C spec has to say. Regarding objects with automatic storage duration (section 6.4.2):
For an object that does not have a variable length array type, its
lifetime extends from entry into the block with which it is associated
until execution of that block ends in anyway.
The same section defines the term "lifetime" as (emphasis mine):
The lifetime of an object is the portion of program execution during
which storage is guaranteed to be reserved for it. An object exists,
has a constant address, and retains its last-stored value throughout
its lifetime. If an object is referred to outside of its lifetime, the
behavior is undefined.
The key word here is, of course, 'guaranteed'. Once you leave the scope of the inner set of braces, the array's lifetime is over. Storage may or may not still be allocated for it (your compiler might re-use the space for something else), but any attempts to access the array invoke undefined behavior and bring about unpredictable results.
The C spec has no notion of stack frames. It speaks only to how the resulting program will behave, and leaves the implementation details to the compiler (after all, the implementation would look quite different on a stackless CPU than it would on a CPU with a hardware stack). There is nothing in the C spec that mandates where a stack frame will or will not end. The only real way to know is to compile the code on your particular compiler/platform and examine the resulting assembly. Your compiler's current set of optimization options will likely play a role in this as well.
If you want to ensure that the array d is no longer eating up memory while your code is running, you can either convert the code in curly braces into a separate function or explicitly malloc and free the memory instead of using automatic storage.
I believe that it does go out of scope, but is not pop-ed off the stack until the function returns. So it will still be taking up memory on the stack until the function is completed, but not accessible downstream of the first closing curly brace.
There has already been given much information on the standard indicating that it is indeed implementation specific.
So, one experiment might be of interest. If we try the following code:
#include <stdio.h>
int main() {
int* x;
int* y;
{
int a;
x = &a;
printf("%p\n", (void*) x);
}
{
int b;
y = &b;
printf("%p\n", (void*) y);
}
}
Using gcc we obtain here two times the same address: Coliro
But if we try the following code:
#include <stdio.h>
int main() {
int* x;
int* y;
{
int a;
x = &a;
}
{
int b;
y = &b;
}
printf("%p\n", (void*) x);
printf("%p\n", (void*) y);
}
Using gcc we obtain here two different addresses: Coliro
So, you can't be really sure what is going on.
Related
If I define a variable anywhere in a function (not at the beginning), when the program is compiled and executed to this function, will space be allocated to the defined variable first or will it be allocated when it runs to the defined statement?
If it runs in order, Will it reduce some overhead when problems arise?
like so:
if(!(Size && Packs))
{
ret = false;
return SendAck(ret);
}
uint8_t *pSrc = (uint8_t *)pRcv->data;
uint8_t crc = 0;
The Standard C Computing Model
The C standard specifies memory reservation requirements in terms of storage duration and lifetime. Objects declared inside functions without extern, static, or _Thread_local have automatic storage duration. (Other storage durations, not discussed here, are static, thread, allocated, and temporary.) This includes parameters of functions, since they are declared inside the function declaration.
Each declaration inside a function has an associated block. Blocks are groups of statements (sometimes just a single statement). A compound statement bracketed with { and } is a block, each selection statement (if, switch) and each loop statement (for, while, do) is a block, and each substatement of those statements is a block. The block associated with a declaration inside a function is the innermost block it is in. For a function parameter, its associated block is the compound statement that defines the function.
For an automatic object that is not a variable length array, its lifetime starts when execution enters the block it is in, and it ends when execution of the block ends. (Calling a function suspends execution of the block b ut does not end it.) So in:
{
Label:
foo();
int A;
}
A exists as soon as execution reaches Label, because execution of A’s block has started.
This means that, as soon as the block is entered, all automatic objects in it other than variable length arrays should have memory reserved for them.
This fact is generally of little use, as there is no way to refer to A at Label. However, if we do this:
{
int i = 0;
int *p;
Label:
if (0 < i)
*p += foo();
int A = 0;
p = &A;
if (++i < 3)
goto Label;
bar(A);
}
then we can use A at Label after the first iteration because p points to it. We could imagine motivation for code like this could arise in a loop that needs to treat its first iteration specially. However, I have never seen it used in practice.
For an automatic object that is a variable length array, its lifetime starts when execution reaches its declaration and ends when execution leaves the scope of the declaration. So with this code:
int N = baz();
{
int i = 0;
Label:
foo();
int A[N];
if (++i < 3)
goto Label;
}
A does not exist at Label. Its lifetime begins each time execution reaches the declaration int A[N]; and ends, in the first few iterations, when the goto Label; transfers execution out of the scope of the declaration or, in the last iteration, when execution of the block ends.
Practical Implementation
In general-purpose C implementations, automatic objects are implemented with a hardware stack. A region of memory is set aside to be used as a stack, and a particular processor register, call the stack pointer, keeps track of how much is in use, by recording the address of the current “top” of stack. (For historic reasons, stacks usually start at high addresses and grow toward lower addresses, so the logical top of a stack is at the bottom of its used addresses.)
Because the lifetimes of automatic objects other than variable length arrays start when execution of their associate blocks begins, a compiler could implement this by adjusting the stack pointer whenever entering or ending a block. This can provide memory efficiency in at least two ways. Consider this code:
if (foo(0))
{
int A[100];
bar(A, x, 0);
}
if (foo(1))
{
int B[100];
bar(B, x, 1);
}
if (foo(2))
{
int C[1000];
bar(C, x, 2);
}
Because A and B do not exist at the same time, the compiler does not have to reserve memory for both of them when the function starts. It can adjust the stack pointer when each block is entered and ended. And for the large array C, the space might never be reserved at all; a compiler could choose to allocate space for 1000 int only if the block is actually entered.
However, I do not think GCC and Clang are taking advantage of this. In practice, I think they generally figure out the maximum space will be needed at any one time in the function and allocate that much space on the stack and use it through the function. This does include optimizations like using the same space for A and B, since they are never in use at the same time, but it does not include optimizating for the possibility that C is never used. However, I could be wrong; I have not checked on this compiler behavior lately.
In the cases of variable length arrays, the compiler generally cannot plan the memory use in advance, since it does not know the array size. So, for a variable length array, space has to be reserved for it on the stack when its declaration is reached.
Additionally, note that the compiler does not have to implement the computing model the C standard uses literally. It can make any optimizations that get the same observable behavior. This means that, for example, if it can tell part of an array is not used, it does not have to allocate memory for that at all. This means the answer to your question, “… will space be allocated to the defined variable first or will it be allocated when it runs to the defined statement?”, is that a compiler designer may choose either method, as long as the observable behavior of the resulting program matches what the C standard specifies.
The observable behavior includes data written to files, input/output interactions, and accesses to volatile objects.
How does a compiler know if something is allocated on the heap or stack, for instance if I made a variable in a function and returned the address of the variable, the compiler warns me that "function returns address of a local variable":
#include <stdio.h>
int* something() {
int z = 21;
return &z;
}
int main() {
int *d = something();
return 0;
}
I understand why this is a warning because when the function exits, the stack frame is no more and if you have a pointer to that memory and you change it's value you will cause a segmentation fault. What I wonder is how the compiler will know if that variable is allocating memory via. malloc, or how it can tell if it's a local variable on the stack?
A compiler builds a syntax tree from which it is able to analyze each part of the source code.
It builds a symbol table which associates to each symbol defined some information. This is required for many aspects:
finding undeclared identifiers
checking that types are convertible
so on
Once you have this symbol table it is quite easy to know if you are trying to return the address of a local variable since you end up having a structure like
ReturnStatement
+ UnaryOperator (&)
+ Identifier (z)
So the compiler can easily check if the identifier is a local stack variable or not.
Mind that this information could in theory propagate along assignments but in practice I don't think many compilers do it, for example if you do
int* something() {
int z = 21;
int* pz = &z;
return pz;
}
The warning goes away. With static code flow analysis you could be able to prove that pz could only refer to a local variable but in practice that doesn't happen.
The example in your question is really easy to figure out.
int* something() {
int z = 21;
return &z;
}
Look at the expression in the return statement. It takes the address of the identifier z.
Find out where z is declared. Oh, it is a local variable.
Not all cases will be as easy as this one and it's likely that you can trick the compiler into giving false positives or negatives if you write sufficiently weird code.
If you're interested in this kind of stuff, you might enjoy watching some of the talks given at CppCon'15 where static analysis of C++ code was a big deal. Some remarkable talks:
Bjarne Stroustrup: “Writing Good C++14”
Herb Sutter: “Writing Good C++14… By Default”
Neil MacIntosh: “Static Analysis and C++: More Than Lint”
The compiler knows what chunk of memory is holding the current stack. Every time a function is called it creates a new stack and moves the previous frame and stack pointers appropriately which effectively give it a beginning and endpoint for the current stack in memory. Checking to see if you're trying to return a pointer to memory that's about to get freed is relatively simple given that setup.
What I wonder is how the compiler will know if that variable is
allocating memory via. malloc, or how it can tell if it's a local
variable on the stack?
The compiler has to analyse all the code and generate machine code from it.
When functions need to be called, the compiler has to push the parameters on the stack (or reserve registers for them), update the stack pointer, look if there are local variables, initialize those on the stack too and update the stack pointer again.
So obviously the compiler knows about local variables being pushed on the stack.
If I define an array in if statement then does memory gets allocated during compile time eg.
if(1)
{
int a[1000];
}
else
{
float b[1000];
}
Then a memory of 2 * 1000 for ints + 4 * 1000 for floats get allocated?
It is reserved on the stack at run-time (assuming a non-trivial condition - in your case, the compiler would just exclude the else part). That means it only exists inside the scope block (between the {}).
In your example, only the memory for the ints gets allocated on the stack (1000 * sizeof(int)).
As you can guess, this is happening at run time. The generated code has instructions to allocate the space on the stack when the corresponding block of code is entered.
Keep in mind that this is happening because of the semantics of the language. The block structure introduces a new scope, and any automatic variables allocated in that scope have a lifetime that lasts as long as the scope does. In C, this is implemented by allocating it on the stack, which collapses as the scope disappears.
Just to drive home the point, note that the allocation would be different had the variables been of different nature.
if(1)
{
static int a[1000];
}
else
{
static float b[1000];
}
In this case, space is allocated for both the ints and the floats. The lifetime of these variables is the program. But the visibility is within the block scope they are allocated in.
Scope
Variables declared inside the scope of a pair of { } are on the stack. This applies to variables declared at the beginning of a function or in any pair of { } within the function.
int myfunc()
{
int i = 0; // On the stack, scoped: myfunc
printf("%i\n");
if (1)
{
int j = 1; // On the stack, scope: this if statement
printf("%i %i\n",i,j);
}
printf("%i %i\n",i,j); // Won't work, no j
}
These days the scope of the variables is limited to the surrounding { }. I recall that some older Microsoft compilers didn't limit the scope, and that in the example above the final printf() would compile.
So Where is it in memory?
The memory of i and j is merely reserved on the stack. This is not the same as memory allocation done with malloc(). That is important, because calling malloc() is very slow in comparison. Also with memory dynamically allocated using malloc() you have to call free().
In effect the compiler knows ahead of time what space is needed for a function's variables and will generate code that refers to memory relative to whatever the stack pointer is when myfunc() is called. So long as the stack is big enough (2MBytes normally, depends on the OS), all is good.
Stack overflow occurs in the situation where myfunc() is called with the stack pointer already close to the end of the stack (i.e. myfunc() is called by a function which in turn had been called by another which it self was called by yet another, etc. Each layer of nested calls to functions moves the stack pointer on a bit more, and is only moved back when functions return).
If the space between the stack pointer and the end of the stack isn't big enough to hold all the variables that are declared in myfunc(), the code for myfunc() will simply try to use locations beyond the end of the stack. That is almost always a bad thing, and exactly how bad and how hard it is to notice that something has gone wrong depends on the operating system. On small embedded micro controllers it can be a nightmare as it usually means some other part of the program's data (eg global variables) get silently overwritten, and it can be very hard to debug. On bigger systems (Linux, Windows) the OS will tell you what's happened, or will merely make the stack bigger.
Runtime Efficiency Considerations
In the example above I'm assigning values to i and j. This does actually take up a small amount of runtime. j is assigned 1 only after evaluation of the if statement and subsequent branch into where j is declared.
Say for example the if statement hadn't evaluated as true; in that case j is never assigned 1. If j was declared at the start of myfunc() then it would always get assigned the value of 1 regardless of whether the if statement was true - a minor waste of time. But consider a less trivial example where a large array is declared an initialised; that would take more execution time.
int myfunc()
{
int i = 0; // On the stack, scoped: myfunc
int k[10000] = {0} // On the stack, scoped: myfunc. A complete waste of time
// when the if statement evaluates to false.
printf("%i\n");
if (0)
{
int j = 1; // On the stack, scope: this if statement
// It would be better to move the declaration of k to here
// so that it is initialised only when the if evaluates to true.
printf("%i %i %i\n",i,j,k[500]);
}
printf("%i %i\n",i,j); // Won't work, no j
}
Placing the declaration of k at the top of myfunc() means that a loop 10,000 long is executed to initialise k every time myfunc() is called. However it never gets used, so that loop is a complete waste of time.
Of course, in these trivial examples compilers will optimise out the unnecessary code, etc. In real code where the compiler cannot predict ahead of time what the execution flow will be then things are left in place.
Memory for the array in the if block will be allocated on stack at run time. else part will be optimized (removed) by the compiler. For more on where the variables will be allocated memory, see Segmentation Fault when writing to a string
As DCoder & paddy corrected me, the memory will be calculated at compile time but allocated at run-time in stack memory segment, but with the scope & lifetime of the block in which the array is defined. The size of memory allocated depends on size of int & float in your system. Read this for an overview on C memory map
What is the difference between the block and function scopes in C99 in terms of what happens on stack when a function / block is entered and left?
In theory, a compiler could generate code to allocate a stack frame on entry to any block that contains local variables. In such a case, there wouldn't be much difference at all.
In practice, most compilers compute the maximum size of local variables that could be used by any path through a function, then allocate that size of stack frame on entry. Variables in any block inside the function are simply different offsets from the stack pointer. Note that in such a case, two (or more) blocks may use the same addresses. For example, with source code like this:
void f(int x) {
if (x) {
long y;
}
else {
float z;
}
}
...chances are quite good that y and z will end up at the same address.
Like this:
void foo(int n) // <-- beginning of function scope
{ // <-- beginning of function body scope
int x = n;
for (;;)
{ // <-- beginning of block scope
int q = n;
x *= q;
} // <-- end of block scope
foo(x);
{ // <-- another block scope
int w = x;
}
} // <-- end of function body scope
// and of function scope
Nothing "happens" when a scope ends, but a variable only lives inside the scope where it is declared (with some arcane exceptions). It is up to the implementation to reuse the space of variables of previous, nested scopes that have ended.
The only thing an implementation is required to do when control enters either a function or a block scope is to behave as if new instances have been created of all data objects directly in that scope with "automatic storage duration." Behave as if means it can do something different as long as the program being compiled can't tell the difference (or could only tell the difference by doing something whose behavior is undefined). For instance, if a variable is declared at function scope but only used within one subblock, the compiler can collapse its live range to that subblock, and probably will, because this makes register allocation easier.
An implementation is not required to do anything when control exits a function or block scope. The lifetimes of all automatic-storage-duration objects directly in that scope end, but no program can tell that this has happened without triggering undefined behavior.
There is no requirement for a C implementation to have a stack, and a stack is not the only way to implement the above requirements. See for instance "Cheney on the M.T.A." and c2:SpaghettiStack.
C implementations that do have a stack will normally try to avoid adjusting the stack pointer in the middle of a function, for reasons too complicated to go into here. This can mean that a value with block scope survives on the stack longer than its declared lifetime, but it's still undefined behavior to access it. The compiler is allowed to recycle storage for values that are no longer in scope, but it is also allowed to recycle storage for values that are still in scope but will not be accessed anymore ("dead" in compiler jargon). Historically compilers have been much more aggressive about doing that for values in registers than for values in stack slots, but again, that's a distinction that doesn't necessarily exist on your implementation.
As we know, local variables have local scope and lifetime. Consider the following code:
int* abc()
{
int m;
return(&m);
}
void main()
{
int* p=abc();
*p=32;
}
This gives me a warning that a function returns the address of a local variable.
I see this as justification:
Local veriable m is deallocated once abc() completes. So we are dereferencing an invalid memory location in the main function.
However, consider the following code:
int* abc()
{
int m;
return(&m);
int p=9;
}
void main()
{
int* p=abc();
*p=32;
}
Here I am getting the same warning. But I guess that m will still retain its lifetime when returning. What is happening? Please explain the error. Is my justification wrong?
First, notice that int p=9; will never be reached, so your two versions are functionally identical. The program will allocate memory for m and return the address of that memory; any code below the return statement is unreacheable.
Second, the local variable m is not actually de-allocated after the function returns. Rather, the program considers the memory free space. That space might be used for another purpose, or it might stay unused and forever hold its old value. Because you have no guarantee about what happens to the memory once the abc() function exits, you should not attempt to access or modify it in any way.
As soon as return keyword is encountered, control passes back to the caller and the called function goes out of scope. Hence, all local variables are popped off the stack. So the last statement in your second example is inconsequential and the warning is justified
Logically, m no longer exists when you return from the function, and any reference to it is invalid once the function exits.
Physically, the picture is a bit more complicated. The memory cells that m occupied are certainly still there, and if you access those cells before anything else has a chance to write to them, they'll contain the value that was written to them in the function, so under the right circumstances it's possible for you to read what was stored in m through p after abc has returned. Do not rely on this behavior being repeatable; it is a coding error.
From the language standard (C99):
6.2.4 Storage durations of objects
...
2 The lifetime of an object is the portion of program execution during which storage is
guaranteed to be reserved for it. An object exists, has a constant address,25) and retains
its last-stored value throughout its lifetime.26) If an object is referred to outside of its
lifetime, the behavior is undefined. The value of a pointer becomes indeterminate when
the object it points to reaches the end of its lifetime.
25) The term ‘‘constant address’’ means that two pointers to the object constructed at possibly different
times will compare equal. The address may be different during two different executions of the same
program.
26) In the case of a volatile object, the last store need not be explicit in the program.
Emphasis mine. Basically, you're doing something that the language definition explicitly calls out as undefined behavior, meaning the compiler is free to handle that situation any way it wants to. It can issue a diagnostic (which your compiler is doing), it can translate the code without issuing a diagnostic, it can halt translation at that point, etc.
The only way you can make m still valid memory (keeping the maximum resemblance with your code) when you exit the function, is to prepend it with the static keyword
int* abc()
{
static int m;
m = 42;
return &m;
}
Anything after a return is a "dead branch" that won't be ever executed.
int m should be locally visible. You should create it as int* m and return it directly.