I came across this [question]: How can I store values into a multi-parameter struct and pass typedef struct to a function?.
I suggested declaring a variable of dtErrorMessage once before the if and use it for both conditions instead of declaring two different variables as shown here:
// Initial version
if(GetTemperature() <= -20)
{
dtErrorMessage low;
low.errorCode=ERROR_CODE_UNDER_TEMP;
low.temperature=GetTemperature();
SendErrorReport(&low);
}
if(GetTemperature() >= 160)
{
dtErrorMessage high;
high.errorCode=ERROR_CODE_OVER_TEMP;
high.temperature=GetTemperature();
SendErrorReport(&high);
}
// My version
dtErrorMessage err;
int8_t temp = GetTemperature(); // Do not consider change during evaluation
if(temp <= -20)
{
err.errorCode = ERROR_CODE_UNDER_TEMP;
err.temperature = temp;
SendErrorReport(&err);
}
else if(temp >= 160)
{
err.errorCode = ERROR_CODE_OVER_TEMP;
err.temperature = temp ;
SendErrorReport(&err);
}
else
{
// Do not send error report
}
My question is: (Under the embedded aspect,) am I right, that this way there will be two local variables created in RAM regardless of the condition? Consequently, it would reduce the reqired RAM using one unconditional variable declaration before the if and then using it for both conditions, right?
I couldn't find the correct terms to search for getting this answered myself.
The lifetime of a variable with automatic storage duration is until the end of the block. Storage will be guaranteed and it will retain constant address (i.e. the address given by e.g. &) until the end of the block, and after that all pointers to that object will become indeterminate.
The C standard does not say that low and high must occupy the same portion of memory! The storage must be guaranteed until the end of the block containing the declaration, but it can be around longer too. On the other hand the as-if rule says that the program needs to only behave as if it was a program compiled for the abstract machine according to the rules of the C standard, so a high-quality implementation would probably
not reserve memory for both low and high simultaneously at different addresses, and
not reserve memory for err in case the else branch is taken.
In effect making the execution behaviour of these 2 virtually identical.
The difference is mostly stylistic: in your version there is less to type, and the original version accommodates for bad compilers.
Related
If I define a variable anywhere in a function (not at the beginning), when the program is compiled and executed to this function, will space be allocated to the defined variable first or will it be allocated when it runs to the defined statement?
If it runs in order, Will it reduce some overhead when problems arise?
like so:
if(!(Size && Packs))
{
ret = false;
return SendAck(ret);
}
uint8_t *pSrc = (uint8_t *)pRcv->data;
uint8_t crc = 0;
The Standard C Computing Model
The C standard specifies memory reservation requirements in terms of storage duration and lifetime. Objects declared inside functions without extern, static, or _Thread_local have automatic storage duration. (Other storage durations, not discussed here, are static, thread, allocated, and temporary.) This includes parameters of functions, since they are declared inside the function declaration.
Each declaration inside a function has an associated block. Blocks are groups of statements (sometimes just a single statement). A compound statement bracketed with { and } is a block, each selection statement (if, switch) and each loop statement (for, while, do) is a block, and each substatement of those statements is a block. The block associated with a declaration inside a function is the innermost block it is in. For a function parameter, its associated block is the compound statement that defines the function.
For an automatic object that is not a variable length array, its lifetime starts when execution enters the block it is in, and it ends when execution of the block ends. (Calling a function suspends execution of the block b ut does not end it.) So in:
{
Label:
foo();
int A;
}
A exists as soon as execution reaches Label, because execution of A’s block has started.
This means that, as soon as the block is entered, all automatic objects in it other than variable length arrays should have memory reserved for them.
This fact is generally of little use, as there is no way to refer to A at Label. However, if we do this:
{
int i = 0;
int *p;
Label:
if (0 < i)
*p += foo();
int A = 0;
p = &A;
if (++i < 3)
goto Label;
bar(A);
}
then we can use A at Label after the first iteration because p points to it. We could imagine motivation for code like this could arise in a loop that needs to treat its first iteration specially. However, I have never seen it used in practice.
For an automatic object that is a variable length array, its lifetime starts when execution reaches its declaration and ends when execution leaves the scope of the declaration. So with this code:
int N = baz();
{
int i = 0;
Label:
foo();
int A[N];
if (++i < 3)
goto Label;
}
A does not exist at Label. Its lifetime begins each time execution reaches the declaration int A[N]; and ends, in the first few iterations, when the goto Label; transfers execution out of the scope of the declaration or, in the last iteration, when execution of the block ends.
Practical Implementation
In general-purpose C implementations, automatic objects are implemented with a hardware stack. A region of memory is set aside to be used as a stack, and a particular processor register, call the stack pointer, keeps track of how much is in use, by recording the address of the current “top” of stack. (For historic reasons, stacks usually start at high addresses and grow toward lower addresses, so the logical top of a stack is at the bottom of its used addresses.)
Because the lifetimes of automatic objects other than variable length arrays start when execution of their associate blocks begins, a compiler could implement this by adjusting the stack pointer whenever entering or ending a block. This can provide memory efficiency in at least two ways. Consider this code:
if (foo(0))
{
int A[100];
bar(A, x, 0);
}
if (foo(1))
{
int B[100];
bar(B, x, 1);
}
if (foo(2))
{
int C[1000];
bar(C, x, 2);
}
Because A and B do not exist at the same time, the compiler does not have to reserve memory for both of them when the function starts. It can adjust the stack pointer when each block is entered and ended. And for the large array C, the space might never be reserved at all; a compiler could choose to allocate space for 1000 int only if the block is actually entered.
However, I do not think GCC and Clang are taking advantage of this. In practice, I think they generally figure out the maximum space will be needed at any one time in the function and allocate that much space on the stack and use it through the function. This does include optimizations like using the same space for A and B, since they are never in use at the same time, but it does not include optimizating for the possibility that C is never used. However, I could be wrong; I have not checked on this compiler behavior lately.
In the cases of variable length arrays, the compiler generally cannot plan the memory use in advance, since it does not know the array size. So, for a variable length array, space has to be reserved for it on the stack when its declaration is reached.
Additionally, note that the compiler does not have to implement the computing model the C standard uses literally. It can make any optimizations that get the same observable behavior. This means that, for example, if it can tell part of an array is not used, it does not have to allocate memory for that at all. This means the answer to your question, “… will space be allocated to the defined variable first or will it be allocated when it runs to the defined statement?”, is that a compiler designer may choose either method, as long as the observable behavior of the resulting program matches what the C standard specifies.
The observable behavior includes data written to files, input/output interactions, and accesses to volatile objects.
I'm trying to understand some things about structures and memory management. I have three versions code whose result is identical. They only differ in the definition/allocation of the structure. In version one, the structure is allocated with this code: resultValue_t *resultVal = malloc(sizeof(resultValue_t));. In version two, the structure is created inside the function: resultValue_t resultVal;. In version three resultValue_t resultVal; is created globally.
I would like to ask how the three definitions actually differ? Which one gives which possibilities? And which one is more appreciated?
I know that if a structure is accessed via a pointer, the -> operator should be used, etc. But here I'm curious about the difference and possibilities between these definitions.
//version_1
typedef struct
{
int valOne;
int valTwo;
}resultValue_t;
void twoSum(int *array, int sum)
{
int baseVal, tempSum;
int arrLen = sizeof(array);
resultValue_t *resultVal = malloc(sizeof(resultValue_t));
//...
if (tempSum == sum)
{
resultVal->valOne = array[index];
resultVal->valTwo = array[nIndex];
printf("%d + %d\n", resultVal->valOne, resultVal->valTwo);
}
//...
free(resultVal);
}
//version_2
typedef struct
{
int valOne;
int valTwo;
}resultValue_t;
void twoSum2(int *array, int sum)
{
int baseVal, tempSum;
resultValue_t resultVal;
//...
if (tempSum == sum)
{
resultVal.valOne = array[index];
resultVal.valTwo = array[nIndex];
printf("%d + %d\n", resultVal.valOne, resultVal.valTwo);
}
//...
}
//version_3
typedef struct
{
int valOne;
int valTwo;
}resultValue_t;
resultValue_t resultVal;
The difference is "storage duration", i.e. the lifetime of object.
In version 1 "allocated storage duration" is used. That means that the object resultVal exists after malloc returns and until some code explicit "kills it" by calling free. On most systems memory is allocated in an area called heap. The allocation is typically slow (i.e. don't use it unless there is a good reason - see later).
In version 2 "automatic storage duration" is used. That means that the object resultVal exists until execution of the associated block ends. In this case until the function returns. In other words - in this case the object resultVal is automatically created when execution enters the function and automatically destroyed when execution leaves the function.
On most systems memory is allocated in an area called stack. The allocation is typically very fast.
In version 3 "static storage duration" is used. That means that the object resultVal exists from execution start to execution end. So the object can be accessed at any time from anywhere in the code. Memory is typically allocated in some special area as part of program start up and it's also very fast.
Obviously there are use cases for all 3 types.
Version 3 shall in general be avoided but there are special cases where static objects make sense. When used it's typically good to limit their use to "file scope" using the keyword static.
In most cases you should go for version 2 as it is fast and secure. However, if the size of the object (in this case resultVal) is very big, version 2 should be avoided due to risk of stack overflow. In that case use version 1.
Version 1 is also useful (actually required) if you want the object to be "alive" after the function returns. That however is not the case for the example in the question as free is called before leaving the function.
Conclusion:
Since resultValue_t; is a rather small object type and the object resultVal doesn't need to be "alive" after the function call, you should go for version 2.
Version 1 allocates memory from the heap.
Version 2 allocates memory from the stack.
Version 3 uses memory from the bss section (In every compiler implementation I've used, but it could come from elsewhere). It's also the only version that initializes the object (in this case, to all zeroes).
Depended of the version of the C compiler and compiler flags it is possible to initialize variables on any place in your functions (As far as I am aware).
I'm used to put it all the variables at the top of the function, but the discussion started about the memory use of the variables if defined in any other place in the function.
Below I have written 2 short examples, and I wondered if anyone could explain me (or verify) how the memory gets allocated.
Example 1: Variable y is defined after a possible return statement, there is a possibility this variable won't be used for that reason, as far as I'm aware this doesn't matter and the code (memory allocation) would be the same if the variable was placed at the top of the function. Is this correct?
Example 2: Variable x is initialized in a loop, meaning that the scope of this variable is only within this loop, but what about the memory use of this variable? Would it be any different if placed on the top of the functions? Or just initialized on the stack at the function call?
Edit: To conclude a main question:
Does reducing the scope of the variable or change the location of the first use (so anywhere else instead of top) have any effects on the memory use?
Code example 1
static void Function(void){
uint8_t x = 0;
//code changing x
if(x == 2)
{
return;
}
uint8_t y = 0;
//more code changing y
}
Code example 2
static void LoopFunction(void){
uint8_t i = 0;
for(i =0; i < 100; i ++)
{
uint8_t x = i;
// do some calculations
uartTxLine("%d", x);
}
//more code
}
I'm used to put it all the variables at the top of the function
This used to be required in the older versions of C, but modern compilers dropped that requirement. As long as they know the type of the variable at the point of its first use, the compilers have all the information they need.
I wondered if anyone could explain me how the memory gets allocated.
The compiler decides how to allocate memory in the automatic storage area. Implementations are not limited to the approach that gives each variable you declare a separate location. They are allowed to reuse locations of variables that go out of scope, and also of variables no longer used after a certain point.
In your first example, variable y is allowed to use the space formerly occupied by variable x, because the first point of use of y is after the last point of use of x.
In your second example the space used for x inside the loop can be reused for other variables that you may declare in the // more code area.
Basically, the story goes like this. When calling a function in raw assembler, it is custom to store everything used by the function on the stack upon entering the function, and clean it up upon leaving. Certain CPUs and ABIs may have a calling convention which involves automatic stacking of parameters.
Likely because of this, C and many other old languages had the requirement that all variables must be declared at the top of the function (or on top of the scope), so that the { } reflect push/pop on the stack.
Somewhere around the 80s/90s, compilers started to optimize such code efficiently, as in they would only allocate room for a local variable at the point where it was first used, and de-allocate it when there was no further use for it. Regardless of where that variable was declared - it didn't matter for the optimizing compiler.
Around the same time, C++ lifted the variable declaration restrictions that C had, and allowed variables to be declared anywhere. However, C did not actually fix this before the year 1999 with the updated C99 standard. In modern C you can declare variables everywhere.
So there is absolutely no performance difference between your two examples, unless you are using an incredibly ancient compiler. It is however considered good programming practice to narrow the scope of a variable as much as possible - though it shouldn't be done at the expense of readability.
Although it is only a matter of style, I would personally prefer to write your function like this:
(note that you are using the wrong printf format specifier for uint8_t)
#include <inttypes.h>
static void LoopFunction (void)
{
for(uint8_t i=0; i < 100; i++)
{
uint8_t x = i;
// do some calculations
uartTxLine("%" PRIu8, x);
}
//more code
}
Old C allowed only to declare (and initialize) variables at the top of a block. You where allowed to init a new block (a pair of { and } characters) anywhere inside a block, so you had then the possibility of declaring variables next to the code using them:
... /* inside a block */
{ int x = 3;
/* use x */
} /* x is not adressabel past this point */
And you where permitted to do this in switch statements, if statements and while and do statements (everywhere where you can init a new block)
Now, you are permitted to declare a variable anywhere where a statement is allowed, and the scope of that variable goes from the point of declaration to the end of the inner nested block you have declared it into.
Compilers decide when they allocate storage for local variables so, you can all of them allocated when you create a stack frame (this is the gcc way, as it allocates local variables only once) or when you enter in the block of definition (for example, Microsoft C does this way) Allocating space at runtime is something that requires advancing the stack pointer at runtime, so if you do this only once per stack frame you are saving cpu cycles (but wasting memory locations). The important thing here is that you are not allowed to refer to a variable location outside of its scoping definition, so if you try to do, you'll get undefined behaviour. I discovered an old bug for a long time running over internet, because nobody take the time to compile that program using Microsoft-C compiler (which failed in a core dump) instead of the commmon use of compiling it with GCC. The code was using a local variable defined in an inner scope (the then part of an if statement) by reference in some other part of the code (as everything was on main function, the stack frame was present all the time) Microsoft-C just reallocated the space on exiting the if statement, but GCC waited to do it until main finished. The bug solved by just adding a static modifier to the variable declaration (making it global) and no more refactoring was neccesary.
int main()
{
struct bla_bla *pointer_to_x;
...
if (something) {
struct bla_bla x;
...
pointer_to_x = &x;
}
/* x does not exist (but it did in gcc) */
do_something_to_bla_bla(pointer_to_x); /* wrong, x doesn't exist */
} /* main */
when changed to:
int main()
{
struct bla_bla *pointer_to_x;
...
if (something) {
static struct bla_bla x; /* now global ---even if scoped */
...
pointer_to_x = &x;
}
/* x is not visible, but exists, so pointer_to_x continues to be valid */
do_something_to_bla_bla(pointer_to_x); /* correct now */
} /* main */
I have a piece of memory I am "guarding", defined by
typedef unsigned char byte;
byte * guardArea;
size_t guardSize;
byte * guardArea = getGuardArea();
size_t guardSize = getGuardSize();
An acceptable implementation for the sake of this would be:
size_t glGuardSize = 1024; /* protect an area of 1kb */
byte * getGuardArea()
{
return malloc( glGuardSize );
}
size_t getGuardSize()
{
return glGuardSize;
}
Can the following snippet return true for any pointer (from a different malloc, from the stack etc)?
if ( ptr >= guardArea && ptr < (guardArea + guardSize)) {
return true;
}
The standard states that:
values within the area will return true. (When ptr was a member, all acts correctly.)
pointers will be distinct (a == b only if they are the same).
all addresses within the byte array can be accessed by incrementing the base.
any pointer can be converted to and from a char *, without damage.
So I can't understand how the result could be true for any pointer from a different object (as it would break the distinct rule for one of the pointers within the area).
Edit:
What is the use case?
The ability to detect whether a pointer is within a region is really important, at some point code is written
if ( isInMyAreaOfInterest( unknownPointer ) ) {
doMySpecialThing( unknownPointer );
} else {
doSomethingElse( unknownPointer );
}
I think the language needs to support the developer by making such constructs simple and obvious, and our interpretation of the standard, is that the developer needs to cast to int. Due to the "undefined behavior" of pointer comparisons of distinct objects.
I was hoping for some clarity of why I can't do what I would like (my snippet), as all the posts on SO I found say that the standard claims undefined behavior, without any explanation, or examples of why the standard is better than how I would like it to work.
At the moment, we have a rule, we are neither understanding why the rule exists, or questioning if the rule is helping us
Example posts:
SO: checking if a pointer is in a malloced area
SO: C compare pointers
It is still possible for an allocation to generate a pointer that satisfies the condition despite the pointer not pointing into the region. This will happen, for example, on an 80286 in protected mode, which is used by Windows 3.x in Standard mode and OS/2 1.x.
In this system, pointers are 32-bit values, split into two 16-bit parts, traditionally written as XXXX:YYYY. The first 16-bit part (XXXX) is the "selector", which chooses a bank of 64KB. The second 16-bit part (YYYY) is the "offset", which chooses a byte within that 64KB bank. (It's more complicated than this, but let's just leave it at that for the purpose of this discussion.)
Memory blocks larger than 64KB are broken up into 64KB chunks. To move from one chunk to the next, you add 8 to the selector. For example, the byte after 0101:FFFF is 0109:0000.
But why do you add 8 to move to the next selector? Why not just increment the selector? Because the bottom three bits of the selector are used for other things.
In particular, the bottom bit of the selector is used to choose the selector table. (Let's ignore bits 1 and 2 since they are not relevant to the discussion. Assume for convenience that they are always zero.)
There are two selector tables, the Global Selector Table (for memory shared across all processes) and the Local Selector Table (for memory private to a single process). Therefore, the selectors available for process private memory are 0001, 0009, 0011, 0019, etc. Meanwhile, the selectors available for global memory are 0008, 0010, 0018, 0020, etc. (Selector 0000 is reserved.)
Okay, now we can set up our counter-example. Suppose guardArea = 0101:0000 and guardSize = 0x00020000. This means that the guarded addresses are 0101:0000 through 0101:FFFF and 0109:0000 through 0109:FFFF. Furthermore, guardArea + guardSize = 0111:0000.
Meanwhile, suppose there is some global memory that happens to be allocated at 0108:0000. This is a global memory allocation because the selector is an even number.
Observe that the global memory allocation is not part of the guarded region, but its pointer value does satisfy the numeric inequality 0101:0000 <= 0108:0000 < 0111:0000.
Bonus chatter: Even on CPU architectures with a flat memory model, the test can fail. Modern compilers take advantage of undefined behavior and optimize accordingly. If they see a relational comparison between pointers, they are permitted to assume that the pointers point into the same array (or one past the last element of that array). Specifically, the only pointers that can legally be compared with guardArea are the ones of the form guardArea, guardArea+1, guardArea+2, ..., guardArea + guardSize. For all of these pointers, the condition ptr >= guardArea is true and can therefore be optimized out, reducing your test to
if (ptr < (guardArea + guardSize))
which will now be satisfied for pointers that are numerically less than guardArea.
Moral of the story: This code is not safe, not even on flat architectures.
But all is not lost: The pointer-to-integer conversion is implementation-defined, which means that your implementation must document how it works. If your implementation defines the pointer-to-integer conversion as producing the numeric value of the pointer, and you know that you are on a flat architecture, then what you can do is compare integers rather than pointers. Integer comparisons are not constrained in the same way that pointer comparisons are.
if ((uintptr_t)ptr >= (uintptr_t)guardArea &&
(uintptr_t)ptr < (uintptr_t)guardArea + (uintptr_t)guardSize)
Yes.
void foo(void) {}
void(*a) = foo;
void *b = malloc(69);
uintptr_t ua = a, ub = b;
ua and ub are in fact permitted to have the same value. This occurred frequently on segmented systems (like MS-DOS) which might put code and data in separate segments.
Code Snippet:
int secret_foo(void)
{
int key = get_secret();
/* use the key to do highly privileged stuff */
....
/* Need to clear the value of key on the stack before exit */
key = 0;
/* Any half decent compiler would probably optimize out the statement above */
/* How can I convince it not to do that? */
return result;
}
I need to clear the value of a variable key from the stack before returning (as shown in the code).
In case you are curious, this was an actual customer requirement (embedded domain).
You can use volatile (emphasis mine):
Every access (both read and write) made through an lvalue expression of volatile-qualified type is considered an observable side effect for the purpose of optimization and is evaluated strictly according to the rules of the abstract machine (that is, all writes are completed at some time before the next sequence point). This means that within a single thread of execution, a volatile access cannot be optimized out or reordered relative to another visible side effect that is separated by a sequence point from the volatile access.
volatile int key = get_secret();
volatile might be overkill sometimes as it would also affect all the other uses of a variable.
Use memset_s (since C11): http://en.cppreference.com/w/c/string/byte/memset
memset may be optimized away (under the as-if rules) if the object modified by this function is not accessed again for the rest of its lifetime. For that reason, this function cannot be used to scrub memory (e.g. to fill an array that stored a password with zeroes). This optimization is prohibited for memset_s: it is guaranteed to perform the memory write.
int secret_foo(void)
{
int key = get_secret();
/* use the key to do highly privileged stuff */
....
memset_s(&key, sizeof(int), 0, sizeof(int));
return result;
}
You can find other solutions for various platforms/C standards here: https://www.securecoding.cert.org/confluence/display/c/MSC06-C.+Beware+of+compiler+optimizations
Addendum: have a look at this article Zeroing buffer is insufficient which points out other problems (besides zeroing the actual buffer):
With a bit of care and a cooperative compiler, we can zero a buffer — but that's not what we need. What we need to do is zero every location where sensitive data might be stored. Remember, the whole reason we had sensitive information in memory in the first place was so that we could use it; and that usage almost certainly resulted in sensitive data being copied onto the stack and into registers.
Your key value might have been copied into another location (like a register or temporary stack/memory location) by the compiler and you don't have any control to clear that location.
If you go with dynamic allocation you can control wiping that memory and not be bound by what the system does with the stack.
int secret_foo(void)
{
int *key = malloc(sizeof(int));
*key = get_secret();
memset(key, 0, sizeof(int));
// other magical things...
return result;
}
One solution is to disable compiler optimizations for the section of the code that you dont want optimizations:
int secret_foo(void) {
int key = get_secret();
#pragma GCC push_options
#pragma GCC optimize ("O0")
key = 0;
#pragma GCC pop_options
return result;
}