This question already has answers here:
Order of local variable allocation on the stack
(10 answers)
Closed 4 years ago.
I'm quite confused about how the local variables are ordered on the stack. I understand, that (on Intel x86) the local variables are stored from higher to lower address as they go in the code. So it's clear, that this code:
int i = 0;
char buffer[4];
strcpy(buffer, "aaaaaaaaaaaaaaa");
printf("%d", i);
produces something like this:
1633771873
The i variable was overwritten by the overflowed buffer.
However, if I swap the first two lines:
char buffer[4];
int i = 0;
strcpy(buffer, "aaaaaaaaaaaaaaa");
printf("%d", i);
the output is absolutely same.
How is it possible? The i's address is lower than the buffer's one and so an overflow of the buffer should overwrite other data, but not i. Or am I missing something?
There is no rule about the order of local variables, so the compiler is generally free to allocate them the way it likes. But on the other hand there are many strategies that a compiler will use to reduce the possibility that could happen what you are voluntarily trying to do.
One of those safety enhancement would be to allocate a buffer always far from other scalar variables because an array can be addressed out of bounds and be more incline to bloat adjacent variables. Another trick is to add some trap empty space after arrays to create a kind of isolation for the bounds problem.
Anyway you can use the debugger to have a look to the assembly for confirmation of variables positioning.
If you want to look at how the local variables are allocated by the compiler try compiling with gcc -S which will output the assembly code. On the assembly code you can see how the compiler has chosen to order the variables.
One thing to keep in mind in how the compiler chooses to order local variables is that each char only needs to align by 1 (which means that it can start at any byte of memory), on the other hand the int has to align by 4 (which means that it can only start on a byte evenly divisible by 4), so depending on the alignment the compiler has it's own logic on how to avoid having empty bytes of data which means that it often groups together variables of similar type in a certain order. So even if you define them like this:
int a;
char c;
int b;
char d;
It is likely that the compiler has grouped together the ints and chars in memory so the memory going from low memory on top to high memory on bottom might look something like:
low memory
| | | char d | char c|
| int b |
| int a |
high memory
each block of || represents one byte and an entire line represents 4 bytes.
Try messing around with the assembly code sometime it is pretty interesting.
Related
I have a question regarding memory allocation on the basic C program.
#include <stdio.h>
int main()
{
int iarray[3];
char carray[3];
printf("%p\n", &iarray); // b8
printf("%p\n", &iarray+1); // c4
printf("%p\n", &carray); // c5
return 0;
}
given the code above, you can see that &iarray+1 and &carray have a difference of one byte which I'm not sure for which purpose or what in it, why does the compiler assign an unused byte between the two arrays?
I thought maybe it uses to know the array size, but I understood that sizeof is a compile-time function that knows the size without allocation of real memory, so there no use for storing the array size
Note: The output can be seen on the comments of each printf.
b8
c4
c5
Playground: https://onlinegdb.com/cTdzccpDvI
Thanks.
Compilers are free to arrange variables in memory any way that they see fit. Typically, they will be placed at memory offsets whose value is a multiple of the variable's size, for example a 4 byte int or int array will start at an address which is a multiple of 4.
In this case, you have an int array starting at an address which is a multiple of 4, followed by an unused byte, followed by a char array of size 3. In theory, an int or long could immediately follow the char array in memory if it was defined as the next available address is a multiple of 8.
From your output it looks like the stack is arranged like this for these local variables:
b8-bb: 1st integer of iarray
bc-bf: 2nd integer of iarray
c0-c3: 3rd integer of iarray
c4: padding probably, only compiler knows
c5-c7: carray
Now when you do &iarray+1 You are taking the address of an array int[3], and adding +1 of that array type to it. In other words, you are getting the address of the next int[3] array, which indeed would be at c4 (but isn't because there's just one int[3]).
This code is actually valid. You must not dereference this pointer, but because it points exactly +1 past the iarray, having the pointer and printing its value is legal (in other words, not Undefined Behavior, like &iarray+2 would be).
If you also print this:
printf("%p\n", iarray+1);
You should get result bc, because now you take pointer of type int (iarray is treated as pointer to int), add 1 to that, getting the next int.
The reason this happens is the particular compiler you are using allocates memory from high addresses to low in this particular situation.
The compiler analyzes the main routine and see it needs 3 int for iarray and 3 char for carray. For whatever reason, it decides to work on carray first.
The compiler starts with a planned stack frame that is required to have 16-byte alignment at certain points. Additional data is needed on the stack with the result that the point where the compiler starts putting local variables is at an address that is 8 modulo 16 (so its hexadecimal representation ends in 8). That is, from some address like 7FFC565E90A8 and up, memory is used for managing the stack frame. The first bytes for local objects will be at 7FFC565E90A7, 7FFC565E90A6, 7FFC565E90A5, 7FFC565E90A4, and so on.
The compiler takes the first three bytes of that space for carray. Recall we are working from high addresses to low addresses. (For historical reasons, that is the direction that stacks grow; some high address is assigned as the starting point, and new data is put in lower addresses.) So carray is put at address 7FFC565E90A5. It fills the bytes at 7FFC565E90A5, 7FFC565E90A6, and 7FFC565E90A7.
Then the compiler needs to assign 12 bytes for iarray. The next available 12 bytes are from 7FFC565E9099 to 7FFC565E90A4. However, the int elements in iarray require 4-byte alignment, so they cannot start at 7FFC565E9099. Therefore, the compiler adjusts to have them start at 7FFC565E9098. Then iarray fills bytes from 7FFC565E9098 to 7FFC565E90A3, and 7FFC565E90A4 is unused.
Note that in other situations, the compiler may arrange local objects in different ways. When you have multiple objects with different alignments, the compiler may choose to cluster all objects with the same alignment to reduce the number of places it needs to insert padding. A compiler could also choose to allocate memory for objects in alphabetical order by their names. Or it could do it in the order it happens to store them in its hash table. Or some combination of these things, such as clustering all objects by alignment requirement but then sorting by name within each cluster.
This behavior is purely (compiler) implementation defined. What probably happens is this:
When a function (main() in this case) is invoked which has local variables, memory for those variables are allocated on the stack. In this case, 15 bytes are needed, but it is likely that 4-byte alignment is required for the stack allocation, so that 16 bytes are allocated.
It is also likely that the int-array must be 4-byte aligned. Hence the address of the int array is a multiple of 4. The char-array does not have any alignment requirements so it can be placed anywhere in the 4 remaining bytes.
So in short, the additional byte is unused, but allocated due to alignment.
In order to understand Buffer Overflow Vulnerabilities, i'm looking for some clarification around what the stack actually looks when you declare an array. For the following code:
int main()
{
int a = 0;
char b[8];
char c[8];
I understand that the stack will look like:
4 bytes a, 8 bytes b, 8 bytes c
but if the code instead is:
int main()
{
int a = 0;
int b[8];
char c[8]
What does the stack look like? I'm thinking that it will be 4 bytes a, then 32 (8*4) bytes b, and then 8 bytes for c. However, i'm not entirely sure how the indexing would work at that point (is it b[0] closest to a, or b[7]?).
Any clarification would help, thanks!
First of all, you may not assume that any specific sequence will be used when placing variables on the stack. The compiler may choose the sequence. This happens often when optimizations are used. In this case the compiler may not allocate space on the stack until the first time the variable is referenced.
Additionally, the compiler may opt not to use the stack at all, if the variable is short-lived, and may store the variable in a CPU register instead.
In terms of how much memory is needed, it will depend on the system. Some systems may allow allocation of 8 bits at a time. Some systems may force everything to 32 bits of width. Some systems may add padding to make sure that larger width variables align properly.
I know that the global variable is stored in memory, and the local variable is saved in the stack, and the stack will be destroyed after each use.
But I do not know how an array is saved in memory.
I tried to declare a global array:
int tab[5]={10,9,12,34,30};
and at the end I read the contents of the memory, I mean, that after the execution of the code, I read the contents of the memory, (e.g I'm working on a microcontroller, and I know where the data is saving) when I declare a global variable for example a = 10; and when I read the contents of the memory I find the value 10 in the memory, but I do not find the content of the table that is 10,9,12,34,30
I want to understand where the array content is it save in memory?
I work on Aurix Infineon, and I use Hightec as a compiler, I execute my code directly on the aurix, I read the memorition like this:
const volatile unsigned char * mem_start = 0xd0000000;
#define size ((ptrdiff_t) 0xfff)
unsigned char bufferf [size];
code ();
main(){
...
for (int e = 0; e < sizeof (bufferf); e ++)
bufferf [e] = * (mem_start + e); // read memory
}
The answer to the question of where arrays, or indeed any variables, are stored depends on whether you are considering the abstract machine or the real hardware. The C standard specifies how things work on the abstract machine, but a conforming compiler is free to do something else on the real hardware (e.g., due to optimization) if the observable effects are the same.
(On the abstract machine, arrays are generally stored in the same place as other variables declared in the same scope would be.)
For example, a variable might be placed in a register instead of the stack, or it might be optimized away completely. As a programmer you generally shouldn't care about this, since you can also just consider the abstract machine. (Admittedly there may be some cases where you do care about it, and on microcontrollers with very limited RAM one reason might be that you have to be very frugal about using the stack, etc.)
As for your code for reading the memory: it cannot possibly work. If size is the size of the memory available to variables, you cannot fit the array bufferf[size] in that memory together with everything else.
Fortunately, copying the contents of the memory to a separate buffer is not needed. Consider your line bufferf[e] = *(mem_start + e) – since you can already read an arbitrary index e from memory at mem_start, you can use *(mem_start + e) (or, better, mem_start[e], which is exactly equivalent) directly everywhere you would use bufferf[e]! Just treat mem_start as pointing to the first element of an array of size bytes.
Also note that if you are programmatically searching for the contents of the array tab, its elements are ints, so they are more than one byte each – you won't simply find five adjacent bytes with those values.
(Then again, you can also just take the address of tab[0] and find out where it is stored, i.e., ((unsigned char *) tab) - mem_start is the index of tab in mem_start. However, here observing it may change things due to the aforementioned optimization.)
global variable is stored in memory, and the local variable is saved in the stack
This is false.
Global variables are sometimes kept in registers, not only in static memory.
If the function is recursive the compiler may choose to use the stack or not. If the function is tail recursive it is no need to use the stack as there is no continuation after the function returns and one can use the current frame for the next call.
There are mechanical methods to convert the code in continuation passing style and in this equivalent form it is evaluated stackless.
There are lots of mathematical models of computation and not all use the stack. The code can be converted from one model to other keeping the same result after evaluation.
You got these informations from books written in the 70s, 80s and in the meantime the process of evaluation of code was much improved (using methods that were theoretical in the 30s but nowadays are implemented in systems).
This program may get you closer to what you want:
int tab[] = {10, 9, 12, 34, 30};
// Used to get the number of elements in an array
#define ARRAY_SIZE(array) \
(sizeof(array) / sizeof(array[0]))
// Used to suppress compiler warnings about unused variables
#define UNUSED(x) \
(void)(x)
int main(void) {
int *tab_ptr;
int tab_copy[ARRAY_SIZE(tab)];
tab_ptr = tab;
UNUSED(tab_ptr);
for (int index = 0; index < ARRAY_SIZE(tab_copy); index++) {
tab_copy[index] = tab[index];
}
return 0;
}
I don't have the Hightec environment to test this, nor do I have an Aurix to run it on. However, this may give you a starting point that helps you obtain the information you desire.
It looks like the Infineon Aurix is a 32-bit little-endian machine. This means that an int will be 32-bits wide and the first byte will store the first 8 bits of the value. For example, the value 10 would be stored like this:
0a 00 00 00
If you try to read this memory as char, you will get 0x0a, 0x00, 0x00, and 0x00.
I'm trying to calculate stack memory used in my program.
should i add 4 for each integer i have defined?
what about something like char str[128], shoudld i add 128 or 129?
#define ARRAY1_LIMIT 200
#define ARRAY2_LIMIT 100
char* array1[ARRAY1_LIMIT];
char* array2[ARRAY2_LIMIT];
int i = 0;
int j = 0
array1[i] = (char *)malloc(sizeof(char)*5);
array2[j] = (char *)malloc(sizeof(char)*10);
I know that the heap memory is 5+15 = 15, but i don't know how to calculate the stack memory? is it 200 + 100?
If you're doing anything more than simply trying to learn about stack usage and how variable allocations affect it, you'll want to use a stack depth analysis tool. Such a tool can help you determine if your program could possibly overflow its stack under any possible sequence of events (excepting unexpected or unbounded recursion). You can write your own (I have, in C#, for embedded programs compiled in C for M16C and MIPS targets using GCC and IAR compilers), but it's really complex and not something for beginners to attempt.
Look for a "stack usage analyzer" or "stack usage analysis tool" for your particular processor and toolchain (e.g. x86/x64/ARM/etc and GCC/VisualStudio/IAR/etc).
If you're using GCC, you may be able to use the -fstack-usage option, but that only gives you the maximum stack usage on a per-function basis. By itself that's not terribly helpful, since to verify that a program won't blow its stack, you have to recursively walk the calltree to see what the maximum stack depth could be at any level of the call tree. If you also use the -Wstack-usage option, you can get a warning if any subprogram's stack usage can possibly exceed a specified stack depth, which is more useful than the information you get with merely the -fstack-usage option.
If you're trying to work out the amount of 'space allocated to those variables in a function, use the sizeof() operator.
Notice that the arrays are arrays of char * which are pointers to characters not characters.
#include <stdio.h>
int main(void) {
size_t total=0;
size_t first_array_size=sizeof(char *[200]);
printf("first array: %zu\n",first_array_size);
total+=first_array_size;
size_t second_array_size=sizeof(char *[100]);
printf("second array: %zu\n",second_array_size);
total+=second_array_size;
size_t int_size=sizeof(int);
printf("int size: %zu * 2 = %zu\n",int_size,int_size*2);
total+=int_size*2;
printf("total=%zu\n",total);
return 0;
}
Typical output (on a 64-bit architecture):
first array: 1600
second array: 800
int size: 4 * 2 = 8
total=2408
Results may vary.
Footnote: It's also worth understanding that the amount of space allocated to the stack frame may be greater.
For example arguments passed in are typically copied to the stack as is the return value along with a pointer representing the execution point to return to after the function is called.
There's also the complexity of alignment. For example on many modern machines space may be left between variables to make sure they're aligned. However optimizations may take some of that space back depending on how it re-orders the variables. It's also possible that values (particularly integer values) may be allocated to registers and not take up space on the stack.
Finally an implementation could in principle allocate the arrays on the heap. That's certainly a possible implementation for variable length arrays but I'm not aware of it being strictly disallowed for fixed size arrays.
I want to write a C code to see the difference between static and dynamic allocation.
That's my idea but it doesn't work.
It simply initializes an array of size 10, but assigns 100 elements instead of 10. I'll then initialize another array large enough hoping to replace the 90 elements that're not part of array1[10], then I print out the 100 elements of array1.
int i;
int array1[10];
int array2[10000];
for(i=0;i<100;i++)
array1[i] = i;
for(i=0;i<10000;i++)
array2[i] = i+1;
for(i=0;i<100;i++)
{
printf("%d \n",array1[i]);
}
What I hope to get is garbage outside then first 10 elements when using static allocation, afterwards, I'll use malloc and realloc to ensure that the 100 elements would be there correctly. But unfortunately, it seems that the memory is large enough so that the rest of the 100 elements wouldn't be replaced!
I tried to run the code on linux and use "ulimit" to limit the memory size, but it didn't work either.
Any ideas please?
Cdoesn't actually do any boundary checking with regards to arrays. It depends on the OS to ensure that you are accessing valid memory.
Accessing outside the array bounds is undefined behavior, from the c99 draft standard section Annex J.2 J.2 Undefined behavior includes the follow point:
An array subscript is out of range, even if an object is apparently accessible with the
given subscript (as in the lvalue expression a[1][7] given the declaration int
a[4][5]) (6.5.6).
In this example you are declaring a stack based array. Accessing out of bound will get memory from already allocated stack space. Currently undefined behavior is not in your favor as there is no Seg fault. Its programmer's responsibility to handle boundary conditions while writing code in C/C++.
You do get garbage after the first 10 elements of array1. All of the data after element 9 should not be considered allocated by the stack and can be written over at any time. When the program prints the 100 elements of array1, you might see the remnants of either for loop because the two arrays are allocated next to each other and normally haven't been written over. If this were implemented in a larger program, other arrays might take up the space after these two example arrays.
When you access array1[10] and higher index values, the program will just keep writing into adjacent memory locations even though they don't "belong" to your array. At some point you might try to access a memory location that's forbidden, but as long as you're mucking with memory that the OS has given to your program, this will run. The results will be unpredictable though. It could happen that this will corrupt data that belongs to another variable in your program, for example. It could also happen that the value that you wrote there will still be there when you go back to read it if no other variable has been "properly assigned" that memory location. (This seems to be what's happening in the specific case that you posted.)
All of that being said, I'm not clear at all how this relates to potential differences between static and dynamic memory allocation since you've only done static allocation in the program and you've deliberately introduced a bug.
Changing the memory size won't resolve your problem, because when you create your two arrays, the second one should be right after the first one in memory.
Your code should do what you think it will, and on my computer, it does.
Here's my output :
0
1
2
3
4
5
6
7
8
9
10
11
1
2
3
4
5
...
What OS are you running your code on ? (I'm on linux 64bit).
Anyway, as everybody told you, DON'T EVER DO THIS IN A REAL PROGRAM. Writing outside an array is an undefined behaviour and could lead your program to crash.
Writing out of bounds of an array will prove nothing and is not well-defined. Generally, there's nothing clever or interesting involved in invoking undefined behavior. The only thing you'll achieve by that is random crashes.
If you wish to know where a variable is allocated, you have to look at addresses. Here's one example:
#include <stdio.h>
#include <stdlib.h>
int main (void)
{
int stack;
static int data = 1;
static int bss = 0;
int* heap = malloc(sizeof(*heap));
printf("stack: %p\n", (void*)&stack);
printf(".data: %p\n", (void*)&data);
printf(".bss: %p\n", (void*)&bss);
printf(".heap: %p\n", (void*)heap);
}
This should print 4 distinctively different addresses (.data and .bss probably close to each other though). To know exactly where a certain memory area starts, you either need to check some linker script or use a system-specific API. And once you know the memory area's offset and size, you can determine if a variable is stored within one of the different memory segments.