static const vs const declaration performance difference on uC - c

Lets say i have a lookuptable, an array of 256 elements defined and declared in a header named lut.h. The array will be accessed multiple times in lifetime of the program.
From my understanding if its defined & declared as static, it will remain in memory until the program is done, i.e. if it is a task running on a uC, the array is in memory the entire time.
Where as without static, it will be loaded into memory when accessed.
In lut.h
static const float array[256] = {1.342, 14.21, 42.312, ...}
vs.
const float array[256] = {1.342, 14.21, 42.312, ...}
Considering the uC has limited spiflash and psram, what would be the most performance oriented approach?

You have some misconceptions here, since a MCU is not a PC. Everything in memory in a MCU will persist for as long as the MCU has power. Programs do not end or return resources to a hosting OS.
"Tasks" on a MCU means you have a RTOS. They use their own stack and that's a topic of it's own, quite unrelated to your question. It is normal that all tasks on a RTOS execute forever, rather than getting allocated/deallocated in run-time like processes in a PC.
static versus automatic on local scope does mean different RAM memory use, but not necessarily more/less memory use. Local variables get pushed/popped on the stack as the program executes. static ones sit on their designated address.
Where as without static, it will be loaded into memory when accessed.
Only if the array you are loading into is declared locally. That is:
void func (void)
{
int my_local_array[] = {1,2,3};
...
}
Here my_local_array will load the values from flash to RAM during execution of that function only. This means two things:
The actual copy down from flash to RAM is slow. First of all, copying something is always slow, regardless of the situation. But in the specific case of copying from RAM to flash, it might be extra slow, depending on MCU.
It will be extra slow on high end MCUs with flash wait states that fail to utilize data cache for the copy. It will be extra slow on weird Harvard architecture MCUs that can't address data directly. Etc.
So naturally if you do this copy down each time a function is called, instead of just once, your program will turn much slower.
Large local objects lead to a need for higher stack size. The stack must be large enough to deal with the worst-case scenario. If you have large local objects, the stack size will need to be set much higher to prevent stack overflows. Meaning this can actually lead to less effective memory use.
So it isn't trivial to tell if you save or lose memory by making an object local.
General good practice design in embedded systems programming is to not allocate large objects on the stack, as they make stack handling much more detailed and the potential for stack overflow increases. Such objects should be declared as static, at file scope. Particularly if speed is important.
static const float array vs const float array
Another misconception here. Making something const in MCU system, while at the same time placing it at file scope ("global"), most likely means that the variable will end up in flash ROM, not in RAM. Regardless of static.
This is most of the time preferred, since in general RAM is a more valuable resource than flash. The role static plays here is merely good program design, as it limits access to the variable to the local translation unit, rather than cluttering up the global namespace.
In lut.h
You should never define variables in header files.
It is bad from a program design point-of-view, as you expose the variable all over the place ("spaghetti programming") and it is bad from a linker point of view, if multiple source files include the same header file - which is extremely likely.
Correctly designed programs places the variable in the .c file and limits access by declaring it static. Access from the outside, if needed, is done through setters/getters.
he uC has limited spiflash
What is "spiflash"? An external serial flash memory accessed through SPI? Then none of this makes sense, since such flash memory isn't memory-mapped and typically the compiler can't utilize it. Access to such memories has to be carried out by your application, manually.

If your arrays are defined on a file level (you mentioned lut.h), and both have const qualifiers, they will not be loaded into RAM¹. The static keyword only limits the scope of the array, it doesn't change its lifetime in any way. If you check the assembly for your code, you will see that both arrays look exactly the same when compiled:
static const int static_array[] = { 1, 2, 3 };
const int extern_array[] = { 1, 2, 3};
extern void do_something(const int * a);
int main(void)
{
do_something(static_array);
do_something(extern_array);
return 0;
}
Resulting assembly:
main:
sub rsp, 8
mov edi, OFFSET FLAT:static_array
call do_something
mov edi, OFFSET FLAT:extern_array
call do_something
xor eax, eax
add rsp, 8
ret
extern_array:
.long 1
.long 2
.long 3
static_array:
.long 1
.long 2
.long 3
On the other hand, if if you declare the arrays inside a function, then the array will be copied to temporary storage (stack) for the duration of the function, unless you add the static qualifier:
extern void do_something(const int * a);
int main(void)
{
static const int static_local_array[] = { 1, 2, 3 };
const int local_array[] = { 1, 2, 3 };
do_something(static_local_array);
do_something(local_array);
return 0;
}
Resulting assembly:
main:
sub rsp, 24
mov edi, OFFSET FLAT:static_local_array
movabs rax, 8589934593
mov QWORD PTR [rsp+4], rax
mov DWORD PTR [rsp+12], 3
call do_something
lea rdi, [rsp+4]
call do_something
xor eax, eax
add rsp, 24
ret
static_local_array:
.long 1
.long 2
.long 3
¹ More precisely, it depends on the compiler. Some compilers will need additional custom attributes to define exactly where you want to store the data. Some compilers will try to place the array into RAM when there is enough spare space, to allow faster reading.

Related

Why does the stack frame also store instructions(besides data)? What is the precise mechanism by which instructions on stack frame get executed?

Short version:
0: 48 c7 c7 ee 4f 37 45 mov $0x45374fee, %rdi
7: 68 60 18 40 00 pushq $0x401860
c: c3 retq
How can these 3 lines of instruction(0,7,c), saved in the stack frame, get executed? I thought stack frame only store data, does it also store instructions? I know data is read to registers, but how do these instructions get executed?
Long version:
I am self-studying 15-213(Computer Systems) from CMU. In the Attack lab, there is an instance (phase 2) where the stack frame gets overwritten with "attack" instructions. The attack happens by then overwriting the return address from the calling function getbuf() with the address %rsp points to, which I know is the top of the stack frame. In this case, the top of the stack frame is in turn injected with the attack code mentioned above.
Here is the question, by reading the book(CSAPP), I get the sense that the stack frame only stores data the is overflown from the registers(including return address, extra arguments, etc.). But I don't get why it can also store instructions(attack code) and be executed. How exactly did the content in the stack frame, which %rsp points to, get executed? I also know that %rsp stores the return address of the calling function, the point being it is an address, not an instruction? So exactly by which mechanism does an supposed address get executed as an instruction? I am very confused.
Edit: Here is a link to the question(4.2 level 2):
http://csapp.cs.cmu.edu/3e/attacklab.pdf
This is a post that is helpful for me in understanding: https://github.com/magna25/Attack-Lab/blob/master/Phase%202.md
Thanks for your explanation!
ret instruction gets a pointer from the current position of the stack and jumps to it. If, while in a function, you modify the stack to point to another function or piece of code that could be used maliciously, the code can return to it.
The code below doesn't necessarily compile, and it is just meant to represent the concept.
For example, we have two functions: add(), and badcode():
int add(int a, int b)
{
return a + b;
}
void badcode()
{
// Some very bad code
}
Let's also assume that we have a stack such as the below when we call add()
...
0x00....18 extra arguments
0x00....10 return address
0x00....08 saved RBP
0x00....00 local variables and etc.
...
If during the execution of add, we managed to change the return address to address of badcode(), on ret instruction we will automatically start executing badcode(). I don't know if this answer your question.
Edit:
An instruction is simply an array of numbers. Where you store them is irrelevant (mostly) to their execution. A stack is essentially an abstract data structure, it is not a special place in RAM. If your OS doesn't mark the stack as non-executable, there is nothing stopping the code on the stack from being returned to by the ret.
Edit 2:
I get the sense that the stack frame only stores data that is overflown
from the registers(including return address, extra arguments, etc.)
I do not think that you know how registers, RAM, stack, and programs are incorporated. The sense that stack frame only stores data that is overflown is incorrect.
Let's start over.
Registers are pieces of memory on your CPU. They are independent of RAM. There are mainly 8 registers on a CPU. a, c, d, b, si, di, sp, and bp. a is for accumulator and it generally used for arithmetic operations, likewise b stands for base, c stands for counter, d stands for data, si stands for source, di stands for destination, sp is the stack pointer, and bp is the base pointer.
On 16 bit computers a, b, c, d, si, di, sp, and bp are 16 bits (2 byte). The a, b, c, and d are often shown as ax, bx, cx, and dx where the x stands for extension from their original 8 bit versions. They can also be referred to as eax, ecx, edx, ebx, esi, edi, esp, ebp for 32 bit (e again stands for extended) and rax, rcx, rdx, rbx, rsi, rdi, rsp, rbp for 64 bit.
Once again these are on your CPU and are independent of RAM. CPU uses these registers to do everything that it does. You wanna add two numbers? put one of them inside ax and another one inside cx and add them.
You also have RAM. RAM (standing for Random Access Memory) is a storage device that allows you to access and modify all of its values using equal computation power or time (hence the term random access). Each value that RAM holds also has an address that determines where on the RAM this value is. CPU can use numbers and treat such numbers as addresses to access memory addresses of RAM. Numbers that are used for such purposes are called pointers.
A stack is an abstract data structure. It has a FILO (first in last out) structure which means that to access the first datum that you have stored you have to access all of the other data. To manipulate the stack CPU provides us with sp which holds the pointer to the current position of the stack, and bp which holds the top of the stack. The position that bp holds is called the top of the stack because the stack usually grows downwards meaning that if we start a stack from the memory address 0x100 and store 4 bytes in it, sp will now be at the memory address 0x100 - 4 = 0x9C. To do such operations automatically we have the push and pop instructions. In that sense a stack could be used to store any type of data regardless of the data's relation to registers are programs.
Programs are pieces of structured code that are placed on the RAM by the operating system. The operating system reads program headers and relevant information and sets up an environment for the program to run on. For each program a stack is set up, usually, some space for the heap is given, and instructions (which are the building blocks of a program) are placed in arbitrary memory locations that are either predetermined by the program itself or automatically given by the OS.
Over the years some conventions have been set to standardize CPUs. For example, on most CPU's ret instruction receives the system pointer size amount of data from the stack and jumps to it. Jumping means executing code at a particular RAM address. This is only a convention and has no relation to being overflown from registers and etc. For that reason when a function is called firstly the return address (or the current address in the program at the time of execution) is pushed onto the stack so that it could be retrieved later by ret. Local variables are also stored in the stack, along with arguments if a function has more than 6(?).
Does this help?
I know it is a long read but I couldn't be sure on what you know and what you don't know.
Yet Another Edit:
Lets also take a look at the code from the PDF:
void test()
{
int val;
val = getbuf();
printf("No exploit. Getbuf returned 0x%x\n", val);
}
Phase 2 involves injecting a small amount of code as part of your exploit string.
Within the file ctarget there is code for a function touch2 having the following C representation:
void touch2(unsigned val)
{
vlevel = 2; /* Part of validation protocol */
if (val == cookie) {
printf("Touch2!: You called touch2(0x%.8x)\n", val);
validate(2);
} else {
printf("Misfire: You called touch2(0x%.8x)\n", val);
fail(2);
}
exit(0);
}
Your task is to get CTARGET to execute the code for touch2 rather than returning to test. In this case,
however, you must make it appear to touch2 as if you have passed your cookie as its argument.
Let's think about what you need to do:
You need to modify the stack of test() so that two things happen. The first thing is that you do not return to test() but you rather return to touch2. The other thing you need to do is give touch2 an argument which is your cookie. Since you are giving only one argument you don't need to modify the stack for the argument at all. The first argument is stored on rdi as a part of x86_64 calling convention.
The final code that you write has to change the return address to touch2()'s address and also call mov rdi, cookie
Edit:
I before talked about RAM being able to store data on addresses and CPU being able to interact with them. There is a secret register on your CPU that you are not able to reach from you assembly code. This register is called ip/eip/rip. It stands for instruction pointer. This register holds a 16/32/64 bit pointer to an address on RAM. this particular address is the address that the CPU will execute in its clock cycle. With that in my we can say that what a ret instruction is doing is
pop rip
which means get the last 64 bits (8 bytes for a pointer) on the stack into this instruction pointer. Once rip is set to this value, the CPU begins executing this code. The CPU doesn't do any checks on rip whatsoever. You can technically do the following thing (excuse me, my assembly is in intel syntax):
mov rax, str ; move the RAM address of "str" into rax
push rax ; push rax into stack
ret ; return to the last pushed qword (8 bytes) on the stack
str: db "Hello, world!", 0 ; define a string
This code can call/execute a string. Your CPU will be very upset tho, that there is no valid instruction there and will probably stop working.

How garbage collection works with data segment?

For below Go syntax in function scope f():
var fruits [5]string
fruits[0] = "Apple"
Below is the memory representation:
My understanding is, string Apple gets stored in data segment and rest of the six string headers(ptr,length) gets allocated in stack segment.
For below code in function scope f():
numbers := [4]int{10, 20, 30, 40}
Memory for {10, 20, 30, 40} gets allocated in data segment but not in stack segment for function scope f.
Go garbage collector cleans heap segment of a process.
Returning from function f(), stack segment pointer clears stack segment of function f()
Edit:
To understand, value semantics & pointer semantics in the aspect of allocating strings,
How data segment memory(for string Apple) gets cleared after returning from function f?
The language definition for Go does not describe actions in terms of segments, stacks, heaps, and so on. So all of this is implementation detail, which might change from one Go implementation to another.
In general, though, Go compilers do live-range analysis for variables and use escape analysis to determine whether to allocate something in GC-able memory ("heap") or automatically-released storage ("stack"). String literals may, depending on too many things to count, be allocated at compile time as text and referenced directly from there, or copied to some data area that's either heap-ish or stack-ish.
Let's assume for argument's sake that you wrote:
func f() {
var fruits [5]string
fruits[0] = "Apple"
}
This function doesn't do anything at all, so it just gets elided from the build.
The string constant "Apple" appears nowhere at all. Let's add a bit more so that it actually does exist:
package main
import "fmt"
func f() {
var fruits [5]string
fruits[0] = "Apple"
fmt.Println(fruits[0])
}
func main() {
f()
fmt.Println("foo")
}
Here is some (hand-trimmed / cleaned-up) disassembly of main.f in the resulting binary. Note that the implementation will almost certainly be different in other versions of Go. This was built with Go 1.13.5 (for amd64).
main.f:
mov %fs:0xfffffffffffffff8,%rcx
cmp 0x10(%rcx),%rsp
jbe 2f
Everything up to here is boilerplate: the entry point for the function checks whether it needs to call the runtime to allocate more stack space, because it's about to use 0x58 bytes of stack space here:
1: sub $0x58,%rsp
mov %rbp,0x50(%rsp)
This is the end of the boilerplate: after the next few instructions, we will be able to return from f with a simple retq. Now we make room on the stack for the array fruits, plus other space the compiler deems appropriate for whatever reason, and update %rbp. Then we store a string header at %(rsp) and %8%(rsp) in order to call convTstring in package runtime:
lea 0x50(%rsp),%rbp
lea 0x35305(%rip),%rax # <go.string.*+0x24d> - the string is here
mov %rax,(%rsp)
movq $0x5,0x8(%rsp) # this is the length of the string
callq 408da0 <runtime.convTstring>
mov 0x10(%rsp),%rax
The function runtime.convTstring actually allocates space (16 bytes on this machine) for another copy of the string header, on "the heap", then copies the header into place. This copy is now ready to be stored into fruits[0] or elsewhere. The calling convention for Go on x86_64 is a bit odd, so the return value is at 0x10(%rsp), which we've now copied into %rax. We'll see where this gets used in a moment:
xorps %xmm0,%xmm0
movups %xmm0,0x40(%rsp)
These instructions zero out 16 bytes starting at 0x40(%rsp). It's not clear to me what this is for, especially since we overwrite them immediately.
lea 0x11a92(%rip),%rcx # <type.*+0x11140>
mov %rcx,0x40(%rsp)
mov %rax,0x48(%rsp)
mov 0xd04a1(%rip),%rax # <os.Stdout>
lea 0x4defa(%rip),%rcx # <go.itab.*os.File,io.Writer>
mov %rcx,(%rsp)
mov %rax,0x8(%rsp)
lea 0x40(%rsp),%rax
mov %rax,0x10(%rsp)
movq $0x1,0x18(%rsp)
movq $0x1,0x20(%rsp)
callq <fmt.Fprintln>
This appears to be the call to fmt.Println: since we pass an interface value, we must package it up as a type and pointer-to-value (perhaps that's why there is a call to runtime.convTstring in the first place). We also have os.stdout and its interface descriptor inserted directly into the call here, via some inlining (note that this call goes directly to fmt.Fprintln).
In any case, we passed the string header, allocated in runtime.convTstring here, to function fmt.Println.
mov 0x50(%rsp),%rbp
add $0x58,%rsp
retq
2: callq <runtime.morestack_noctxt>
jmpq 1b
This is how we return from a function—the constants 0x50 and 0x58 depend on how much stack space we allocated—and, after the label that the start of the function can jump to, the rest of the function-entry boilerplate.
Anyway, the point of all of the above is to show that:
The five byte sequence Apple is not allocated at runtime at all. Instead, it exists in the rodata segment known as go.string.*. This rodata segment is in effect program text: the OS places it into read-only memory, if at all possible. It's just separated from the executable instructions for organizational purposes.
The fruits array never actually got used at all. The compiler could see that, while we wrote to it, we didn't use it other than the one call, so we didn't need it after all.
But a string header, by which one can find both the length of the string and the data (in that rodata segment), did get heap-allocated.
It didn't need to be, as fmt.Println is not going to save this pointer, but the compiler didn't spot that. Eventually, the runtime gc will free the heap-allocated string header data, unless the program exits entirely first.
Memory for {10, 20, 30, 40} gets allocated in data segment but not in
stack segment for function scope f.
No, it's still gonna be allocated on the stack. [4]int is an array type. It's a value type. int is a value type. So the whole array would be on the stack, GC wouldn't need to deal with it.
But if something is allocated on the heap (I guess that what you mean by data segment) then the GC would kick in. The internals are an implementation detail and may change in the future but to put it simply current version would just start from roots (global variables, stack, registers) and mark all the live objects. Everything that's unmarked would be collected.
Edit:
If we're talking about string literals in particular. Strings work similar to slices - it's a value type, struct with two fields - pointer to the backing array of bytes and a length. String literals are special, they actually point to read-only data segment that contains actual string contents. So at runtime no allocation occurs and GC has nothing to collect.

AVR C compilers behavior. Memory management

Do AVR C compilers make program memorize the address in SRAM where function started to store its data (variables, arrays) in data stack in one of index registers in order to get absolute address of local variable by formula:
absoluteAdr = functionDataStartAdr + localShiftOfVariable.
And do they increase data stack point when variable declared by it's length or stack pointer increased in end/start of function for all it's variables lengths.
Let's have a look at avr-gcc, which is freely available including its ABI:
Do AVR C compilers make program memorize the address in SRAM where function started to store its data (variables, arrays) in data stack in one of index registers in order to get absolute address of local variable by formula:
Yes, no, it depends:
Static Storage
For variables in static storage, i.e. variables as defined by
unsigned char func (void)
{
static unsigned char var;
return ++var;
}
the compiler generates a symbol like var.123 with appropriate size (1 byte in this case). The linker / locator will then assign the address.
func:
lds r24,var.1505
subi r24,lo8(-(1))
sts var.1505,r24
ret
.local var.1505
.comm var.1505,1,1
Automatic
Automatic variables are held in registers if possible, otherwise the compiler allocates space in the frame of the function. It may even be the case that variables are optimized out, and in that case they do not exist anywhere in the program:
int add (void)
{
int a = 1;
int b = 2;
return a + b;
}
→
add:
ldi r24,lo8(3)
ldi r25,0
ret
There are 3 types of entities that are stored in the frame of a function, all of which might be present or absent depending on the program:
Callee-saved registers that are saved (PUSH'ed) by the function prologue and restored (POP'ed) by the epilogue. This is needed when local variables are allocated to callee-saved registers.
Space for local variables that cannot be allocated to registers. This happens when the variable is too big to be held in registers, there are too many auto variables, or the address of a variable is taken (and taking the address cannot be optimized out). This is because you cannot take the address of a register1.
void use_address (int*);
void func (void)
{
int a;
use_address (&a);
}
The space for these variables is allocated in the prologue and deallocated in the epilogue. Shrink-wrapping is not implemented:
func:
push r28
push r29
rcall .
in r28,__SP_L__
in r29,__SP_H__
/* prologue: function */
/* frame size = 2 */
/* stack size = 4 */
movw r24,r28
adiw r24,1
rcall use_address
pop __tmp_reg__
pop __tmp_reg__
pop r29
pop r28
ret
In this example, a occupies 2 bytes which are allocated by rcall . (it was compiled for a device with 16-bit program counter). Then the compiler initialized the frame-pointer Y (R29:R28) with the value of the stack pointer. This is needed because on AVR, you cannot access memory via SP; the only memory operations that involve SP are PUSH and POP. Then the address of that variable which is Y+1 is passed in R24. After the call of the function, the epilogue frees the frame and restores R28 and R29.
Arguments that have to be passed on the stack:
void xfunc (int, ...);
void call_xfunc (void)
{
xfunc (42);
}
These arguments are pushed and the callee is picking them up from the stack. These arguments are pushed / popped around the call, but can also be accumulated by means of -maccumulate-args.
call_func:
push __zero_reg__
ldi r24,lo8(42)
push r24
rcall xfunc
pop __tmp_reg__
pop __tmp_reg__
ret
In this example, the argument has to be passed on the stack because the ABI says that all arguments of a varargs function have to be passed on the stack, including the named ones.
For a description on how exactly the frame is being layed out and arguments are being passed, see [Frame Layout and Argument Passing] (https://gcc.gnu.org/wiki/avr-gcc#Frame_Layout).
1 Some AVRs actually allow this, but you never (like in NEVER) want to pass around the address of a general purpose register!
Compilers are not managing the RAM, compilers at compilation time calculate the required size for each data sections like bss, data, text, rodata, .. etc and generate relocatable object file for each translation unit
The linker comes after and generate one object file and assign the relocatable addresses to absolute ones mapped according to the Linker configuration File LCF.
In run time, the mechanism depends on the architecture itself. normally, each function call has a frame in the stack where it's arguments, return address and local variables are defined. the stack extend with a creation of variables and for low cost AVR microcontrollers, there is no memory management protection regarding the stack increase or the overlapping between the stack and another memory section -normally the heap-. even if there is OS managing the protection from the tasks to exceed its allocated stack, without a memory management unit, all what OS can do is to assert a RESET with illegal memory access reason.

How avoid cache line invalidation from multiple threads writing to a shared array?

Context of the problem:
I am writing a code that creates 32 threads, and set affinity of them to each one of the 32 cores in my multi-core-multi-processor system.
Threads simply execute the RDTSCP instruction and the value is stored in a shared array at a non-overlapping position, this is the shared array:
uint64_t rdtscp_values[32];
So, every thread is going to write to the specific array position based on its core number.
Up to know, everything is working properly with the exception that I know that I may not be using the right data structure to avoid cache line bouncing.
P.S: I have checked already that my processor's cache line is 64-bytes wide.
Because I am using a simple uint64_t array, it implies that a single cache line is going to store 8 positions of this array, because of the read-ahead.
Question:
Because of this simple array, although the threads write to different indexes, my understanding tells that every write to this array will cause a cache invalidation to all other threads?
How could I create a structure that is aligned to the cache line?
EDIT 1
My system is: 2x Intel Xeon E5-2670 2.30GHz (8 cores, 16 threads)
Yes you definitely want to avoid "false sharing" and cache-line ping-pong.
But this probably doesn't make sense: if these memory locations are thread-private more often than they're collected by other threads, they should be stored with other per-thread data so you're not wasting cache footprint on 56 bytes of padding. See also Cache-friendly way to collect results from multiple threads. (There's no great answer; avoid designing a system that needs really fine-grained gathering of results if you can.)
But let's just assume for a minute that unused padding between slots for different threads is actually what you want.
Yes, you need the stride to be 64 bytes (1 cache line), but you don't actually need the 8B you're using to be at the start of each cache line. Thus, you don't need any extra alignment as long as the uint64_t objects are naturally-aligned (so they aren't split across a cache-line boundary).
It's fine if each thread is writing to the 3rd qword of its cache line instead of the 1st. OTOH, aligning to 64B makes sure nothing else is sharing a cache line with the first element, and it's easy so we might as well.
Static storage: aligning static storage is very easy in ISO C11 using alignas(), or with compiler-specific stuff.
With a struct, padding is implicit to make the size a multiple of the required alignment. Having one member with an alignment requirement implies that the whole struct requires at least that much alignment. The compiler takes care of this for you with static and automatic storage, but you have to use aligned_alloc or an alternative for over-aligned dynamic allocation.
#include <stdalign.h> // for #define alignas _Alignas for C++ compat
#include <stdint.h> // for uint64_t
// compiler knows the padding is just padding
struct { alignas(64) uint64_t v; } rdtscp_values[32];
int foo(unsigned t) {
rdtscp_values[t].v = 1;
return sizeof(rdtscp_values[0]); // yes, this is 64
}
Or with an array as suggested by # Eric Postpischil:
alignas(64) // optional, stride will still be 64B without this.
uint64_t rdtscp_values_2d[32][8]; // 8 uint64_t per cache line
void bar(unsigned t) {
rdtscp_values_2d[t][0] = 1;
}
alignas() is optional if you don't care about the whole thing being 64B aligned, just having 64B stride between elements you use. You could also use __attribute__((aligned(64))) in GNU C or C++, or __declspec(align(64)) for MSVC, using #ifdef to define an ALIGN macro that's portable across the major x86 compilers.
Either way produces the same asm. We can check compiler output to verify that we got what we wanted. I put it up on the Godbolt compiler explorer. We get:
foo: # and same for bar
mov eax, edi # zero extend 32-bit to 64-bit
shl rax, 6 # *64 is the same as <<6
mov qword ptr [rax + rdtscp_values], 1 # store 1
mov eax, 64 # return value = 64 = sizeof(struct)
ret
Both arrays are declared the same way, with the compiler requesting 64B alignment from the assembler/linker with the 3rd arg to .comm:
.comm rdtscp_values_2d,2048,64
.comm rdtscp_values,2048,64
Dynamic storage:
If the number of threads is not a compile-time constant, then you can use an aligned allocation function to get aligned dynamically-allocated memory (especially if you want to support a very high number of threads). See How to solve the 32-byte-alignment issue for AVX load/store operations?, but really just use C11 aligned_alloc. It's perfect for this, and returns a pointer that's compatible with free().
struct { alignas(64) uint64_t v; } *dynamic_rdtscp_values;
void init(unsigned nthreads) {
size_t sz = sizeof(dynamic_rdtscp_values[0]);
dynamic_rdtscp_values = aligned_alloc(nthreads*sz, sz);
}
void baz(unsigned t) {
dynamic_rdtscp_values[t].v = 1;
}
baz:
mov rax, qword ptr [rip + dynamic_rdtscp_values]
mov ecx, edi # same code as before to scale by 64 bytes
shl rcx, 6
mov qword ptr [rax + rcx], 1
ret
The address of the array is no longer a link-time constant, so there's an extra level of indirection to access it. But the pointer is read-only after it's initialized, so it will stay shared in cache in each core and reloading it when needed is very cheap.
Footnote: In the i386 System V ABI, uint64_t only has 4B-alignment inside structs by default (without alignas(8) or __attribute__((aligned(8)))), so if you put an int before a uint64_t and didn't do any alignment of the whole struct, it would be possible to get cache-line splits. But compilers align it by 8B whenever possible, so your struct-with padding is still fine.
SOLUTION
So, I followed the comments here and I must say thanks for all contributions.
Finally I got what I expected: cache lines being used properly per each thread.
Here is the shared structure:
typedef struct align_st {
uint64_t v;
uint64_t padding[7];
} align_st_t __attribute__ ((aligned (64)));
I am using a padding uint64_t padding[7] inside the structure to fill the remaining bytes in the cache line when this structure is loaded to the L1 cache. Nonetheless, I am asking to the compiler to use 64-bytes memory alignment when compiling it __attribute__ ((aligned (64))).
So, I allocate this structure dynamically based on the number of cores, using the memalign() for this:
align_st_t *al = (align_st_t*) memalign(64, n_cores * sizeof(align_st_t));
To compare it, I wrote one code version (V1) that uses these aligned mechanisms, and other code version (V2) that uses the simple array method.
By executing with perf, and I got these numbers:
V1: 7.184 cache-misses;
V2: 2.621.347 cache-misses.
P.S.: Each thread is writing 1-thousand times to the same address of the shared structure just to increase the numbers

C - How to create a pattern in code segment to recognize it in memory dump?

I dump my RAM (a piece of it - code segment only) in order to find where is which C function being placed. I have no map file and I don't know what boot/init routines exactly do.
I load my program into RAM, then if I dump the RAM, it is very hard to find exactly where is what function. I'd like to use different patterns build in the C source, to recognize them in the memory dump.
I've tryed to start every function with different first variable containing name of function, like:
char this_function_name[]="main";
but it doesn't work, because this string will be placed in the data segment.
I have simple 16-bit RISC CPU and an experimental proprietary compiler (no GCC or any well-known). The system has 16Mb of RAM, shared with other applications (bootloader, downloader). It is almost impossible to find say a unique sequence of N NOPs or smth. like 0xABCD. I would like to find all functions in RAM, so I need unique identificators of functions visible in RAM-dump.
What would be the best pattern for code segment?
If it were me, I'd use the symbol table, e.g. "nm a.out | grep main". Get the real address of any function you want.
If you really have no symbol table, make your own.
struct tab {
void *addr;
char name[100]; // For ease of searching, use an array.
} symtab[] = {
{ (void*)main, "main" },
{ (void*)otherfunc, "otherfunc" },
};
Search for the name, and the address will immediately preceed it. Goto address. ;-)
If your compiler has inline asm you can use it to create a pattern. Write some NOP instructions which you can easily recognize by opcodes in memory dump:
MOV r0,r0
MOV r0,r0
MOV r0,r0
MOV r0,r0
How about a completely different approach to your real problem, which is finding a particular block of code: Use diff.
Compile the code once with the function in question included, and once with it commented out. Produce RAM dumps of both. Then, diff the two dumps to see what's changed -- and that will be the new code block. (You may have to do some sort of processing of the dumps to remove memory addresses in order to get a clean diff, but the order of instructions ought to be the same in either case.)
Numeric constants are placed in the code segment, encoded in the function's instructions. So you could try to use magic numbers like 0xDEADBEEF and so on.
I.e. here's the disassembly view of a simple C function with Visual C++:
void foo(void)
{
00411380 push ebp
00411381 mov ebp,esp
00411383 sub esp,0CCh
00411389 push ebx
0041138A push esi
0041138B push edi
0041138C lea edi,[ebp-0CCh]
00411392 mov ecx,33h
00411397 mov eax,0CCCCCCCCh
0041139C rep stos dword ptr es:[edi]
unsigned id = 0xDEADBEEF;
0041139E mov dword ptr [id],0DEADBEEFh
You can see the 0xDEADBEEF making it into the function's source. Note that what you actually see in the executable depends on the endianness of the CPU (tx. Richard).
This is a x86 example. But RISC CPUs (MIPS, etc) have instructions moving immediates into registers - these immediates can have special recognizable values as well (although only 16-bit for MIPS, IIRC).
Psihodelia - it's getting harder and harder to catch your intention. Is it just a single function you want to find? Then can't you just place 5 NOPs one after another and look for them? Do you control the compiler/assembler/linker/loader? What tools are at your disposal?
As you noted, this:
char this_function_name[]="main";
... will end up setting a pointer in your stack to a data segment containing the string. However, this:
char this_function_name[]= { 'm', 'a', 'i', 'n' };
... will likely put all these bytes in your stack so you will be able to recognize the string in your code (I just tried it on my platform).
Hope this helps
Why not get each function to dump its own address. Something like this:
void* fnaddr( char* fname, void* addr )
{
printf( "%s\t0x%p\n", fname, addr ) ;
return addr ;
}
void test( void )
{
static void* fnaddr_dummy = fnaddr( __FUNCTION__, test ) ;
}
int main (int argc, const char * argv[])
{
static void* fnaddr_dummy = fnaddr( __FUNCTION__, main ) ;
test() ;
test() ;
}
By making fnaddr_dummy static, the dump is done once per-function. Obviously you would need to adapt fnaddr() to support whatever output or logging means you have on your system. Unfortunately, if the system performs lazy initialisation, you'll only get the addresses of the functions that are actually called (which may be good enough).
You could start each function with a call to the same dummy function like:
void identifyFunction( unsigned int identifier)
{
}
Each of your functions would call the identifyFunction-function with a different parameter (1, 2, 3, ...). This will not give you a magic mapfile, but when you inspect the code dump you should be able to quickly find out where the identifyFunction is because there will be lots of jumps to that address. Next scan for those jump and check before the jump to see what parameter is passed. Then you can make your own mapfile. With some scripting this should be fairly automatic.

Resources