I feel kind of dumb, but I'm struggling with dereferencing a pointer (+ adding an offset) in C.
What I want to recreate in C is this behavior:
movabs rax, 0xdeadbeef
add rax, 0xa
mov rax, QWORD PTR [rax]
So at the end rax should be: *(0xdeadbeef+0xa)
Especially the equivalent to mov rax, QWORD PTR [rax] would be improtant, as I need to use the calculated value and retrieve the data (=a different address) that is being stored at that point.
I tried so many things, but here is my current stage:
void *ptr = (void*)0xdeadbeef;
void *ptr2 = *(void*)(ptr+0xa);
Which translates to sth like this:
0x7ffff7fe6050: mov QWORD PTR [rbp-0x38],rax
0x7ffff7fe6054: mov rax,QWORD PTR [rbp-0x38]
0x7ffff7fe6058: add rax,0xa
EDIT: It does not actually compile, I made a mistake with the provided C code here and can't figure out which code actually compiled to this. It's not that important anyways as the main target was the translation of ASM to C and the problem is solved now. Thanks everyone for participating.
So the first 2 lines are basically useless and just the value is added to my address and nothing more. I need it to be interpreted as an address and retrieve the value at that point though.
The data stored at those places doesn't matter at this point. Essentially what I want to do is find a specific value in memory and I know a way of adding offsets and dereferencing pointers to get to my goal. The final step will just be a typecast from my address to the actual datatype at that point.
I know this may seem trivial to some of you, but I'm not super familiar with C, so I'm struggling here...
You can simplify your asm to a single instruction, with the math done at assemble time. movabs rax, [0xdeadbeef + 0xa] can use the AL/AX/EAX/RAX-only form of mov that loads from a 64-bit absolute address (https://felixcloutier.com/x86/MOV.html). (It won't fit in a 32-bit sign-extended disp32, because the high bit of the low 32 is set, unlike normal static addresses in position-dependent code). Regular mov with a 32-bit address-size override would work, too, in about 7 bytes, because your address does fit in a zero-extended 32-bit integer.
In C you can also do the whole thing with a single statement. No need to overcomplicate things: your address is a pointer to a pointer, so you need to cast your integer to a x ** type.
void *ptr = *(const void**)(0xdeadbeefUL + 0xa);
In asm pointers are just integers, so it makes sense to do your math using integers instead of char*. Making it unsigned guarantees it zero-extends to pointer-width instead of sign-extending.
(Numeric literals in C have a type wide enough to represent the value, though, so 0xdeadbeef on an x86-64 compiler would be an int64_t (long long). You wouldn't actually get 0xdeadbeef being a negative 32-bit int that sign-extended to 0xffffffffdeadbeef.)
Since void doesn't have a size, you can't add / subtract integers to a void*. And pointer-math on void ** would be in chunks of sizeof(void*).
To avoid undefined behaviour from dereferencing a void** that's not aligned by 8 = alignof(void*) (in both mainstream x86-64 ABIs), you'd want to use memcpy. But I assume your example address is just a fake example. The mainstream x86 compilers like gcc don't do anything weird with unaligned addresses to punish programmers for UB, so the compiler output will contain unaligned loads which work fine on x86. But when auto-vectorizing you can run into problems from this kind of UB. Why does unaligned access to mmap'ed memory sometimes segfault on AMD64?
But if you did for some reason want to break things up into multiple asm statements, you could transliterate it into multiple C statements like this:
uintptr_t wheres_the_beef = 0xdeadbeef; // mov eax, 0xdeadbeef
wheres_the_beef += 0xa; // add eax, 0xa
void **address = (void**)wheres_the_beef; // purely a cast, no asm instructions;
void *ptr = *address; // mov rax, [rax]
You could mess around with char* if you wanted to add byte offsets to pointers, but there's really no point here.
Again, this still has undefined behaviour on most C implementations, where alignof(void*) is greater than 1 so void **address = (void**)wheres_the_beef creates a misaligned pointer.
(Fun fact: even creating misaligned pointers is UB in ISO C. But all x86 compilers that support Intel's intrinsics must support creating of misaligned pointers for passing them to intrinsics like _mm_loadu_ps(), so only actually dereferencing them is a potential problem on x86 compilers.)
Example Showing the gcc Optimization and User Code that May Fault
The function 'foo' in the snippet below will load only one of the struct members A or B; well at least that is the intention of the unoptimized code.
typedef struct {
int A;
int B;
} Pair;
int foo(const Pair *P, int c) {
int x;
if (c)
x = P->A;
else
x = P->B;
return c/102 + x;
}
Here is what gcc -O3 gives:
mov eax, esi
mov edx, -1600085855
test esi, esi
mov ecx, DWORD PTR [rdi+4] <-- ***load P->B**
cmovne ecx, DWORD PTR [rdi] <-- ***load P->A***
imul edx
lea eax, [rdx+rsi]
sar esi, 31
sar eax, 6
sub eax, esi
add eax, ecx
ret
So it appears that gcc is allowed to speculatively load both struct members in order to eliminate branching. But then, is the following code considered undefined behavior or is the gcc optimization above illegal?
#include <stdlib.h>
int naughty_caller(int c) {
Pair *P = (Pair*)malloc(sizeof(Pair)-1); // *** Allocation is enough for A but not for B ***
if (!P) return -1;
P->A = 0x42; // *** Initializing allocation only where it is guaranteed to be allocated ***
int res = foo(P, 1); // *** Passing c=1 to foo should ensure only P->A is accessed? ***
free(P);
return res;
}
If the load speculation will happen in the above scenario there is a chance that loading P->B will cause an exception because the last byte of P->B may lie in unallocated memory. This exception will not happen if the optimization is turned off.
The Question
Is the gcc optimization shown above of load speculation legal? Where does the spec say or imply that it's ok?
If the optimization is legal, how is the code in 'naughtly_caller' turn out to be undefined behavior?
Reading a variable (that was not declared as volatile) is not considered to be a "side effect" as specified by the C standard. So the program is free to read a location and then discard the result, as far as the C standard is concerned.
This is very common. Suppose you request 1 byte of data from a 4 byte integer. The compiler may then read the whole 32 bits if that's faster (aligned read), and then discard everything but the requested byte. Your example is similar to this but the compiler decided to read the whole struct.
Formally this is found in the behavior of "the abstract machine", C11 chapter 5.1.2.3. Given that the compiler follows the rules specified there, it is free to do as it pleases. And the only rules listed are regarding volatile objects and sequencing of instructions. Reading a different struct member in a volatile struct would not be ok.
As for the case of allocating too little memory for the whole struct, that's undefined behavior. Because the memory layout of the struct is usually not for the programmer to decide - for example the compiler is allowed to add padding at the end. If there's not enough memory allocated, you might end up accessing forbidden memory even though your code only works with the first member of the struct.
No, if *P is allocated correctly P->B will never be in unallocated memory. It might not be initialized, that is all.
The compiler has every right to do what they do. The only thing that is not allowed is to oops about the access of P->B with the excuse that it is not initialized. But what and how they do all of this is under the discretion of the implementation and not your concern.
If you cast a pointer to a block returned by malloc to Pair* that is not guaranteed to be wide enough to hold a Pair the behavior of your program is undefined.
This is perfectly legal because reading some memory location isn't considered an observable behavior in the general case (volatile would change this).
Your example code is indeed undefined behavior, but I can't find any passage in the standard docs that explicitly states this. But I think it's enough to have a look at the rules for effective types ... from N1570, ยง6.5 p6:
If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value.
So, your write access to *P actually gives that object the type Pair -- therefore it just extends into memory you didn't allocate, the result is an out of bounds access.
A postfix expression followed by the -> operator and an identifier designates a member of a structure or union object. The value is that of the named member of the object to which the first expression points
If invoking the expression P->A is well-defined, then P must actually point to an object of type struct Pair, and consequently P->B is well-defined as well.
A -> operator on a Pair * implies that there's a whole Pair object fully allocated. (#Hurkyl quotes the standard.)
x86 (like any normal architecture) doesn't have side-effects for accessing normal allocated memory, so x86 memory semantics are compatible with the C abstract machine's semantics for non-volatile memory. Compilers can speculatively load if/when they think that will be a performance win on target microarchitecture they're tuning for in any given situation.
Note that on x86 memory protection operates with page granularity. The compiler could unroll a loop or vectorize with SIMD in a way that reads outside an object, as long as all pages touched contain some bytes of the object. Is it safe to read past the end of a buffer within the same page on x86 and x64?. libc strlen() implementations hand-written in assembly do this, but AFAIK gcc doesn't, instead using scalar loops for the leftover elements at the end of an auto-vectorized loop even where it already aligned the pointers with a (fully unrolled) startup loop. (Perhaps because it would make runtime bounds-checking with valgrind difficult.)
To get the behaviour you were expecting, use a const int * arg.
An array is a single object, but pointers are different from arrays. (Even with inlining into a context where both array elements are known to be accessible, I wasn't able to get gcc to emit code like it does for the struct, so if it's struct code is a win, it's a missed optimization not to do it on arrays when it's also safe.).
In C, you're allowed to pass this function a pointer to a single int, as long as c is non-zero. When compiling for x86, gcc has to assume that it could be pointing to the last int in a page, with the following page unmapped.
Source + gcc and clang output for this and other variations on the Godbolt compiler explorer
// exactly equivalent to const int p[2]
int load_pointer(const int *p, int c) {
int x;
if (c)
x = p[0];
else
x = p[1]; // gcc missed optimization: still does an add with c known to be zero
return c + x;
}
load_pointer: # gcc7.2 -O3
test esi, esi
jne .L9
mov eax, DWORD PTR [rdi+4]
add eax, esi # missed optimization: esi=0 here so this is a no-op
ret
.L9:
mov eax, DWORD PTR [rdi]
add eax, esi
ret
In C, you can pass sort of pass an array object (by reference) to a function, guaranteeing to the function that it's allowed to touch all the memory even if the C abstract machine doesn't. The syntax is int p[static 2]
int load_array(const int p[static 2], int c) {
... // same body
}
But gcc doesn't take advantage, and emits identical code to load_pointer.
Off topic: clang compiles all versions (struct and array) the same way, using a cmov to branchlessly compute a load address.
lea rax, [rdi + 4]
test esi, esi
cmovne rax, rdi
add esi, dword ptr [rax]
mov eax, esi # missed optimization: mov on the critical path
ret
This isn't necessarily good: it has higher latency than gcc's struct code, because the load address is dependent on a couple extra ALU uops. It is pretty good if both addresses aren't safe to read and a branch would predict poorly.
We can get better code for the same strategy from gcc and clang, using setcc (1 uop with 1c latency on all CPUs except some really ancient ones), instead of cmovcc (2 uops on Intel before Skylake). xor-zeroing is always cheaper than an LEA, too.
int load_pointer_v3(const int *p, int c) {
int offset = (c==0);
int x = p[offset];
return c + x;
}
xor eax, eax
test esi, esi
sete al
add esi, dword ptr [rdi + 4*rax]
mov eax, esi
ret
gcc and clang both put the final mov on the critical path. And on Intel Sandybridge-family, the indexed addressing mode doesn't stay micro-fused with the add. So this would be better, like what it does in the branching version:
xor eax, eax
test esi, esi
sete al
mov eax, dword ptr [rdi + 4*rax]
add eax, esi
ret
Simple addressing modes like [rdi] or [rdi+4] have 1c lower latency than others on Intel SnB-family CPUs, so this might actually be worse latency on Skylake (where cmov is cheap). The test and lea can run in parallel.
After inlining, that final mov probably wouldn't exist, and it could just add into esi.
This is always allowed under the "as-if" rule if no conforming program can tell the difference. For example, an implementation could guarantee that after each block allocated with malloc, there are at least eight bytes that can be accessed without side effects. In that situation, the compiler can generate code that would be undefined behaviour if you wrote it in your code. So it would be legal for the compiler to read P[1] whenever P[0] is correctly allocated, even if that would be undefined behaviour in your own code.
But in your case, if you don't allocate enough memory for a struct, then reading any member is undefined behaviour. So here the compiler is allowed to do this, even if reading P->B crashes.
I'm trying to replicate an x86 mov instruction, such as mov %ecx,-0x4(%ebp) in C and am confused about how to do it. I have an int array for the registers and an int displacement. How would I move the value of %ecx into the memory address 4 less than the value stored in %ebp?
I have:
int* destAddress=(int*)(displacement + registers[destination]);
*destAddress=registers[source];
I'm getting a Warning: cast to pointer from integer of different size.
mov %ecx,-0x4(%ebp)
or, in Intel syntax:
mov DWORD PTR [ebp-4], ecx
is storing the value in ECX into the memory location [ebp-4].
EBP is the "base pointer" and is commonly used (in unoptimized code) to access data on the stack. Based on the negative offset, this instruction is almost certainly storing the value of ECX into the first DWORD-sized local variable.
If you wanted to translate this to C, it would be:
int local = value;
assuming that value is mapped to the ECX register, and local is a local variable allocated on the stack. Really, that's it.
[Except that a C compiler would generally put a local variable like this in a register, so this would really translate to something more like mov edx, ecx. The only time it would spill to stack would be if it ran out of registers (which isn't uncommon in the very register-poor x86 ISA).Alternatively, you could force it to spill by making the variable volatile: volatile int local = value;.But there is no good reason for doing that in real code.]
There is pointer dereferencing going on here under the hood, of course, as you see in the assembly-language instruction, but it doesn't manifest in the C representation.
If you wanted to get some pointer notation in there, say you had an array of values allocated on the stack, and wanted to initialize its first member:
int array[4];
array[0] = value; // set first element of array to 'value' (== ECX)
The displacement (-4) won't appear at all in the C code. The C compiler handles that.
If ECX holds a pointer to a structure or an array, is MOV DWORD ptr ds:[ECX], ECX equivalent to MOV DWORD ptr ds:ECX, ECX
I have attached this image:
When I first saw that line, it got me to think, in the high level language context if it is indeed:
1) a pointer to a function which does nothing at all, but calls itself, thus achieving recursion instead.
2) an array/structure of x datatype whose first index is a pointer to the array/structure itself OR the structure/array's first entry. And hence my question in the title.
Other than that it seems to me pretty arbitrary: (an address storing the value of the said address which happens the address itself?). Any thought on that?
Under MASM MOV DWORD ptr ds:[ECX], ECX is not equivalent to MOV DWORD ptr ds:ECX, ECX because the former is syntactically valid while the later is not.
As for your other questions:
1) If a function pointer is stored in ECX then this instruction would modify the function, which generally isn't permitted under most operating systems and so cause a crash. Even the write were allowed it would change the instruction or instructions at the start of function that when executed wouldn't do anything useful and probably crash. It would never result in recursion.
2) Probably something like this.
It doesn't look arbitrary to me.
I want to understand how space allocation is done for variables on stack.
Here for this C program with no variables
main() { return 0; }
It's disassembly is
push ebp
mov ebp, esp
sub esp, 0c0h
main() {
int i = 10; }
The dis-assembly for this program is
push ebp
mov ebp, esp
sub esp, 0cch
I am initializing an INT variable, whose size is 4 bytes. But in the above dis-assembly compiler is allocating 12 bytes (0cc-0c0).
For the following program
main() { long long int i = 10LL; }
The disassembly is
push ebp
mov ebp, esp
sub esp, 0D0h
In the above disassembly compiler is allocating 16 bytes(0D0 - 0C0) for long long int, whose size is 8 bytes.
Why is compiler assigning 12 bytes(4 bytes extra allocated. It should be 8 byte or 16 byte aligned) for INT, whose size is 4 bytes and 16 bytes for LONG LONG INT, whose size is 8 bytes?
Can someone please clarify this.
Thanks.
The compiler is free to allocate as much extra storage as it wants. The C standard does not dictate constraints on the stack allocation.
EDIT:
I did some experimentation on godbolt with the ICC compiler, the only compiler that generates code like your example. I disproved myself about the arguments to main thing I mentioned before. I also tried creating some character arrays and found that the stack will always allocate in increments of 16 bytes. A char array of 1-16 bytes all cause a 16-byte allocation. Next 17-32 will cause a 32-byte allocation and so on.