Try to understand calling process in assembly code - c

I wrote a very simple program in C and try to understand the function calling process.
#include "stdio.h"
void Oh(unsigned x) {
printf("%u\n", x);
}
int main(int argc, char const *argv[])
{
Oh(0x67611c8c);
return 0;
}
And its assembly code seems to be
0000000100000f20 <_Oh>:
100000f20: 55 push %rbp
100000f21: 48 89 e5 mov %rsp,%rbp
100000f24: 48 83 ec 10 sub $0x10,%rsp
100000f28: 48 8d 05 6b 00 00 00 lea 0x6b(%rip),%rax # 100000f9a <_printf$stub+0x20>
100000f2f: 89 7d fc mov %edi,-0x4(%rbp)
100000f32: 8b 75 fc mov -0x4(%rbp),%esi
100000f35: 48 89 c7 mov %rax,%rdi
100000f38: b0 00 mov $0x0,%al
100000f3a: e8 3b 00 00 00 callq 100000f7a <_printf$stub>
100000f3f: 89 45 f8 mov %eax,-0x8(%rbp)
100000f42: 48 83 c4 10 add $0x10,%rsp
100000f46: 5d pop %rbp
100000f47: c3 retq
100000f48: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
100000f4f: 00
0000000100000f50 <_main>:
100000f50: 55 push %rbp
100000f51: 48 89 e5 mov %rsp,%rbp
100000f54: 48 83 ec 10 sub $0x10,%rsp
100000f58: b8 8c 1c 61 67 mov $0x67611c8c,%eax
100000f5d: c7 45 fc 00 00 00 00 movl $0x0,-0x4(%rbp)
100000f64: 89 7d f8 mov %edi,-0x8(%rbp)
100000f67: 48 89 75 f0 mov %rsi,-0x10(%rbp)
100000f6b: 89 c7 mov %eax,%edi
100000f6d: e8 ae ff ff ff callq 100000f20 <_Oh>
100000f72: 31 c0 xor %eax,%eax
100000f74: 48 83 c4 10 add $0x10,%rsp
100000f78: 5d pop %rbp
100000f79: c3 retq
Well, I don't quite understand the argument passing process, since there is only one parameter passed to Oh function, I could under stand this
100000f58: b8 8c 1c 61 67 mov $0x67611c8c,%eax
So what does the the code below do? Why rbp? Isn't it abandoned in X86-64 assembly? If it is a x86 style assembly, how can I generate the x86-64 style assembly using clang? If it is x86, it doesn't matter, could any one explains the below code line by line for me?
100000f5d: c7 45 fc 00 00 00 00 movl $0x0,-0x4(%rbp)
100000f64: 89 7d f8 mov %edi,-0x8(%rbp)
100000f67: 48 89 75 f0 mov %rsi,-0x10(%rbp)
100000f6b: 89 c7 mov %eax,%edi
100000f6d: e8 ae ff ff ff callq 100000f20 <_Oh>

You might get cleaner code if you turned optimizations on, or you might not. But, here’s what that does.
The %rbp register is being used as a frame pointer, that is, a pointer to the original top of the stack. It’s saved on the stack, stored, and restored at the end. Far from being removed in x86_64, it was added there; the 32-bit equivalent was %ebp.
After this value is saved, the program allocates sixteen bytes off the stack by subtracting from the stack pointer.
There then is a very inefficient series of copies that sets the first argument of Oh() as the second argument of printf() and the constant address of the format string (relative to the instruction pointer) as the first argument of printf(). Remember that, in this calling convention, the first argument is passed in %rdi (or %edi for 32-bit operands) and the second in %rsi This could have been simplified to two instructions.
After calling printf(), the program (needlessly) saves the return value on the stack, restores the stack and frame pointers, and returns.
In main(), there’s similar code to set up the stack frame, then the program saves argc and argv (needlessly), then it moves around the constant argument to Oh into its first argument, by way of %eax. This could have been optimized into a single instruction. It then calls Oh(). On return, it sets its return value to 0, cleans up the stack, and returns.
The code you’re asking about does the following: stores the constant 32-bit value 0 on the stack, saves the 32-bit value argc on the stack, saves the 64-bit pointer argv on the stack (the first and second arguments to main()), and sets the first argument of the function it is about to call to %eax, which it had previously loaded with a constant. This is all unnecessary for this program, but would have been necessary had it needed to use argc and argv after the call, when those registers would have been clobbered. There’s no good reason it used two steps to load the constant instead of one.

As Jester mentions you still have frame pointers on (to aid debugging)so stepping through main:
0000000100000f50 <_main>:
First we enter a new stack frame, we have to save the base pointer and move the stack to the new base. Also, in x86_64 the stack frame has to be aligned to a 16 byte boundary (hence moving the stack pointer by 0x10).
100000f50: push %rbp
100000f51: mov %rsp,%rbp
100000f54: sub $0x10,%rsp
As you mention, x86_64 passes parameters by register, so load the param in to the register:
100000f58: mov $0x67611c8c,%eax
??? Help needed
100000f5d: movl $0x0,-0x4(%rbp)
From here: "Registers RBP, RBX, and R12-R15 are callee-save registers", so if we want to save other resisters then we have to do it ourselves ....
100000f64: mov %edi,-0x8(%rbp)
100000f67: mov %rsi,-0x10(%rbp)
Not really sure why we didn't just load this in %edi where it needs to be for the call to begin with, but we better move it there now.
100000f6b: mov %eax,%edi
Call the function:
100000f6d: callq 100000f20 <_Oh>
This is the return value (passed in %eax), xor is a smaller instruction than load 0, so is a cmmon optimization:
100000f72: xor %eax,%eax
Clean up that stack frame we added earlier (not really sure why we saved those registers on it when we didn't use them)
100000f74: add $0x10,%rsp
100000f78: pop %rbp
100000f79: retq

Related

How does the transfer of parameters between functions work in assembly?

So I'm trying to understand how assembly programming works with stack frames etc.
I did some exercises and disassembled some C-code with GDB. The task now is to find out how the transfer of parameters between the 'main' and functions works. I just started learning and kinda got lost on what next example is actually doing. Any ideas or tips on where to get started?
It's a recursive program working with faculty.
The assembly code looks like this:
1149: f3 0f 1e fa endbr64
114d: 55 push rbp
114e: 48 89 e5 mov rbp,rsp
1151: 48 83 ec 10 sub rsp,0x10
1155: 89 7d fc mov DWORD PTR [rbp-0x4],edi
1158: 83 7d fc 01 cmp DWORD PTR [rbp-0x4],0x1
115c: 76 13 jbe 1171 <f+0x28>
115e: 8b 45 fc mov eax,DWORD PTR [rbp-0x4]
1161: 83 e8 01 sub eax,0x1
1164: 89 c7 mov edi,eax
1166: e8 de ff ff ff call 1149 <f>
116b: 0f af 45 fc imul eax,DWORD PTR [rbp-0x4]
116f: eb 05 jmp 1176 <f+0x2d>
1171: b8 01 00 00 00 mov eax,0x1
1176: c9 leave
1177: c3 ret
1178: f3 0f 1e fa endbr64
117c: 55 push rbp
117d: 48 89 e5 mov rbp,rsp
1180: 48 83 ec 10 sub rsp,0x10
1184: c7 45 f8 05 00 00 00 mov DWORD PTR [rbp-0x8],0x5
118b: c7 45 fc 00 00 00 00 mov DWORD PTR [rbp-0x4],0x0
1192: 8b 45 f8 mov eax,DWORD PTR [rbp-0x8]
1195: 89 c7 mov edi,eax
1197: e8 ad ff ff ff call 1149 <f>
119c: 89 45 fc mov DWORD PTR [rbp-0x4],eax
Study the calling convention for your environment.  An overview of the many calling conventions for a number of architectures: https://www.dyncall.org/docs/manual/manualse11.html
The calling convention specifies:
Where parameters and return values must appear at the one single point of transfer of control of the instruction stream from the caller to the callee.  For parameters being passed, that single point is after the call is made and before the first instruction of the callee (and for return values, at the point where the callee finishes and just before execution resumes in the caller).
Many conventions combine parameter passing in CPU registers with stack memory for parameters that don't fit in CPU registers.  And even some that don't use CPU registers for parameters still use CPU registers for return values.
What registers a function is allowed to clobber vs. must preserve.  Call-clobbered registers can be assigned new values without concern.  Call-preserved registers can be used but must be restored to the value they had upon entry before returning to the caller.  The advantage of call-preserved registers is that since they are preserved by a call, you can use them for variables that need to survive another call.
The meaning & treatment of the stack pointer, regarding memory below and above the current pointer, and alignment requirements for stack allocation.
If the function allocates stack space in some manner, then memory parameters will appear to move farther away from the top of the stack (they don't actually move, of course, but become larger offsets from the current stack or frame pointer).  Compilers know this and adjust their access to stack memory accordingly.
Some compilers set up frame pointers to refer to stack memory.  A frame pointer is a copy of the stack pointer made at some point in the prologue.  Frame pointers are not always necessary but facilitate exception handling and stack unwinding, as well as dynamic stack allocation.

How does this assembly function return a value?

I have made a function in C which is pretty straightforward, it uses strlen() from <string.h> to return the length of a char* variable:
int length(char *str) {
return strlen(str);
}
Here is the corresponding x86_64 assembly from objdump -M intel -d a.out:
00000000000011a8 <length>:
11a8: f3 0f 1e fa endbr64
11ac: 55 push rbp
11ad: 48 89 e5 mov rbp,rsp
11b0: 48 83 ec 10 sub rsp,0x10
11b4: 48 89 7d f8 mov QWORD PTR [rbp-0x8],rdi
11b8: 48 8b 45 f8 mov rax,QWORD PTR [rbp-0x8]
11bc: 48 89 c7 mov rdi,rax
11bf: e8 ac fe ff ff call 1070 <strlen#plt>
11c4: c9 leave
11c5: c3 ret
Here is my current understanding of the code (please correct me if anything seems wrong):
00000000000011a8 <length>:
11a8: f3 0f 1e fa endbr64
11ac: 55 push rbp // stack setup, old rbp of previous frame pushed
11ad: 48 89 e5 mov rbp,rsp // rbp and rsp point to same place
11b0: 48 83 ec 10 sub rsp,0x10 // space is made for arguments
11b4: 48 89 7d f8 mov QWORD PTR [rbp-0x8],rdi // rdi stores argument and is moved into the space made on the line 11b0
11b8: 48 8b 45 f8 mov rax,QWORD PTR [rbp-0x8] // value at memory address rbp-0x8 aka argument is stored in rax
11bc: 48 89 c7 mov rdi,rax // move the value into rdi for function call
11bf: e8 ac fe ff ff call 1070 <strlen#plt> // strlen() is called
11c4: c9 leave // stack clear up
11c5: c3 ret // return address popped and control flow resumes
If anything above is incorrect please correct me, secondly how does call 1070 <strlen#plt> return a value? because the strlen() function returns the length of a string and i would have thought that something would have been moved into the rax register (which i believe is commonly used for return values). But nothing is moved into rax and it does not show a value returned in the assembly.
Lastly here is the code at address 1070 (from call 1070 strlen#plt)
0000000000001070 <strlen#plt>:
1070: f3 0f 1e fa endbr64
1074: f2 ff 25 45 2f 00 00 bnd jmp QWORD PTR [rip+0x2f45] # 3fc0 <strlen#GLIBC_2.2.5>
107b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
how does call 1070 strlen#plt return a value?
The strlen puts its result into rax register, which conveniently is also where your length() function should put its return value.
Under optimization your length() could be compiled into a single instruction: jmp strlen -- the parameter is already in rdi, and the return value will be in rax.
P.S.
Lastly here is the code at address 1070
That isn't the actual code of strlen. This is a "PLT jump stub". To understand what that is, you could read this blog post.
Also, from that small address, you can see this is a PIE executable: those are just offsets from the image base address; the runtime address will be something like 0x55...

What happens to arguments pushed on the stack when a function call returns?

Assume this C code:
int add(int a, int b){
int c = a+b;
return c;
}
int main(){
int c = add(3,4);
printf("%d", c);
return 0;
}
When calling add the following happens:
Push 4 on the stack
Push 3 on the stack
Push the address of `printf` on the stack (return address)
call `add`
// do stuff under add
pop the stack and and goto `printf`
But values 4 and 3 under add are still on the stack -0x18(%rbp). Are they not supposed to be cleared?
00000000000005fa <add>:
5fa: 55 push %rbp
5fb: 48 89 e5 mov %rsp,%rbp
5fe: 89 7d ec mov %edi,-0x14(%rbp)
601: 89 75 e8 mov %esi,-0x18(%rbp)
604: 8b 55 ec mov -0x14(%rbp),%edx
607: 8b 45 e8 mov -0x18(%rbp),%eax
60a: 01 d0 add %edx,%eax
60c: 89 45 fc mov %eax,-0x4(%rbp)
60f: 8b 45 fc mov -0x4(%rbp),%eax
612: 5d pop %rbp
613: c3 retq
0000000000000614 <main>:
614: 55 push %rbp
615: 48 89 e5 mov %rsp,%rbp
618: be 04 00 00 00 mov $0x4,%esi
61d: bf 03 00 00 00 mov $0x3,%edi
622: e8 d3 ff ff ff callq 5fa <add>
627: 5d pop %rbp
628: c3 retq
But values 4 and 3 under add are still on the stack
Not really, because stack no longer covers the memory where 4 and 3 are stored. They are in the memory beyond the stack pointer. Although the values are still in the place in memory that used to be part of stack, they are now part of uninitialized garbage. Accessing them after the stack has moved on, e.g. by storing their addresses, would be undefined behavior.
Are they not supposed to be cleared?
No, clearing these values is unnecessary, and could impact run-time efficiency of the program.
When you call the pop instruction, the value at the top of the stack is stored in the specified register and the SP (stack pointer) register is incremented (on x86 the stack grows down). It does NOT modify the memory location that was previously the top of the stack as there's no need to do so.
Whatever value was at the top of the stack will remain there until a new value is pushed on top of it.

Understanding how parameters are passed to functions in Assembly

I am reading this writeup on how to perform a ret2libc exploit. It states that the arguments passed to a function are stored at ebp+8.
Now, if I take a simple C program
#include <stdlib.h>
int main() {
system("/bin/sh");
}
And compile it:
gcc -m32 -o test_sh test_sh.c
and look at the disassembly of it with
objdump -d -M intel test_sh
0804840b <main>:
804840b: 8d 4c 24 04 lea ecx,[esp+0x4]
804840f: 83 e4 f0 and esp,0xfffffff0
8048412: ff 71 fc push DWORD PTR [ecx-0x4]
8048415: 55 push ebp
8048416: 89 e5 mov ebp,esp
8048418: 51 push ecx
8048419: 83 ec 04 sub esp,0x4
804841c: 83 ec 0c sub esp,0xc
804841f: 68 c4 84 04 08 push 0x80484c4
8048424: e8 b7 fe ff ff call 80482e0 <system#plt>
8048429: 83 c4 10 add esp,0x10
804842c: b8 00 00 00 00 mov eax,0x0
8048431: 8b 4d fc mov ecx,DWORD PTR [ebp-0x4]
8048434: c9 leave
8048435: 8d 61 fc lea esp,[ecx-0x4]
8048438: c3 ret
8048439: 66 90 xchg ax,ax
804843b: 66 90 xchg ax,ax
804843d: 66 90 xchg ax,ax
804843f: 90 nop
The line
804841f: 68 c4 84 04 08 push 0x80484c4
Pushes the address of the string "/bin/sh" onto the stack. Immediately afterwards the system#plt function is called. So how does one arrive to ebp+8 from the above output?
Help would be appreciated!
The arguments passed to a function are stored at ebp+8.
That's from the called function's point of view, not from the calling function's point of view. The calling function has its own arguments at ebp+8, and your main() does not use any of its arguments, hence, you don't see any use of ebp+8 in your main().
You can see ebp+8 being used as follows:
Try writing a second function, with arguments, and calling it from main(), instead of invoking system(). You still won't see any use of ebp+8 within main(), but you will see it being used in your second function.
Try declaring your main() to accept its char** argv argument, then try sending argv[0] to printf().
You don't because EBP + 8 is only relevant after the prologue in system#plt after creating a new procedure frame.
push ebp
mov ebp, esp
at this point in system#plt the contents of memory location pointed to by EBP + 8 will equal 0x80484C4.

Assembly - why is %rsp decremented by so much, and why are arguments stored at the top of the stack?

Assembly newbie here... I wrote the following simple C program:
void fun(int x, int* y)
{
char arr[4];
int* sp;
sp = y;
}
int main()
{
int i = 4;
fun(i, &i);
return 0;
}
I compiled it with gcc and ran objdump with -S, but the Assembly code output is confusing me:
000000000040055d <fun>:
void fun(int x, int* y)
{
40055d: 55 push %rbp
40055e: 48 89 e5 mov %rsp,%rbp
400561: 48 83 ec 30 sub $0x30,%rsp
400565: 89 7d dc mov %edi,-0x24(%rbp)
400568: 48 89 75 d0 mov %rsi,-0x30(%rbp)
40056c: 64 48 8b 04 25 28 00 mov %fs:0x28,%rax
400573: 00 00
400575: 48 89 45 f8 mov %rax,-0x8(%rbp)
400579: 31 c0 xor %eax,%eax
char arr[4];
int* sp;
sp = y;
40057b: 48 8b 45 d0 mov -0x30(%rbp),%rax
40057f: 48 89 45 e8 mov %rax,-0x18(%rbp)
}
400583: 48 8b 45 f8 mov -0x8(%rbp),%rax
400587: 64 48 33 04 25 28 00 xor %fs:0x28,%rax
40058e: 00 00
400590: 74 05 je 400597 <fun+0x3a>
400592: e8 a9 fe ff ff callq 400440 <__stack_chk_fail#plt>
400597: c9 leaveq
400598: c3 retq
0000000000400599 <main>:
int main()
{
400599: 55 push %rbp
40059a: 48 89 e5 mov %rsp,%rbp
40059d: 48 83 ec 10 sub $0x10,%rsp
int i = 4;
4005a1: c7 45 fc 04 00 00 00 movl $0x4,-0x4(%rbp)
fun(i, &i);
4005a8: 8b 45 fc mov -0x4(%rbp),%eax
4005ab: 48 8d 55 fc lea -0x4(%rbp),%rdx
4005af: 48 89 d6 mov %rdx,%rsi
4005b2: 89 c7 mov %eax,%edi
4005b4: e8 a4 ff ff ff callq 40055d <fun>
return 0;
4005b9: b8 00 00 00 00 mov $0x0,%eax
}
4005be: c9 leaveq
4005bf: c3 retq
First, in the line:
400561: 48 83 ec 30 sub $0x30,%rsp
Why is the stack pointer decremented so much in the call to 'fun' (48 bytes)? I assume it has to do with alignment issues, but I cannot visualize why it would need so much space (I only count 12 bytes for local variables (assuming 8 byte pointers))?
Second, I thought that in x86_64, the arguments to a function are either stored in specific registers, or if there are a lot of them, just 'above' (with a downward growing stack) the base pointer, %rbp. Like in the picture at http://en.wikipedia.org/wiki/Call_stack#Structure except 'upside-down'.
But the lines:
400565: 89 7d dc mov %edi,-0x24(%rbp)
400568: 48 89 75 d0 mov %rsi,-0x30(%rbp)
suggest to me that they are being stored way down from the base of the stack (%rsi and %edi are where main put the arguments, right before calling 'fun', and 0x30 down from %rbp is exactly where the stack pointer is pointing...). And when I try to do stuff with them , like assigning their values to local variables, it grabs them from those locations near the head of the stack:
sp = y;
40057b: 48 8b 45 d0 mov -0x30(%rbp),%rax
40057f: 48 89 45 e8 mov %rax,-0x18(%rbp)
... what is going on here?! I would expect them to grab the arguments from either the registers they were stored in, or just above the base pointer, where I thought they are 'supposed to be', according to every basic tutorial I read. Every answer and post I found on here related to stack frame questions confirms my understanding of what stack frames "should" look like, so why is my Assembly output so darn weird?
Because that stuff is a hideously simplified version of what really goes on. It's like wondering why Newtonian mechanics doesn't model the movement of the planets down to the millimeter. Compilers need stack space for all sorts of things. For example, saving callee-saved registers.
Also, the fundamental fact is that debug-mode compilations contain all sorts of debugging and checking machinery. The compiler outputs all sorts of code that checks that your code is correct, for example the call to __stack_chk_fail.
There are only two ways to understand the output of a given compiler. The first is to implement the compiler, or be otherwise very familiar with the implementation. The second is to accept that whatever you understand is a gross simplification. Pick one.
Because you're compiling without optimization, the compiler does lots of extra stuff to maybe make things easier to debug, which use lots of extra space.
it does not attempt to compress the stack frame to reuse memory for anything, or get rid of any unused things.
it redundantly copies the arguments into the stack frame (which requires still more memory)
it copies a 'canary' on to the stack to guard against stack smashing buffer overflows (even though they can't happen in this code).
Try turning on optimization, and you'll see more real code.
This is 64 bit code. 0x30 of stack space corresponds to 6 slots on the stack. You have what appears to be:
2 slots for function arguments (which happen also to be passed in registers)
2 slots for local variables
1 slot for saving the AX register
1 slot looks like a stack guard, probably related to DEBUG mode.
Best thing is to experiment rather than ask questions. Try compiling in different modes (DEBUG, optimisation, etc), and with different numbers and types of arguments and variables. Sometimes asking other people is just too easy -- you learn better by doing your own experiments.

Resources