Pass values from C program to Assembly language - c

Question

I would like to pass values from C program to Assembly using the linked assembly method instead of inline assembly method in C.
Below is the Assembly program(GCD) which is am working on.
;gcdasm.nasm
bits 64
section .text
global gcdasm
gcdasm:
push rbp
mov rbp, rsp
mov rax, [rbp+4] ;load rax with x
mov rbx, [rbp+8] ;load rbx with y
top:
cmp rax, rbx ;x(rax) has to be larger than y(rbx)
je exit ;if x=y then exit and return value y
jb xchange ;if x<y then swap x and y
modulo:
cqo ;RDX:RAX sign extend
div rbx ;div rdx:rax with rbx
cmp rdx, 0 ;check remider if its 0
je exit ;if reminder is 0 then exit return return y
mov rax, rdx ;reminder rdx as next dividend
jmp modulo ;loop
xchange:
xchg rax, rbx ;swap x and y
jmp modulo
exit:
mov rax, rbx ;Return c program with the divisor y
mov rsp, rbp
pop rbp
ret
And this is the C program from with I am trying to pass the values to assembly program
//gcd.c
#include<stdio.h>
extern int gcdasm(int x, int y);
int main(void){
int x=0;
int y=0;
int result=0;
x = 46;
y = 90;
printf("%d and %d have a gcd of %d\n", x,y,gcdasm(x,y));
x = 55;
y = 66;
printf("%d and %d have a gcd of %d\n", x,y,gcdasm(x,y));
return 0;
}
When I compile using the below method and run it. I get either error Floating point exception or an empty prompt waiting for input
$ nasm -felf64 gcdasm.nasm -o gcdasm.o
$ gcc gcdasm.o gcd.c -o gcd
$ ./gcd
Floating point exception
$ ./gcd
I am unable to figure out the error. Kindly help me out.
Thank you.

Answer 1

Passing arguments to gcdasm()
The two int arguments are passed through registers, not the stack. The first and second arguments are passed in the lower-half of rdi and rsi (i.e.: edi and esi), respectively. So, by sign extending edi and esi into rax and rbx respectively, you load the passed arguments into those registers:
movsx rax, edi ;load rax with x
movsx rbx, esi ;load rbx with y
However, note that rbx is not a scratch register, therefore the callee needs to save it before modifying it and then restore it back before leaving the gcdasm function.
You can simply replace rbx by rcx (which isn't a callee-saved register) everywhere in your code. You don't need rbp at all, so you can remove all the instructions where rbp appears.
Other problems
There is also a problem with the logic of the program with:
mov rax, rdx ;reminder rdx as next dividend
Instead of this, the divisor (rcx) should become the dividend (rax) and the remainder (rdx) should become the divisor (rcx), that is:
mov rax, rcx
mov rcx, rdx
When dividing signed values, you have to use the idiv instruction, not div.
Improvement
There are also some reasons regarding performance and code size to use test rdx, rdx instead of cmp rdx, 0 for comparing rdx against zero.
With all that above in mind:
;gcdasm.nasm
bits 64
section .text
global gcdasm
gcdasm:
movsx rax, edi ;load rax with x
movsx rcx, esi ;load rcx with y
top:
cmp rax, rcx ;x(rax) has to be larger than y(rcx)
je exit ;if x=y then exit and return value y
jb xchange ;if x<y then swap x and y
modulo:
cqo ;sign extend RDX:RAX
idiv rcx ;rdx:rax/rcx (signed values)
test rdx, rdx ;check whether remainder is zero
je exit ;if reminder is 0 then exit return y
mov rax, rcx ;divisor becomes dividend
mov rcx, rdx ;remainder becomes divisor
jmp modulo ;loop
xchange:
xchg rax, rcx ;swap x and y
jmp modulo
exit:
mov rax, rcx ;Return c program with the divisor y
ret