X86 Assembly Problems - c

Question

I'm attempting to get through a book on X86 that was written using examples from Visual C++ and Visual Studio. I'm trying to convert the examples for use with gcc. After a number of problems, I finally wound up with code that would at least compile, but now I'm getting segfaults. Here's the code:
assembly.s:
.intel_syntax noprefix
.section .text
.globl CalcSum
.type CalcSum, #function
// extern "C" int CalcSum_(int a, int b, int c)
CalcSum:
// Initialize a stack frame pointer
pushq rbp
mov ebp,esp
// Load the argument values
mov eax,[ebp+8]
mov ecx,[ebp+12]
mov edx,[ebp+16]
// Calculate the sum
add eax, ecx
add eax, edx
// Restore the caller's stack frame pointer
popq rbp
ret
test.c:
#include <stdio.h>
extern int CalcSum(int a, int b, int c);
int main() {
int sum = CalcSum(5,6,7);
printf(" result: %d\n",sum);
return 0;
}
I'm using gcc -o execute test.c assembly.s to compile. If I change all the 32 bit instructions to 64 bit (i.e. ebp to rbp) it will run but give completely random output. Could anyone point out what I'm doing wrong here? Thanks!

Answer 1

As hinted in the comments, it's a matter of calling convention. 32-bit C functions follow the CDECL calling convention in Windows and in Linux. In 64-bit Linux you have to use the System V AMD64 ABI. The 64-bit calling convention of Windows is different. There might be specifics to use functions of the operating system.
32-bit C (GCC):
.intel_syntax noprefix
.section .text
.globl CalcSum
.type CalcSum, #function
// extern "C" int CalcSum_(int a, int b, int c)
CalcSum: // with underscore in Windows: _CalcSum
// Initialize a stack frame pointer
push ebp
mov ebp,esp
// Load the argument values
mov eax,[ebp+8]
mov ecx,[ebp+12]
mov edx,[ebp+16]
// Calculate the sum
add eax, ecx
add eax, edx
// Restore the caller's stack frame pointer
pop ebp
ret
64-bit Linux (GCC):
.intel_syntax noprefix
.section .text
.globl CalcSum
.type CalcSum, #function
// extern "C" int CalcSum_(int a, int b, int c)
CalcSum:
// Load the argument values
mov rax, rdi
add rax, rsi
add rax, rdx
ret
64-bit Windows (MingW-GCC):
.intel_syntax noprefix
.section .text
.globl CalcSum
// .type CalcSum, #function
// extern "C" int CalcSum_(int a, int b, int c)
CalcSum:
// Load the argument values
mov rax, rcx
add rax, rdx
add rax, r8
ret