Why does this assembly cause a segmentation fault? [duplicate] - c

This question already has answers here:
How to write assembly language hello world program for 64 bit Mac OS X using printf?
(1 answer)
Why does the x86-64 / AMD64 System V ABI mandate a 16 byte stack alignment?
(1 answer)
Closed 1 year ago.
I'm working on a hobby compiler project, and am getting frustrated because my generated assembly makes perfect sense to me but causes a segmentation fault on running. For the input printf("%d", strcmp("1", "2"));, the compiler generates:
.globl _main
_main:
pushq %rbp
movq %rsp, %rbp
subq $16, %rsp
leaq S0(%rip), %rax
movq %rax, %rdi
pushq %rdi
leaq S1(%rip), %rax
movq %rax, %rdi
leaq S2(%rip), %rax
movq %rax, %rsi
callq _strcmp
popq %rdi
movq %rax, %rsi
callq _printf
addq $16, %rsp
popq %rbp
retq
S0:
.asciz "%d"
S1:
.asciz "1"
S2:
.asciz "2"
The compiler starts by generating code to call printf, since that's what it encounters first. However, the second argument is a function call, so I thought it would be a good idea to push all the arguments I've already parsed, hence the pushq %rdi (and later popq %rdi). However, running the program causes a segfault without printing anything, and I'm not sure why. I'm guessing it's either due to stack alignment somehow or it's due to the offset based adressing (leaq S0(%rip), %rax) not being stored or retrieved correctly. Commenting out the instructions that push and pop %rdi and after the call to strcmp re-inserting leaq S0(%rip), %rdi before callq _printf causes the program to run as expected.
I've been banging my head against the wall trying to figure this out. If anyone can help figure out exactly what's wrong with the given assembly, it would be much appreciated.
System info: Working on a 2015 Mac running Mac OSX 10.15.6.
$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Related

subq $40 %rsp crash with AS but GCC not

I meet a strange phenomenon, I record the code in following.
My test bed is x86_64 and gcc is 5.3.0
When I reserve some space in the stack for local value, sometimes it would crash.
| AS and LD | gcc |
--------------------------------------------
40 bytes in stack | crash | ok |
--------------------------------------------
32 bytes in stack | ok | crash |
--------------------------------------------
.section .data
fmt:
.ascii "0x%lx\n\0"
.section .text
.global _start
_start:
subq $40, %rsp # subq $32, %rsp is OK
# I want to reserve some place for local value.
movq $8, %rsi
movq $fmt, %rdi
call printf #print something
addq $40, %rsp
movq $1, %rax
int $0x80
as tsp.s -o tsp.o
ld -lc -I /lib64/ld-linux-x86-64.so.2 tsp.o -o tsp
./tsp
Segmentation fault (core dumped)
This time I use gcc to compile and link.
It is ok, when I reserve 40 bytes in the stack.
It crash, when I reserve 32 bytes in the stack.
.section .data
fmt:
.ascii "0x%lx\n\0"
.section .text
.global main
main:
subq $40, %rsp # if subq $32, %rsp, it would crash.
movq $8, %rsi
movq $fmt, %rdi
call printf
addq $40, %rsp
movq $1, %rax
int $0x80
gcc tsp.s -o tsp
./tsp
0x8
When I tested your code printf crashed when accessing xmm registers. There are two reasons for it. When you let gcc do the compilation and linking it will actually have additional code before main. That code will correctly align the stack and then call main.
Since main was called like a normal function the stack will be aligned at 8 mod 16 because of the call instruction, but when calling a function the stack has to be correctly aligned (0 mod 16). The reason for the alignment requirement is because of xmm registers (among others).
Now, why did printf touch xmm registers in the first place? Because you called printf incorrectly. The ABI for amd64 says:
When a function taking variable-arguments is called, %rax must be set to the total number of floating point parameters passed to the function in SSE registers.
Your rax probably has some non-zero value in it.
So, two things to fix your problems. xorl %eax, %eax to zero %rax before the call to printf. And be aware of how you have been called and how to align the stack. If you've been called as a normal function, you need to subtract 8+n*16 (n can be 0) from your stack pointer before doing a call. If you've been called as an entry point to be safe you need to properly align your stack pointer because I'm not sure if the kernel always guarantees that your stack pointer will be aligned.

Passing 128 bit register to C function from Assembly [duplicate]

This question already has an answer here:
Printing floating point numbers from x86-64 seems to require %rbp to be saved
(1 answer)
Closed 5 years ago.
I am attempting to test passing a floating point value to a C function from assembly on 64-bit Linux. The C file containing my C function looks like this:
#include <stdio.h>
extern void printer(double k){
printf("%f\n",k);
}
Its expected behavior is to simply print the floating point number passed to it. I am trying to accomplish this from an AT&T-syntax assembly file. If I am not mistaken, in 64-bit linux, the calling convention is to pass floating point arguments on the XMM registers. My .s file is the following:
.extern printer
.data
var:
.double 120.1
.global main
main:
movups (var),%xmm0
call printer
mov $60,%rax
syscall
What I'm hoping this could do is have a variable (var) with value 120.1. This is then moved to the xmm0 register, which I expect is what is used to pass the argument k. This understanding of the calling convention is also backed up by the assembly code generated from the C file, a portion of which is below:
printer:
.LFB0:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset 6, -16
movq %rsp, %rbp
.cfi_def_cfa_register 6
subq $16, %rsp
movsd %xmm0, -8(%rbp)
movq -8(%rbp), %rax
movq %rax, -16(%rbp)
movsd -16(%rbp), %xmm0
movl $.LC0, %edi
movl $1, %eax
call printf
leave
.cfi_def_cfa 7, 8
ret
.cfi_endproc
My .s file assembles to an executable, but running it only gives a segmentation fault, and doesn't print the floating point value. I can only assume this is because I'm not properly moving the value to xmm0 and/or using the register to pass it to the function. Can somebody explain how I should pass the value to the function?
You have defined main in the data section, which makes it non-executable. Add a .text directive before main.

Setting up local stack according to x86-64 calling convention on linux

I am doing some extended assembly optimization on gnu C code running on 64 bit linux. I wanted to print debugging messages from within the assembly code and that's how I came accross the following. I am hoping someone can explain what I am supposed to do in this situation.
Take a look at this sample function:
void test(int a, int b, int c, int d){
__asm__ volatile (
"movq $0, %%rax\n\t"
"pushq %%rax\n\t"
"popq %%rax\n\t"
:
:"m" (a)
:"cc", "%rax"
);
}
Since the four agruments to the function are of class INTEGER, they will be passed through registers and then pushed onto the stack. The strange thing to me is how gcc actually does it:
test:
pushq %rbp
movq %rsp, %rbp
movl %edi, -4(%rbp)
movl %esi, -8(%rbp)
movl %edx, -12(%rbp)
movl %ecx, -16(%rbp)
movq $0, %rax
pushq %rax
popq %rax
popq %rbp
ret
The passed arguments are pushed onto the stack, but the stack pointer is not decremented. Thus, when I do pushq %rax, the values of a and b are overwritten.
What I am wondering: is there a way to ask gcc to properly set up the local stack? Am I simply not supposed to use push and pop in function calls?
x86-64 abi provides a 128 byte red zone under the stack pointer, and the compiler decided to use that. You can turn that off using -mno-red-zone option.

Why no corresponding subroutine call in machine code dump when calling strcpy()?

I am working through the book 'Hacking: The Art of Exploitation' by Jon Erickson.
In one part of the book he gives C code and then walks through the corresponding assembly using gdb, explaining the instructions and memory activity.
I am working along in Mac OS X, so things are a bit different than he presents in the book (he is using Linux).
Anyway, I have this C program:
1 #include <stdio.h>
2 #include <string.h>
3
4 int main()
5 {
6 char str_a[20];
7
8 strcpy(str_a, "Hello, world!\n");
9 printf(str_a);
10 }
Here is the corresponding otool object dump (I have just included main):
_main:
0000000100000ea0 pushq %rbp
0000000100000ea1 movq %rsp,%rbp
0000000100000ea4 subq $0x30,%rsp
0000000100000ea8 movq 0x00000189(%rip),%rax
0000000100000eaf movq (%rax),%rax
0000000100000eb2 movq %rax,0xf8(%rbp)
0000000100000eb6 leaq 0xd4(%rbp),%rax
0000000100000eba movq %rax,%rcx
0000000100000ebd movq $0x77202c6f6c6c6548,%rdx
0000000100000ec7 movq %rdx,(%rcx)
0000000100000eca movb $0x00,0x0e(%rcx)
0000000100000ece movw $0x0a21,0x0c(%rcx)
0000000100000ed4 movl $0x646c726f,0x08(%rcx)
0000000100000edb movq %rcx,0xe8(%rbp)
0000000100000edf xorb %cl,%cl
0000000100000ee1 movq %rax,%rdi
0000000100000ee4 movb %cl,%al
0000000100000ee6 callq 0x100000f1e ; symbol stub for: _printf
0000000100000eeb movl 0xf4(%rbp),%eax
0000000100000eee movq 0x00000143(%rip),%rcx
0000000100000ef5 movq (%rcx),%rcx
0000000100000ef8 movq 0xf8(%rbp),%rdx
0000000100000efc cmpq %rdx,%rcx
0000000100000eff movl %eax,0xd0(%rbp)
0000000100000f02 jne 0x100000f0d
0000000100000f04 movl 0xd0(%rbp),%eax
0000000100000f07 addq $0x30,%rsp
0000000100000f0b popq %rbp
0000000100000f0c ret
0000000100000f0d callq 0x100000f12 ; symbol stub for: ___stack_chk_fail
OK. You will notice the subroutine call to printf():
0000000100000ee6 callq 0x100000f1e ; symbol stub for: _printf
But where is the call to strcpy()?
There are two further anomalies. First of all, if I set a breakpoint in gdp for strcpy():
break strcpy
The program zips through its execution without stopping. It seems as if strcpy() isn't actually getting called.
Second, when I compiled the code:
gcc -g -o char_array2 char_array2.c
I got a warning:
char_array2.c: In function ‘main’:
char_array2.c:9: warning: format not a string literal and no format arguments
char_array2.c:9: warning: format not a string literal and no format arguments
I'm not sure if that is related to the missing subroutine call, but I thought I would include it as a data point anyway.
It almost seems to me as if the compiler has decided strcpy() is not necessary and has optimised the code to work without it. The program does work as expected, printing 'Hello, world!' to the standard output, but this missing call to strcpy() has me wondering exactly what is happening.
In Erickson's example in the book there is a call to strcpy() so perhaps there is a difference in how his compiler and my compiler are working. I am on LLVM:
i686-apple-darwin11-llvm-gcc-4.2
Any ideas would be gratefully received!
Thanks in advance, and I hope you find this one interesting.
Tom
It's right here:
0000000100000ebd movq $0x77202c6f6c6c6548,%rdx
0000000100000ec7 movq %rdx,(%rcx)
0000000100000eca movb $0x00,0x0e(%rcx)
0000000100000ece movw $0x0a21,0x0c(%rcx)
0000000100000ed4 movl $0x646c726f,0x08(%rcx)

Not getting Segmentation Fault in C

here is the c code:
char **s;
s[334]=strdup("test");
printf("%s\n",s[334]);`
i know that strdup does the allocation of "test", but the case s[334] where we will put the pointer to the string "test" is not allocated,however,this code works like a charm
Your code exhibits undefined behavior. That does not mean it will crash. All it means is that you can't predict anything about what will happen.
A crash is rather likely, but not guaranteed at all, in this case.
"Undefined behaviour" doesn't mean you'll get a segfault, it means you might get a segfault. A conforming implementation might also decide to display ASCII art of a puppy.
You might like to check this code with a tool like Valgrind.
You don't always get segmentation fault if you access uninitialized memory.
You do access uninitialized memory here.
The compiler is too smart for us! It knows that printf("%s\n", some_string) is exactly the same as puts(some_string), so it can simplify
char **s;
s[334]=strdup("test");
printf("%s\n",s[334]);
into
char **s;
s[334]=strdup("test");
puts(s[334]);
and then (assuming no UB) that is again equivalent to
puts(strdup("test"));
So, by chance the segment fault didn't happen (this time).
I get a segfault without optimisations, but when compiled with optimisations, gcc doesn't bother with the s at all, it's eliminated as dead code.
gcc -Os -S:
.cfi_startproc
subq $8, %rsp
.cfi_def_cfa_offset 16
movl $.LC0, %edi # .LC0 is where "test" is at
call strdup
addq $8, %rsp
.cfi_def_cfa_offset 8
movq %rax, %rdi
jmp puts
.cfi_endproc
gcc -S -O (same for -O2, -O3):
.LFB23:
.cfi_startproc
subq $8, %rsp
.cfi_def_cfa_offset 16
movl $5, %edi
call malloc
movq %rax, %rdi
testq %rax, %rax
je .L2
movl $1953719668, (%rax)
movb $0, 4(%rax)
.L2:
call puts
addq $8, %rsp
.cfi_def_cfa_offset 8
ret
.cfi_endproc

Resources