Why does a C compiler generate NOPs after a RET instruction? [duplicate] - c

This question already has answers here:
Why does GCC pad functions with NOPs?
(3 answers)
Closed 7 years ago.
On OSX 64bit, compiling a dummy C program like that:
#include <stdio.h>
void foo1() {
}
void foo2() {
}
int main() {
printf("Helloooo!\n");
foo1();
foo2();
return 0;
}
Produces the following ASM code (obtained disassembling the binary with otool):
(__TEXT,__text) section
_foo1:
0000000100000f10 55 pushq %rbp
0000000100000f11 4889e5 movq %rsp, %rbp
0000000100000f14 897dfc movl %edi, -0x4(%rbp)
0000000100000f17 5d popq %rbp
0000000100000f18 c3 retq
0000000100000f19 0f1f8000000000 nopl (%rax)
_foo2:
0000000100000f20 55 pushq %rbp
0000000100000f21 4889e5 movq %rsp, %rbp
0000000100000f24 5d popq %rbp
0000000100000f25 c3 retq
0000000100000f26 662e0f1f840000000000 nopw %cs:(%rax,%rax)
_main:
0000000100000f30 55 pushq %rbp
0000000100000f31 4889e5 movq %rsp, %rbp
0000000100000f34 4883ec10 subq $0x10, %rsp
0000000100000f38 488d3d4b000000 leaq 0x4b(%rip), %rdi ## literal pool for: "Helloooo!\n"
0000000100000f3f c745fc00000000 movl $0x0, -0x4(%rbp)
0000000100000f46 b000 movb $0x0, %al
0000000100000f48 e81b000000 callq 0x100000f68 ## symbol stub for: _printf
0000000100000f4d bf06000000 movl $0x6, %edi
0000000100000f52 8945f8 movl %eax, -0x8(%rbp)
0000000100000f55 e8b6ffffff callq _foo1
0000000100000f5a e8c1ffffff callq _foo2
0000000100000f5f 31c0 xorl %eax, %eax
0000000100000f61 4883c410 addq $0x10, %rsp
0000000100000f65 5d popq %rbp
0000000100000f66 c3 retq
What are the "nop" instructions found right after the "ret" on functions foo1() and foo2()? They are, of course, never executed since the "ret" instructions return from the function call. Is that any kind of padding or it has a different meaning?

From the Assembly language for x86 processors, Kip R. Irvine
The safest (and the most useless) instruction you can write is called NOP (no operation). It takes up 1 byte of program storage and doesn’t do any work. It is sometimes used by compilers and assemblers to align code to even-address boundaries
00000000 66 8B C3 mov ax,bx
00000003 90 nop ; align next instruction
00000004 8B D1 mov edx,ecx

What are the "nop" instructions found right after the "ret" on functions foo1() and foo2()?
The nop is a no-operation instruction (do nothing), from the linked Wikipedia page (emphasis mine)
A NOP is most commonly used for timing purposes, to force memory alignment, to prevent hazards, to occupy a branch delay slot, to render void an existing instruction such as a jump, or as a place-holder to be replaced by active instructions later on in program development (or to replace removed instructions when refactoring would be problematic or time-consuming).

nop is short for No Operation. The nop instructions in this case are providing execution code alignment. Notice that labels are on 16 byte boundaries. On OSX, the linker (ld) should have a -segalign option that will affect this behavior.

Related

Why gdb shows different addresses in RIP-relative mode from absolute address?

Having this in c:
#include <stdio.h>
#include <stdlib.h>
int x;
int main(){
printf("eneter x\n");
scanf("%i",&x);
printf("you enetered: %i\n", x);
return 0;
}
in gdb:
starti
disas main
0x0000555555555155 <+0>: push %rbp
0x0000555555555156 <+1>: mov %rsp,%rbp
0x0000555555555159 <+4>: lea 0xea4(%rip),%rdi # 0x555555556004
0x0000555555555160 <+11>: callq 0x555555555030 <puts#plt>
0x0000555555555165 <+16>: lea 0x2ed8(%rip),%rsi # 0x555555558044 <x>
0x000055555555516c <+23>: lea 0xe9a(%rip),%rdi # 0x55555555600d
0x0000555555555173 <+30>: mov $0x0,%eax
0x0000555555555178 <+35>: callq 0x555555555050 <__isoc99_scanf#plt>
0x000055555555517d <+40>: mov 0x2ec1(%rip),%eax # 0x555555558044 <x>
0x0000555555555183 <+46>: mov %eax,%esi
0x0000555555555185 <+48>: lea 0xe84(%rip),%rdi # 0x555555556010
0x000055555555518c <+55>: mov $0x0,%eax
0x0000555555555191 <+60>: callq 0x555555555040 <printf#plt>
0x0000555555555196 <+65>: mov $0x0,%eax
0x000055555555519b <+70>: pop %rbp
0x000055555555519c <+71>: retq
here the relative address of x variable is $rip+0x2ed8 (from instruction lea 0x2ed8(%rip),%rsi # 0x555555558044). But as you can see in the comment #, the absolute address is 0x555555558044. Ok will I get that address when try to read from the relative one? Lets see:
x $rip+0x2ed8
0x555555558055: 0x00000000
nop - relative address did not use the absolute address, where the x var is really stored (0x555555558055 != 0x555555558044) the difference is 17 bytes. Is it the number of bytes of the instruction itself (lea + operands)? I do not know, but do not think so. So why does relative and absolute addressing differ in gdb?
PS, generated assembly:
.file "a.c"
.comm x,4,4
.section .rodata
.LC0:
.string "eneter x"
.LC1:
.string "%i"
.LC2:
.string "you enetered: %i\n"
.text
.globl main
.type main, #function
main:
pushq %rbp #
movq %rsp, %rbp #,
# a.c:5: printf("eneter x\n");
leaq .LC0(%rip), %rdi #,
call puts#PLT #
# a.c:6: scanf("%i",&x);
leaq x(%rip), %rsi #,
leaq .LC1(%rip), %rdi #,
movl $0, %eax #,
call __isoc99_scanf#PLT #
# a.c:7: printf("you enetered: %i\n", x);
movl x(%rip), %eax # x, x.0_1
movl %eax, %esi # x.0_1,
leaq .LC2(%rip), %rdi #,
movl $0, %eax #,
call printf#PLT #
# a.c:8: return 0;
movl $0, %eax #, _6
# a.c:9: }
popq %rbp #
ret
.size main, .-main
.ident "GCC: (Debian 8.3.0-6) 8.3.0"
.section .note.GNU-stack,"",#progbits
Here, the RIP-relative mode is used:
# a.c:6: scanf("%i",&x);
leaq x(%rip), %rsi #,
where the x is position of the x symbol. But in comments, someone said, that $rip+0x2ed8 is not the same, and the offset 0x2ed8 does not lead to the address of the x. But why those two differ? but should be RIP-relative mode addressing and both should gain the same offset (and thus address).
0x0000555555555165 <+16>: lea 0x2ed8(%rip),%rsi # 0x555555558044 <x>
0x000055555555516c <+23>: lea 0xe9a(%rip),%rdi # 0x55555555600d
A RIP relative address in an instruction is relative to the address just after the current instruction (i.e. the address of the instruction plus the size of the instruction, or the address of the following instruction). This is because when the instruction has been loaded into the processor, the RIP register is advanced by the size of the current instruction just before it is executed. (At least that is the model that is followed even though modern processors use all sorts of tricks behind the scenes to speed up execution.) (Note: The above is true for several CPU architectures, including x86 variants, but some other CPU architectures differ in the point from which PC-relative addresses are measured1.)
The first instruction above is at address 0x555555555165 and the following instruction is at address 0x55555555516c (the instruction is 7 bytes long). In the first instruction, the RIP relative address 0x2ed8(%rip) refers to 0x2ed8 + 0x000055555555516c = 0x555555558044.
Note that if you set a breakpoint on an instruction in a debugger and show the registers when the breakpoint is reached, RIP will point to the current instruction, not the next one, because the current instruction is not being executed yet.
1 Thanks to Peter Cordes for details about PC-relative addressing for ARM and RISC-V CPU architectures.

What do the instructions mov %edi and mov %rsi do?

I've written a basic C program that defines an integer variable x, sets it to zero and returns the value of that variable:
#include <stdio.h>
int main(int argc, char **argv) {
int x;
x = 0;
return x;
}
When I dump the object code using objdump (compiled on Linux X86-64 with gcc):
0x0000000000400474 <main+0>: push %rbp
0x0000000000400475 <main+1>: mov %rsp,%rbp
0x0000000000400478 <main+4>: mov %edi,-0x14(%rbp)
0x000000000040047b <main+7>: mov %rsi,-0x20(%rbp)
0x000000000040047f <main+11>: movl $0x0,-0x4(%rbp)
0x0000000000400486 <main+18>: mov -0x4(%rbp),%eax
0x0000000000400489 <main+21>: leaveq
0x000000000040048a <main+22>: retq
I can see the function prologue, but before we set x to 0 at address 0x000000000040047f there are two instructions that move %edi and %rsi onto the stack. What are these for?
In addition, unlike where we set x to 0, the mov instruction as shown in GAS syntax does not have a suffix.
If the suffix is not specified, and there are no memory operands for the instruction, GAS infers the operand size from the size of the destination register operand.
In this case, are -0x14(%rsbp) and -0x20(%rbp) both memory operands and what are their sizes? Since %edi is a 32 bit register, are 32 bits moved to -0x14(%rsbp) whereas since %rsi is a 64 bit register, 64 bits are moved to %rsi,-0x20(%rbp)?
In this simple case, why don't you ask your compiler directly? For GCC, clang and ICC there's the -fverbose-asm option.
main:
pushq %rbp #
movq %rsp, %rbp #,
movl %edi, -20(%rbp) # argc, argc
movq %rsi, -32(%rbp) # argv, argv
movl $0, -4(%rbp) #, x
movl -4(%rbp), %eax # x, D.2607
popq %rbp #
ret
So, yes, they save argv and argv onto the stack by using the "old" frame pointer method since new architectures allow subtracting/adding from/to the stack pointer directly, thus omitting the frame pointer (-fomit-frame-pointer).
Purpose of ESI & EDI registers?
Based on this and the context, I'm not an expert, but my guess is these are capturing the main() input parameters. EDI takes a standard width, which would match the int argc, whereas RSI takes a long, which would match the char **argv pointer.

Grabbing 128bit key from shellcode

I'm trying to take out a 128bit key from shellcode. I have compiled shellcode as a C code within an array which is like
#include <stdio.h>
#include <stdlib.h>
/*shellcode.c*/
char code[] = "\x31\xC0\x50\x68\x75\x70\x25\x75\x68\x23\x78\x27\x78\x68\x25\x74\x72\x20\x68\x79\x24\x73\x77\x68\x71\x72\x77\x76\x68\x20\x25\x22\x70\x68\x23\x78\x75\x27\x68\x75\x20\x23\x75\x68\x76\x72\x79\x79\x68\x20\x70\x72\x73\x68\x71\x71\x24\x25\x68\x79\x27\x76\x77\x68\x24\x77\x71\x72\x68\x27\x79\x70\x70\x68\x74\x24\x24\x75\x68\x79\x73\x23\x23\x68\x74\x22\x75\x79\x68\x23\x74\x70\x27\x68\x20\x74\x24\x79\x68\x74\x77\x24\x78\x68\x25\x27\x70\x75\x68\x74\x77\x74\x78\x68\x23\x23\x71\x76\x68\x77\x70\x73\x71\x68\x27\x20\x77\x24\x68\x22\x72\x78\x75\x68\x25\x72\x79\x77\x68\x23\x75\x79\x76\x68\x72\x71\x72\x24\x68\x71\x23\x23\x79\x68\x79\x23\x79\x70\x68\x20\x20\x76\x77\x54\x5E\x8B\xFE\x8B\xD7\xFC\xB9\x80\x00\x00\x00\xBB\x41\x00\x00\x00\x31\xC0\x50\xAC\x33\xC3\xAA\xE2\xFA\x54\x5E\xCC";
int main(int argc, char **argv)
{
int (*func)();
func = (int (*)()) code;
(int)(*func)();
}
when I compiled it using GCC under linux. After disassembling it using GDB, as the shellcode located within code array;
(gdb) disas &code
Dump of assembler code for function code:
0x0000000000600840 <+0>: xor %eax,%eax
0x0000000000600842 <+2>: push %rax
0x0000000000600843 <+3>: pushq $0x75257075
0x0000000000600848 <+8>: pushq $0x78277823
0x000000000060084d <+13>: pushq $0x20727425
0x0000000000600852 <+18>: pushq $0x77732479
0x0000000000600857 <+23>: pushq $0x76777271
0x000000000060085c <+28>: pushq $0x70222520
0x0000000000600861 <+33>: pushq $0x27757823
0x0000000000600866 <+38>: pushq $0x75232075
0x000000000060086b <+43>: pushq $0x79797276
0x0000000000600870 <+48>: pushq $0x73727020
0x0000000000600875 <+53>: pushq $0x25247171
0x000000000060087a <+58>: pushq $0x77762779
0x000000000060087f <+63>: pushq $0x72717724
0x0000000000600884 <+68>: pushq $0x70707927
0x0000000000600889 <+73>: pushq $0x75242474
0x000000000060088e <+78>: pushq $0x23237379
0x0000000000600893 <+83>: pushq $0x79752274
0x0000000000600898 <+88>: pushq $0x27707423
0x000000000060089d <+93>: pushq $0x79247420
0x00000000006008a2 <+98>: pushq $0x78247774
0x00000000006008a7 <+103>: pushq $0x75702725
0x00000000006008ac <+108>: pushq $0x78747774
0x00000000006008b1 <+113>: pushq $0x76712323
0x00000000006008b6 <+118>: pushq $0x71737077
0x00000000006008bb <+123>: pushq $0x24772027
0x00000000006008c0 <+128>: pushq $0x75787222
0x00000000006008c5 <+133>: pushq $0x77797225
0x00000000006008ca <+138>: pushq $0x76797523
0x00000000006008cf <+143>: pushq $0x24727172
0x00000000006008d4 <+148>: pushq $0x79232371
0x00000000006008d9 <+153>: pushq $0x70792379
0x00000000006008de <+158>: pushq $0x77762020
0x00000000006008e3 <+163>: push %rsp
0x00000000006008e4 <+164>: pop %rsi
0x00000000006008e5 <+165>: mov %esi,%edi
0x00000000006008e7 <+167>: mov %edi,%edx
0x00000000006008e9 <+169>: cld
0x00000000006008ea <+170>: mov $0x80,%ecx
0x00000000006008ef <+175>: mov $0x41,%ebx
0x00000000006008f4 <+180>: xor %eax,%eax
0x00000000006008f6 <+182>: push %rax
0x00000000006008f7 <+183>: lods %ds:(%rsi),%al
0x00000000006008f8 <+184>: xor %ebx,%eax
0x00000000006008fa <+186>: stos %al,%es:(%rdi)
0x00000000006008fb <+187>: loop 0x6008f7 <code+183>
0x00000000006008fd <+189>: push %rsp
0x00000000006008fe <+190>: pop %rsi
0x00000000006008ff <+191>: int3
0x0000000000600900 <+192>: add %al,(%rax)
End of assembler dump.
Looking at disassembly 128bit key will be calculated after loop instructions at 6008fb. I'm not very comfortable with GDB. How can I obtain 128bit key out of this shellcode I suspect I need to put a pointer to right after loop and view content ? But I don't know how to do it.
Thank you very much in advance ...
If you can successfully run this code, it will stop at the int3 instruction at the end. At that point you'll have the decrypted string in memory starting at rsp+8 (which is the same as rsi+8). You can print it from gdb by x/s $rsp+8, for example. The +8 comes from the push %rax at 0x6008f6.
Note that this code seems to be wrongly ported 32 bit code. All the pushq instructions will store the immediate constant given in the instruction followed by 4 zero bytes. Decrypting that will result in every 4 letters coming out as AAAA (and only getting half of the key). The mov %esi,%edi at 0x6008e5 will zero the top bits of rdi so the code will only work if the stack is in the first 4GB of the address space (which it normally isn't on a 64 bit system).
Furthermore the code assumes it's on the stack, but your C wrapper puts it in the data section. In any case, you will need execute permissions too.
Alternatively, since this code does nothing beyond XOR-ing the values by 0x41 you can easily do that by hand, remembering that x86 is little endian so the constants printed in the disassembly have to be byte-swapped.

Why does LLVM add two extra instructions for the same program?

I am compiling this C program and comparing the generated assembly code:
int main(){ return 0; }
GCC gives this main function (cc hello.c -S):
_main:
LFB2:
pushq %rbp
LCFI0:
movq %rsp, %rbp
LCFI1:
movl $0, %eax
leave
ret
LLVM gives this main function (clang hello.c -S):
_main:
Leh_func_begin0:
pushq %rbp
Ltmp0:
movq %rsp, %rbp
Ltmp1:
movl $0, %eax
movl $0, -4(%rbp)
popq %rbp
ret
Leh_func_end0:
What are movl $0, -4(%rbp) and popq %rbp needed for? Moving something on the stack and popping it directly afterwards seems useless to me.
The movl $0, -4(%rbp) instruction is dead, because this is unoptimized code. Try passing in -O to both compilers to see what changes.
Actually, they're comparable. Leave is a high level instruction:
From the Intel manual:
16-bit: C9 LEAVE A Valid Valid Set SP to BP, then pop BP.
32-bit: C9 LEAVE A N.E. Valid Set ESP to EBP, then pop EBP.
64-bit: C9 LEAVE A Valid N.E. Set RSP to RBP, then pop RBP.
basically, leave is equivalent to
movq %rbp, %rsp
popq %rbp
It looks like LLVM is using a traditional function prolog/epilog, whereas GCC is taking advantage of the fact that the entry point doesn't need to clean up

Understand the assembly code generated by a simple C program

I am trying to understand the assembly level code for a simple C program by inspecting it with gdb's disassembler.
Following is the C code:
#include <stdio.h>
void function(int a, int b, int c) {
char buffer1[5];
char buffer2[10];
}
void main() {
function(1,2,3);
}
Following is the disassembly code for both main and function
gdb) disass main
Dump of assembler code for function main:
0x08048428 <main+0>: push %ebp
0x08048429 <main+1>: mov %esp,%ebp
0x0804842b <main+3>: and $0xfffffff0,%esp
0x0804842e <main+6>: sub $0x10,%esp
0x08048431 <main+9>: movl $0x3,0x8(%esp)
0x08048439 <main+17>: movl $0x2,0x4(%esp)
0x08048441 <main+25>: movl $0x1,(%esp)
0x08048448 <main+32>: call 0x8048404 <function>
0x0804844d <main+37>: leave
0x0804844e <main+38>: ret
End of assembler dump.
(gdb) disass function
Dump of assembler code for function function:
0x08048404 <function+0>: push %ebp
0x08048405 <function+1>: mov %esp,%ebp
0x08048407 <function+3>: sub $0x28,%esp
0x0804840a <function+6>: mov %gs:0x14,%eax
0x08048410 <function+12>: mov %eax,-0xc(%ebp)
0x08048413 <function+15>: xor %eax,%eax
0x08048415 <function+17>: mov -0xc(%ebp),%eax
0x08048418 <function+20>: xor %gs:0x14,%eax
0x0804841f <function+27>: je 0x8048426 <function+34>
0x08048421 <function+29>: call 0x8048340 <__stack_chk_fail#plt>
0x08048426 <function+34>: leave
0x08048427 <function+35>: ret
End of assembler dump.
I am seeking answers for following things :
how the addressing is working , I mean (main+0) , (main+1), (main+3)
In the main, why is $0xfffffff0,%esp being used
In the function, why is %gs:0x14,%eax , %eax,-0xc(%ebp) being used.
If someone can explain , step by step happening, that will be greatly appreciated.
The reason for the "strange" addresses such as main+0, main+1, main+3, main+6 and so on, is because each instruction takes up a variable number of bytes. For example:
main+0: push %ebp
is a one-byte instruction so the next instruction is at main+1. On the other hand,
main+3: and $0xfffffff0,%esp
is a three-byte instruction so the next instruction after that is at main+6.
And, since you ask in the comments why movl seems to take a variable number of bytes, the explanation for that is as follows.
Instruction length depends not only on the opcode (such as movl) but also the addressing modes for the operands as well (the things the opcode are operating on). I haven't checked specifically for your code but I suspect the
movl $0x1,(%esp)
instruction is probably shorter because there's no offset involved - it just uses esp as the address. Whereas something like:
movl $0x2,0x4(%esp)
requires everything that movl $0x1,(%esp) does, plus an extra byte for the offset 0x4.
In fact, here's a debug session showing what I mean:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
c:\pax> debug
-a
0B52:0100 mov word ptr [di],7
0B52:0104 mov word ptr [di+2],8
0B52:0109 mov word ptr [di+0],7
0B52:010E
-u100,10d
0B52:0100 C7050700 MOV WORD PTR [DI],0007
0B52:0104 C745020800 MOV WORD PTR [DI+02],0008
0B52:0109 C745000700 MOV WORD PTR [DI+00],0007
-q
c:\pax> _
You can see that the second instruction with an offset is actually different to the first one without it. It's one byte longer (5 bytes instead of 4, to hold the offset) and actually has a different encoding c745 instead of c705.
You can also see that you can encode the first and third instruction in two different ways but they basically do the same thing.
The and $0xfffffff0,%esp instruction is a way to force esp to be on a specific boundary. This is used to ensure proper alignment of variables. Many memory accesses on modern processors will be more efficient if they follow the alignment rules (such as a 4-byte value having to be aligned to a 4-byte boundary). Some modern processors will even raise a fault if you don't follow these rules.
After this instruction, you're guaranteed that esp is both less than or equal to its previous value and aligned to a 16 byte boundary.
The gs: prefix simply means to use the gs segment register to access memory rather than the default.
The instruction mov %eax,-0xc(%ebp) means to take the contents of the ebp register, subtract 12 (0xc) and then put the value of eax into that memory location.
Re the explanation of the code. Your function function is basically one big no-op. The assembly generated is limited to stack frame setup and teardown, along with some stack frame corruption checking which uses the afore-mentioned %gs:14 memory location.
It loads the value from that location (probably something like 0xdeadbeef) into the stack frame, does its job, then checks the stack to ensure it hasn't been corrupted.
Its job, in this case, is nothing. So all you see is the function administration stuff.
Stack set-up occurs between function+0 and function+12. Everything after that is setting up the return code in eax and tearing down the stack frame, including the corruption check.
Similarly, main consist of stack frame set-up, pushing the parameters for function, calling function, tearing down the stack frame and exiting.
Comments have been inserted into the code below:
0x08048428 <main+0>: push %ebp ; save previous value.
0x08048429 <main+1>: mov %esp,%ebp ; create new stack frame.
0x0804842b <main+3>: and $0xfffffff0,%esp ; align to boundary.
0x0804842e <main+6>: sub $0x10,%esp ; make space on stack.
0x08048431 <main+9>: movl $0x3,0x8(%esp) ; push values for function.
0x08048439 <main+17>: movl $0x2,0x4(%esp)
0x08048441 <main+25>: movl $0x1,(%esp)
0x08048448 <main+32>: call 0x8048404 <function> ; and call it.
0x0804844d <main+37>: leave ; tear down frame.
0x0804844e <main+38>: ret ; and exit.
0x08048404 <func+0>: push %ebp ; save previous value.
0x08048405 <func+1>: mov %esp,%ebp ; create new stack frame.
0x08048407 <func+3>: sub $0x28,%esp ; make space on stack.
0x0804840a <func+6>: mov %gs:0x14,%eax ; get sentinel value.
0x08048410 <func+12>: mov %eax,-0xc(%ebp) ; put on stack.
0x08048413 <func+15>: xor %eax,%eax ; set return code 0.
0x08048415 <func+17>: mov -0xc(%ebp),%eax ; get sentinel from stack.
0x08048418 <func+20>: xor %gs:0x14,%eax ; compare with actual.
0x0804841f <func+27>: je <func+34> ; jump if okay.
0x08048421 <func+29>: call <_stk_chk_fl> ; otherwise corrupted stack.
0x08048426 <func+34>: leave ; tear down frame.
0x08048427 <func+35>: ret ; and exit.
I think the reason for the %gs:0x14 may be evident from above but, just in case, I'll elaborate here.
It uses this value (a sentinel) to put in the current stack frame so that, should something in the function do something silly like write 1024 bytes to a 20-byte array created on the stack or, in your case:
char buffer1[5];
strcpy (buffer1, "Hello there, my name is Pax.");
then the sentinel will be overwritten and the check at the end of the function will detect that, calling the failure function to let you know, and then probably aborting so as to avoid any other problems.
If it placed 0xdeadbeef onto the stack and this was changed to something else, then an xor with 0xdeadbeef would produce a non-zero value which is detected in the code with the je instruction.
The relevant bit is paraphrased here:
mov %gs:0x14,%eax ; get sentinel value.
mov %eax,-0xc(%ebp) ; put on stack.
;; Weave your function
;; magic here.
mov -0xc(%ebp),%eax ; get sentinel back from stack.
xor %gs:0x14,%eax ; compare with original value.
je stack_ok ; zero/equal means no corruption.
call stack_bad ; otherwise corrupted stack.
stack_ok: leave ; tear down frame.
Pax has produced a definitive answer. However, for completeness, I thought I'd add a note on getting GCC itself to show you the assembly it generates.
The -S option to GCC tells it to stop compilation and write the assembly to a file. Normally, it either passes that file to the assembler or for some targets writes the object file directly itself.
For the sample code in the question:
#include <stdio.h>
void function(int a, int b, int c) {
char buffer1[5];
char buffer2[10];
}
void main() {
function(1,2,3);
}
the command gcc -S q3654898.c creates a file named q3654898.s:
.file "q3654898.c"
.text
.globl _function
.def _function; .scl 2; .type 32; .endef
_function:
pushl %ebp
movl %esp, %ebp
subl $40, %esp
leave
ret
.def ___main; .scl 2; .type 32; .endef
.globl _main
.def _main; .scl 2; .type 32; .endef
_main:
pushl %ebp
movl %esp, %ebp
subl $24, %esp
andl $-16, %esp
movl $0, %eax
addl $15, %eax
addl $15, %eax
shrl $4, %eax
sall $4, %eax
movl %eax, -4(%ebp)
movl -4(%ebp), %eax
call __alloca
call ___main
movl $3, 8(%esp)
movl $2, 4(%esp)
movl $1, (%esp)
call _function
leave
ret
One thing that is evident is that my GCC (gcc (GCC) 3.4.5 (mingw-vista special r3)) doesn't include the stack check code by default. I imagine that there is a command line option, or that if I ever got around to nudging my MinGW install up to a more current GCC that it could.
Edit: Nudged to do so by Pax, here's another way to get GCC to do more of the work.
C:\Documents and Settings\Ross\My Documents\testing>gcc -Wa,-al q3654898.c
q3654898.c: In function `main':
q3654898.c:8: warning: return type of 'main' is not `int'
GAS LISTING C:\DOCUME~1\Ross\LOCALS~1\Temp/ccLg8pWC.s page 1
1 .file "q3654898.c"
2 .text
3 .globl _function
4 .def _function; .scl 2; .type
32; .endef
5 _function:
6 0000 55 pushl %ebp
7 0001 89E5 movl %esp, %ebp
8 0003 83EC28 subl $40, %esp
9 0006 C9 leave
10 0007 C3 ret
11 .def ___main; .scl 2; .type
32; .endef
12 .globl _main
13 .def _main; .scl 2; .type 32;
.endef
14 _main:
15 0008 55 pushl %ebp
16 0009 89E5 movl %esp, %ebp
17 000b 83EC18 subl $24, %esp
18 000e 83E4F0 andl $-16, %esp
19 0011 B8000000 movl $0, %eax
19 00
20 0016 83C00F addl $15, %eax
21 0019 83C00F addl $15, %eax
22 001c C1E804 shrl $4, %eax
23 001f C1E004 sall $4, %eax
24 0022 8945FC movl %eax, -4(%ebp)
25 0025 8B45FC movl -4(%ebp), %eax
26 0028 E8000000 call __alloca
26 00
27 002d E8000000 call ___main
27 00
28 0032 C7442408 movl $3, 8(%esp)
28 03000000
29 003a C7442404 movl $2, 4(%esp)
29 02000000
30 0042 C7042401 movl $1, (%esp)
30 000000
31 0049 E8B2FFFF call _function
31 FF
32 004e C9 leave
33 004f C3 ret
C:\Documents and Settings\Ross\My Documents\testing>
Here we see an output listing produced by the assembler. (Its name is GAS, because it is Gnu's version of the classic *nix assembler as. There's humor there somewhere.)
Each line has most of the following fields: a line number, an address in the current section, bytes stored at that address, and the source text from the assembly source file.
The addresses are offsets into that portion of each section provided by this module. This particular module only has content in the .text section which stores executable code. You will typically find mention of sections named .data and .bss as well. Lots of other names are used and some have special purposes. Read the manual for the linker if you really want to know.
It will be better to try the -fno-stack-protector flag with gcc to disable the canary and see your results.
I'd like to add that for simple stuff, GCC's assembly output is often easier to read if you turn on a little optimization. Here's the sample code again...
void function(int a, int b, int c) {
char buffer1[5];
char buffer2[10];
}
/* corrected calling convention of main() */
int main() {
function(1,2,3);
return 0;
}
this is what I get without optimization (OSX 10.6, gcc 4.2.1+Apple patches)
.globl _function
_function:
pushl %ebp
movl %esp, %ebp
pushl %ebx
subl $36, %esp
call L4
"L00000000001$pb":
L4:
popl %ebx
leal L___stack_chk_guard$non_lazy_ptr-"L00000000001$pb"(%ebx), %eax
movl (%eax), %eax
movl (%eax), %edx
movl %edx, -12(%ebp)
xorl %edx, %edx
leal L___stack_chk_guard$non_lazy_ptr-"L00000000001$pb"(%ebx), %eax
movl (%eax), %eax
movl -12(%ebp), %edx
xorl (%eax), %edx
je L3
call ___stack_chk_fail
L3:
addl $36, %esp
popl %ebx
leave
ret
.globl _main
_main:
pushl %ebp
movl %esp, %ebp
subl $24, %esp
movl $3, 8(%esp)
movl $2, 4(%esp)
movl $1, (%esp)
call _function
movl $0, %eax
leave
ret
Whew, one heck of a mouthful! But look what happens with -O on the command line...
.text
.globl _function
_function:
pushl %ebp
movl %esp, %ebp
leave
ret
.globl _main
_main:
pushl %ebp
movl %esp, %ebp
movl $0, %eax
leave
ret
Of course, you do run the risk of your code being rendered completely unrecognizable, especially at higher optimization levels and with more complicated stuff. Even here, we see that the call to function has been discarded as pointless. But I find that not having to read through dozens of unnecessary stack spills is generally more than worth a little extra scratching my head over the control flow.

Resources