I know that correctly disassemble a COST binary is still an issue. But, given the symbols and debug infomation, can a disassembler achieve a 100% accuracy in disassembling any binary? If no, I would like to know what are the failing cases.
Because on some platforms, disassembling may not have only a single solution. Check out this code, for example:
mov rax, 0x1111111111E8
call get_eip
get_eip:
pop rax
sub rax, 13
jmp rax
Assembled into the following:
48 B8 E8 11 11 11 11 11 00 00 E8 00 00 00 00 58 48 2D 0D 00 00 00 FF E0
The jmp rax will actually jump to the middle of the mov rax, 0x1111111111E8 opcode, specifically to the bytes: E8 11 11 11 11 which form a valid relative call opcode.
So, how do you disassembly the above binary? :)
On other platforms (such as ARM), the value or eip (pc on ARM) determine the architecture. On some ARMs, having the LSB of PC on means you're running thumb mode (a different instruction set), while opcodes are always 4 bytes long (on aarch64. On ARMv7, they're 4 bytes long on regular mode and 2 bytes long on thumb mode iirc).
However, in practice, most code is produced by compilers, where such nasty tricks can't take place. So compiler code is actually easy disassembled.
Related
I have this code in C:
int main(void)
{
int a = 1 + 2;
return 0;
}
When I objdump -x86-asm-syntax=intel -d a.out which is compiled with -O0 flag with GCC 9.3.0_1, I get:
0000000100000f9e _main:
100000f9e: 55 push rbp
100000f9f: 48 89 e5 mov rbp, rsp
100000fa2: c7 45 fc 03 00 00 00 mov dword ptr [rbp - 4], 3
100000fa9: b8 00 00 00 00 mov eax, 0
100000fae: 5d pop rbp
100000faf: c3 ret
and with -O1 flag:
0000000100000fc2 _main:
100000fc2: b8 00 00 00 00 mov eax, 0
100000fc7: c3 ret
which removes the unused variable a and stack managing altogether.
However, when I use Apple clang version 11.0.3 with -O0 and -O1, I get
0000000100000fa0 _main:
100000fa0: 55 push rbp
100000fa1: 48 89 e5 mov rbp, rsp
100000fa4: 31 c0 xor eax, eax
100000fa6: c7 45 fc 00 00 00 00 mov dword ptr [rbp - 4], 0
100000fad: c7 45 f8 03 00 00 00 mov dword ptr [rbp - 8], 3
100000fb4: 5d pop rbp
100000fb5: c3 ret
and
0000000100000fb0 _main:
100000fb0: 55 push rbp
100000fb1: 48 89 e5 mov rbp, rsp
100000fb4: 31 c0 xor eax, eax
100000fb6: 5d pop rbp
100000fb7: c3 ret
respectively.
I never get the stack managing part stripped off as in GCC.
Why does (Apple) Clang keep unnecessary push and pop?
This may or may not be a separate question, but with the following code:
int main(void)
{
// return 0;
}
GCC creates a same ASM with or without the return 0;.
However, Clang -O0 leaves this extra
100000fa6: c7 45 fc 00 00 00 00 mov dword ptr [rbp - 4], 0
when there is return 0;.
Why does Clang keep these (probably) redundant ASM codes?
I suspect you were trying to see the addition happen.
int main(void)
{
int a = 1 + 2;
return 0;
}
but with optimization say -O2, your dead code went away
00000000 <main>:
0: 2000 movs r0, #0
2: 4770 bx lr
The variable a is local, it never leaves the function it does not rely on anything outside of the function (globals, input variables, return values from called functions, etc). So it has no functional purpose it is dead code it doesn't do anything so an optimizer is free to remove it and did.
So I assume you went to use no or less optimization and then saw it was too verbose.
00000000 <main>:
0: cf 93 push r28
2: df 93 push r29
4: 00 d0 rcall .+0 ; 0x6 <main+0x6>
6: cd b7 in r28, 0x3d ; 61
8: de b7 in r29, 0x3e ; 62
a: 83 e0 ldi r24, 0x03 ; 3
c: 90 e0 ldi r25, 0x00 ; 0
e: 9a 83 std Y+2, r25 ; 0x02
10: 89 83 std Y+1, r24 ; 0x01
12: 80 e0 ldi r24, 0x00 ; 0
14: 90 e0 ldi r25, 0x00 ; 0
16: 0f 90 pop r0
18: 0f 90 pop r0
1a: df 91 pop r29
1c: cf 91 pop r28
1e: 08 95 ret
If you want to see addition happen instead first off don't use main() it has baggage, and the baggage varies among toolchains. So try something else
unsigned int fun ( unsigned int a, unsigned int b )
{
return(a+b);
}
now the addition relies on external items so the compiler cannot optimize any of this away.
00000000 <_fun>:
0: 1d80 0002 mov 2(sp), r0
4: 6d80 0004 add 4(sp), r0
8: 0087 rts pc
If we want to figure out which one is a and which one is b then.
unsigned int fun ( unsigned int a, unsigned int b )
{
return(a+(b<<1));
}
00000000 <_fun>:
0: 1d80 0004 mov 4(sp), r0
4: 0cc0 asl r0
6: 6d80 0002 add 2(sp), r0
a: 0087 rts pc
Want to see an immediate value
unsigned int fun ( unsigned int a )
{
return(a+0x321);
}
00000000 <fun>:
0: 8b 44 24 04 mov eax,DWORD PTR [esp+0x4]
4: 05 21 03 00 00 add eax,0x321
9: c3 ret
you can figure out what the compilers return address convention is, etc.
But you will hit some limits trying to get the compiler to do things for you to learn asm likewise you can easily take the code generated by these compilations
(using -save-temps or -S or disassemble and type it in (I prefer the latter)) but you can only get so far on your operating system in high level/C callable functions. Eventually you will want to do something bare-metal (on a simulator at first) to get maximum freedom and to try instructions you cant normally try or try them in a way that is hard or you don't quite understand yet how to use in the confines of an operating system in a function call. (please do not use inline assembly until down the road or never, use real assembly and ideally the assembler not the compiler to assemble it, down the road then try those things).
The one compiler was built for or defaults to using a stack frame so you need to tell the compiler to omit it. -fomit-frame-pointer. Note that one or both of these can be built to default not to have a frame pointer.
../gcc-$GCCVER/configure --target=$TARGET --prefix=$PREFIX --without-headers --with-newlib --with-gnu-as --with-gnu-ld --enable-languages='c' --enable-frame-pointer=no
(Don't assume gcc nor clang/llvm have a "standard" build as they are both customizable and the binary you downloaded has someone's opinion of the standard build)
You are using main(), this has the return 0 or not thing and it can/will carry other baggage. Depends on the compiler and settings. Using something not main gives you the freedom to pick your inputs and outputs without it warning that you didn't conform to the short list of choices for main().
For gcc -O0 is ideally no optimization although sometimes you see some. -O3 is max give me all you got. -O2 is historically where folks live if for no other reason than "I did it because everyone else is doing it". -O1 is no mans land for gnu it has some items not in -O0 but not a lot of good ones in -O2, so depends heavily on your code as to whether or not you landed in one/some of the optimizations associated with -O1. These numbered optimization things if your compiler even has a -O option is just a pre-defined list 0 means this list 1 means that list and so on.
There is no reason to expect any two compilers or the same compiler with different options to produce the same code from the same sources. If two competing compilers were able to do that most if not all of the time something very fishy is going on...Likewise no reason to expect the list of optimizations each compiler supports, what each optimization does, etc, to match much less the -O1 list to match between them and so on.
There is no reason to assume that any two compilers or versions conform to the same calling convention for the same target, it is much more common now and further for the processor vendor to create a recommended calling convention and then the competing compilers to often conform to that because why not, everyone else is doing it, or even better, whew I don't have to figure one out myself, if this one fails I can blame them.
There are a lot of implementation defined areas in C in particular, less so in C++ but still...So your expectations of what come out and comparing compilers to each other may differ for this reason as well. Just because one compiler implements some code in some way doesn't mean that is how that language works sometimes it is how that compiler author(s) interpreted the language spec or had wiggle room.
Even with full optimizations enabled, everything that compiler has to offer there is no reason to assume that a compiler can outperform a human. Its an algorithm with limits programmed by a human, it cannot outperform us. With experience it is not hard to examine the output of a compiler for sometimes simple functions but often for larger functions and find missed optimizations, or other things that could have been done "better" for some opinion of "better". And sometimes you find the compiler just left something in that you think it should have removed, and sometimes you are right.
There is education as shown above in using a compiler to start to learn assembly language, and even with decades of experience and dabbling with dozens of assembly languages/instruction sets, if there is a debugged compiler available I will very often start with disassembling simple functions to start learning that new instruction set, then look those up then start to get a feel from what I find there for how to use it.
Very often starting with this one first:
unsigned int fun ( unsigned int a )
{
return(a+5);
}
or
unsigned int fun ( unsigned int a, unsigned int b )
{
return(a+b);
}
And going from there. Likewise when writing a disassembler or a simulator for fun to learn the instruction set I often rely on an existing assembler since it is often the documentation for a processor is lacking, the first assembler and compiler for that processor are very often done with direct access to the silicon folks and then those that follow can also use existing tools as well as documentation to figure things out.
So you are on a good path to start learning assembly language I have strong opinions on which ones to or not to start with to improve the experience and chances of success, but I have been in too many battles on Stack Overflow this week, I'll let that go. You can see that I chose an array of instruction sets in this answer. And even if you don't know them you can probably figure out what the code is doing. "standard" installs of llvm provide the ability to output assembly language for several instruction sets from the same source code. The gnu approach is you pick the target (family) when you compile the toolchain and that compiled toolchain is limited to that target/family but you can easily install several gnu toolchains on your computer at the same time be they variations on defaults/settings for the same target or different targets. A number of these are apt gettable without having to learn to build the tools, arm, avr, msp430, x86 and perhaps some others.
I cannot speak to the why does it not return zero from main when you didn't actually have any return code. See comments by others and read up on the specs for that language. (or ask that as a separate question, or see if it was already answered).
Now you said Apple clang not sure what that reference was to I know that Apple has put a lot of work into llvm in general. Or maybe you are on a mac or in an Apple supplied/suggested development environment, but check Wikipedia and others, clang had a lot of corporate help not just Apple, so not sure what the reference was there. If you are on an Apple computer then the apt gettable isn't going to make sense, but there are still lots of pre-built gnu (and llvm) based toolchains you can download and install rather than attempt to build the toolchain from sources (which isn't difficult BTW).
Why stripped binary shows _cxa_finalize instead of libc_start_main?
I am trying to locate and disassemble main() in a very simple C program on Linux (Ubuntu). The binary is stripped. Below you can see disassembly (not stripped) vs disassembly (stripped) of the same instructions.
Question: what is _cxa_finalize in the stripped version? Why is libc_start_main is replaced by _cxa_finalize?
Not stripped:
106d: 48 8d 3d c1 00 00 00 lea rdi,[rip+0xc1] # 1135 <main>
1074: ff 15 66 2f 00 00 call QWORD PTR [rip+0x2f66] # 3fe0 <__libc_start_main#GLIBC_2.2.5>
Stripped:
106d: 48 8d 3d c1 00 00 00 lea rdi,[rip+0xc1] # 1135 <__cxa_finalize#plt+0xf5>
1074: ff 15 66 2f 00 00 call QWORD PTR [rip+0x2f66] # 3fe0 <__cxa_finalize#plt+0x2fa0>
It's not __cxa_finalize. It's __cxa_finalize#plt+0xf5 and __cxa_finalize#plt+0x2fa0 (notice the significant offsets). The disassembler has no information about the symbol main or __libc_start_main because you removed the symbol table, but for technical reasons it is still aware of the symbols assocated with PLT thunks (because they're needed for binding at dynamic linking time, and the disassembler probably falls back to using that information when it lacks s symbol table). In general, the disassembler works backward from an address until it finds an address named by a symbol, and assumes (wrongly, here) that the address being disassembled is part of that function.
When compiling this:
#include <inttypes.h>
void foo(void)
{
*(uint16_t *) (0xb8000) = 0xf61;
}
with
gcc test.c -c -m16 -O1
I get the following warning:
/tmp/ccyziKm4.s: Assembler messages:
/tmp/ccyziKm4.s:9: Warning: 753664 shortened to 32768
And when I drop the -O1 switch, I get none and gcc uses the 0x67 prefix to switch address size as expected (-m16 basically emits prefixed 32-bit code):
00000000 <foo>:
0: 66 55 push %ebp
2: 66 89 e5 mov %esp,%ebp
5: 66 b8 00 80 0b 00 mov $0xb8000,%eax
b: 67 c7 00 61 0f movw $0xf61,(%eax)
10: 90 nop
11: 66 5d pop %ebp
13: 66 c3 retl
So, obviously this has something to do with the optimization switch -O1. The gcc man page describes all the options it sets and I wrote a script to single out every one of them and pass them to gcc, but it doesn't really work. Now, gcc does not show the warning at all, even with the whole bunch of them.
I appreciate any suggestions on how to resolve this.
I'd say that it is bug in gcc, but I see some logic behind this behavior:
GCC without optimization produces quite straightforward code with 2 instructions (I prefer intel syntax):
mov eax, 0xb8000 # move value 0xb8000 to eax
movw [eax], 0xf61 # move value 0xf61 to address stored in eax
Binary view:
66 b8 00 80 0b 00
^ operation: move 16 bit value to 16 register ax
^ size override prefix to indicate that 32 bit data is used instead of 16 bit, so eax should be used instead of ax
66 c7 00 61 0f
^ operation: move 16 bit value to 16 address in ax
^ size override prefix
GCC with optimization tries to optimize, so it generates following code:
movw [0xb8000], 0xf61 # mov value 0xf61 directly to 32 bit address 0xb8000 without any intermediate registers
Binary view:
66 c7 05 00 80 0b 00 61 0f
^ operation: move 16 bit value to 16 bit address
^ size override prefix
So, 32 bit op codes are actually the same opcodes as 16 bit, but with 66/67 prefix.
And here is problem:
operation movw [REGISTER], 0xf61 is legal and officially supported in both 16/32 modes
operation movw [0xb8000], 0xf61 is legal, but values > 16 bit (0xffff) are not officially supported in 16 real mode, in 32 protected - they are officially supported
This is why compiler emits warning and truncates value 0xb8000 to 0x8000 to generate legal and officially supported instruction.
Note: I believe that gcc should emit warning in first case too, as it does not work as you'd expected in 16 bit:
In real mode such instruction allowed, but eax cannot exceed 0xffff (effectively it does not use eax but only ax part).
in protected/unreal mode such instruction allowed and full eax will be used.
I don't know why gcc allows you to use m16 flag, while not supporting 16 bit code generation and real mode memory models properly. I suggest you to switch to something else. 20 years ago watcom was very cool.
If you're in unreal mode, it automatically means that you can and should use m32 instructions.
I'm understanding the basics of assembly and c programming.
I compiled following simple program in C,
#include <stdio.h>
int main()
{
int a;
int b;
a = 10;
b = 88
return 0;
}
Compiled with following command,
gcc -ggdb -fno-stack-protector test.c -o test
The disassembled code for above program with gcc version 4.4.7 is:
5 push %ebp
89 e5 mov %esp,%ebp
83 ec 10 sub $0x10,%esp
c7 45 f8 0a 00 00 00 movl $0xa,-0x8(%ebp)
c7 45 fc 58 00 00 00 movl $0x58,-0x4(%ebp)
b8 00 00 00 00 mov $0x0,%eax
c9 leave
c3 ret
90 nop
However disassembled code for same program with gcc version 4.3.3 is:
8d 4c 23 04 lea 0x4(%esp), %ecx
83 e4 f0 and $0xfffffff0, %esp
55 push -0x4(%ecx)
89 e5 mov %esp,%ebp
51 push %ecx
83 ec 10 sub $0x10,%esp
c7 45 f4 0a 00 00 00 00 movl $0xa, -0xc(%ebp)
c7 45 f8 58 00 00 00 00 movl $0x58, -0x8(%ebp)
b8 00 00 00 00 mov $0x0, %eax
83 c4 10 add $0x10,%esp
59 pop %ecx
5d pop %ebp
8d 61 fc lea -0x4(%ecx),%esp
c3 ret
Why there is difference in the assembly code?
As you can see in second assembled code, Why pushing %ecx on stack?
What is significance of and $0xfffffff0, %esp?
note: OS is same
Compilers are not required to produce identical assembly code for the same source code. The C standard allows the compiler to optimize the code as they see fit as long as the observable behaviour is the same. So, different compilers may generate different assembly code.
For your code, GCC 6.2 with -O3 generates just:
xor eax, eax
ret
because your code essentially does nothing. So, it's reduced to a simple return statement.
To give you some idea, how many ways exists to create valid code for particular task, I thought this example may help.
From time to time there are size coding competitions, obviously targetting Assembly programmers, as you can't compete with compiler against hand written assembly at this level at all.
The competition tasks are fairly trivial to make the entry level and total effort reasonable, with precise input and output specifications (down to single byte or pixel perfection).
So you have almost trivial exact task, human produced code (at the moment still outperforming compilers for trivial task), with single simple rule "minimal size" as a goal.
With your logic it's absolutely clear every competitor should produce the same result.
The real world answer to this is for example:
Hugi Size Coding Competition Series - Compo29 - Random Maze Builder
12 entries, size of code (in bytes): 122, 122, 128, 135, 136, 137, 147, ... 278 (!).
And I bet the first two entries, both having 122B are probably different enough (too lazy to actually check them).
Now producing valid machine code from high level programming language and by machine (compiler) is lot more complex task. And compilers can't compete with humans in reasoning, most of the "how good code is produced by c++ compiler" stems from C++ language itself being defined quite close to machine code (easy to compile) and from brute CPU power allowing the compilers to work on thousands of variants for particular code path, searching for near-optimal solution mostly by brute force.
Still the numerical "reasoning" behind the optimizers are state of art in their own way, getting to the point where human are still unreachable, but more like in their own way, just as humans can't achieve the efficiency of compilers within reasonable effort for full-sized app compilation.
At this point reasoning about some debug code being different in few helper prologue/epilogue instructions... Even if you would find difference in optimized code, and the difference being "obvious" to human, it's still quite a feat the compiler can produce at least that, as compiler has to apply universal rules on specific code, without truly understanding the context of task.
I am working through a lab where I have to defuse a "bomb" by providing the correct input for each phase. I do not have access to the source code, so I have to step through the assembly code with GDB. Right now, I'm stuck on phase 2 and would really appreciate some help. Here is the x86 assembly code - I've added some comments that describe what I think is happening, but these could be horribly wrong because we only started learning assembly code a few days ago and I'm still quite shaky. As far as I can tell right now, this phase reads in 6 numbers from the user (that's what read_six_numbers does) and seems to go through some type of loop.
0000000000400f03 <phase_2>:
400f03: 41 55 push %r13 // save values
400f05: 41 54 push %r12
400f07: 55 push %rbp
400f08: 53 push %rbx
400f09: 48 83 ec 28 sub $0x28,%rsp // decrease stack pointer
400f0d: 48 89 e6 mov %rsp,%rsi // move rsp to rsi
400f10: e8 5a 07 00 00 callq 40166f <read_six_numbers> // read in six numbers from the user
400f15: 48 89 e3 mov %rsp,%rbx // move rsp to rbx
400f18: 4c 8d 64 24 0c lea 0xc(%rsp),%r12 // ?
400f1d: bd 00 00 00 00 mov $0x0,%ebp // set ebp to 0?
400f22: 49 89 dd mov %rbx,%r13 // move rbx to r13
400f25: 8b 43 0c mov 0xc(%rbx),%eax // ?
400f28: 39 03 cmp %eax,(%rbx) // compare eax and rbx
400f2a: 74 05 je 400f31 <phase_2+0x2e> // if equal, skip explode
400f2c: e8 1c 07 00 00 callq 40164d <explode_bomb> // bomb detonates (fail)
400f31: 41 03 6d 00 add 0x0(%r13),%ebp // add r13 and ebp (?)
400f35: 48 83 c3 04 add $0x4,%rbx // add 4 to rbx
400f39: 4c 39 e3 cmp %r12,%rbx // compare r12 and rbx
400f3c: 75 e4 jne 400f22 <phase_2+0x1f> // loop? if not equal, jump to 400f22
400f3e: 85 ed test %ebp,%ebp // compare ebp with itself?
400f40: 75 05 jne 400f47 <phase_2+0x44> // skip explosion if not equal
400f42: e8 06 07 00 00 callq 40164d <explode_bomb> // bomb detonates (fail)
400f47: 48 83 c4 28 add $0x28,%rsp
400f4b: 5b pop %rbx
400f4c: 5d pop %rbp
400f4d: 41 5c pop %r12
400f4f: 41 5d pop %r13
400f51: c3 retq
Any help is greatly appreciated - especially advice on how I would go about translating something like this into C code. Thanks in advance!
especially advice on how I would go about translating something like this into C code
Don't literally translate it into C.
Learn to think in terms of how algorithms are implemented in terms of changes to registers and memory. C and asm are just different ways of expressing what you actually want the machine to do.
Every instruction makes a well-defined modification to the architectural state of the machine, so just follow that chain of steps and see the result. Any good debugger (e.g. gdb in layout reg mode) can show you which register was modified as you single-step. The insn ref manual (links in the x86 tag wiki) has full documentation on exactly what every instruction does.
If you're ever surprised by something, look it up. There are many SO questions from people that didn't do that, and then posted silly questions about div results when they didn't set rdx first.
You need to find connections between insns that modify or overwrite a register or memory location, and later instructions that read from that register or memory location.
You can often get clues from how a register is being used, e.g. add $0x4,%rbx is probably a pointer increment to an int *. It's rare to increment a 64bit integer by 4 if it isn't a pointer, or involved in memory addressing somehow.
If you look at surrounding code and find mov 0xc(%rbx),%eax (loading 4B from an offset from %rbx), that confirms the theory that it's a pointer.
The cmp %r12,%rbx / jcc tells you that it's also part of the loop condition, and that %r12 is the end pointer. You check it's just a simple do{}while(p < end) loop by verifying that %r12 isn't modified in the loop, and that it's initialized to something sensible before the loop.
mov $0x0,%ebp tells you that this is compiler output from -O0 or -O1, because every x86 compiler knows the "peephole" optimization that xor %ebp,%ebp is the best way to zero registers. Fortunately this looks like -O1 compiler output, so it doesn't store everything to memory after every C statement and reload after. That makes code that's hard to follow, because a value doesn't stay live in the same register for long.
If you have any specific questions about that binary bomb code, you should ask them. I just answered the "how to read asm" part.