Consider the mappings in the virtual memory of a small test program I wrote:
0000000000400000 20K r-x-- /home/arvidj/dev/examples/testgen.c.out
0000000000604000 4K r---- /home/arvidj/dev/examples/testgen.c.out
0000000000605000 4K rw--- /home/arvidj/dev/examples/testgen.c.out
0000000000606000 16K rw--- [ anon ]
00007ffff7a12000 1780K r-x-- /lib/x86_64-linux-gnu/libc-2.17.so
00007ffff7bcf000 2048K ----- /lib/x86_64-linux-gnu/libc-2.17.so
00007ffff7dcf000 16K r---- /lib/x86_64-linux-gnu/libc-2.17.so
00007ffff7dd3000 8K rw--- /lib/x86_64-linux-gnu/libc-2.17.so
00007ffff7dd5000 20K rw--- [ anon ]
00007ffff7dda000 140K r-x-- /lib/x86_64-linux-gnu/ld-2.17.so
00007ffff7fde000 12K rw--- [ anon ]
00007ffff7ff8000 8K rw--- [ anon ]
00007ffff7ffa000 8K r-x-- [ anon ]
00007ffff7ffc000 4K r---- /lib/x86_64-linux-gnu/ld-2.17.so
00007ffff7ffd000 8K rw--- /lib/x86_64-linux-gnu/ld-2.17.so
00007ffffffde000 132K rw--- [ stack ]
ffffffffff600000 4K r-x-- [ anon ]
How flexible is this layout? Which parts can be moved (statically or dynamically), and how? If I want the stack to start at 0x0000000001000000, how can I change this? Conversely, can I move up the DATA/TEXT segments, am I at liberty to do so?
If I manage to move the stack (perhaps via makecontext, or by fiddling with sp in asm), how can I re-use that space? Can I use map over it, or can I have to unmap it using munmap? You can find my motivation in this question.
Related
For instance, lets suppose that instead of buffers growing in the opposite direction of the stack, they grow in the same direction. If I have a character buffer containing the string "Hello world", instead of 'H' being placed at the lowest address, it is placed at the highest address, and so on.
If an input string copied to a buffer were to overflow it could not overwrite the return address of the function, but certainly there are other things it could overwrite. My question is -- if the input string was long enough, what things could be overwritten? Are there library functions that exist between the heap and the stack that could be overwritten? Can heap variables be overwritten? I assume that variables in the data and bss sections can be overwritten, but is the text segment protected from writes?
The layout of processes in memory varies from system to system. This answer covers Linux under x86_64 processors.
There is a nice article illustrating the memory layout for Linux processes here.
If the buffer is a local variable, then it will be on the stack, along with other local variables. The first thing you are likely to hit if you overflow the buffer is other local variables in the same function.
When you reach the end of the stack, there is a randomly sized offset before the next used segment of memory. If you continue writing into this address space you would trigger a segfault (since that address space is not mapped to any physical RAM).
Assuming you managed to skip over the random offset without crashing, and continued overwriting, the next thing it might hit is the memory mapping segment. This segment contains file mappings, including those used to map dynamic shared libraries into the address space, and anonymous mappings. The dynamic libraries are going to be read-only, but if the process had any RW mappings in place you could perhaps overwrite data in them.
After this segment comes another random offset before you hit the heap. Again if you tried to write into the address space of the random offset you would trigger a crash.
Below the heap comes another random offset, followed by the BSS, Data and finally text segments. Static variables within BSS and Data could be overwritten. The text segment should not be writable.
You can inspect the memory map of a process using the pmap command.
The answer to your question depends entirely on what operating system is being used, as well as what hardware architecture. The operating system lays out logical memory in a certain fashion, and the architecture sometimes reserves (very low) memory for specific purposes as well.
One thing to understand is that traditional processes can access their entire logical memory space, but very little of this capacity is typically used. The most likely effect of what you describe is that you'll try to access some unallocated memory and you'll get a segfault in response, crashing your program.
That said, you definitely can modify these other segments of memory, but what happens when you do so depends on their read/write permissions. For example, the typical memory layout you learn in school is:
Low memory to high memory:
.text - program code
.data - initialized static variables
.bss - uninitialized static variables
.heap - grows up
memory map segments - dynamic libraries
.stack - grows down
The .text segment is marked read only / executable by default, so if you attempt to write to a .text memory location you'll get a segmentation fault. It's possible to change .text to writeable, but this is in general a terrible idea.
The .data, .bss, .heap, and .stack segments are all readable/writeable by default, so you can overwrite those sections without any program errors.
The memory map segment(s) all have their own permissions to deal with as well. Some of these segments are writeable, most are not (so writing to them creates segfaults).
The last thing to note is that most modern OSes will randomize the locations of these segments to make things more difficult for hackers. This may introduce gaps between different segments (which will again cause segfaults if you try to access them).
On Linux, you can print out a process' memory map with the command pmap. The following is the output of this program on an instance of vim:
10636: vim hello.text
0000000000400000 2112K r-x-- vim
000000000080f000 4K r---- vim
0000000000810000 88K rw--- vim
0000000000826000 56K rw--- [ anon ]
0000000000851000 2228K rw--- [ anon ]
00007f7df24c6000 8212K r--s- passwd
00007f7df2ccb000 32K r-x-- libnss_sss.so.2
00007f7df2cd3000 2044K ----- libnss_sss.so.2
00007f7df2ed2000 4K r---- libnss_sss.so.2
00007f7df2ed3000 4K rw--- libnss_sss.so.2
00007f7df2ed4000 48K r-x-- libnss_files-2.17.so
00007f7df2ee0000 2044K ----- libnss_files-2.17.so
00007f7df30df000 4K r---- libnss_files-2.17.so
00007f7df30e0000 4K rw--- libnss_files-2.17.so
00007f7df30e1000 24K rw--- [ anon ]
00007f7df30e7000 103580K r---- locale-archive
00007f7df960e000 8K r-x-- libfreebl3.so
00007f7df9610000 2044K ----- libfreebl3.so
00007f7df980f000 4K r---- libfreebl3.so
00007f7df9810000 4K rw--- libfreebl3.so
00007f7df9811000 8K r-x-- libutil-2.17.so
00007f7df9813000 2044K ----- libutil-2.17.so
00007f7df9a12000 4K r---- libutil-2.17.so
00007f7df9a13000 4K rw--- libutil-2.17.so
00007f7df9a14000 32K r-x-- libcrypt-2.17.so
00007f7df9a1c000 2044K ----- libcrypt-2.17.so
00007f7df9c1b000 4K r---- libcrypt-2.17.so
00007f7df9c1c000 4K rw--- libcrypt-2.17.so
00007f7df9c1d000 184K rw--- [ anon ]
00007f7df9c4b000 88K r-x-- libnsl-2.17.so
00007f7df9c61000 2044K ----- libnsl-2.17.so
00007f7df9e60000 4K r---- libnsl-2.17.so
00007f7df9e61000 4K rw--- libnsl-2.17.so
00007f7df9e62000 8K rw--- [ anon ]
00007f7df9e64000 88K r-x-- libresolv-2.17.so
00007f7df9e7a000 2048K ----- libresolv-2.17.so
00007f7dfa07a000 4K r---- libresolv-2.17.so
00007f7dfa07b000 4K rw--- libresolv-2.17.so
00007f7dfa07c000 8K rw--- [ anon ]
00007f7dfa07e000 152K r-x-- libncurses.so.5.9
00007f7dfa0a4000 2044K ----- libncurses.so.5.9
00007f7dfa2a3000 4K r---- libncurses.so.5.9
00007f7dfa2a4000 4K rw--- libncurses.so.5.9
00007f7dfa2a5000 16K r-x-- libattr.so.1.1.0
00007f7dfa2a9000 2044K ----- libattr.so.1.1.0
00007f7dfa4a8000 4K r---- libattr.so.1.1.0
00007f7dfa4a9000 4K rw--- libattr.so.1.1.0
00007f7dfa4aa000 144K r-x-- liblzma.so.5.0.99
00007f7dfa4ce000 2044K ----- liblzma.so.5.0.99
00007f7dfa6cd000 4K r---- liblzma.so.5.0.99
00007f7dfa6ce000 4K rw--- liblzma.so.5.0.99
00007f7dfa6cf000 384K r-x-- libpcre.so.1.2.0
00007f7dfa72f000 2044K ----- libpcre.so.1.2.0
00007f7dfa92e000 4K r---- libpcre.so.1.2.0
00007f7dfa92f000 4K rw--- libpcre.so.1.2.0
00007f7dfa930000 1756K r-x-- libc-2.17.so
00007f7dfaae7000 2048K ----- libc-2.17.so
00007f7dface7000 16K r---- libc-2.17.so
00007f7dfaceb000 8K rw--- libc-2.17.so
00007f7dfaced000 20K rw--- [ anon ]
00007f7dfacf2000 88K r-x-- libpthread-2.17.so
00007f7dfad08000 2048K ----- libpthread-2.17.so
00007f7dfaf08000 4K r---- libpthread-2.17.so
00007f7dfaf09000 4K rw--- libpthread-2.17.so
00007f7dfaf0a000 16K rw--- [ anon ]
00007f7dfaf0e000 1548K r-x-- libperl.so
00007f7dfb091000 2044K ----- libperl.so
00007f7dfb290000 16K r---- libperl.so
00007f7dfb294000 24K rw--- libperl.so
00007f7dfb29a000 4K rw--- [ anon ]
00007f7dfb29b000 12K r-x-- libdl-2.17.so
00007f7dfb29e000 2044K ----- libdl-2.17.so
00007f7dfb49d000 4K r---- libdl-2.17.so
00007f7dfb49e000 4K rw--- libdl-2.17.so
00007f7dfb49f000 20K r-x-- libgpm.so.2.1.0
00007f7dfb4a4000 2048K ----- libgpm.so.2.1.0
00007f7dfb6a4000 4K r---- libgpm.so.2.1.0
00007f7dfb6a5000 4K rw--- libgpm.so.2.1.0
00007f7dfb6a6000 28K r-x-- libacl.so.1.1.0
00007f7dfb6ad000 2048K ----- libacl.so.1.1.0
00007f7dfb8ad000 4K r---- libacl.so.1.1.0
00007f7dfb8ae000 4K rw--- libacl.so.1.1.0
00007f7dfb8af000 148K r-x-- libtinfo.so.5.9
00007f7dfb8d4000 2048K ----- libtinfo.so.5.9
00007f7dfbad4000 16K r---- libtinfo.so.5.9
00007f7dfbad8000 4K rw--- libtinfo.so.5.9
00007f7dfbad9000 132K r-x-- libselinux.so.1
00007f7dfbafa000 2048K ----- libselinux.so.1
00007f7dfbcfa000 4K r---- libselinux.so.1
00007f7dfbcfb000 4K rw--- libselinux.so.1
00007f7dfbcfc000 8K rw--- [ anon ]
00007f7dfbcfe000 1028K r-x-- libm-2.17.so
00007f7dfbdff000 2044K ----- libm-2.17.so
00007f7dfbffe000 4K r---- libm-2.17.so
00007f7dfbfff000 4K rw--- libm-2.17.so
00007f7dfc000000 132K r-x-- ld-2.17.so
00007f7dfc1f8000 40K rw--- [ anon ]
00007f7dfc220000 4K rw--- [ anon ]
00007f7dfc221000 4K r---- ld-2.17.so
00007f7dfc222000 4K rw--- ld-2.17.so
00007f7dfc223000 4K rw--- [ anon ]
00007ffcb46e7000 132K rw--- [ stack ]
00007ffcb475f000 8K r-x-- [ anon ]
ffffffffff600000 4K r-x-- [ anon ]
total 163772K
The segment starting at 0x851000 is actually the start of the heap (which pmap will tell you with more verbose reporting modes, but the more verbose mode didn't fit).
I think your question reflects a fundamental misunderstanding of how things work in an operating system. Things like "buffers" and "stack" tend not to be defined by the operating system.
The operating system divides memory into kernel and user areas (and some systems have additional, protected areas).
The layout of the user area is usually defined by the linker. The linker creates executables that instruct the loader how to set up the address space. Various linkers have different levels of control. Generally, the default linker settings group the sections of program as something like:
-Read/execute
-Read/no execute
-Read/write/initialized
-Read/write/demand zero
One some linkers you can create multiple program sections with these attributes.
You ask:
"If I have a character buffer containing the string "Hello world", instead of 'H' being placed at the lowest address, it is placed at the highest address, and so on."
In a van neumann machine, memory is independent of its usage. The same memory block can simultaneously be interpreted as a string, floating point, integer, or instruction. You can put your letter in any order you want but most software libraries would not recognize them in reverse order. IF your own libraries can handle the strings stored backwards, knock yourself out.
"My question is -- if the input string was long enough, what things could be overwritten?"
It could be anything.
"Are there library functions that exist between the heap and the stack that could be overwritten?"
That depends upon what your linker did.
"Can heap variables be overwritten?"
Heap can be overwritten.
"I assume that variables in the data and bss sections can be overwritten, but is the text segment protected from writes?
Generally, yes.
I was seeing where the stack , heap and shared library's address ranges starts. I'm seeing 2 values for shared library(which i created) and a.out. 3 values for ld and libc. rest is anonymous and stack region starting addresses.
kg>pmap 24545
24545: ./a.out
003d3000 4K r-x-- [ anon ]
004d9000 4K r-x-- /home/trng3/sh/POC/libfile_sys.so
004da000 4K rwx-- /home/trng3/sh/POC/libfile_sys.so
08048000 4K r-x-- /home/trng3/sh/POC/a.out
08049000 4K rwx-- /home/trng3/sh/POC/a.out
46f46000 100K r-x-- /lib/ld-2.5.so
46f5f000 4K r-x-- /lib/ld-2.5.so
46f60000 4K rwx-- /lib/ld-2.5.so
46f68000 1244K r-x-- /lib/libc-2.5.so
4709f000 8K r-x-- /lib/libc-2.5.so
470a1000 4K rwx-- /lib/libc-2.5.so
470a2000 12K rwx-- [ anon ]
b7f8a000 4K rw--- [ anon ]
b7fa1000 4K rw-s- /dev/zero (deleted)
b7fa2000 8K rw--- [ anon ]
bfc0f000 84K rw--- [ stack ]
Why is that we have 2 copies instead of one. Is the one is from the disk and the other one is currently in memory. What is the purpose of having the two copies of the same data in memory ?
They are not multiple copies, they are just different segments with different permissions. Look at the executable:
08048000 4K r-x-- /home/trng3/sh/POC/a.out
08049000 4K rwx-- /home/trng3/sh/POC/a.out
You can see that the first mapping has r-x permissions and the second mapping has rwx permissions. Ordinarily, the second mapping would have rw permissions but maybe your processor isn't capable of setting no-execute permissions, or maybe the feature is turned off, maybe the program was compiled with an executable data segment, or maybe the processor doesn't have the required granularity.
I think i386 without PAE has very coarse granularity for the NX-bit, so that might explain why the data segments are executable but the stack isn't.
46f46000 100K r-x-- /lib/ld-2.5.so
46f5f000 4K r-x-- /lib/ld-2.5.so
46f5f000 - 46f46000 = 25 * 4k = 100k. Its the last segment of the file. I still cant explain why but i found this.
In top, I noticed that my c program (using CUDA 3.2) has a virtual size of 28g or more (looking at VIRT), on every run right from the beginning. This doesn't make ANY sense to me. The resident memory makes sense and is only around 2g on my largest data set. I know at some point in the past the virtual size was not so large, but I'm not sure when the change occurred.
Why would my process use 28g of virtual memory (or why would top's VIRT be so large)? I understand that VIRT includes the executable binary (only 437K), shared libraries, and "data area". What is the "data area"? How can I find out how much memory the shared libraries require? What about other elements of my process's total memory?
contents of /proc/< pid >/smaps (1022 lines) here: http://pastebin.com/fTJJneXr
One of the entries from smaps show that one of them accounts for MOST of it, but has no label... how can I find out what this "blank" entry is that has 28gb?
200000000-900000000 ---p 00000000 00:00 0
Size: 29360128 kB
Rss: 0 kB
Pss: 0 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 0 kB
Anonymous: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
--
ubuntu 11.04 64-bit
16 GB RAM
UVA requires CUDA to allocate enough virtual memory to map all of both GPU and system memory. Please see post #5 in the following thread on the NVIDIA forums:
These two regions would be the culprit:
200000000-900000000 ---p 00000000 00:00 0
Size: 29360128 kB
Rss: 0 kB
Pss: 0 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 0 kB
Anonymous: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
7f2e9deec000-7f2f131ec000 rw-s 33cc0c000 00:05 12626 /dev/nvidia0
Size: 1920000 kB
Rss: 1920000 kB
Pss: 1920000 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 1920000 kB
Referenced: 1920000 kB
Anonymous: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
The first segment is a 30GB anonymous private segment, with no access to it allowed, mapped from 0x200000000-0x900000000. A bit mysterious, indeed - probably something to do with the nvidia driver's internal workings (maybe it wants to prevent allocations with those specific addresses?). It's not actually occupying any memory though - Rss is zero, and the access flags (---p) are set to deny all access, so (at the moment) actually allocating any memory to it won't happen. It's just a reserved section in your address space.
The other bit is the /dev/nvidia0 mapping, of two gigabytes. This is likely a direct mapping of part of the video card's RAM. It's not occupying memory as such - it's just reserving part of your address space to use to communicate with hardware.
So it's not really something to worry about. If you want to know how much memory you're really using, add up the Rss figures for all other memory segments (use the Private_* entries instead if you want to skip shared libraries and such).
In my code I tried to print the address of printf() function.
22834: ./a.out
00250000 1372K r-x-- /lib/libc-2.12.1.so
003a7000 8K r---- /lib/libc-2.12.1.so
003a9000 4K rw--- /lib/libc-2.12.1.so
003aa000 12K rw--- [ anon ]
00a14000 112K r-x-- /lib/ld-2.12.1.so
00a30000 4K r---- /lib/ld-2.12.1.so
00a31000 4K rw--- /lib/ld-2.12.1.so
00fb9000 4K r-x-- [ anon ]
08048000 4K r-x-- /home/anirudh/Documents/DUMP/a.out
08049000 4K r---- /home/anirudh/Documents/DUMP/a.out
0804a000 4K rw--- /home/anirudh/Documents/DUMP/a.out
08068000 132K rw--- [ anon ]
b7898000 4K rw--- [ anon ]
b78ac000 8K rw--- [ anon ]
bfc9a000 132K rw--- [ stack ]
total 1808K
Address of the function "printf()" in HEX = 8048408
I was expecting its address to be a part of
00250000 1372K r-x-- /lib/libc-2.12.1.so
rather than as shown that its code is in this section
08048000 4K r-x-- /home/anirudh/Documents/DUMP/a.out i.e code segment of my code.
I even tried printing the address of a few more functions like getpid() and scanf() but they were all shown to be part of my program's code segment rather than the libc-2.12.1.so
What is the reason for this ?. Thanks in advance.
Look at the code that is at the printf address; it is probably just an indirect jump or call that calls into libc. Typically, calls to shared libraries are turned into references to a dispatch function that is patched with (or looks up) the actual address where printf was loaded. If you run readelf -a on your executable and look for the address where you found printf, it will probably be marked as a relocation to be pointed to the actual address in libc.
Not sure, but it can be that you are printing the chunk that does the actual printf call?
Like, when you call a symbol that has to be resolved at dynamic link time there has to be some code there, like a trampoline or something like that, so when you call it the resolution happens and the actual library call gets called.
The following is the result after run on solaris, it shows there are two heaps, but in my understanding, for a process, there is only one heap which is a large continuous memory which can be managed by brk to expand or shrink the size. And for anon memory, a process can have many anon memory which can be managed by mmap/munmap. Is my understanding correct? or I read the result of the pmap wrongly?
sol9# pmap -sx pgrep testprog
...
00022000 3960 3960 3960 - 8K rwx-- [ heap ]
00400000 131072 131072 131072 - 4M rwx-- [ heap ]
...
FF390000 8 8 - - 8K r-x-- libc_psr.so.1
FF3B0000 8 8 8 - 8K rwx-- [ anon ]
...
total Kb 135968 135944 135112 -
You are both correct and misreading the pmap output. If you had done pmap -x the results would probably be less confusing, showing the heap just once, but since you added the -s flag, it breaks down the heap into segments with different page mappings.
The addresses starting at 0x0022000 are not aligned properly to be mapped to a 4Mb page, so they use 3960kb of 8k pages. 0x0022000+(3960*1024) = 0x00400000
At 0x00400000 the address is properly aligned for 4Mb pages, so the heap switches to using the larger pages with fewer page table entries.
If you wanted to ensure that your heap started at the proper alignment to use 4Mb pages for the whole thing instead of starting with 8k until it reached an alignment boundary, then you would link your program with -M /usr/lib/ld/map.bssalign to do that.
A slightly more in-depth explanation can be found in the Page Size and Memory Layout blog post from Solaris Application Programming author Darryl Gove.