I remember reading a long time ago that if I want to test for buffer overflows on my linux box that I need to set something in the system to allow it to happen. I can't remember exactly what it was for, but I was hoping some one knew what I was talking about.
I want to be able to test my programs for vulnerabilities, and see if the registers are overwritten.
EDIT: I am running ubuntu 10.04
One option is to use a memory debugger such as Valgrind. Note, however, that Valgrind only tracks for buffer overflows on dynamically-allocated memory.
If you have the option to use C++ instead of C, then you can switch to using containers rather than raw arrays, and harness GCC's "checked container" mode (see GCC STL bound checking). I'm sure other compilers offer similar tools.
Another hint (in addition of Oli's answer), when chasing memory bugs with the gdb debugger, is to disable address space layout randomization, with e.g.
echo 0 > /proc/sys/kernel/randomize_va_space
After doing that, two consecutive runs of the same deterministic program will usually mmap regions at the same addresses (from one run to another), and this helps a lot debugging with gdb (because then malloc usually gives the same result from one run to another, at the same given location in the run).
You can also use the watch command of gdb. In particular, if in a first run (with ASLR disabled) you figure that the location 0x123456 is changing unexepectedly, you could give gdb the following command in its second run:
watch * (void**) 0x123456
Then gdb will break when this location changes (sadly, it has to be mmap-ed already).
Related
I'm trying to optimize a C code project.
I would like to count how many times a global variable was used (read or write) in order to place it at the most suitable memory type.
For example, to store commonly used variables at the fast access memory type.
Data cache is disabled for determenistic reasons.
Is there a way to count how many times a variable was used without inserting counters or adding extra code? for example, using the assembly code?
The code is written in C.
In my possession:
A) (.map) file, generated by the GCC compiler from which I extracts the global variables names, addresses and sizes.
B) The assembly code of the project generated using the GCC compiler -S flag.
Thanks a lot,
GDB has something called watchpoints: https://sourceware.org/gdb/onlinedocs/gdb/Set-Watchpoints.html
Set a watchpoint for an expression. GDB will break when the expression
expr is written into by the program and its value changes. The
simplest (and the most popular) use of this command is to watch the
value of a single variable:
(gdb) watch foo
awatch [-l|-location] expr [thread thread-id] [mask maskvalue]
Set a watchpoint that will break when expr is either read from or written
into by the program.
Commands can be attached to watchpoints: https://sourceware.org/gdb/onlinedocs/gdb/Break-Commands.html#Break-Commands
You can give any breakpoint (or watchpoint or catchpoint) a series of commands to execute when your program stops due to that breakpoint… For example, here is how you could use breakpoint commands to print
the value of x at entry to foo whenever x is positive.
break foo if x>0
commands
silent
printf "x is %d\n",x
cont
end
The command should typically increment a variable or print "read/write" to a file, but you can really add other stuff too such as a backtrace. Unsure about the best way for outward communication using gdb. Maybe it is good enough for you to run it in interactive mode.
You can do this, using Visual Studio (or an other IDE): search for all places where your variable is used in the source code, put a conditional breakpoint, logging some information, attach to process, and launch the features which use that variable. You can count the instances in the output window.
I think, what you need is automatic instrumentation and/or profiling. GCC can actually do profile-guided optimization for you. As well as other types of instrumentation, the documentation even mentions a hook for implementing your own custom instrumentation.
There are several performance analysis tools out there such as perf and gprof profilers.
Also, execution inside a virtual machine could (at least in theory) do what you are after. valgrind comes to mind. I think, valgrind actually knows about all memory accesses. I'd look for ways to obtain this informaiton (and then corellate that with the map files).
I don't know if any of the above tools solves exactly your problem, but you definitely could use, say, perf (if it's available for your platform) to see in what areas of code significant time is spent. Then probably there are either a lot of expensive memory accesses, or just intensive computations, you can figure out which is the case by staring at the code.
Note that the compiler already allocates frequently accessed variables to registers, so the kind of information you are after won't give you an accurate picture. I.e. while some variable might be accessed a lot, cache-allocating it might not improve things much if its value already lives on a register most of the time.
Also consider that optimization affects your program greatly on the assembly level. So any performance statistics such as memory accesses counters would be different with and without optimization. And what should be of interest to you is the optimized case. On the other hand, restoring the information about what location corresponds to which variable is harder with the optimized program if feasible at all.
I think I am experiencing a stack overflow issue when running my unit-tests through gdb within emacs on Windows.
I run exactly the same unit-tests on linux with no problems
I am also aware that I am using a horrendously memory inefficient stack-based .ini file parser within these unit-tests so it would seem to be a reasonable possibility that stack overflow is occuring.
I have noticed a few unit tests faling on Windows that passed on Linux. Further investigation reveals a (stack-based) counter in a for loop resetting itself to zero at random points in the for loops execution and the (stack-based) values in the arrays that the for loop is inspecting change for the same index value
I noticed that gdb seems to be allocated its own thread of execution under Windows - is there any way of finding out how much stack space the thread is allocated?
One of the differences between Linux and Windows is that on Windows the stack-size has to be set at compile-time (there are two sizes, initial and a predefined reserved limit). Not sure what the default is with the compiler you are using, but you might try to increase it using --stack parameter (gcc).
On Linux the stack-size is dynamic up to the limits, generally set by the sysadmin.
Well then, perhaps Windows has tighter restrictions on the maximum amount of stack per process than Linux does?
This page details how to debug a stack overflow in Windows. It's not based on gdb, but perhaps you can still infer something.
Recently I'm doing some Return-to-libc attack experiment base on the paper Bypassing non-executable-stack during exploitation using return-to-libc with my Ubuntu11.10.
Before my experiment I closed the ALSR.
According to the paper, I can find address of the environment variable SHELL="/bin/bash" in gdb(use gdb to debug the program I want to attack):
But I found that this address is wrong when I try to use it to Return-to-libc experiment.
And then I write a simple program to get the environment variable address:
When I run this program in the Terminal, I get the right address:
With this address I can do the attack.
I also find the related question about this. But the answers doesn't really make sense(the second one may be better).
Just tell me some details about this, please.
From your screenshots, I'll assume you're running on an 32-bit intel platform. I haven't spent the time to fully research an answer to this, but these are points worth noting:
I'll bet that your entire environment is in about the same place, and is packed together tightly as c-style strings. (try x/100s **(char***)&environ).
When I tried ths on my x86-64 installation, the only thing I saw after the environment was my command line, and some empty strings.
At 0xBffff47A, you're very close to the top of user address space (which ends at 0xC0000000).
So, my guess is that what's going on here is that:
The environment block and command line parameters are, at some point during startup, shoved in a packed form right at the end of user address space.
The contents of your environment are different when you run your program in GDB or in the terminal. For example, I notice "_=/usr/bin/gdb" when running under GDB, and I'll just bet that's only there when running under GDB.
The result is that, while your fixed pointer tends to land somewhere in the middle of the environment block, it doesn't land in the same place every time, since the environment itself is changing between runs.
I have built a plain C code on Linux (Fedora) using code-sorcery tool-chain. This is for ARM Cortex-A8 target. This code is running on a Cortex A8 board, running embedded Linux.
When I run this code for some test case, which does dynamic memory allocation (malloc) for some large size (10MB), it crashes after some time giving error message as below:
select 1 (init), adj 0, size 61, to kill
select 1030 (syslogd), adj 0, size 64, to kill
select 1032 (klogd), adj 0, size 74, to kill
select 1227 (bash), adj 0, size 378, to kill
select 1254 (ppp), adj 0, size 1069, to kill
select 1255 (TheoraDec_Corte), adj 0, size 1159, to kill
send sigkill to 1255 (TheoraDec_Corte), adj 0, size 1159
Program terminated with signal SIGKILL, Killed.
Then, when I debug this code for the same test case using gdb built for the target, the point where this dynamic memory allocation happens, code fails to allocate that memory and malloc returns NULL. But during normal stand-alone run, I believe malloc should be failing to allocate but it strangely might not be returning NULL, but it crashes and the OS kills my process.
Why is this behaviour different when run under gdb and when without debugger?
Why would malloc fails yet not return a NULL. Could this be possible, or the reason for the error message I am getting is else?
How do I fix this?
thanks,
-AD
So, for this part of the question, there is a surefire answer:
Why would malloc fails yet not return a NULL. Could this be possible, or the reason for the error message i am getting is else?
In Linux, by default the kernel interfaces for allocating memory almost never fail outright. Instead, they set up your page table in such a way that on the first access to the memory you asked for, the CPU will generate a page fault, at which point the kernel handles this and looks for physical memory that will be used for that (virtual) page. So, in an out-of-memory situation, you can ask the kernel for memory, it will "succeed", and the first time you try to touch that memory it returned back, this is when the allocation actually fails, killing your process. (Or perhaps some other unfortunate victim. There are some heuristics for that, which I'm not incredibly familiar with. See "oom-killer".)
Some of your other questions, the answers are less clear for me.
Why is this behaviour different when run under gdb and when without debugger?It could be (just a guess really) that GDB has its own malloc, and is tracking your allocations somehow. On a somewhat related point, I've actually frequently found that heap bugs in my code often aren't reproducible under debuggers. This is frustrating and makes me scratch my head, but it's basically something I've pretty much figured one has to live with...
How do i fix this?
This is a bit of a sledgehammer solution (that is, it changes the behavior for all processes rather than just your own, and it's generally not a good idea to have your program alter global state like that), but you can write the string 2 to /proc/sys/vm/overcommit_memory. See this link that I got from a Google search.
Failing that... I'd just make sure you're not allocating more than you expect to.
By definition running under a debugger is different than running standalone. Debuggers can and do hide many of the bugs. If you compile for debugging you can add a fair amount of code, similar to compiling completely unoptimized (allowing you to single step or watch variables for example). Where compiling for release can remove debugging options and remove code that you needed, there are many optimization traps you can fall into. I dont know from your post who is controlling the compile options or what they are.
Unless you plan to deliver the product to be run under the debugger you should do your testing standalone. Ideally do your development without the debugger as well, saves you from having to do everything twice.
It sounds like a bug in your code, slowly re-read your code using new eyes as if you were explaining it to someone, or perhaps actually explain it to someone, line by line. There may be something right there that you cannot see because you have been looking at it the same way for too long. It is amazing how many times and how well that works.
I could also be a compiler bug. Doing things like printing out the return value, or not can cause the compiler to generate different code. Adding another variable and saving the result to that variable can kick the compiler to do something different. Try changing the compiler options, reduce or remove any optimization options, reduce or remove the debugger compiler options, etc.
Is this a proven system or are you developing on new hardware? Try running without any of the caches enabled for example. Working in a debugger and not in standalone, if not a compiler bug can be a timing issue, single stepping flushes the pipline, mixes the cache up differently, gives the cache and memory system an eternity to come up with a result which it doesnt have in real time.
In short there is a very long list of reasons why running under a debugger hides bugs that you cannot find until you test in the final deliverable like environment, I have only touched on a few. Having it work in the debugger and not in standalone is not unexpected, it is simply how the tools work. It is likely your code, the hardware, or your tools based on the description you have given so far.
The fastest way to eliminate it being your code or the tools is to disassemble the section and inspect how the passed values and return values are handled. If the return value is optimized out there is your answer.
Are you compiling for a shared C library or static? Perhaps compile for static...
I wrote a C program in which I did some pretty heavy stack allocation, around 2 MiB. Since I use the poor man's IDE* I was automatically running the program via make in order to test it, each time I compiled.
I had pretty much wrapped everything up, but for some reason, during some of the final optimization, I ran it directly from the shell. Instant segfault! Running it with make still worked, and running it by hand always produced the same segfault.
I eventually reduced the amount of stack allocation I was doing to 256 KiB, which solved the problem. My rationale was that make was probably exec-ing the process, and thus it was inheriting some weird parameters that allowed it to use more stack space.
Although everything is fine now, I have no way of testing my theory. Can anyone confirm or deny, or suggest some way of testing?
* zsh, vim, gcc, gdb, and some nutty makefiles
You can try setting the maximum stack size with ulimit(1) and see if it works:
# Limit stack to 1024 KiB
ulimit -s 1024; ./myprogram
# Now no limit
ulimit -s unlimited; ./myprogram
Personally, my first step would be to try to locate where in the code the segfault occurred, either with gdb or debugging printfs or whatever. (This is why you always check return values from malloc, it cuts down on the possible sources of segfaults ;-p) For one thing, finding the exact source of the problem could give you evidence for or against your theory about stack allocation; also, it'll let you insert error-checking code so the program can exit gracefully with an informative error message rather than segfaulting.
More information is needed to diagnose this. Namely, it'd be nice to see the makefile and the shell scripts which you were using.