How do you properly implement a robust stack in C - c

Prior Knowledge
I've read and understand a bit about stacks and data structures, but couldn't find the answer to this specific question. I know that any programmer worth anything will implement security exception handling beyond the default in their programs.
Situation
I would like to understand how a program could be written in C that establishes a relatively robust stack with exception handling for some arbitrary case S.
My goal is to discern from this very specific information why (from what I understand) it is always possible to exploit the SEH in a program and execute arbitrary code.
The issue is not that I do not understand the concept of overflowing a buffer - I do not understand why (in very specific, CPU architecture related reasons) the security implemented on the stack (canaries, etc) cannot sufficiently address these issues (like heap overflows cannot be stopped?).
Reference Information
"There is no sane way to alter the layout of data within a structure; structures are expected to be the same between modules, especially with shared libraries. Any data in a structure after a buffer is impossible to protect with canaries; thus, programmers must be very careful about how they organize their variables and use their structures. In C and C++, structures with buffers should either be malloc()ed or obtained with new."
- via http://en.wikipedia.org/wiki/Buffer_overflow_protection#Implementations
Also:
http://blogs.msdn.com/b/michael_howard/archive/2006/08/16/702707.aspx
If anyone knows a good resource for understanding this material or can provide a code snippet, I would be very grateful.

The security of the stack can be made secure enough to cope with stack overflows:
http://msdn.microsoft.com/en-us/library/9a89h429(VS.80).aspx
The question as to why you need registered exception handlers (safe-SEH) and why normal exception handlers won't cut it is because of the case where you get very large stack overflows.
Let's suppose I have the function which begins
try {
char buffer[N];
strcpy(&buffer, &attacker);
} __except(...) { }
This might translate into the assembly code
push ebp
mov ebp, esp
; GS if you want to here
; install the exception handler:
push lbl_Exceptionhandler
push dword ptr [fs:0]
mov dword ptr[fs:0], esp
; setup the locals inside the stack
sub esp, LOCALS
; GS if you want to here
; call strcpy
lea ecx [ebp + offset_to_buffer];
push ecx
lea edx, [ebp + offset_to_attacker]
push edx
call _strcpy
add esp, 8
; uninstall the locals
mov esp, ebp
; uninstall the exception handler
pop dword ptr [fs:0]
; return
pop ebp
; optionally check GS cookies that we might have also inserted at any point in this function.
call _checksecuritycookie
ret
Or in other words, the stack looks like this:
RET PTR
/GS1
SAVED EBP
/GS2
SAVED FS:0
/GS3
LOCAL char buffer[N]
GS1, GS2 and GS3 are locations where stack canaries might choose to write the stack cookie. Note that the cookie will only be checked at the end of the function (This is important in computer security. When you introduce a check you need to think not only whether the check will detect the overflow, but whether it will detect it before it is already too late; and that requires thinking where the check will take place. For stack cookies, the cookie is only inspected on function exits, because stack cookies are generally only there to protect the return address, not to protect local variables).
The problem with normal exception handlers is what happens if the attacker buffer is really huge. Let's suppose it's so huge it destroys the entire stack, writes onto the guard page for the thread and triggers a fault?
Well, the kernel calls back into ntdll and tells it to sort it's process out, and ntdll's first port of call is to see if there are any registered exception handlers. Now how does it find what exception handler to call? Well, it looks at fs:0, which points to the exception handler on the stack, and calls the exception handler pointer. Except that exception handler's on the stack that the attacker just destroyed.
Oops. Now the attacker has control of EIP and you lose.
Safe-SEH solves this problem by noting that the list of exception handlers that you might ever want to call is in fact a finite list entirely determined at compile time. By burning this list into the PE file itself, ntdll has a chance to double check that the exception handler that it should jump to is, in fact, a real exception handler and not the cause of some evil attacker's plot to take over your EIP.
There's a cost to Safe-SEH (hence why it is opt in), but that cost is that it becomes more expensive to catch an exception, since ntdll will now do more work before your exception handler takes over.
Despite this, my advice is that SafeSEH should always be on. Making it easier to lose your customer's credit card details because your app is critically performance dependent on the speed of throwing exceptions suggests a mentality so horrendously broken in the developers mind that they should be immediately put into a cannon and fired into the sun to avoid their awful code from damaging society.

The normal way to implement a stack in C is to use a linked list. For example:
struct stack_entry {
struct stack_entry *previous;
/* other fields for the actual data */
}
struct stack_entry *stack_top = NULL;
void push(struct stack_entry *entry) {
entry->previous = stack_top;
stack_top = entry;
}
struct stack_entry *pop(void) {
struct stack_entry *entry;
entry = stack_top;
if(entry != NULL) stack_top = entry->previous;
return entry;
}
This is as robust and difficult to exploit as any other normal code.

If you're not implementing a stack in C, but are implementing a C compiler (in any language), then..
It would be possible to create a C compiler that detects programming errors and generates safe code. For example, for every read or write the compiler could insert checks to ensure that the read or write is contained within one storage area (e.g. and you're not trying to write 4 bytes at the end of an array of char or something); where a signal is raised (e.g. "SIGSEGV") if one of these checks fail.
Due to the nature of C, these checks would involve scanning things like the heap, and it'd require more code to be insert to track the sizes of things on the stack.
The main reason this hasn't been implemented is that it'd create massive performance problems, and would therefore defeat the purpose of using C to begin with.
However, there are debugging tools (e.g. valgrind) that do this type of checking by running the application inside a virtual machine (where the virtual machine tracks the sizes of storage areas and is able to check reads/writes before they're performed).

Related

Why not trap stack writes above ebp to avoid overflow exploits?

I've been looking at StackGuard and similar, and also Intel's new technology preview on "Control flow enforcement" (basically a shadow stack), here: https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf.
Obviously there is a reason why what I'm wondering will either break everything or not protect against buffer overflows, but its simple so I'm sure someone can explain why I'm barking up the wrong tree.
Why not implement in CPU hardware an optional feature to abort/trap when writing to a stack address higher than or equal to ebp? This would protect the return address and function parameters from being overwritten via a buffer overflow.
Use of ebp as frame pointer is optional, but of course that could be changed. Worse problem is that you may legally write outside of your stack frame, such as if you got a pointer to a variable belonging to a caller:
int foo;
scanf("%d", &foo);
Obviously &foo points outside of the frame of scanf.
Function parameters don't need to be protected, they can be legally modified too. This could also be changed, however.

Allocating a new call stack

(I think there's a high chance of this question either being a duplicate or otherwise answered here already, but searching for the answer is hard thanks to interference from "stack allocation" and related terms.)
I have a toy compiler I've been working on for a scripting language. In order to be able to pause the execution of a script while it's in progress and return to the host program, it has its own stack: a simple block of memory with a "stack pointer" variable that gets incremented using the normal C code operations for that sort of thing and so on and so forth. Not interesting so far.
At the moment I compile to C. But I'm interested in investigating compiling to machine code as well - while keeping the secondary stack and the ability to return to the host program at predefined control points.
So... I figure it's not likely to be a problem to use the conventional stack registers within my own code, I assume what happens to registers there is my own business as long as everything is restored when it's done (do correct me if I'm wrong on this point). But... if I want the script code to call out to some other library code, is it safe to leave the program using this "virtual stack", or is it essential that it be given back the original stack for this purpose?
Answers like this one and this one indicate that the stack isn't a conventional block of memory, but that it relies on special, system specific behaviour to do with page faults and whatnot.
So:
is it safe to move the stack pointers into some other area of memory? Stack memory isn't "special"? I figure threading libraries must do something like this, as they create more stacks...
assuming any area of memory is safe to manipulate using the stack registers and instructions, I can think of no reason why it would be a problem to call any functions with a known call depth (i.e. no recursion, no function pointers) as long as that amount is available on the virtual stack. Right?
stack overflow is obviously a problem in normal code anyway, but would there be any extra-disastrous consequences to an overflow in such a system?
This is obviously not actually necessary, since simply returning the pointers to the real stack would be perfectly serviceable, or for that matter not abusing them in the first place and just putting up with fewer registers, and I probably shouldn't try to do it at all (not least due to being obviously out of my depth). But I'm still curious either way. Want to know how these sorts of things work.
EDIT: Sorry of course, should have said. I'm working on x86 (32-bit for my own machine), Windows and Ubuntu. Nothing exotic.
All of these answer are based on "common processor architectures", and since it involves generating assembler code, it has to be "target specific" - if you decide to do this on processor X, which has some weird handling of stack, below is obviously not worth the screensurface it's written on [substitute for paper]. For x86 in general, the below holds unless otherwise stated.
is it safe to move the stack pointers into some other area of memory?
Stack memory isn't "special"? I figure threading libraries
must do something like this, as they create more stacks...
The memory as such is not special. This does however assume that it's not on an x86 architecture where the stack segment is used to limit the stack usage. Whilst that is possible, it's rather rare to see in an implementation. I know that some years ago Nokia had a special operating system using segments in 32-bit mode. As far as I can think of right now, that's the only one I've got any contact with that uses the stack segment for as x86-segmentation mode describes.
Assuming any area of memory is safe to manipulate using the stack
registers and instructions, I can think of no reason why it would be a
problem to call any functions with a known call depth (i.e. no
recursion, no function pointers) as long as that amount is available
on the virtual stack. Right?
Correct. Just as long as you don't expect to be able to get back to some other function without switching back to the original stack. Limited level of recursion would also be acceptable, as long as the stack is deep enough [there are certain types of problems that are definitely hard to solve without recursion - binary tree search for example].
stack overflow is obviously a problem in normal code anyway,
but would there be any extra-disastrous consequences to an overflow in
such a system?
Indeed, it would be a tough bug to crack if you are a little unlucky.
I would suggest that you use a call to VirtualProtect() (Windows) or mprotect() (Linux etc) to mark the "end of the stack" as unreadable and unwriteable so that if your code accidentally walks off the stack, it crashes properly rather than some other more subtle undefined behaviour [because it's not guaranteed that the memory just below (lower address) is unavailable, so you could overwrite some other useful things if it does go off the stack, and that would cause some very hard to debug bugs].
Adding a bit of code that occassionally checks the stack depth (you know where your stack starts and ends, so it shouldn't be hard to check if a particular stack value is "outside the range" [if you give yourself some "extra buffer space" between the top of the stack and the "we're dead" zone that you protected - a "crumble zone" as they would call it if it was a car in a crash]. You can also fill the entire stack with a recognisable pattern, and check how much of that is "untouched".
Typically, on x86, you can use the existing stack without any problems so long as:
you don't overflow it
you don't increment the stack pointer register (with pop or add esp, positive_value / sub esp, negative_value) beyond what your code starts with (if you do, interrupts or asynchronous callbacks (signals) or any other activity using the stack will trash its contents)
you don't cause any CPU exception (if you do, the exception handling code might not be able to unwind the stack to the nearest point where the exception can be handled)
The same applies to using a different block of memory for a temporary stack and pointing esp to its end.
The problem with exception handling and stack unwinding has to do with the fact that your compiled C and C++ code contains some exception-handling-related data structures like the ranges of eip with the links to their respective exception handlers (this tells where the closest exception handler is for every piece of code) and there's also some information related to identification of the calling function (i.e. where the return address is on the stack, etc), so you can bubble up exceptions. If you just plug in raw machine code into this "framework", you won't properly extend these exception-handling data structures to cover it, and if things go wrong, they'll likely go very wrong (the entire process may crash or become damaged, despite you having exception handlers around the generated code).
So, yeah, if you're careful, you can play with stacks.
You can use any region you like for the processor's stack (modulo the memory protections).
Essentially, you simply load the ESP register ("MOV ESP, ...") with a pointer to the new area, however you managed to allocate it.
You have to have enough for your program, and whatever it might call (e.g., a Windows OS API), and whatever funny behaviours the OS has. You might be able to figure out how much space your code needs; a good compiler can easily do that. Figuring how much is needed by Windows is harder; you can always allocate "way too much" which is what Windows programs tend to do.
If you decide to manage this space tightly, you'll probably have to switch stacks to call Windows functions. That won't be enough; you'll likely get burned by various Windows surprises. I describe one of them here Windows: avoid pushing full x86 context on stack. I have mediocre solutions, but not good solutions for this.

Recognizing stack frames in a stack using saved EBP values

I would like to divide a stack to stack-frames by looking on the raw data on the stack. I thought to do so by finding a "linked list" of saved EBP pointers.
Can I assume that a (standard and commonly used) C compiler (e.g. gcc) will always update and save EBP on a function call in the function prologue?
pushl %ebp
movl %esp, %ebp
Or are there cases where some compilers might skip that part for functions that don't get any parameters and don't have local variables?
The x86 calling conventions and the Wiki article on function prologue don't help much with that.
Is there any better method to divide a stack to stack frames just by looking on its raw data?
Thanks!
Some versions of gcc have a -fomit-frame-pointer optimization option. If memory serves, it can be used even with parameters/local variables (they index directly off of ESP instead of using EBP). Unless I'm badly mistaken, MS VC++ can do roughly the same.
Offhand, I'm not sure of a way that's anywhere close to universally applicable. If you have code with debug info, it's usually pretty easy -- otherwise though...
Even with the framepointer optimized out, stackframes are often distinguishable by looking through stack memory for saved return addresses instead. Remember that a function call sequence in x86 always consists of:
call someFunc ; pushes return address (instr. following `call`)
...
someFunc:
push EBP ; if framepointer is used
mov EBP, ESP ; if framepointer is used
push <nonvolatile regs>
...
so your stack will always - even if the framepointers are missing - have return addresses in there.
How do you recognize a return address ?
to start with, on x86, instruction have different lengths. That means return addresses - unlike other pointers (!) - tend to be misaligned values. Statistically 3/4 of them end not at a multiple of four.
Any misaligned pointer is a good candidate for a return address.
then, remember that call instructions on x86 have specific opcode formats; read a few bytes before the return address and check if you find a call opcode there (99% most of the time, it's five bytes back for a direct call, and three bytes back for a call through a register). If so, you've found a return address.
This is also a way to distinguish C++ vtables from return addresses by the way - vtable entrypoints you'll find on the stack, but looking "back" from those addresses you don't find call instructions.
With that method, you can get candidates for the call sequence out of the stack even without having symbols, framesize debugging information or anything.
The details of how to piece the actual call sequence together from those candidates are less straightforward though, you need a disassembler and some heuristics to trace potential call flows from the lowest-found return address all the way up to the last known program location. Maybe one day I'll blog about it ;-) though at this point I'd rather say that the margin of a stackoverflow posting is too small to contain this ...

How can I create a parallel stack and run a coroutine on it?

I decided I should try to implement coroutines (I think that's how I should call them) for fun and profits. I expect to have to use assembler, and probably some C if I want to make this actually useful for anything.
Bear in mind that this is for educational purposes. Using an already built coroutine library is too easy (and really no fun).
You guys know setjmp and longjmp? They allow you to unwind the stack up to a predefined location, and resumes execution from there. However, it can't rewind to "later" on the stack. Only come back earlier.
jmpbuf_t checkpoint;
int retval = setjmp(&checkpoint); // returns 0 the first time
/* lots of stuff, lots of calls, ... We're not even in the same frame anymore! */
longjmp(checkpoint, 0xcafebabe); // execution resumes where setjmp is, and now it returns 0xcafebabe instead of 0
What I'd like is a way to run, without threading, two functions on different stacks. (Obviously, only one runs at a time. No threading, I said.) These two functions must be able to resume the other's execution (and halt their own). Somewhat like if they were longjmping to the other. Once it returns to the other function, it must resume where it left (that is, during or after the call that gave control to the other function), a bit like how longjmp returns to setjmp.
This is how I thought it:
Function A creates and zeroes a parallel stack (allocates memory and all that).
Function A pushes all its registers to the current stack.
Function A sets the stack pointer and the base pointer to that new location, and pushes a mysterious data structure indicating where to jump back and where to set the instruction pointer back.
Function A zeroes most of its registers and sets the instruction pointer to the beginning of function B.
That's for the initialization. Now, the following situation will indefinitely loop:
Function B works on that stack, does whatever work it needs to.
Function B comes to a point where it needs to interrupt and give A control again.
Function B pushes all of its registers to its stack, takes the mysterious data structure A gave it at the very beginning, and sets the stack pointer and the instruction pointer to where A told it to. In the process, it hands back A a new, modified data structure that tells where to resume B.
Function A wakes up, popping back all the registers it pushed to its stack, and does work until it comes to a point where it needs to interrupt and give B control again.
All this sounds good to me. However, there is a number of things I'm not exactly at ease with.
Apparently, on good ol' x86, there was this pusha instruction that would send all registers to the stack. However, processor architectures evolve, and now with x86_64 we've got a lot more general-purpose registers, and likely several SSE registers. I couldn't find any evidence that pusha does push them. There are about 40 public registers in a mordern x86 CPU. Do I have to do all the pushes myself? Moreover, there is no push for SSE registers (though there's bound to be an equivalent—I'm new to this whole "x86 assembler" thing).
Is changing the instruction pointer as easy as saying it? Can I do, like, mov rip, rax (Intel syntax)? Also, getting the value from it must be somewhat special as it constantly changes. If I do like mov rax, rip (Intel syntax again), will rip be positioned on the mov instruction, to the instruction after it, or somewhere between? It's just jmp foo. Dummy.
I've mentioned a mysterious data structure a few times. Up to now I've assumed it needs to contain at least three things: the base pointer, the stack pointer and the instruction pointer. Is there anything else?
Did I forget anything?
While I'd really like to understand how things work, I'm pretty sure there are a handful of libraries that do just that. Do you know any? Is there any POSIX- or BSD-defined standard way to do it, like pthread for threads?
Thanks for reading my question textwall.
You are correct in that PUSHA wont work on x64 it will raise the exception #UD, as PUSHA only pushes the 16-bit or 32-bit general purpose registers. See the Intel manuals for all the info you ever wanted to know.
Setting RIP is simple, jmp rax will set RIP to RAX. To retrieve RIP, you could either get it at compile time if you already know all the coroutine exit origins, or you could get it at run time, you can make a call to the next address after that call. Like this:
a:
call b
b:
pop rax
RAX will now be b. This works because CALL pushes the address of the next instruction. This technique works on IA32 as well (although I'd suppose there's a nicer way to do it on x64, as it supports RIP-relative addressing, but I don't know of one). Of course if you make a function coroutine_yield, it can just intercept the caller address :)
Since you can't push all the registers to the stack in a single instruction, I wouldn't recommend storing the coroutine state on the stack, as that complicates things anyways. I think the nicest thing to do would be to allocate a data structure for every coroutine instance.
Why are you zeroing things in function A? That's probably not necessary.
Here's how I would approach the entire thing, trying to make it as simple as possible:
Create a structure coroutine_state that holds the following:
initarg
arg
registers (also contains the flags)
caller_registers
Create a function:
coroutine_state* coroutine_init(void (*coro_func)(coroutine_state*), void* initarg);
where coro_func is a pointer to the coroutine function body.
This function does the following:
allocate a coroutine_state structure cs
assign initarg to cs.initarg, these will be the initial argument to the coroutine
assign coro_func to cs.registers.rip
copy current flags to cs.registers (not registers, only flags, as we need some sane flags to prevent an apocalypse)
allocate some decent sized area for the coroutine's stack and assign that to cs.registers.rsp
return the pointer to the allocated coroutine_state structure
Now we have another function:
void* coroutine_next(coroutine_state cs, void* arg)
where cs is the structure returned from coroutine_init which represents a coroutine instance, and arg will be fed into the coroutine as it resumes execution.
This function is called by the coroutine invoker to pass in some new argument to the coroutine and resume it, the return value of this function is an arbitrary data structure returned (yielded) by the coroutine.
store all current flags/registers in cs.caller_registers except for RSP, see step 3.
store the arg in cs.arg
fix the invoker stack pointer (cs.caller_registers.rsp), adding 2*sizeof(void*) will fix it if you're lucky, you'd have to look this up to confirm it, you probably want this function to be stdcall so no registers are tampered with before calling it
mov rax, [rsp], assign RAX to cs.caller_registers.rip; explanation: unless your compiler is on crack, [RSP] will hold the instruction pointer to the instruction that follows the call instruction that called this function (ie: the return address)
load the flags and registers from cs.registers
jmp cs.registers.rip, efectively resuming execution of the coroutine
Note that we never return from this function, the coroutine we jump to "returns" for us (see coroutine_yield). Also note that inside this function you may run into many complications such as function prologue and epilogue generated by the C compiler, and perhaps register arguments, you have to take care of all this. Like I said, stdcall will save you lots of trouble, I think gcc's -fomit-frame_pointer will remove the epilogue stuff.
The last function is declared as:
void coroutine_yield(void* ret);
This function is called inside the coroutine to "pause" execution of the coroutine and return to the caller of coroutine_next.
store flags/registers in cs.registers
fix coroutine stack pointer (cs.registers.rsp), once again, add 2*sizeof(void*) to it, and you want this function to be stdcall as well
mov rax, arg (lets just pretend all the functions in your compiler return their arguments in RAX)
load flags/registers from cs.caller_registers
jmp cs.caller_registers.rip This essentially returns from the coroutine_next call on the coroutine invoker's stack frame, and since the return value is passed in RAX, we returned arg. Let's just say if arg is NULL, then the coroutine has terminated, otherwise it's an arbitrary data structure.
So to recap, you initialize a coroutine using coroutine_init, then you can repeatedly invoke the instantiated coroutine with coroutine_next.
The coroutine's function itself is declared:
void my_coro(coroutine_state cs)
cs.initarg holds the initial function argument (think constructor). Each time my_coro is called, cs.arg has a different argument that was specified by coroutine_next. This is how the coroutine invoker communicates with the coroutine. Finally, every time the coroutine wants to pause itself, it calls coroutine_yield, and passes one argument to it, which is the return value to the coroutine invoker.
Okay, you may now think "thats easy!", but I left out all the complications of loading the registers and flags in the correct order while still maintaining a non corrupt stack frame and somehow keeping the address of your coroutine data structure (you just overwrote all your registers), in a thread-safe manner. For that part you will need to find out how your compiler works internally... good luck :)
Good learning reference: libcoroutine, especially their setjmp/longjmp implementation. I know its not fun to use an existing library, but you can at least get a general bearing on where you are going.
Simon Tatham has an interesting implementation of coroutines in C that doesn't require any architecture-specific knowledge or stack fiddling. It's not exactly what you're after, but I thought it might nonetheless be of at least academic interest.
boost.coroutine (boost.context) at boost.org does all for you

How to get address of base stack pointer

I am in the process of porting an application from x86 to x64. I am using Visual Studio 2009; most of the code is C++ and some portions are plain C. The __asm keyword is not supported when compiling towards x64 and our application contains a few portions of inline assembler. I did not write this code so I don't know exactly what et is supposed to do:
int CallStackSize() {
DWORD Frame;
PDWORD pFrame;
__asm
{
mov EAX, EBP
mov Frame, EAX
}
pFrame = (PDWORD)Frame;
/*... do stuff with pFrame here*/
}
EBP is the base pointer to the stack of the current function. Is there some way to obtain the stack pointer without using inline asm? I have been looking at the intrinsics that Microsoft offers as a substitute for inline asm but I could not find anything that gave me something usefull. Any ideas?
Andreas asked what stuff is done with pFrame. Here is the complete function:
int CallStackSize(DWORD frameEBP = 0)
{
DWORD pc;
int tmpint = 0;
DWORD Frame;
PDWORD pFrame, pPrevFrame;
if(!frameEBP) // No frame supplied. Use current.
{
__asm
{
mov EAX, EBP
mov Frame, EAX
}
}
else Frame = frameEBP;
pFrame = (PDWORD)Frame;
do
{
pc = pFrame[1];
pPrevFrame = pFrame;
pFrame = (PDWORD)pFrame[0]; // precede to next higher frame on stack
if ((DWORD)pFrame & 3) // Frame pointer must be aligned on a DWORD boundary. Bail if not so.
break;
if (pFrame <= pPrevFrame)
break;
// Can two DWORDs be read from the supposed frame address?
if(IsBadWritePtr(pFrame, sizeof(PVOID)*2))
break;
tmpint++;
} while (true);
return tmpint;
}
The variable pc is not used. It looks like this function walks down the stack until it fails. It assumes that it can't read outside the applications stack so when it fails it has measured the depth of the call stack. This code does not need to compile on _EVERY_SINGLE compiler out there. Just VS2009. The application does not need to run on EVERY_SINGLE computer out there. We have complete control of deployment since we install/configure it ourselves and deliver the whole thing to our customers.
The really right thing to do would be to rewrite whatever this function does so that it does not require access to the actual frame pointer. That is definitely bad behavior.
But, to do what you are looking for you should be able to do:
int CallStackSize() {
__int64 Frame = 0; /* MUST be the very first thing in the function */
PDWORD pFrame;
Frame++; /* make sure that Frame doesn't get optimized out */
pFrame = (PDWORD)(&Frame);
/*... do stuff with pFrame here*/
}
The reason this works is that in C usually the first thing a function does is save off the location of the base pointer (ebp) before allocating local variables. By creating a local variable (Frame) and then getting the address of if, we're really getting the address of the start of this function's stack frame.
Note: Some optimizations could cause the "Frame" variable to be removed. Probably not, but be careful.
Second Note: Your original code and also this code manipulates the data pointed to by "pFrame" when "pFrame" itself is on the stack. It is possible to overwrite pFrame here by accident and then you would have a bad pointer, and could get some weird behavior. Be especially mindful of this when moving from x86 to x64, because pFrame is now 8 bytes instead of 4, so if your old "do stuff with pFrame" code was accounting for the size of Frame and pFrame before messing with memory, you'll need to account for the new, larger size.
You can use the _AddressOfReturnAddress() intrinsic to determine a location in the current frame pointer, assuming it hasn't been completely optimized away. I'm assuming that the compiler will prevent that function from optimizing away the frame pointer if you explicitly refer to it. Or, if you only use a single thread, you can use the IMAGE_NT_HEADER.OptionalHeader.SizeOfStackReserve and IMAGE_NT_HEADER.OptionalHeader.SizeOfStackCommit to determine the main thread's stack size. See this for how to access the IMAGE_NT_HEADER for the current image.
I would also recommend against using IsBadWritePtr to determine the end of the stack. At the very least you will probably cause the stack to grow until you hit the reserve, as you'll trip a guard page. If you really want to find the current size of the stack, use VirtualQuery with the address you are checking.
And if the original use is to walk the stack, you can use StackWalk64 for that.
There is no guarantee that RBP (the x64's equivalent of EBP) is actually a pointer to the current frame in the callstack. I guess Microsoft decided that despite several new general purpose registers, that they needed another one freed up, so RBP is only used as framepointer in functions that call alloca(), and in certain other cases. So even if inline assembly were supported, it would not be the way to go.
If you just want to backtrace, you need to use StackWalk64 in dbghelp.dll. It's in the dbghelp.dll that's shipped with XP, and pre-XP there was no 64-bit support, so you shouldn't need to ship the dll with your application.
For your 32-bit version, just use your current method. Your own methods will likely be smaller than the import library for dbghelp, much less the actual dll in memory, so it is a definite optimization (personal experience: I've implemented a Glibc-style backtrace and backtrace_symbols for x86 in less than one-tenth the size of the dbghelp import library).
Also, if you're using this for in-process debugging or post-release crash report generation, I would highly recommend just working with the CONTEXT structure supplied to the exception handler.
Maybe some day I'll decide to target the x64 seriously, and figure out a cheap way around using StackWalk64 that I can share, but since I'm still targeting x86 for all my projects I haven't bothered.
Microsoft provides a library (DbgHelp) which takes care of the stack walking, and you should use instead of relying on assembly tricks. For example, if the PDB files are present, it can walk optimized stack frames too (those that don't use EBP).
CodeProject has an article which explains how to use it:
http://www.codeproject.com/KB/threads/StackWalker.aspx
If you need the precise "base pointer" then inline assembly is the only way to go.
It is, surprisingly, possible to write code that munges the stack with relatively little platform-specific code, but it's hard to avoid assembly altogether (depending on what you're doing).
If all you're trying to do is avoid overflowing the stack, you can just take the address of any local variable.
.code
PUBLIC getStackFrameADDR _getStackFrameADDR
getStackFrameADDR:
mov RAX, RBP
ret 0
END
Something like that could work for you.
Compile it with ml64 or jwasm and call it using this in your code
extern "C" void getstackFrameADDR(void);

Resources