about buffer overflow - c

Question

I am new to the ethical hacking world, and one of the most important things is the stack overflow, anyway I coded a vulnerable C program which has a char name [400] statement, and when I try to run the program with 401A's it doesn't overflow, but the book which I am following says it must overflow and the logic sense says so, so what's wrong???

Answer 1

If you've defined a buffer:
char buf[400];
And wrote 401 bytes into it, the buffer has overflown. The rest, however, depends on the structure of your code:
How is the buffer allocated (statically, dynamically, on the stack)
What comes before and after it in memory
Your architecture's calling convention and ABI (in case of a stack buffer)
some more...
Things are more complex than they seem. To quote Wikipedia:
In computer security and programming,
a buffer overflow, or buffer overrun,
is an anomaly where a process stores
data in a buffer outside the memory
the programmer set aside for it. The
extra data overwrites adjacent memory,
which may contain other data,
including program variables and
program flow control data. This may
result in erratic program behavior,
including memory access errors,
incorrect results, program termination
(a crash), or a breach of system
security.
Note the multiple instances of the word may in this quote. All of this may happen, and it may not. Again, this depends on other factors.

Answer 2

C doesn't check about buffer overflow (overflowing the buffer is an undefined behavior). Usually the system will just allow you (and the hacker) to write beyond the buffer, and this is the reason why buffer overflow is vulnerable.
For example if the code is
char name[400];
char secret_password[400];
...
The memory may be layout as
[John ][12345 ]
name secret_password
Now if you write 401 A followed by a NULL to name, the extra A\0 will be written to secret_password, which basically changed the password from your luggage combination to just "A":
[AAAAAAAAA...AAAAA][A␀345 ]
name secret_password

Answer 3

Stackoverflow and bufferoverflow are different concepts.
Stackoverflow:
The size of a programs stack is static, it never changes at runtime. Since it is not possible to know how much memory your stack will need at runtime a reasonable big memory block is reserved. However some programs exeed this by calling a rekursive function.
A function call reserves as much space as it needs to store lokal variables on the stack and releases the memory once it exits. A recursive function will reserve new memory each time it is entered and release it once it exits. If the recursion never ends due to a programming error, more and more memory on the stack is reserved until the stack is full.
Trying to reserve memory on a full stack will cause an error, the stackoverflow.
Example code:
volatile bool args = false;
int myoverflow(int i){
int a[500];
if(args)
return a[i%500];
else
return myoverflow(i+1);
}
This should overflow the stack. It will reserve 500 * sizeof(int) every time it enters the function.
Bufferoverflow:
You have two variables, an array a and an array b. a can hold 4 elements and b can hold 2.
Now you write 5 elements into a, the 5th element lands in b.
Example:
void main(int ,char**)
{
int a[4];
int b[2];
a[5] = 22;
std::cout<<b[0];
}
This should print 22. it will write outside of a, into the memory used by b.
Note: None of my example functions are guaranteed to work, the compiler is free to optimize function calls and to arrange the memory used on the stack as it wants. It may even print a compile error on accessing memory out of bounds for array a.

Answer 4

Here's a good example in C showing how a buffer overflow can be used to execute arbitrary code. Its objective is to find an input string that will overwrite a return address causing a target function to be executed.
For a very good explanation of buffer overflows I would recommend chapter 5 of Writing Secure Code 2nd Edition.
Other good info on buffer overflows:
Secure programmer: Countering buffer overflows by David Wheeler
Smashing the Stack for Fun and Profit The classic article from Phrack Magazine