malloc memory corruption, fopen - c

I've tried using GDB and Valgrind, but I can't seem to pinpoint the problem.
Interestingly, the program crashes during normal execution and GDB, but not Valgrid.
To help you follow along with the code, heres the basic point of the program:
Communicate with a server via sockets and UDP to transfer a file, and handle some basic packet loss.
I won't share the server's code, because I know the issue isn't there.
The point that might confuse some, is that I'm implementing packet loss myself, with a number generator. Right now it doesn't do anything really, besides make the program use another recvfrom.
To guide you throught the programs output, the client tells the server what file it wants, the server tells the client how big the file is it's going to send, and then sends it in chunks (of 10 characters at a time).
The output shows what chunk is sent, how many characters were received, and what the concatenated string is.
The file transfer succeeds from what i can tell, its just the fopen call that I use to write the received file that is giving me trouble. Not sure if it's to do with my malloc call or not.
Here is the source code:
pastebin.com/Z79hvw6L
Here are the outputs from CLI execution, and Valgrind (GDB doesn't seem to give any more info):
Notice the CLI gives a malloc memory corruption error, and Valgrind doesn't.
CLI: http://pastebin.com/qdTKMCD2
VALGRIND: http://pastebin.com/8inRygnU
Thanks for any help!
Added the GDB Backtrace results
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x6b961)[0x19a961]
/lib/i386-linux-gnu/libc.so.6(+0x6e15d)[0x19d15d]
/lib/i386-linux-gnu/libc.so.6(__libc_malloc+0x63)[0x19ef53]
/lib/i386-linux-gnu/libc.so.6(+0x5c2b8)[0x18b2b8]
/lib/i386-linux-gnu/libc.so.6(fopen+0x2c)[0x18b38c]
/home/---/client[0x8048dc2]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x145e37]
/home/---/client[0x8048871]
Maybe this could give someone an insight as to what part of the program the error is in?

char chunk[10];
chunk[10] = '\0';
is wrong, chunk[10] is one past the array.
And in general, be careful with doing this
char filename[25];
scanf("%s",filename);
If you enter a long filename, you'll trash memory. using fgets() would be better. You also would at least want to check if scanf succeeds, else the following strlen() on filename isn't valid.
line 93, buf[strlen(buf)-1]='\0'; is dangerous, you can't use strlen if the buffer isn't already nul terminated, and you trash memory if buf is an empty string, as you index buf[-1].
Edit.
Your other problem is strcat(fullstring,chunk); , you've no control in your loop that stops appending to this string if you happen to receive more data than it can hold. The size is also likely off by one, as you need room for the last nul terminator. Make it at least char * fullstring = malloc(sizeof(char)*filesize + 1 ); But your loop really needs to check that's it is not writing past the end of that buffer.
As for adding a nul terminator to buf , the recv call returns how many bytes you've read, so if you've checked recv for errors, do buf[numbytes] = 0 , but this will be off by one as well, as you've allocated 10 bytes for buf and you try to read 10 bytes into it as well - but in C, a string needs room for a nul terminator too. Make buf 11 bytes big. Or recv() only 9 bytes.
In fact, you're off by one many places, so start counting how many bytes you need, and were you put stuff into them. Remember that in C, arrays starts with index zero, and an array of 10 can only be indexed by index 0 to 9.

This (line 93) is suspect:
buf[strlen(buf)-1]='\0';
UPDATE This (line 99,100) is also wrong:
char chunk[10];
chunk[10] = '\0';
UPDATE2: The buffer is too small
char * fullstring = malloc(sizeof(char)*filesize); // line 103
...
strcat(fullstring,chunk); // line 124
UPDATE3:
UDP is unreliable. Transmission of a packet may fail (packets may be dropped anywhere between sender and receiver) , and the packets may be received in a different order than in which you sent them.

Well, it shouldn't be a problem on modern OS:es but you don't check the returned value from malloc() for NULL. On what line does it crash and with what signal?

Related

How many bytes should I read/write to a socket?

I'm having some doubts about the number of bytes I should write/read through a socket in C on Unix. I'm used to sending 1024 bytes, but this is really too much sometimes when I send short strings.
I read a string from a file, and I don't know how many bytes this string is, it can vary every time, it can be 10, 20 or 1000. I only know for sure that it's < 1024. So, when I write the code, I don't know the size of bytes to read on the client side, (on the server I can use strlen()). So, is the only solution to always read a maximum number of bytes (1024 in this case), regardless of the length of the string I read from the file?
For instance, with this code:
read(socket,stringBuff,SIZE);
wouldn't it be better if SIZE is 10 instead of 1024 if I want to read a 10 byte string?
In the code in your question, if there are only 10 bytes to be read, then it makes no difference whether SIZE is 10 bytes, 1,024 bytes, or 1,000,024 bytes - it'll still just read 10 bytes. The only difference is how much memory you set aside for it, and if it's possible for you to receive a string up to 1,024 bytes, then you're going to have to set aside that much memory anyway.
However, regardless of how many bytes you are trying to read in, you always have to be prepared for the possibility that read() will actually read a different number of them. Particularly on a network, when you can get delays in transmission, even if your server is sending a 1,024 byte string, less than that number of bytes may have arrived by the time your client calls read(), in which case you'll read less than 1,024.
So, you always have to be prepared for the need to get your input in more than one read() call. This means you need to be able to tell when you're done reading input - you can't rely alone on the fact that read() has returned to tell you that you're done. If your server might send more than one message before you've read the first one, then you obviously can't hope to rely on this.
You have three main options:
Always send messages which are the same size, perhaps padding smaller strings with zeros if necessary. This is usually suboptimal for a TCP stream. Just read until you've received exactly this number of bytes.
Have some kind of sentinel mechanism for telling you when a message is over. This might be a newline character, a CRLF, a blank line, or a single dot on a line followed by a blank line, or whatever works for your protocol. Keep reading until you have received this sentinel. To avoid making inefficient system calls of one character at a time, you need to implement some kind of buffering mechanism to make this work well. If you can be sure that your server is sending you lines terminated with a single '\n' character, then using fdopen() and the standard C I/O library may be an option.
Have your server tell you how big the message is (either in an initial fixed length field, or using the same kind of sentinel mechanism from point 2), and then keep reading until you've got that number of bytes.
The read() system call blocks until it can read one or more bytes, or until an error occurs.
It DOESN'T guarantee that it will read the number of bytes you request! With TCP sockets, it's very common that read() returns less than you request, because it can't return bytes that are still propagating through the network.
So, you'll have to check the return value of read() and call it again to get more data if you didn't get everything you wanted, and again, and again, until you have everything.

Reading the last chunk of a file open for un-cached direct I/O doesn't produce EOF, normal behavior?

I'm opening a file using CreateFile() with the flags FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH for several reasons, and I've noticed a strange behavior:
Since for using those flags we have to allocate memory aligned to the sector size, let's say the sector size is 512.
Now, if I allocate 512 bytes with _aligned_malloc() and I read from the file, everything works fine if the file size is exactly a multiple of the sector size, let's say 512*4, or 2048. I read pieces of 512 bytes and the last piece makes ReadFile() to return the EOF code, that is, to return FALSE and GetLastError() set as ERROR_HANDLE_EOF.
The problem arise when the file size it not aligned to the sector size, that is, the file's size is let's say 2048+13, or 2061 bytes.
I can successfully read the first 4 512-sized chunks from the file, and a 5th call to ReadFile() lets me to read the latest 13 surplus bytes from the file, but this is the strange thing: in such a case ReadFile() doesn't return the EOF code! Even if I told to ReadFile() to read 512 bytes, and it read only 13 bytes (so it surpassed the end of file), it doesn't tell me that, and returns just 13 bytes read, without no other further information.
So, when I read the last 13 bytes and my loop is set to read until EOF, it will call ReadFile() again for a 6th time, causing an error: ERROR_INVALID_PARAMETER and I guess this is correct, because I'm trying to read after I had surpassed the end of file!
My question is: is this a normal behavior or am I doing something wrong? When using non-buffered I/O, I should expect to not having EOF code when I read the last non-sector-aligned chunk of file? Or there is another way to do that?
How I can understand that I've just passed the EOF?
I guess that I could solve this problem by modifying the loop: instead of reading until EOF, I could read until EOF OR until the actually returned bytes are less than the requested bytes for the reading. Is this a correct assumption?
NOTE: this does not happen when using files with normal flags, it only happens when I use FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH.
NOTE 2: I'm using I/O Completion Ports for reading files, but I guess this happens also without using them, by just using blocking I/O.
EOF is surprisingly hard. Even C's feof function is often misunderstood.
Basically, you get ERROR_HANDLE_EOF in the first case to distinguish the "512 bytes read, more to read" and "512 bytes read, nothing left" cases.
In the seconds case, this is not needed. "512 bytes requested, 13 bytes read, no error" already means that you're at EOF. Any other reason for a partial read would have been an error.

detecting end of http header with \r\n\r\n

Using recv I want to get the http header so I can parse for a content length. However I'm having trouble detecting the line break. Or actually do I even have to detect line break or will the first time I read into the buffer always be the complete header (assuming I have a long enough buffer).
This is written in C.
edit: looking at some of the related questions one of the things I am worried about is
"...the "\r\n" of the header break might be pulled into your buffer by two different calls to recv() which would prevent your code from recognizing the header break."
You should call recv() repeatedly and each time it gives you x bytes you increase the buffer-pointer you give to it by x bytes (and decrease the cb it is allowed to write also by x bytes). You do this until your buffer either contains a \r\n\r\n or is completely full, in which case you just close the socket and ignore the malicious client from then on. Buffer-size should be about 3000 bytes.
But: this ignores the general problem that your server seems to be a polling-server. If you have some experience you should try to make an epoll-server instead.
In addition to the problem of identifying "\r\n\r\n" across packet boundaries, you have the problem of identifying "Content-Length: xxxx\r\n" across packet boundaries. I suggest recieving and parsing one byte at a time. When you get a recv() of '\r' followed by a recv() of '\n', followed by a recv() of '\r' followed by a recv() of '\n', you can be sure the header has ended. Once you've grasped this, adapt your solution to recieve and parse n bytes at a time where n is a preprocessor definition defined to 1 initially, and change n.
In the end I did something like this:
while ( recv... > 0 ) {
if rnrn is inside the buffer using strstr
look for content length, output error if content length doesn't exist
else
keep on reading into the buffer
}
and then once the header is found I keep on reading for the message body.
anyway thanks guys, ended up doing a really inefficient way to get my answer but what must be done is done.

How send function works

I am using ssize_t send(int sockfd, const void *buf, size_t len, int flags); from socket.h file. I have some doubts about it.
Doubt 1:If suppose the string I am passing to send is of length 10 and the length i specified in the third parameter is 15. Then what will send only send 10 chars or it will send 15 chars (which it gets by reading unallocated memory for last 5 chars).
What will happen in the reverse case means if length of second parameter is more than third parameter.
Doubt 2:I am assuming the length of second parameter is equal to third parameter. Now if the second parameter is say - "abc\0def\0qw". Its length is 11. Will send send the whole string or \0 have any of its effect. I think it will send the whole string. How really send works.
If someone know any good source about send, recv which discuss these function in depth pls share.
The function send will try to send exactly as much as you tell it. If the buffer is big enough, nothing special happens. If it's not, what does happen is not defined: it could send garbage or it could crash or it could do anything else
The function send does not care about buffer contents - it only cares about the specified number of bytes to write
The send function knows nothing about "strings". If you give it a pointer, and tell it to send the next 15 bytes after that pointer, then that is EXACTLY what it will try to do. (You may well encounter a Seg-Fault or similar if you give it an inappropriate len value).
There is no justification for believing that it would stop just because it finds a byte with value 0x00. After all, many network protocols are FILLED with 0x00 all over the place. You can't have send stopping every time it happens to find that value.
send doesn't know anything about NUL-terminated strings (there's a clue in the parameter types - it takes a void* rather than a char*).
It simply sends the number of bytes you give it, from the address you give it. If that means reading unallocated memory, then that's what it'll do, possibly crashing in the process.

Interacting with shell program in C to feed a custom Buffer into its gets() function (and more)

If you dont want the long sschpeal head the the last paragraph-->
I found a buffer overflow vulnerability in a program that is using gets() to fill a function's local 1024-char* buffer. It's on Sparc Solaris 5.8 (sun4u) 32-bit.
The first obstacle to overcome was the tch was not letting me manually input > 257 chars
(256 if I want to be able to hit enter ;)
To bypass this, I have been executing /bin/sh and stty raw and I can effectively overflow the buffer now with > 1095 chars.
(Note : I have to use Ctrl-J to do line-feeds/enter , though I haven't researched stty raw to examine why this change occurs.
My issue is this: it is now time to not only overflow the buffer but also write new return address / preserve %fp in hex codes. But since I know of no way to manually enter hex codes from inside a terminal program, I figured I could find a way to use C and have it execute/interact with the vulnerable program and eventually send it my custom buffer.
HOWEVER, if I had a way to manually enter / copy paste hex bytes, I could just do something EASY like this!!!
perl -e 'print "n" . "A"x1094 . "\xff\xbe\xf5\x58" . "\xff\xbe\xff\x68" . "\0"'
(if you're wondering why I am printing 'n' it is because the vulnerable program checks for a yes/no # index 0 of the string)
because I know no way to manually paste such hex-information, I have been trying in C.
In C, I craft the special buffer and have been learning to popen() the vulnerable program ("w") and fputs my buffer, but it has been working iffy at best. (popen and IPC is all new to me)
(I also tried piping/dup2ing and i got NO results, no evidence of effective string output/input) not sure what is going wrong, and I experimented much with the code and later abandoned it.
The best to depict the output from my 'popen' program is that there is a segfault in the vulnerable program only by delimiting the buffer at indexes [1096->1099], this is effectively the location of the function's %fp, so it seemed normal # first. However, delimiting the string at indexes HIGHER than this leaves the programing working fine (WTF)!!! And that sort of behavior makes me think WTF!!? That is not the same behavior as manually pasting, as going more chars most definitely changes seg fault -> bus error, because I will be next overwriting the return address followed by whatever possibly important info in that stack frame and beyond!!
Is the whole string not actually getting sent in one bang?!?!? I heard something about buffer fflush() issues from the popen() manpage, but I dont understand that talk!!
It's my first time using popen(), and there is more behavior that I have deemed strange-> if i stop fputs()ing data , the vulnerable program goes into an infinite loop, repeatedly printing the last output string that it NORMALLY would
only print once,
but in this case, whenever i stop fputs'ing, the thing starts infinitely printing out. Now, I expected that if I am not outputting, wouldn't the program just sit and wait for more input like a good duck. ??? apparently not. apparently it has to keep on pissing and moaning that I need to enter the next string!! is this normal behavior with popen?! Perhaps it is due to my popen' program exiting and closing with pclose(), before actually finishing (but i was expecting a buffer overflow and i dont know why I am not getting it like I could when pasting manually)
Note: I am using "\r\n" to signal the vulnerable program to do a 'return' , I am not sure the equivalent of CTRL-J / Enter key (which enter key does not work in raw tty). I am also not sure if raw tty is even necessary when piping a buffer.
then I thought I try to be clever and cat the strings to a file and then do a pipe via command line. I have no idea if u can pipe like this to a program expecting inputs
in this form, I could not even get a single overflow!! i.e.
printf "\r\n" > derp && perl -e 'print "n" . "A"x1025' >> derp && printf "\r\n" >> derp
cat derp | ./vuln
Now, rewind <-> back in tsh, i said I have a 257 char limit, and i needed to do ONE LESS THAN THAT if i wanted to be able to hit enter and have the program continue operation. So, perhaps \r\n is not right here, cause that's 2 chars. either that or you just Cannot cat into a program like this. But I AM using \r\n in my C programs to tell the vulnerable program that I have hit enter, and they are at least mildly more functional (not really), though still not overflowing the buffer in the same fashion as manually pasting my trash buffer.
ARGh!!!
Also, using just one or the other: '\r' or '\n' was most definitely not working! is there another control char out there I am missing out on? And is it possible that this could be one of my issues with my programs???
but basically my whole problem is I cant' seem to understand how to create a program to run and interface with a command-line executable and say hey!!! Take this whole buffer into your gets(), i know you'd really love it!! just as I would if I was running the program from terminal myself.
And i know of no way to manually paste / write hex codes into the terminal, is the whole reason why i am trying to write an interacting program to
craft a string with hext bytes in C and send to that program's gets()!!!!
If you jumped to this paragraph, i want you also to know that I am using specifically /bin/bash and stty raw so that I could manually input more than 257 chars (not sure if I NEED to continue doing this if I can successfully create an interacting program to send the vulnerable program the buffer. maybe sending a buffer in that way bypasses tch' terminal 257 char limit)
Can anyone help me!?!?!?!?!
The popen call is probably the call you want. Make sure to call pclose after the test is finished so that the child process is properly reaped.
Edit Linux man page mentioned adding "b" to the mode was possible for binary mode, but POSIX says anything other than "r" or "w" is undefined. Thanks to Dan Moulding for pointing this out.
FILE *f = popen("./vuln", "w");
fwrite(buf, size, count, f);
pclose(f);
If the shell is reading with gets(), it is reading its standard input.
In your exploit code, therefore, you need to generate an appropriate overlong string. Unless you're playing at being expect, you simply write the overlong buffer to a pipe connected from your exploit program to the victim's standard input. You just need to be sure that your overlong string doesn't contain any newlines (CR or LF). If you pipe, you avoid the vagaries of terminal settings and control-J for control-M etc; the pipe is a transparent 8-bit transport mechanism.
So, your program should:
Create a pipe (pipe()).
Fork.
Child:
connect the read end of the pipe to standard input (dup2()).
close the read and write ends of the pipe.
exec the victim program.
report an error and exit if it fails to exec the victim.
Parent:
close the read end of the pipe.
generates the string to overflow the victim's input buffer.
write the string to the victim down the pipe.
Sit back and watch the fireworks!
You might be able to simplify this with popen() and the "w" option (since the parent process will want to write to the child).
You might need to consider what to do about signal handling. There again, it is simpler not to do so, though if you write to a pipe when the receiver (victim) has exited, you will get a SIGPIPE signal which will terminate the parent.
Nothing is yielding results.
Let me make highlights of what I suspect are issues.
the string that I pipe includes a \n at the beginning to acknowledge the "press enter to continue" of the vulnerable program.
The buffer I proceed to overflow is declared char c[1024]; now I fill this up with over 1100 bytes. I don't get it; sometimes it works, sometimes it doesn't. Wavering factor is if I am in gdb (being in gdb yields better results). but sometimes it doesn't overflow there either. DUE TO THIS, I really believe this to be some sort of issue with the shell / terminal settings on how my buffer is getting transferred. But I have no idea how to fix this :(
I really appreciate the help everybody. But I am not receiving consistent results. I have done a number of things and have learned a lot of rough material, but I think it might be time to abandon this effort. Or, at least wait longer until someone comes through with answers.
p.s.
installed Expect, :) but I could not receive an overflow from within it...
I seemed to necessitate Expect anyways, because after the pipe is done doing its work I need to regain control of the streams. Expect made this very simple, aside from that fact that I can't get the program to overflow.
I swear this has to do something with the terminal shell settings but I don't have a clue.
Another update
It's teh strangest.
I have actually effectively overwritten the return address with the address of a shellcode environment variable.
That was last night, Oddly enough, the program crashed after going to the end of the environment variable, and never gave me a shell. The shellcode is handwritten, and works (in an empty program that alters main's return address to the addr of the shellcode and returns, simply for test purposes to ensure working shellcode). In this test program Main returns into my SPARC shellcode and produces a shell.
...so.... idk why it didn't work in the new context. but thats the least of my problems. because the overflow it's strange.....
I couldn't seem to reproduce the overflow after some time, as I had stated in my prior post. So, i figured hey why not, let's send a bigger,more dangerous 4000 char buffer filled with trash "A"s like #JonathanLeffler recommended, to ensure segfaulting. And ok let's just say STRANGE results.
If I send less than 3960 chars, there will NOT be an overflow (WTF?!?!), although earlier i could get overflow at times when doing only about 1100 chars, which is significantly less, and that smaller buffer would overwrite the exact spot of return address (when it worked .*cough)
NOW THE strangest part!!!
this 'picky' buffer seems to segfault only for specific lengths. But i tried using gdb after sending the big 4000 char buffer, and noticed something strange. Ok yes it segfaulted, but there were 'preserved areas,' including the return address i previously was able to overflow, is somehow unharmed, and u can see from the image (DONT CLICK IT YET) Read the next paragraph to understand everything so u can properly view it. I am sure it looks a mess without proper understanding. parts of my crafted buffer are not affecting certain areas of memory that I have affected in the past with a smaller buffer! How or why this is happening. I do not know yet. I have no idea how regular this behavior is. but i will try to find out .
That image takes place about 1000 bytes in from the buffer's start address. you can see the 'preserved memory segments', embedded between many 0x41's from my buffer ("A" in hex) . In actuality, address 0xffbef4bc holds the return address of 0x0001136c, which needs to be overwritten, it is the return address of the function that called this one, 'this one' = the function that holds the vulnerable buffer. we cannot write (*the function that vulnerable buffer belongs to)*'s return address due to the nature of stack windows in SPARC -- that return address is actually BELOW the address of the buffer, unreachable, so therefore we must overwrite the return address of the function above us. aka our caller ;)
Anyways the point is that I was also able to previously overflow that return address sometimes with a smaller buffer. So WTF is up with these gaps!!?!??! Shouldnt a larger buffer be able to overflow these, esp. if the smaller buffer could (though not consistently).. Whatever, here's the image.
[image] http://s16.postimage.org/4l5u9g3c3/Screen_shot_2012_06_26_at_11_29_38_PM.png

Resources