I am developing a client that needs to parse Chunked-type HTTP transfers. I've beat my head against the wall trying to figure out the error with the following, and would appreciate it if someone might be able to catch my error a bit quicker. To sum up the issue: it seems as though, the client does not receive ALL of the chunk, thereby screwing up the rest of the process. Thanks in advance!
while(cflag){
pfile_chunk = malloc(CHUNK_SIZE+1);
memset(pfile_chunk, 0, CHUNK_SIZE);
cPtr = pfile_chunk;
cPtr2 = NULL;
k=0;
while(*(cPtr-1) != '\n'){
k++;
recv(sock, cPtr, 1, 0);
cPtr = pfile_chunk+k;
}
cPtr2 = strchr(pfile_chunk, '\r');
*cPtr2 = '\0';
sscanf(pfile_chunk, "%x", &l);
if(l == 0)
break;
printf("\nServer wants to deliver %ld bytes.\n", l);
pfile_chunk = realloc(pfile_chunk, l+1);
memset(pfile_chunk, 0, l);
recv(sock, pfile_chunk, l, 0);
fputs(pfile_chunk, f);
printf("GOT THIS, SIZE %ld:\n%s\n", strlen(pfile_chunk), pfile_chunk);
//get next \r\n bytes.
recv(sock, NULL, 2, 0);
}
At the very least, you should check the return value of recv to see if you are getting the number of bytes you are expecting to get.
A short read is definitely possible on the network, since the system call will return whatever is available in the socket receive buffer at the time you make the call.
Implement a loop until you have read in your entire chunk, or pass the MSG_WAITALL flag to recv in the last parameter. However, you still need to check for an error from recv.
ssize_t r = recv(sock, pfile_chunk, l, MSG_WAITALL);
if (r < l) {
/* check for errors ... */
} else {
/* got the data */
}
It looks as though your very first dereference for the check in your while loop will access before the beginning of your array, which is likely not to be desired behavior. Hopefully, that memory location usually won't contain \n. That could mess up your read. I expect it probably contains some information to do with your malloc, which is unlikely to be \n, so you might never see a problem from that.
Also, hopefully you can trust the other end of the socket not to send more than CHUNK_SIZE+1 before they give you a \n. Otherwise, it could seg-fault out. Normally, though, I would expect a sender to just send 10 or fewer ASCII numeric characters and a CRLF for a chunk header anyways, but they could theoretically send a bunch of long chunk extension header fields with it.
Apart from that, there's just the more important issue already found by user315052 that you should either tell the recv method to wait for all the data you requested, or check how much data it actually read.
Related
I'm trying to write a code for a socket where I first read the stdin and send to a socket then read from said socket to send to stdout and so far I have something that looks more or less like this (before you attack me I also don't know why I have to do this) :
void read_write(int socket_descriptor_file){
int n = 1;
char buffer_in[1024];
while(n>0){
n = fread(&buffer_in,sizeof(buffer_in),1, stdin);
if(n==0) break;
ssize_t sent_status = send(socket_description_file, buffer_in, sizeof(buffer_in), 0);
if(sent_status == -1){
printf("nothing sent");
}
ssize_t receive_status = recv(socket_descriptor_file,buffer_in,sizeof(buffer_in), 0);
if(receive_status == -1){
printf("nothing received ");
}
fwrite(&buffer_in,sizeof (char), sizeof(buffer_in), stdout);
fflush(stdout);
}
}
I'm unsure as to if the said buffer when applying the send function will automatically clear so that I can use the buffer to store the message from the recv function.
The objective of this code is to simulate a chat between a host 1 and a host 2. So it is necessary to send treat the message immediately.
I'm also apparently supposed to use the poll function but I don't really know how to use it.
I'm not really well informed in this subject so please let me know if there are any further problems with my code. I would be happy to hear them :)
The buffer is only memory. And after a successfull call to send the content is not needed any more. So it is ok to reuse the buffer for recv. Probably you want to store the reveived number of bytes (reveive_status) and not the size of the buffer to your file.
In C I had working code but have no idea why it worked, so I started rewriting it so I could actually understand what is going on.
So far so good! I rewrote and am 90% sure I understand everything that is going on now; the issue however, is that I have no idea how to store the data chunk received by recv (databff) into my pre-allocated buffer (htmlbff).
Consider the following code (note that I stripped this down quite a bit, so it only includes the basics, e.g. no memory reallocation or leak protection, etc...):
#define BUFFERSIZE 4096
#define MAXDATASIZE 256
char *htmlbff, databff[MAXDATASIZE];
int c, i = BUFFERSIZE, q = 0;
if(!(htmlbff = malloc(i)))
{
printf("\nError! Memory allocation failed!");
return 0x00;
}
while((c = recv(sock, databff, MAXDATASIZE, 0)) > 0)
{
/*memory checks stripped out since they are irrelevent for this post*/
/*store data to the appropriate area in htmlbff*/
q += c;
}
So (if I am doing this right, and things are going as I think they are) c is the size of the current data chunk, and q is the total amount of data received so far (q is incremented by c each time the loop repeats). At the moment I am using q for memory handling (in case anybody was wondering) but I believe that it will also have purpose in the solution to this problem.
At any rate the question I am asking is in regards to the second comment. How do I store the data from recv into htmlbff correctly?
Use memcpy() to copy (append) data to the htmlbff but you also need to ensure you do not exceed the size of htmlbff. Either stop receving data when BUFFERSIZE bytes have been received or use realloc() to extend htmlbff to contain more data.
For example:
char* htmlbff;
size_t htmlbff_size = BUFFERSIZE;
htmlbff = malloc(htmlbff_size);
if (htmlbff)
{
while((c = recv(sock, databff, MAXDATASIZE, 0)) > 0)
{
if (c + q > htmlbff_size)
{
htmlbff_size *= 2; /* Arbitrary doubling of size. */
char* tmp = realloc(htmlbff, htmlbff_size);
if (tmp)
{
htmlbff = tmp;
}
else
{
/* memory allocation failure. */
free(htmlbff);
htmlbff = 0;
break;
}
}
memcpy(htmlbff + q, databff, c);
q += c;
}
}
Use memcpy, and offset htmlbff by q:
memcpy(htmlbff + q, databff, c);
You can similarly recv directly into htmlbff:
c = recv(sock, htmlbff + q, MAXDATASIZE, 0));
But it's fine to keep a separate buffer, and depending upon your full code, it may make things clearer.
Be sure that you add checks against BUFFERSIZE so that you don't copy past the bounds of htmlbff. You mentioned that you've stripped out realloc handling, so maybe you're already handling this.
Your constant names are a bit confusing, when buffering data I would use BUFFERSIZE to indicate the size of each chunk, i.e. the size of databff.
What I would do is recv() data directly into htmlbff, unless you need to do more processing on it.
Make sure that you realloc() htmlbff when i - q is less than MAXDATASIZE so that there is always room for another recv().
Then you would call recv(sock, htmlbff + q, MAXDATASIZE, 0)
You need to keep reallocating/expanding the buffer to fit all the data (if the data read off the socket exceeds MAXDATASIZE) = That way as recv reads data into the databff, your htmlbff can grow in memory and then the new read can be appended to your overall htmlbff.
q and c are like cursors to keep track of where you are up to and how far you have to go.
memcpy(htmlbff+q, databff, c); //Do this in your whle loop to append the data
Hi i have written a server application which accepts a name from the client which usually is a file name.It opens the file ,reads the contents into a buffer and then transmits the buffer over the ethernet using send().But the problem arises in the client side where all the bytes are not received successfully.I receive only a part of what i send.
For your reference ,here's the code snippet for the server side:
Server:
fp = fopen(filename,"r+");
strcpy(str,"");
fseek(fp, 0L, SEEK_END);
size = ftell(fp);
fseek(fp, 0L, SEEK_SET);
fread(str, size, 1,fp);
fclose(fp);
printf("Size of the file is : %d\n",size);
sprintf(filename, "%d", size);
n = send(nsd, filename, strlen(filename), 0);
while(size > 0){
n = send(nsd, str, strlen(str), 0);
printf("%d bytes sent successfully\n",n);
if(n == 0) break;
sentbytes = sentbytes + n;
size = size - sentbytes;
}
Please help me with writing the client app.I am currently confused about how to go about writing it.Shall i place the recv() part in a while(1) loop so that the client keeps running until all the bytes have been received successfully?
EDITED
For starters, you could both read from the file and write to the socket in chunks at the same time.
Since, you are transferring data over TCP, remember that data is transferred reliably as a stream and not as messages. So, don't make assumptions about how the data is recv'd except for the order.
Here is how it could be written:
open socket
open file
size_of_file = read_file_size(file);
send(socket, &size_of_file, sizeof(int), ...)
while (all is written)
read fixed chunk from file
write as much was read to the socket
cleanup // close file, socket
As for the recv part, I think it is best you send the file size over as an integer and keep reading in a while loop until you have recv'd as many bytes as you are sending from the server.
It's like this:
recv(socket, &size_of_msg, sizeof(int), ...)
while(all is read)
read fixed chunk from the socket
cleanup
Well I see atleast some issue with the way you are sending message over socket.
First from the man page of fread:
The function fread() reads nmemb elements of data, each size bytes
long, from the stream pointed to by stream, storing them at the loca-
tion given by ptr.
and what you are trying is this:
fread(str, size, 1,fp);
I assume what you meant was
fread(str, 1,size,fp);
Though it shold not casue the issue.
But the problem lies here:
n = send(nsd, str, strlen(str), 0);
printf("%d bytes sent successfully\n",n);
if(n == 0) break;
sentbytes = sentbytes + n;
size = size - sentbytes;
Though you are decreasing 'size' by decreasing by number of bytes successfully send, where are you extending str to point to new buffer location where data will be send.This will only resend initial bytes of the buffer repeatedly.
str += n; //Assuming str is char*
will solve your issue.
Using strlen doesn't seem appropriate. You've read the file, you know how long it is, so why do strlen? Either you'll just get the same result (so it's redundant) or you'll get something else (so it's a bug).
"Shall i place the recv() part in a while(1) loop so that the client keeps running until all the bytes have been received successfully?"
Something like that. Never presume that a recv() call got everything that was sent -- tcp/ip breaks messages into packets at a lower level, and recv() will return after reading whatever amount of data has actually been received at whatever point. You don't have to worry about that directly, except in so far as you do need to use some kind of protocol to indicate how long a message is so the receiver knows how much to read, then eg.:
char buffer[4096];
int msgsz = 600, // see below
sofar = 0,
cur;
while (sofar < msgsz) {
cur = recv (
socket_fd,
&buffer[sofar],
msgsz - sofar,
0
);
if (cur == -1) {
// error
break;
} else if (cur == 0) {
// disconnected
break;
}
sofar += cur;
}
WRT msgsz, you would include this somewhere in a fixed length header, which is read first. A simple version of that might be just 4 bytes containing a uint32_t, ie, an int with the length. You could also use a null terminated string with a number in it, but that means reading until '\0' is found.
This loop is supposed to take data from a socket line by line and put it in a buffer. For some reason, when there is no new data to return, recv returns the last couple lines it got. I was able to stop the bug by commenting out the first recv, but then I cant tell how long the next line will be. I know it's not a
while(this->connected){
memset(buf, '\0', sizeof(buf));
recv(this->sock, buf, sizeof(buf), MSG_PEEK); //get length of next message
ptr = strstr(buf, "\r\n");
if (ptr == NULL) continue;
err = recv(this->sock, buf, (ptr-buf), NULL); //get next message
printf("--%db\n%s\n", err, buf);
tok[0] = strtok(buf, " ");
for(i=1;tok[i-1]!=NULL;i++) tok[i] = strtok(NULL, " ");
//do more stuff
}
The manual states:
MSG_PEEK
This flag causes the receive operation to return data from the
beginning of the receive queue without
removing that data from the queue.
Thus, a subsequent receive call will
return the same data.
So I think you're getting the correct behavior, but perhaps expecting something else.
Your problem is that when you use recv with MSG_PEEK, you are giving recv the whole size of your buffer, if there are two lines already there, like "HELLO\r\nHELLO\r\n" it will read them into your buff.
ptr would point to the first \r\n, then you call recv with (ptr - buff) which will make recv to read only the first HELLO, into buf, but since you already READ that info into buff, you will process the two lines, but leaving \r\nHELLO\r\n in your queue, because you did not fully read them.
Next time you would peek into it and have info hanging that you already processed, leading you to believe that you are getting repeated data.
(I hope I wrote this clear enough, it is a very confusing bug you got there :)
I needed to add 2 to the length of the second recv so I'd take the "\r\n". Otherwise, it sees the first "\r\n" and thinks the line of the end is buf[0].
Hi i find the solution :
void receiver(int accepted_client) {
// Ready to receive data from client.
while (true) {
char buffer[256];
recv(accepted_client, &buffer, 255, 0);
int sum = 0;
for (int i = 0; i < 256; i++) // Check that buffer value is zero or not.
sum |= buffer[i];
if (sum != 0) {// If buffer value is not zero then start to print the new received message.
string string_message(buffer);
cout << string_message << endl;
}
memset(&buffer, 0, 256); // Clear the buffer.
}
}
I'm writing myself a small server daemon in C, and the basic parts like processing connects, disconnects and receives are already in, but a problem in receiving still persists.
I use "recv" to read 256 bytes at once into a char array, and because it can contain multiple lines of data as one big chunk, I need to be able to split each line separatly to process it.
That alone wouldn't be the problem, but because of the possibility that a line could be cut off because it didn't fit into the buffer anymore, I also need to be able to see if a line has been cut off. Not that bad, too, just check the last char for \r or \n, but what if the line was cut off? My code does not allow for easy "just keep reading more data" because I'm using select() to handle multiple requests.
Basically, this is my situation:
//This is the chunk of code ran after select(), when a socket
//has readable data
char buf[256] = { 0 };
int nbytes;
if ((nbytes = recv(i, buf, sizeof(buf) - 1, 0)) <= 0)
{
if (nbytes == 0)
{
struct remote_address addr;
get_peername(i, &addr);
do_log("[Socket #%d] %s:%d disconnected", i, addr.ip, addr.port);
}
else
do_log("recv(): %s", strerror(errno));
close(i);
FD_CLR(i, &clients);
}
else
{
buf[sizeof(buf) - 1] = 0;
struct remote_address addr;
get_peername(i, &addr);
do_log("[Socket #%d] %s:%d (%d bytes): %s", i, addr.ip, addr.port, nbytes, buf);
// split "buf" here, and process each line
// but how to be able to get the rest of a possibly cut off line
// in case it did not fit into the 256 byte buffer?
}
I was thinking about having a higher scoped temporary buffer variable (possibly malloc()'d) to save the current buffer in, if it was too long to fit in at once, but I always feel bad about introducing unnecessarily high scoped variables if there's a better solution :/
I appreciate any pointers (except for the XKCD ones :))!
I guess you need to add another per-stream buffer that holds the incomplete line until the line feed that comes after is received.
I'd use some kind of dynamically expanding buffer like GString to accumulate data.
The other thing that might help would be putting the socket into nonblocking mode using fcntl(). Then you can recv() in a loop until you get a -1. Check errno, it will be either EAGAIN or EWOULDBLOCK (and those aren't required to have the same value: check for both).
Final remark: I found that using libev (google it; I can't post multiple links) was more fun than using select().