I'm writing a simple server for my networks class and I am having trouble getting data transferred to the client (provided by the prof) correctly.
Once I get through all the setup and the connection is established I start reading chunks of the file and writing them to the socket checking to see that the return from the read and write functions match. I'm also keeping a running total of the bytes that are read/written and comparing that to the total file size (via stat.st_size) and they all match up.
No matter how many times I request the same file the log on the server side always has the correct metrics. The client sporadically loses the end of the file. The difference between the actual and expected size is almost never the same from one invocation to the next and it appears to always be the end of the file that's missing, no pieces from the middle. The size of the arrived file is also a multiple of 512 (the chunk size).
So, it seems that some number of whole chunks are making it and then the rest are getting lost somehow.:w
#define CHUNK_SIZE 512
/* other definitions */
int main()
{
/* basic server setup: socket(), bind(), listen() ...
variable declarations and other setup */
while(1)
{
int cliSock = accept(srvSock, NULL, NULL);
if(cliSock < 0)
; /* handle error */
read(cliSock, filename, FILE_NAME_SIZE - 1);
int reqFile = open(filename, O_RDONLY);
if( reqFile == -1)
; /* handle error */
struct stat fileStat;
fstat(reqFile, &fileStat);
int fileSize = fileStat.st_size;
int bytesRead, totalBytesRead = 0;
char chunk[CHUNK_SIZE];
while((bytesRead = read(reqFile, chunk, CHUNK_SIZE)) > 0)
{
totalBytesRead += byteasRead;
if(write(cliSock, chunk, bytesRead) != bytesRead)
{
/* perror(...) */
/* print an error to the log file */
bytesRead = -1;
break;
}
}
if (bytesRead == -1)
{
/* perror(...) */
/* print an error to the log file */
close(cliSock);
continue;
}
/* more code to write transfer metrics etc to the log file */
}
}
All of the removed error handling code is some flavor of printing an error message to the log file and getting back to the top of the loop.
Edit flipped a < that should have been >
Presumably you are unceremoniously closing the socket with close() when you have written all the data you wanted to the socket (or perhaps just exiting the process, which does the same thing).
This is not right - if the other side has sent some data that you haven't read1, the connection will be reset. The reset can cause unread data to be lost.
Instead, you should use shutdown() to gracefully shutdown the writing side of the socket, then wait for the client to close. Something like:
ssize_t bytesRead;
char chunk[CHUNK_SIZE];
shutdown(cliSock, SHUT_WR);
while((bytesRead = read(cliSock, chunk, CHUNK_SIZE)) != 0)
{
if (bytesRead < 0)
{
if (errno != EINTR)
{
/* perror() */
/* print error in log file */
break;
}
}
else
{
/* maybe log data from client */
}
}
close(cliSock);
1. This can include EOF, if the other side has closed its writing channel.
Related
Extremely new to socket programming and C in general. I am trying to write a basic program to send and receive data between two machines. I understand that recv will not get all your data at once -- you essentially have to loop it until it has read the whole message.
In lieu of just setting a limit on both machines, I have created a simple Message struct on the client side:
struct Message {
size_t length;
char contents[1024 - sizeof(size_t)];
} message;
message.length = sizeof(struct Message);
message.contents = information_i_want_to_send;
When it arrives at the server, I have recv read into a buffer: received = recv(ioSock, &buffer, 1024, 0) (Which coincidentally is the same size as my Message struct -- but assuming it wasn't...).
I then extract Message.length from the buffer like this:
size_t messagelength;
messagelength = *((size_t *) &buffer);
Then I loop recv into the buffer while received < messagelength.
This works, but I can't help feeling it's really ugly and it feels hacky. (Especially if the first recv call reads less than sizeof(size_t) or the machines are different bit architectures, in which case the size_t cast won't work..). Is there a better way to do this?
You have a fixed-size message, so you can use something like this:
#include <errno.h>
#include <limits.h>
// Returns the number of bytes read.
// EOF was reached if the number of bytes read is less than requested.
// On error, returns -1 and sets errno.
ssize_t recv_fixed_amount(int sockfd, char *buf, size_t size) {
if (size > SSIZE_MAX) {
errno = EINVAL;
return -1;
}
ssize_t bytes_read = 0;
while (size > 0) {
ssize_t rv = recv(sockfd, buf, size, 0);
if (rv < 0)
return -1;
if (rv == 0)
return bytes_read;
size -= rv;
bytes_read += rv;
buf += rv;
}
return bytes_read;
}
It would be used something like this:
typedef struct {
uint32_t length;
char contents[1020];
} Message;
Message message;
ssize_t bytes_read = recv_fixed_amount(sockfd, &(message.length), sizeof(message.length));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(message.length)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
bytes_read = recv_fixed_amount(sockfd, &(message.content), sizeof(message.content));
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != msg_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
size_t is not going to be the same everywhere, so I switched to a uint32_t.
I read the fields independently because the padding within the struct can vary between implementations. They would need to be sent that way as well.
The receiver is populating message.length with the information from the stream, but doesn't actually use it.
A malicious or buggy sender could provide a value for message.length that's too large and crash the receiver (or worse) if it doesn't validate it. Same goes for contents. It might not be NUL-terminated if that's expected.
But what if the length wasn't fixed? Then the sender would need to somehow communicate how much the reader needs to read. A common approach is a length prefix.
typedef struct {
uint32_t length;
char contents[];
} Message;
uint32_t contents_size;
ssize_t bytes_read = recv_fixed_amount(sockfd, &contents_size, sizeof(contents_size));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(contents_size)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Message *message = malloc(sizeof(Message)+contents_size);
if (!message) {
perror("malloc");
exit(EXIT_FAILURE);
}
message->length = contents_size;
bytes_read = recv_fixed_amount(sockfd, &(message->contents), contents_size);
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != contents_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
message->length contains the size of message->contents instead of the size of the structure. This is far more useful.
Another approach is to use a sentinel value. This is a value that tells the reader the message is over. This is what the NUL that terminates C strings is. This is more complicated because you don't know how much to read in advance. Reading byte-by-byte is too expensive, so one normally uses a buffer.
while (1) {
extend_buffer_if_necessary();
recv_into_buffer();
while (buffer_contains_a_sentinel()) {
// This also shifts the remainder of the buffer's contents.
extract_contents_of_buffer_up_to_sentinel();
process_extracted_message();
}
}
The advantage of using a sentinel value is that one doesn't need to know the length of the message in advance (so the sender can start sending it before it's fully created.)
The disadvantage is the same as for C strings: The message can't contain the sentinel value unless some form of escaping mechanism is used. Between this and the complexity of the reader, you can see why a length prefix is usually preferred over a sentinel value. :)
Finally, there's a better solution than sentinel values for large messages that you want to start sending before they are fully created: A sequence of length-prefixed chunks. One keeps reading chunks until a chunk of size 0 is encountered, signaling the end.
HTTP supports both length-prefixed messages (in the form of Content-Length: <length> header) and this approach (in the form of the Transfer-Encoding: chunked header).
There are Two ways to do that...
1.)
Use Binary Synchronous protocol. (Use of STX - Start of Text and ETX - End of Text ) for identification of the Text start and end.
2.)
Attach the number of bytes of data being sent at the start of Data. The socket will read those number of bytes and will get the number of bytes to be received from the socket. Then read all data and get the amount of data required.
Hmm... Seems tough...?? Let me give you an example.
Actual Data need to be sent: ABCDEFGHIJ
New Data format : 0010ABCDEFGHIJ
Data required in server side: ABCDE
recv function will read the first 4 bytes to get the number of bytes of actual data(In loop untill it gets 4 bytes):
int received1= recv(ioSock, recvbuf, 4, 0);
As per the above case, 'recvbuf' will be 0010 converted to an integer will give value as '10' which can be stored in some integer variable. So we have :
int toReadVal = 10
Now all we need is to read these 10 digits in next recv call :
int received= recv(ioSock, recvbuf1, toReadVal, 0);
Finally, we get the value of recvbuf1 as ABCDEFGHIG. Now you can truncate the value as per your requirement.
I'm sending a .mp3 file (.jpg and other extensions as well, size between 1K to 50MB) using winsock over http. The client sends file and the server receives the file .
I have serveral questions:
Should send() buffer size and recv() buffer size be the same?
What size should I choose, fixed size or a size big enough to contain all data?
Here comes the code.
Client:
... //POST /index.html HTTP/1.1 and so on
sprintf_s(header, "%sContent-Length: %d\r\n", header, sizeof(szFileData ));
sprintf_s(header, "%s%s\r\n", header , szFileData );
... //Content-Disposition: form-data; name=\"submit\" and so on
while(send(sock, header, strlen(header), 0) == SOCKET_ERROR)
{
Sleep(1000);
}
Server：
recv(sock, recvbuf , 4096 , 0); //neither strlen(recvbuf) nor 4096 works
...//extract file name, content and so on
FILE *pFile;
pFile = fopen ( filename , "wb" );
if ( fwrite(filedata, sizeof(filedata), 1, pfile) != 1)
{
MessageBox(NULL, "Failed!", "MSG", MB_OK);
}
I get a result that either is empty or loses some content.
Two problems.
1) send() doesn't necessarily send everything. There are situations where it might only send some of what you give it. So be prepared for that.
2) recv() returns the number of bytes actually read or -1 on error. You should copy the those bytes from the recvbuf. Also check the return value. If it's -1 then there was an error and you need to use errno to find out what the error is.
To ensure everything is sent use sendall (from Beej's guide):
#include <sys/types.h>
#include <sys/socket.h>
int sendall(SOCKET s, char *buf, int *len)
{
int total = 0; // how many bytes we've sent
int bytesleft = *len; // how many we have left to send
int n;
while(total < *len) {
n = send(s, buf+total, bytesleft, 0);
if (n == -1) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually sent here
return n==-1?-1:0; // return -1 on failure, 0 on success
}
For receiving something like:
int nbytes = 0;
while ((nbytes = recv(sock, recvbuf , 4096 , 0)) > 0){
// From here, valid bytes are from recvbuf to recvbuf + nbytes.
// You could simply fwrite(fp, recvbuf, nbytes) or similar.
}
Make sure you don't get confused between application buffers and socket buffers. They are different things. The socket buffers are in kernel space. App buffers are what you're using here. There is potentially an optimal size but it various depending on what you're doing. Making them 4K is usually reasonable.
If anything isn't clear, consult Beej's guide. This is all very simple but in a real app I'd tend to use non-blocking sockets and/or epoll if on Linux.
I have a client/server program. The client sends a message to the server and the server processes this message and gets back to the client.
The first four bytes of the client message is the size of the whole message. I also know the maximum size a message can be. However, messages are of variable sizes.
I am using the read function in a while loop to read the client messages
while((n = read(socket, buffer, MAX_SIZE)) != 0){
process_message();
write(socket, ...);
}
I was wondering if there is any harm in reading more bytes than what the client is sending? I can first read the size of the message and then read the exact number of bytes but I was wondering if this is necessary.
As mentioned in some other comments, reading from a socket can return any number of bytes, up to the maximum requested.
A better loop, although still not without problems, would be something along these lines:
/* 4 byte message header that contains the length */
#define MSG_HEADER_SIZE 4
#define MAX_MESSAGE_LENGTH 128
struct message {
uint32_t length;
char body[MAX_MESSAGE_LENGTH];
};
#define BUFFER_SIZE 1024
char buffer[BUFFER_SIZE];
uint32_t buf_used = 0;
/* main loop */
while (1) {
n = recv(socket, buffer + buf_used, sizeof(buffer) - buf_used, 0);
if (n == -1) {
/* handle error */
exit(1);
}
if (n == 0) {
/* connection closed, do something. */
exit(1);
}
buf_used += n;
/* check for partial/completed message(s) */
while (buf_used >= MSG_HEADER_SIZE) {
struct message *cur_msg = (struct message *) buffer;
uint32_t total_msg_length;
total_msg_length = cur_msg->length + MSG_HEADER_SIZE;
/* is this message completed yet? */
if (buf_used >= total_msg_length) {
process_message(cur_msg);
/* remove message since it has been processed */
buf_used -= total_msg_length;
/* this could potentially be optimized */
memmove(buffer, buffer + total_msg_length, buf_used);
} else {
/* have incomplete message */
break;
}
}
}
For an introduction to socket programming, I would recommend checking out Beej's Guide to Network Programming.
I'm programming in C an IRC chat client. everything it's working well except I can't read the whole answer sent by the server. here's the code:
char buffer[2048];
write_on_screen(current_page(), "LOG COMMAND", command);
write(sockfd, command, strlen(command)); //write to socket
bzero(buffer, sizeof(buffer));
read(sockfd, buffer, sizeof(buffer));
write_on_screen(current_page(), "RESPONSE", buffer);
return buffer;
most of the time buffer will contain just a piece of the response (which is shorter than 2048 bytes) and other times it contains nothing. in both cases if I do another read() after the first one, it returns me the rest of the answer or another small piece (and then I've to do another read() again). if I put a sleep(1) between write() and read() I get the whole answer, but I'm sure this not a good pratice.
Is there some way I can avoid this?
thank you in advance
You're making the usual mistakes. It is impossible to write correct network code without storing the result of read() or recv() into a variable. You have to:
Check it for -1, and if so look at errno to see whether was fatal, which it almost always is except for EAGAIN/EWOULDBLOCK, and if fatal close the socket and abandon the process.
Check it for zero, which means the peer disconnected. Again you must close the socket and abandon the process.
Use it as the count of bytes actually received. These functions are not obliged nor guaranteed to fill the buffer. Their contract in blocking mode is that they block until an error, end of stream, or at least one byte is transferred. If you're expecting more than one byte, you normally have to loop until you get it.
According to RFC-1459, a single line of text in IRC can contain up to 512 characters and is terminated by a CRLF (\r\n) pair. However:
You're not guaranteed to receive exactly 512 bytes each time. For example, you might receive a comparatively short message from someone else one in the channel: Hi!
Related to the above: A group of 512 bytes might represent more than one message. For example, the buffer might contain a whole line, plus part of the next line: PRIVMSG <msgtarget> <message>\r\nPRIVMS
Given that you could have zero-or-more complete lines plus zero-or-one incomplete lines in your buffer[] at any time, you could try doing something along the lines of:
char buffer[2048];
while(keep_going)
{
char **lines;
int i, num_lines;
// Receive data from the internet.
receiveData(buffer);
// Create an array of all COMPLETE lines in the buffer (split on \r\n).
lines = getCompleteLines(buffer, &num_lines);
removeCompleteLinesFromBuffer(buffer);
// Handle each COMPLETE line in the array.
for (i = 0; i < num_lines; ++i) { handle_line(lines[i]); }
freeLines(lines);
}
This would allow you to handle zero or more complete lines in one go, with any incomplete line (i.e anything after the final \r\n pair) being kept around until the next call to receiveData().
You need to loop around read() until a CRLF had been detected.
A possible way to do this would be:
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt)
{
ssize_t bytes_read = 0;
ssize_t result = 0;
int read_cr = 0;
int read_crlf = 0;
while (bytes_read < s)
{
result = read(sd, p + bytes_read, 1);
if (-1 == result)
{
if ((EAGAIN == errno) || (EWOULDBLOCK == errno))
{
continue;
}
else if (EINTR == errno)
{
if (break_on_interupt)
{
break;
}
continue;
}
else
{
perror("read() failed");
break;
}
}
else if (0 == result)
{
break; /* peer disconnected */
}
if ('\r' == p[bytes_read])
{
read_cr = 1;
}
else if (('\n' == p[bytes_read]) && read_cr)
{
read_crlf = 1;
break; /* CRLF detected */
}
else
{
read_cr = 0;
}
++bytes_read;
}
if (!read_crlf)
{
result = -1; /* Buffer full without having read a CRLF. */
errno = ENOSPC; /* ... or whatever might suite. */
}
return (0 >= result) ?result :bytes_read;
}
Call it like this:
#include <stdio.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt);
int main(void)
{
int sd = -1;
/* init sd here */
{
char line[2048] = "";
ssize_t result = read_until_crlf(sd, line, sizeof line, 0);
if (-1 == result)
{
perror("read_until_newline() failed");
}
printf("read '%s'\n", line);
}
return 0;
}
I'm writing a small and simple server (in C language for Linux stations).
A client requests a file to my server, my server asks this file to another server which sends it to my server.
My server should NOT receive ALL the file before sending it to the client BUT must send the bytes of the file so as they arrive.
This is an exercise in school so I can not dissociate myself from this requirement.
I have implemented the function explained below. The problem is that the client receives a non-deterministic number of bytes and NEVER the entire file.
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
size_t n;
ssize_t nread;
ssize_t nwritten;
char c;
for (n=1; n<file_size; n++)
{
nread=recv(socketa, &c, 1, 0);
if (nread == 1)
{
nwritten = send(socketb,&c,1,0);
}
else if (nread == 0)
{
*buffer = 0;
return (-1); /* Errore */
}
else
return (-1); /* Errore */
}
}
*buffer = 0;
return (n);
}
Someone could kindly tell me where I'm wrong?
Is it an stupid idea to change the values SO_SNDBUF and SO_RCVBUF on both the server and the client?
Assuming the file_size is the total number of bytes you want to send, then your for loop will only send file_size - 1 bytes. In other words, you are off by one. Start from 0 instead to fix this:
for (n=0; n<file_size; n++)
{ //..
You capture the return value of send(), but you do not check to see if it was successful or not.
You are treating a 0 return value from recv() the same as an error. Since you do not show what you do after returning -1 from your function, I don't know if this may be contributing to your problem or not.
Certain errors on send() and recv() are "soft", in that you are allowed to retry the operation for those particular errors. One such error is EINTR, but check the documentation on your system to see if there are others.
In order to optimize performance and simplify your code, you can use splice()+pipes. Sendfile enables you to "forward" data between file descriptors, without the copy to user space.
Are you sure you have copied the correct code? That part as it is would not compile, there is a } in the last else which don't match with a corresponding {.
Also, how you get to know the file size? if it's send thru the socket as an integer, bear in mind the possible byte order of the source and destination machines.
Anyway, you are reading one byte at a time, you should improve it this way:
EDIT: use buffer and not the extra buff[2048];
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
ssize_t nread;
ssize_t nwritten;
ssize_t bLeft=file_size;
while (bLeft > 0)
{
nread=recv(socketa, buffer, bleft, 0);
if (nread > 0)
{
nwritten = send(socketb, buffer, nread, 0);
bLeft -= nread;
buffer+=nread;
}
else if (nread == 0)
{
// I think this could raise a memory exception, read below
*buffer = 0;
return (-1); /* Errore */
}
else
{
return (-1); /* Errore */
}
}
// If buffer is allocated with file_size bytes this one will raise a memory exception
// *buffer = 0;
return (file_size-bLeft);
}