I'm writing a small and simple server (in C language for Linux stations).
A client requests a file to my server, my server asks this file to another server which sends it to my server.
My server should NOT receive ALL the file before sending it to the client BUT must send the bytes of the file so as they arrive.
This is an exercise in school so I can not dissociate myself from this requirement.
I have implemented the function explained below. The problem is that the client receives a non-deterministic number of bytes and NEVER the entire file.
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
size_t n;
ssize_t nread;
ssize_t nwritten;
char c;
for (n=1; n<file_size; n++)
{
nread=recv(socketa, &c, 1, 0);
if (nread == 1)
{
nwritten = send(socketb,&c,1,0);
}
else if (nread == 0)
{
*buffer = 0;
return (-1); /* Errore */
}
else
return (-1); /* Errore */
}
}
*buffer = 0;
return (n);
}
Someone could kindly tell me where I'm wrong?
Is it an stupid idea to change the values SO_SNDBUF and SO_RCVBUF on both the server and the client?
Assuming the file_size is the total number of bytes you want to send, then your for loop will only send file_size - 1 bytes. In other words, you are off by one. Start from 0 instead to fix this:
for (n=0; n<file_size; n++)
{ //..
You capture the return value of send(), but you do not check to see if it was successful or not.
You are treating a 0 return value from recv() the same as an error. Since you do not show what you do after returning -1 from your function, I don't know if this may be contributing to your problem or not.
Certain errors on send() and recv() are "soft", in that you are allowed to retry the operation for those particular errors. One such error is EINTR, but check the documentation on your system to see if there are others.
In order to optimize performance and simplify your code, you can use splice()+pipes. Sendfile enables you to "forward" data between file descriptors, without the copy to user space.
Are you sure you have copied the correct code? That part as it is would not compile, there is a } in the last else which don't match with a corresponding {.
Also, how you get to know the file size? if it's send thru the socket as an integer, bear in mind the possible byte order of the source and destination machines.
Anyway, you are reading one byte at a time, you should improve it this way:
EDIT: use buffer and not the extra buff[2048];
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
ssize_t nread;
ssize_t nwritten;
ssize_t bLeft=file_size;
while (bLeft > 0)
{
nread=recv(socketa, buffer, bleft, 0);
if (nread > 0)
{
nwritten = send(socketb, buffer, nread, 0);
bLeft -= nread;
buffer+=nread;
}
else if (nread == 0)
{
// I think this could raise a memory exception, read below
*buffer = 0;
return (-1); /* Errore */
}
else
{
return (-1); /* Errore */
}
}
// If buffer is allocated with file_size bytes this one will raise a memory exception
// *buffer = 0;
return (file_size-bLeft);
}
Related
Extremely new to socket programming and C in general. I am trying to write a basic program to send and receive data between two machines. I understand that recv will not get all your data at once -- you essentially have to loop it until it has read the whole message.
In lieu of just setting a limit on both machines, I have created a simple Message struct on the client side:
struct Message {
size_t length;
char contents[1024 - sizeof(size_t)];
} message;
message.length = sizeof(struct Message);
message.contents = information_i_want_to_send;
When it arrives at the server, I have recv read into a buffer: received = recv(ioSock, &buffer, 1024, 0) (Which coincidentally is the same size as my Message struct -- but assuming it wasn't...).
I then extract Message.length from the buffer like this:
size_t messagelength;
messagelength = *((size_t *) &buffer);
Then I loop recv into the buffer while received < messagelength.
This works, but I can't help feeling it's really ugly and it feels hacky. (Especially if the first recv call reads less than sizeof(size_t) or the machines are different bit architectures, in which case the size_t cast won't work..). Is there a better way to do this?
You have a fixed-size message, so you can use something like this:
#include <errno.h>
#include <limits.h>
// Returns the number of bytes read.
// EOF was reached if the number of bytes read is less than requested.
// On error, returns -1 and sets errno.
ssize_t recv_fixed_amount(int sockfd, char *buf, size_t size) {
if (size > SSIZE_MAX) {
errno = EINVAL;
return -1;
}
ssize_t bytes_read = 0;
while (size > 0) {
ssize_t rv = recv(sockfd, buf, size, 0);
if (rv < 0)
return -1;
if (rv == 0)
return bytes_read;
size -= rv;
bytes_read += rv;
buf += rv;
}
return bytes_read;
}
It would be used something like this:
typedef struct {
uint32_t length;
char contents[1020];
} Message;
Message message;
ssize_t bytes_read = recv_fixed_amount(sockfd, &(message.length), sizeof(message.length));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(message.length)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
bytes_read = recv_fixed_amount(sockfd, &(message.content), sizeof(message.content));
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != msg_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
size_t is not going to be the same everywhere, so I switched to a uint32_t.
I read the fields independently because the padding within the struct can vary between implementations. They would need to be sent that way as well.
The receiver is populating message.length with the information from the stream, but doesn't actually use it.
A malicious or buggy sender could provide a value for message.length that's too large and crash the receiver (or worse) if it doesn't validate it. Same goes for contents. It might not be NUL-terminated if that's expected.
But what if the length wasn't fixed? Then the sender would need to somehow communicate how much the reader needs to read. A common approach is a length prefix.
typedef struct {
uint32_t length;
char contents[];
} Message;
uint32_t contents_size;
ssize_t bytes_read = recv_fixed_amount(sockfd, &contents_size, sizeof(contents_size));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(contents_size)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Message *message = malloc(sizeof(Message)+contents_size);
if (!message) {
perror("malloc");
exit(EXIT_FAILURE);
}
message->length = contents_size;
bytes_read = recv_fixed_amount(sockfd, &(message->contents), contents_size);
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != contents_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
message->length contains the size of message->contents instead of the size of the structure. This is far more useful.
Another approach is to use a sentinel value. This is a value that tells the reader the message is over. This is what the NUL that terminates C strings is. This is more complicated because you don't know how much to read in advance. Reading byte-by-byte is too expensive, so one normally uses a buffer.
while (1) {
extend_buffer_if_necessary();
recv_into_buffer();
while (buffer_contains_a_sentinel()) {
// This also shifts the remainder of the buffer's contents.
extract_contents_of_buffer_up_to_sentinel();
process_extracted_message();
}
}
The advantage of using a sentinel value is that one doesn't need to know the length of the message in advance (so the sender can start sending it before it's fully created.)
The disadvantage is the same as for C strings: The message can't contain the sentinel value unless some form of escaping mechanism is used. Between this and the complexity of the reader, you can see why a length prefix is usually preferred over a sentinel value. :)
Finally, there's a better solution than sentinel values for large messages that you want to start sending before they are fully created: A sequence of length-prefixed chunks. One keeps reading chunks until a chunk of size 0 is encountered, signaling the end.
HTTP supports both length-prefixed messages (in the form of Content-Length: <length> header) and this approach (in the form of the Transfer-Encoding: chunked header).
There are Two ways to do that...
1.)
Use Binary Synchronous protocol. (Use of STX - Start of Text and ETX - End of Text ) for identification of the Text start and end.
2.)
Attach the number of bytes of data being sent at the start of Data. The socket will read those number of bytes and will get the number of bytes to be received from the socket. Then read all data and get the amount of data required.
Hmm... Seems tough...?? Let me give you an example.
Actual Data need to be sent: ABCDEFGHIJ
New Data format : 0010ABCDEFGHIJ
Data required in server side: ABCDE
recv function will read the first 4 bytes to get the number of bytes of actual data(In loop untill it gets 4 bytes):
int received1= recv(ioSock, recvbuf, 4, 0);
As per the above case, 'recvbuf' will be 0010 converted to an integer will give value as '10' which can be stored in some integer variable. So we have :
int toReadVal = 10
Now all we need is to read these 10 digits in next recv call :
int received= recv(ioSock, recvbuf1, toReadVal, 0);
Finally, we get the value of recvbuf1 as ABCDEFGHIG. Now you can truncate the value as per your requirement.
I am trying to write and read Integer value into/from C socket. Sometimes ntohs() return very big values like 55000 , 32000 etc...Though client is always sending value <1500. If I run the program it happens after 10-15 minutes...Sometimes after 20-30 minutes.
Can you please check below code and tell me
Why this line getting printed ?
printf("Garbage value - ntohs problem ..Exiting... ");
// write exactly n byte
inline int write_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = write(fd, buf, left)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
// send exactly n byte
inline int send_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = send(fd, buf, left, MSG_NOSIGNAL)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
uint16_t nread, len, plength, nsend;
int MTU = 1500;
char buffer[2000];
// Server receive ( Linux 64 bit)
while (1) {
// read packet length
nread = read_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nread <=0) {
break;
}
len = ntohs(plength);
if (len <=0 || len > 1500 ) {
**printf("Garbage value - ntohs problem ..Exiting... "); // WHY ?**
break;
}
// read packat data
nread = read_n(SOCKFD, buffer, len);
if (nread != len) {
break;
}
}
//---------------------
// CLIENT send ( Android 5 )
while (1) {
nread = read(tunfd, buffer, MTU);
if (nread <= 0 || nread > 1500) { // always <=1500
break;
}
plength = htons(nread);
// send packet lenght
nsend = send_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nsend != sizeof(plength)) {
break;
}
// send packet data
nsend = send_n(TCP_SOCKFD, buffer, nread);
if (nsend != nread) {
break;
}
}
Thank you
We cannot tell you with certainty what's happening because you cannot provide a verifiable example. Additionally, you've not presented the implementation of read_n(), but supposing that it follows the same model as write_n() and send_n(), we can nevertheless perform some analysis.
Each of the data transfer functions returns a short count in the event that data transfer is interrupted by an error. The client code watches for this, and breaks out of its loop if it detects it. Well and good. The server code does not do this when reading plength, however. Since plength, as a uint16_t, is two bytes in size, a partial read is possible and would go unnoticed by your server code.
In your example, plength is modified only via the one read_n() call presented. Network byte order is big-endian, so the most-significant byte is read first. It is possible that the combination of that byte with the stale one left over from the previous read would represent a number exceeding 1500. For example, if a 221(0x00dd)-byte packet is followed by a 1280(0x0500)-byte packet, and a partial read occurs on the second packet size, then the combined result will be 1501(0x05dd).
I don't presently see any reason to think that the client sends data different in nature than you think it does, and I don't presently see any other way that your server code could give the appearance of receiving different data than the client sends, especially since client and server each abort at the first recognized sign of trouble.
Do note, however, that this code could still be made more robust. In particular, consider that read(), write(), and send() can fail even when there is no problem with the underlying socket or data transfer request. In particular, they can fail with EINTR if the call is interrupted by a signal, and if the socket is in non-blocking mode then they can fail with EAGAIN. There may be others. It does not seem useful to operate your socket in non-blocking mode, but you might indeed want to watch for EINTR and resume reading after receiving it.
I would also suggest that, at least during development, you emit more data about the nature of the error. Call perror(), for example, and afterward print the bad data. You might even consider logging data sent and received.
I need to write a small client/server application in C on Linux.
I have built a short example in order to explore a little bit more since i am new to network programming.
I am basically trying to send an array of double dynamically allocated by the client.
I found the following way to do it ( client side ) :
write(sd,&datas.size,sizeof(int)); /* send size */
write(sd,datas.yi,datas.size*sizeof(double));/* send array */
and on the server side :
read(sd_cli,&datas.size,sizeof(int)); /* receive size */
datas.yi=(double *)malloc(datas.size*sizeof(double));
read(sd_cli,datas.yi,datas.size*sizeof(double)); /* receiving datas */
At first sight my code seems to work fine.
But since the write calls are non blocking, i ask myself if the read sequence can receive , for example the array of double before its size ?
Is there any guarantee that this can never happen ?
Thanks.
Sockets of type SOCK_STREAM provide reliable, in-order data transmission, though details depend on the nature of the underlying transport. The reader will receive all the data successfully written (if it in fact chooses to read them all), byte-for-byte in the order they were written, but not necessarily in the same size chunks.
Blocking vs. non-blocking has nothing to do with it, though I don't actually see what makes you say your writes are non-blocking. Perhaps you're remarking on the fact that neither write() nor read() promises to transfer the full number of bytes requested on any given call. That in itself provides no guarantee against blocking, but you absolutely do need to account for it correctly, especially with sockets, and even more especially if you really have put one or both ends of the socket in non-blocking mode. The original version of your question seemed to claim that you do account for it.
In any case, barring some kind of kernel bug, your client will never read the array size after any part of the array, nor otherwise receive bytes in a different relative order than they were written.
To be perfectly clear, however, here are reasonable implementations for reading and writing variable-size double arrays via a stream socket. They assume that sender and receiver have identical representations of type double, which will certainly be the case for UNIX-domain sockets. They are not at all trivial, though the helper functions comprising around half the code are suitable for reuse:
/******
* helper functions
*/
/*
* Returns the number of bytes written, which may be zero, or a number
* less than zero on failure.
*/
ssize_t write_fully(int fd, const void *buf, size_t count) {
const unsigned char *next = buf;
size_t remaining = count;
while (remaining) {
ssize_t n_written = write(fd, next, remaining);
if (n_written < 0) {
/* error */
return n_written;
} else {
assert(n_written <= remaining);
next += n_written;
remaining -= n_written;
}
}
/* all bytes successfully written */
return count;
}
/*
* Returns the number of bytes read on success, or a number less
* than zero on error. It is accounted "success" if the end of the stream
* is reached before the requested number of bytes is read; that case
* can be distinguished by the return value, but no recovery is
* possible.
*/
ssize_t read_fully(int fd, void *buf, size_t count) {
unsigned char *next = buf;
size_t remaining = count;
while (remaining) {
ssize_t n_read = read(fd, next, remaining);
if (n_read < 0) {
/* error */
return n_read;
} else if (n_read) {
assert(n_read <= remaining);
next += n_read;
remaining -= n_read;
} else {
/* premature end of file */
return count - remaining;
}
}
/* all bytes successfully read */
return count;
}
/******
* Array-transfer functions
*/
/* returns 0 on success, else nonzero */
int write_double_array(int fd, unsigned n, double d[n]) {
ssize_t bytes_written;
bytes_written = write_fully(fd, &n, sizeof(n));
if (bytes_written < 0) return bytes_written;
bytes_written = write_fully(fd, d, n * sizeof(double));
return (bytes_written < 0) ? bytes_written : 0;
}
/*
* returns 0 on success, else nonzero.
* On success, the caller takes responsibility for freeing the
* dynamically-allocated result array.
*/
int read_double_array(int fd, unsigned *n, double **d) {
unsigned temp_n;
ssize_t bytes_read = read_fully(fd, &temp_n, sizeof(temp_n));
if (bytes_read < 0) {
return -1;
} else if (bytes_read != sizeof(temp_n)) {
return 1;
} else if (temp_n) {
size_t n_bytes = temp_n * sizeof(double);
double *temp = malloc(n_bytes);
if (!temp) return -1; /* allocation failure */
if (read_fully(fd, temp, n_bytes) < n_bytes) {
free(temp);
return -1;
}
/* success */
*d = temp;
}
*n = temp_n;
return 0;
}
You could implement the array-transfer protocol differently, but that approach sends the data in the same form that you claim to do. You cannot safely do it any more simply than that.
I'm programming in C an IRC chat client. everything it's working well except I can't read the whole answer sent by the server. here's the code:
char buffer[2048];
write_on_screen(current_page(), "LOG COMMAND", command);
write(sockfd, command, strlen(command)); //write to socket
bzero(buffer, sizeof(buffer));
read(sockfd, buffer, sizeof(buffer));
write_on_screen(current_page(), "RESPONSE", buffer);
return buffer;
most of the time buffer will contain just a piece of the response (which is shorter than 2048 bytes) and other times it contains nothing. in both cases if I do another read() after the first one, it returns me the rest of the answer or another small piece (and then I've to do another read() again). if I put a sleep(1) between write() and read() I get the whole answer, but I'm sure this not a good pratice.
Is there some way I can avoid this?
thank you in advance
You're making the usual mistakes. It is impossible to write correct network code without storing the result of read() or recv() into a variable. You have to:
Check it for -1, and if so look at errno to see whether was fatal, which it almost always is except for EAGAIN/EWOULDBLOCK, and if fatal close the socket and abandon the process.
Check it for zero, which means the peer disconnected. Again you must close the socket and abandon the process.
Use it as the count of bytes actually received. These functions are not obliged nor guaranteed to fill the buffer. Their contract in blocking mode is that they block until an error, end of stream, or at least one byte is transferred. If you're expecting more than one byte, you normally have to loop until you get it.
According to RFC-1459, a single line of text in IRC can contain up to 512 characters and is terminated by a CRLF (\r\n) pair. However:
You're not guaranteed to receive exactly 512 bytes each time. For example, you might receive a comparatively short message from someone else one in the channel: Hi!
Related to the above: A group of 512 bytes might represent more than one message. For example, the buffer might contain a whole line, plus part of the next line: PRIVMSG <msgtarget> <message>\r\nPRIVMS
Given that you could have zero-or-more complete lines plus zero-or-one incomplete lines in your buffer[] at any time, you could try doing something along the lines of:
char buffer[2048];
while(keep_going)
{
char **lines;
int i, num_lines;
// Receive data from the internet.
receiveData(buffer);
// Create an array of all COMPLETE lines in the buffer (split on \r\n).
lines = getCompleteLines(buffer, &num_lines);
removeCompleteLinesFromBuffer(buffer);
// Handle each COMPLETE line in the array.
for (i = 0; i < num_lines; ++i) { handle_line(lines[i]); }
freeLines(lines);
}
This would allow you to handle zero or more complete lines in one go, with any incomplete line (i.e anything after the final \r\n pair) being kept around until the next call to receiveData().
You need to loop around read() until a CRLF had been detected.
A possible way to do this would be:
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt)
{
ssize_t bytes_read = 0;
ssize_t result = 0;
int read_cr = 0;
int read_crlf = 0;
while (bytes_read < s)
{
result = read(sd, p + bytes_read, 1);
if (-1 == result)
{
if ((EAGAIN == errno) || (EWOULDBLOCK == errno))
{
continue;
}
else if (EINTR == errno)
{
if (break_on_interupt)
{
break;
}
continue;
}
else
{
perror("read() failed");
break;
}
}
else if (0 == result)
{
break; /* peer disconnected */
}
if ('\r' == p[bytes_read])
{
read_cr = 1;
}
else if (('\n' == p[bytes_read]) && read_cr)
{
read_crlf = 1;
break; /* CRLF detected */
}
else
{
read_cr = 0;
}
++bytes_read;
}
if (!read_crlf)
{
result = -1; /* Buffer full without having read a CRLF. */
errno = ENOSPC; /* ... or whatever might suite. */
}
return (0 >= result) ?result :bytes_read;
}
Call it like this:
#include <stdio.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt);
int main(void)
{
int sd = -1;
/* init sd here */
{
char line[2048] = "";
ssize_t result = read_until_crlf(sd, line, sizeof line, 0);
if (-1 == result)
{
perror("read_until_newline() failed");
}
printf("read '%s'\n", line);
}
return 0;
}
I am a new C programmer and so you will have to excuse my lack of knowledge. I am trying to use sockets in C on a windows machine to send data back and forth between a client and server. I am using the tools of cygwin with the codeblocks IDE. Simple send and receives were not working and so after some searching I was under the impression my problem was I needed a send_all and recv_all function. I have written the following two functions but receive seems to always get stuck in an infinite loop. I am not really sure why.
void send_all(int socket, void *buffer, int length) {
size_t i = 0;
for (i = 0; i < length; i += send(socket, buffer, length - i, 0)){
printf("Completed: %d bytes \r", i);
}
printf("Send Completed: %d bytes \n", length);
}
void recv_all(int sockfd, void *buffer, int length){
size_t i = 0;
for (i = 0; i < length; i+= recv(sockfd, buffer + i, length - i, 0)){
printf("Completed: %d bytes \r", i);
}
printf("Receive Completed: %d bytes \n", length);
}
I am wondering if it is because the receive doesn't know how many bytes the send is sending it. All advice is appreciated but please keep it constructive. Thanks.
recv() actually returns a signed value (int in Winsock, ssize_t in POSIX). Its return value can be a negative number if a read error occurred, OR if the socket is in non-blocking mode and no data is available. Its return value is zero if the socket was closed gracefully (this would cause an infinite loop in your code).
You will need to check the return value before you add it to your byte counter, to detect both of these conditions.
If your socket is in blocking mode (the default), your code will block indefinitely until the required amount of data has been received, or an error occurs (once you add code to check for that). Given the name of your function this seems to be the behavior you want. If so, your general approach is sound.
ssize_t bytesRead = 0;
while (bytesRead < length)
{
ssize_t rv = recv(/* ... */);
if (rv == 0)
{
printf("Socket closed gracefully before enough data received\n");
break;
}
else if (rv < 0)
{
// if your socket is non-blocking, check for EAGAIN which
// would mean no data is currently available; in this case you
// could do something like call select() on the socket to
// go to sleep until more data comes in
printf("Read error occurred before enough data received\n");
break;
}
bytesRead += rv;
}