I wonder, If i do 4 send() with my client, written in C, will my server need to read() 4 times, or could it be that the first read will read all 4 send()'s all together?
To help you out a little, you've basically answered your own question already.
You need to do it 'in tiers', so to speak.
Client:
int len = strlen(Filename) + 1; //Mind the terminating 0.
send(sock, (const char *)&len, sizeof(int), 0);
send(sock, Filename, len, 0); //Sending the filename
send(sock, &FileSize, sizeof(int), 0);
send(sock, FileBuf, FileSize, 0);
This code will send the entire data on the way (assuming that the entire file is in the 'FileBuf'-Variable).
Server:
int len;
char *FileBuf, FileName[20];
recv(sock, &len, sizeof(int), 0); //Receives the filename length. (4 Bytes)
recv(sock, FileName, len, 0); //Receives the filename (x bytes)
recv(sock, &len, sizeof(int), 0); //Receives the file length (again, 4 bytes)
FileBuf = new char[len]; //Creates sufficient space in memory.
recv(sock, FileBuf, len, 0); //Receives the file into the appropriate variable.
This is the absolute barebones variant, not very solid, but you should get the idea.
A more robust approach requires you to check the return values of recv() and send(). Both will return the amount of bytes, which have been processed with this call. If this amount equals '0', it means that the connection has been closed by the other end. (Mainly for recv()). If it equals -1, it means that something went wrong and you should check the errno variable.
If all goes well, it equals the exact amount of bytes you sent/tried to receive.
However in case it isn't 'len' (or 0 or -1), you could write a little wrapper like this.
unsigned char Recv(int sock, void *target, int Len) {
unsigned char *ptr = (unsigned char *)target, Ret = 0;
int RecvBytes = 1;
while(Len && !Ret) {
RecvBytes = recv(sock, ptr, Len, 0);
if(!RecvBytes) Ret = 1;
else if(RecvBytes == -1) Ret = errno;
else {
Len -= RecvBytes;
ptr += RecvBytes;
}
}
return Ret;
}
What this code does: It keeps receiving, until you either have received all the data you were expecting (the Len parameter) or an error occurred. If all goes well, it returns '0', which you can check with if(!Recv()).
Another useful wrapper function (a shortcut, so to speak) is this one:
uint32_t RecvInt(int sock) {
uint32_t Ret;
Recv(sock, &Ret, sizeof(Ret));
return ntohl(Ret);
}
This function receives exactly one unsigned int and corrects the endianess from network byte order to host byte order. (Network byte order is always big endian, host byte order is often: Little endian)
Using these wrapper functions, the code may be changed like this:
uint32_t len;
char *FileBuf, FileName[20];
len = RecvInt(sock); //Receives the filename length. (4 Bytes)
Recv(sock, FileName, len); //Receives the filename (x bytes)
len = RecvInt(sock); //Receives the file length (again, 4 bytes)
FileBuf = new char[len]; //Creates sufficient space in memory.
Recv(sock, FileBuf, len); //Receives the file into the appropriate variable.
For stream sockets (e.g. TCP): it makes no difference how many send() or write() calls were made on the sending end. The data could be returned in as few as one chunk, as many as n chunks of 1 byte each (where n is the number of bytes that were sent), or anything in between.
For datagram sockets (e.g. UDP): each recv() or recvmsg() call will return one complete datagram that was sent from the other end. The number of recv() or recvmsg() calls should be the same as the number of datagrams that were sent. recv() is preferred when reading from datagram sockets, but I believe read() should behave the same.
The number of write()s and read() need not be the same - it's possible that the write() writes all the data in one piece but on the other computer read() only manages to receive it in several chunks, and vice versa. That's why you should always check the return value of these functions and if only partial data transfer occurred, then continue with sending/receiving the rest.
Related
This is for a Linux system, in C. It involves network programming. It is for a file transfer program.
I've been having this problem where this piece of code works unpredictably. It either is completely successful, or the while loop in the client never ends. I discovered that this is because the fileLength variable would sometimes be a huge (negative or positive) value, which I thought was attributed to making some mistake with ntohl. When I put in a print statement, it seemed to work perfectly, without error.
Here is the client code:
//...here includes relevant header files
int main (int argc, char *argv[]) {
//socket file descriptor
int sockfd;
if (argc != 2) {
fprintf (stderr, "usage: client hostname\n");
exit(1);
}
//...creates socket file descriptor, connects to server
//create buffer for filename
char name[256];
//recieve filename into name buffer, bytes recieved stored in numbytes
if((numbytes = recv (sockfd, name, 255 * sizeof (char), 0)) == -1) {
perror ("recv");
exit(1);
}
//Null terminator after the filename
name[numbytes] = '\0';
//length of the file to recieve from server
long fl;
memset(&fl, 0, sizeof fl);
//recieve filelength from server
if((numbytes = recv (sockfd, &fl, sizeof(long), 0)) == -1) {
perror ("recv");
exit(1);
}
//convert filelength to host format
long fileLength = ntohl(fl);
//check to make sure file does not exist, so that the application will not overwrite exisitng files
if (fopen (name, "r") != NULL) {
fprintf (stderr, "file already present in client directory\n");
exit(1);
}
//open file called name in write mode
FILE *filefd = fopen (name, "wb");
//variable stating amount of data recieved
long bytesTransferred = 0;
//Until the file is recieved, keep recieving
while (bytesTransferred < fileLength) {
printf("transferred: %d\ntotal: %d\n", bytesTransferred, fileLength);
//set counter at beginning of unwritten segment
fseek(filefd, bytesTransferred, SEEK_SET);
//buffer of 256 bytes; 1 byte for byte-length of segment, 255 bytes of data
char buf[256];
//recieve segment from server
if ((numbytes = recv (sockfd, buf, sizeof buf, 0)) == -1) {
perror ("recv");
exit(1);
}
//first byte of buffer, stating number of bytes of data in recieved segment
//converting from char to short requires adding 128, since the char ranges from -128 to 127
short bufLength = buf[0] + 128;
//write buffer into file, starting after the first byte of the buffer
fwrite (buf + 1, 1, bufLength * sizeof (char), filefd);
//add number of bytes of data recieved to bytesTransferred
bytesTransferred += bufLength;
}
fclose (filefd);
close (sockfd);
return 0;
}
This is the server code:
//...here includes relevant header files
int main (int argc, char *argv[]) {
if (argc != 2) {
fprintf (stderr, "usage: server filename\n");
exit(1);
}
//socket file descriptor, file descriptor for specific client connections
int sockfd, new_fd;
//...get socket file descriptor for sockfd, bind sockfd to predetermined port, listen for incoming connections
//...reaps zombie processes
printf("awaiting connections...\n");
while(1) {
//...accepts any incoming connections, gets file descriptor and assigns to new_fd
if (!fork()) {
//close socket file discriptor, only need file descriptor for specific client connection
close (sockfd);
//open a file for reading
FILE *filefd = fopen (argv[1], "rb");
//send filename to client
if (send (new_fd, argv[1], strlen (argv[1]) * sizeof(char), 0) == -1)
{ perror ("send"); }
//put counter at end of selected file, and find length
fseek (filefd, 0, SEEK_END);
long fileLength = ftell (filefd);
//convert length to network form and send it to client
long fl = htonl(fileLength);
//Are we sure this is sending all the bytes??? TEST
if (send (new_fd, &fl, sizeof fl, 0) == -1)
{ perror ("send"); }
//variable stating amount of data unsent
long len = fileLength;
//Until file is sent, keep sending
while(len > 0) {
printf("remaining: %d\ntotal: %d\n", len, fileLength);
//set counter at beginning of unread segment
fseek (filefd, fileLength - len, SEEK_SET);
//length of the segment; 255 unless last segment
short bufLength;
if (len > 255) {
len -= 255;
bufLength = 255;
} else {
bufLength = len;
len = 0;
}
//buffer of 256 bytes; 1 byte for byte-length of segment, 255 bytes of data
char buf[256];
//Set first byte of buffer as the length of the segment
//converting short to char requires subtracting 128
buf[0] = bufLength - 128;
//read file into the buffer starting after the first byte of the buffer
fread(buf + 1, 1, bufLength * sizeof(char), filefd);
//Send data too client
if (send (new_fd, buf, sizeof buf, 0) == -1)
{ perror ("send"); }
}
fclose (filefd);
close (new_fd);
exit (0);
}
close (new_fd);
}
return 0;
}
Note: I've simplified the code a bit, to make it clearer I hope.
Anything beginning with //... represents a bunch of code
You seem to be assuming that each send() will either transfer the full number of bytes specified or will error out, and that each one will will pair perfectly with a recv() on the other side, such that the recv() receives exactly the number of bytes sent by the send() (or error out), no more and no less. Those are not safe assumptions.
You don't show the code by which you set up the network connection. If you're using a datagram-based protocol (i.e. UDP) then you're more likely to get the send/receive boundary matching you expect, but you need to account for the possibility that packets will be lost or corrupted. If you're using a stream-based protocol (i.e. TCP) then you don't have to be too concerned with data loss or corruption, but you have no reason at all to expect boundary-matching behavior.
You need at least three things:
An application-level protocol on top of the network-layer. You've got parts of that already, such as in how you transfer the file length first to advise the client about much content to expect, but you need to do similar for all data transferred that are not of pre-determined, fixed length. Alternatively, invent another means to communicate data boundaries.
Every send() / write() that aims to transfer more than one byte must be performed in a loop to accommodate transfers being broken into multiple pieces. The return value tells you how many of the requested bytes were transferred (or at least how many were handed off to the network stack), and if that's fewer than requested you must loop back to try to transfer the rest.
Every recv() / read() that aims to transfer more than one byte must be performed in a loop to accommodate transfers being broken into multiple pieces. I recommend structuring that along the same lines as described for send(), but you also have the option of receiving data until you see a pre-arranged delimiter. The delimiter-based approach is more complicated, however, because it requires additional buffering on the receiving side.
Without those measures, your server and client can easily get out of sync. Among the possible results of that are that the client interprets part of the file name or part of the file content as the file length.
Even though you removed it from that code I'll make an educated guess and assume that you're using TCP or some other stream protocol here. This means that the data that the servers sends is a stream of bytes and the recv calls will not correspond in the amount of data they get with the send calls.
It is equally legal for your first recv call to just get one byte of data, as it is to get the file name, file size and half of the file.
You say
When I put in a print statement,
but you don't say where. I'll make another educated guess here and guess that you did it on the server before sending the file length. And that happened to shake things enough that the data amounts that were sent on the connection just accidentally happened to match what you were expecting on the client.
You need to define a protocol. Maybe start with a length of the filename, then the filename, then the length of the file. Or always send 256 bytes for the filename regardless of how long it is. Or send the file name as a 0-terminated string and try to figure out the data from that. But you can never assume that just because you called send with X bytes that the recv call will get X bytes.
I believe the issue is actually a compound of everything you and others have said. In the server code you send the name of the file like this:
send (new_fd, argv[1], strlen (argv[1]) * sizeof(char), 0);
and receive it in the client like this:
recv (sockfd, name, 255 * sizeof (char), 0);
This will cause an issue when the filename length is anything less than 255. Since TCP is a stream protocol (as mentioned by #Art), there are no real boundaries between the sends and recvs, which can cause you to receive data in odd places where you are not expecting them.
My recommendation would be to first send the length of the filename, eg:
// server
long namelen = htonl(strlen(argv[1]));
send (new_fd, &namelen, 4, 0);
send (new_fd, argv[1], strlen (argv[1]) * sizeof(char), 0);
// client
long namelen;
recv (sockfd, &namelen, 4, 0);
namelen = ntohl(namelen);
recv (sockfd, name, namelen * sizeof (char), 0);
This will ensure that you are always aware of exactly how long your filename is and makes sure that you aren't accidentally reading your file length from somewhere in the middle of your file (which is what I expect is happening currently).
edit.
Also, be cautious when you are sending sized numbers. If you use the sizeof call on them, you may be sending and receiving different sizes. This is why I hard-coded the sizes in the send and recv for the name length so that there is no confusion on either side.
Well, after some testing, I discovered that the issue causing the problem did have something to do with htonl(), though I had still read the data incorrectly in the beginning. It wasn't that htonl() wasn't working at all, but that I didn't realize a 'long' has different lengths depending on system architecture (thanks #tofro). That is to say the length of a 'long' integer on 32-bit and 64-bit operating systems is 4 bytes and 8 bytes, respectively. And the htonl() function (from arpa/inet.h) for 4-byte integers. I was using a 64-bit OS, which explains why the value was being fudged. I fixed the issue by using the int32_t variable (from stdint.h) to store the file length. So the main issue in this case was not that it was becoming out of sync (I think). But as for everyone's advice towards developing an actual protocol, I think I know what exactly you mean, I definitely understand why it's important, and I'm currently working towards it. Thank you all for all your help.
EDIT: Well now that it has been several years, and I know a little more, I know that this explanation doesn't make sense. All that would result from long being larger than I expected (8 bytes rather than 4) is that there's some implicit casting going on. I used sizeof(long) in the original code rather than hardcoding it to assume 4 bytes, so that particular (faulty) assumption of mine shouldn't have produced the bug I saw.
The problem is almost certainly what everyone else said: one call to recv was not getting all of the bytes representing the file length. At the time I doubted this was the real cause of the behaviour I saw, because the file name (of arbitrary length) I was sending through was never partially sent (i.e. the client always created a file of the correct filename). Only the file length was messed up. My hypothesis at the time was that recv mostly respected message boundaries, and while recv can possibly only send part of the data, it was more likely that it was sending it all and there was another bug in my code. I now know this isn't true at all, and TCP doesn't care.
I'm a little curious as to why I didn't see other unexpected behaviour as well (e.g. the file name being wrong on the receiving end), and I wanted to investigate further, but despite managing to find the files, I can't seem to reproduce the problem now. I suppose I'll never know, but at least I understand the main issue here.
I read in MSDN about the send() and recv() function, and there is one thing that I'm not sure I understand.
If I send a buffer of size 256 for example, and receive first 5 bytes, so the next time I call the recv() function, it will point to the 6th byte and get the data from there?
for example :
char buff[256];
memcpy(buff,"hello world",12);
send(sockfd, buffer, 100) //sending 100 bytes
//server side:
char buff[256];
recv(sockfd, buff, 5) // now buffer contains : "Hello"?
recv(socfd, buff,5) // now I ovveride the data and the buffer contains "World"?
thanks!
The correct way to receive into a buffer in a loop from TCP in C is as follows:
char buffer[8192]; // or whatever you like, but best to keep it large
int count = 0;
int total = 0;
while ((count = recv(socket, &buffer[total], sizeof buffer - total, 0)) > 0)
{
total += count;
// At this point the buffer is valid from 0..total-1, if that's enough then process it and break, otherwise continue
}
if (count == -1)
{
perror("recv");
}
else if (count == 0)
{
// EOS on the socket: close it, exit the thread, etc.
}
You have missed the principal detail - what kind of socket is used and what protocol is requested. With TCP, data is octet granulated, and, yes, if 256 bytes was sent and you have read only 5 bytes, rest 251 will wait in socket buffer (assuming buffer is larger, which is true for any non-embedded system) and you can get them on next recv(). With UDP and without MSG_PEEK, rest of a single datagram is lost, but, if MSG_PEEK is specified, next recv() will give the datagram from the very beginning. With SCTP or another "sequential packet" protocol, AFAIK, the same behavior as with UDP is got, but I'm unsure in Windows implementation specifics.
Hi i have written a server application which accepts a name from the client which usually is a file name.It opens the file ,reads the contents into a buffer and then transmits the buffer over the ethernet using send().But the problem arises in the client side where all the bytes are not received successfully.I receive only a part of what i send.
For your reference ,here's the code snippet for the server side:
Server:
fp = fopen(filename,"r+");
strcpy(str,"");
fseek(fp, 0L, SEEK_END);
size = ftell(fp);
fseek(fp, 0L, SEEK_SET);
fread(str, size, 1,fp);
fclose(fp);
printf("Size of the file is : %d\n",size);
sprintf(filename, "%d", size);
n = send(nsd, filename, strlen(filename), 0);
while(size > 0){
n = send(nsd, str, strlen(str), 0);
printf("%d bytes sent successfully\n",n);
if(n == 0) break;
sentbytes = sentbytes + n;
size = size - sentbytes;
}
Please help me with writing the client app.I am currently confused about how to go about writing it.Shall i place the recv() part in a while(1) loop so that the client keeps running until all the bytes have been received successfully?
EDITED
For starters, you could both read from the file and write to the socket in chunks at the same time.
Since, you are transferring data over TCP, remember that data is transferred reliably as a stream and not as messages. So, don't make assumptions about how the data is recv'd except for the order.
Here is how it could be written:
open socket
open file
size_of_file = read_file_size(file);
send(socket, &size_of_file, sizeof(int), ...)
while (all is written)
read fixed chunk from file
write as much was read to the socket
cleanup // close file, socket
As for the recv part, I think it is best you send the file size over as an integer and keep reading in a while loop until you have recv'd as many bytes as you are sending from the server.
It's like this:
recv(socket, &size_of_msg, sizeof(int), ...)
while(all is read)
read fixed chunk from the socket
cleanup
Well I see atleast some issue with the way you are sending message over socket.
First from the man page of fread:
The function fread() reads nmemb elements of data, each size bytes
long, from the stream pointed to by stream, storing them at the loca-
tion given by ptr.
and what you are trying is this:
fread(str, size, 1,fp);
I assume what you meant was
fread(str, 1,size,fp);
Though it shold not casue the issue.
But the problem lies here:
n = send(nsd, str, strlen(str), 0);
printf("%d bytes sent successfully\n",n);
if(n == 0) break;
sentbytes = sentbytes + n;
size = size - sentbytes;
Though you are decreasing 'size' by decreasing by number of bytes successfully send, where are you extending str to point to new buffer location where data will be send.This will only resend initial bytes of the buffer repeatedly.
str += n; //Assuming str is char*
will solve your issue.
Using strlen doesn't seem appropriate. You've read the file, you know how long it is, so why do strlen? Either you'll just get the same result (so it's redundant) or you'll get something else (so it's a bug).
"Shall i place the recv() part in a while(1) loop so that the client keeps running until all the bytes have been received successfully?"
Something like that. Never presume that a recv() call got everything that was sent -- tcp/ip breaks messages into packets at a lower level, and recv() will return after reading whatever amount of data has actually been received at whatever point. You don't have to worry about that directly, except in so far as you do need to use some kind of protocol to indicate how long a message is so the receiver knows how much to read, then eg.:
char buffer[4096];
int msgsz = 600, // see below
sofar = 0,
cur;
while (sofar < msgsz) {
cur = recv (
socket_fd,
&buffer[sofar],
msgsz - sofar,
0
);
if (cur == -1) {
// error
break;
} else if (cur == 0) {
// disconnected
break;
}
sofar += cur;
}
WRT msgsz, you would include this somewhere in a fixed length header, which is read first. A simple version of that might be just 4 bytes containing a uint32_t, ie, an int with the length. You could also use a null terminated string with a number in it, but that means reading until '\0' is found.
could be any difference in solving errors between this two functions?:
To this question brought me another question ... is number of characters always same as number of bytes?
For more info: I use it in C on Linux for TCP socket comunication(sys/socket.h)
Thanks for your responses.
send()
write()
Return:
write():
On success, the number of bytes written are returned (zero indicates nothing was written). On error, -1 is returned, and errno is set appropriately. If count is zero and the file descriptor refers to a regular file, 0 will be returned without causing any other effect. For a special file, the results are not portable.
send():
The calls return the number of characters sent, or -1 if an error occurred.
Question from stackoverflow which says that this methods should be same with using flag zero.
here
int client_sockfd;
char* msg;
int length = strlen(msg);
//first option
if(send(client_sockfd, msg, length, 0) != length) return 1;
else return 0;
//second option
if(write(client_sockfd, msg, length) != length) return 1;
else return 0;
They will both return the same number of written bytes (== characters in this case. EXCEPT note this:
If the message is too long to pass atomically through the underlying protocol, the
error EMSGSIZE is returned, and the message is not transmitted.
In other words, depending on the size of the data being written, write() may succeed where send() may fail.
Number of bytes == number of characters, since the C standard reuires that char be an 1-byte integer.
write():
Yes, it returns the number of bytes written. But: it's not always an error if it doesn't return as many bytes as it should heva written. Especially not for TCP communication. A socket may be nonblocking or simply busy, in which case you'll need to rewrite the not-yet-written bytes. This behavior can be achieved like this:
char *buf = (however you acquire your byte buffer);
ssize_t len = (total number of bytes to be written out);
while (len > 0)
{
ssize_t written = write(sockfd, buf, len);
if (written < 0)
{
/* now THAT is an error */
break;
}
len -= written;
buf += written; /* tricky pointer arythmetic */
}
read():
Same applies here, with the only difference that EOF is indicated by returning 0, and it's not an error. Again, you have to retry reading if you want to receive all the available data from a socket.
int readbytes = 0;
char buf[512];
do {
readbytes = read(sockfd, buf, 512);
if (readbytes < 0)
{
/* error */
break;
}
if (readbytes > 0)
{
/* process your freshly read data chunk */
}
} while (readbytes > 0); /* until EOF */
You can see my implementation of a simple TCP helper class using this technique at https://github.com/H2CO3/TCPHelper/blob/master/TCPHelper.m
I need to send a set of bit fields along with a string of characters from a client to a server.
So given I have:
#define YES 1
#define FLAG 2
int main(int argc, char* argv[])
{
return sendToServer("The Message", YES | FLAG);
}
int sendToServer(char* msg, int bitfields)
{
/* create socket and connect to server */
/* Assume sock is set */
send(sock, msg, strlen(msg), 0);
return 0;
}
What would be the best way to send the bitfields? Is there anyway to send the bitfields along with the string?
EDIT: Ok I'm trying to implement Vlad's method. My client is pretty much identical to what he wrote. I have put the flag at the beginning data[0] and I used htonl instead of bswap. My server:
int main(int argc, char* argv[])
{
/* create socket and wait for connection */
char buffer[BUFFERSIZE];
size_t rcvdB = recv(clntSock, buffer, sizeof(int),0);
int flags = ntohl((int) buffer);
rcvdB = recv(clntSock, buffer, sizeof(size_t),0);
size_t msgSize = ntohl((size_t) buffer);
rcvdB = recv(clntSock,buffer,msgSize,0);
/* Then I send back to the client */
ssize_t sntB = send(clntSock,buffer,msgSize,0);
}
When the client prints the message there are multiple ascii characters at the end of the message.
EDIT2:
The issue seems to occur when I read more than 8 bytes of data
As others have pointed out, it depends on a protocol. I'd use something like this:
int sendToServer(char* msg, int bitfields)
{
unsigned int bits = bswap32 (bitfields); // Convert host to network byte order.
size_t len = strlen (msg); // Calculate string length.
size_t nlen = bswap64 (len); // Convert length's byte order from host to network.
iovec data[3]; // Prepare 3 I/O buffers to send data by performing a single system call (they are expensive).
// Send length of the string first.
data[0].iov_base = &nlen;
data[0].iov_len = sizeof (nlen);
// Send string...
data[1].iov_base = msg;
data[1].iov_len = len;
// And, of course, send bits.
data[2].iov_base = &bits;
data[2].iov_len = sizeof (bits);
// Write all of those to the socket.
writev (fd, &data, sizeof (data) / sizeof (data[0]));
}
On the receiving side, you can read the first sizeof (size_t) bytes, convert from network to host byte order, cast to size_t. That will tell you length of string. Then read buffer of that length - that will be your string. Finally, read another sizeof (int) bytes - that will be your bitfield.
See also:
Endianess
writev
This depends on your network protocol, of course. If you're designing it, and you have a maximum of 8 flags to send, put them in the first byte of the message. If you have at most 16, but them in the first two bytes, etc.
int sendToServer(int sock, char* msg, int flags)
{
size_t siz = strlen(msg) + 1; // +1 for NUL
unsigned char *buf = malloc_or_die(siz + 1);
buf[0] = flags;
memcpy(buf + 1, msg, size);
send(sock, msg, strlen(msg), 0);
free(buf);
return 0;
}
(I changed bitfields to flags since that's what they are. A bitfield is a collection of flags.)
It's probably best to send the bitfields as the bytes for a raw int (e.g., send(sock, &bitfields, sizeof(bitfields), 0)), making sure that the size & endianness is the same for both the client & server. It'd probably be easiest to send the bits before the string, as the server could then easily extract the string after a fixed number of bytes.
The bitfield forms a number, for instance 1|2 gives 3 since %01 | %10 = %11.
So all you have to send is that number. Kind of
snprintf(buffer, buffer_size, "%s-%d", message, number);
send(sockfd, buffer, buffer_size, 0);