1: The server copy the file size into the buffer and send it:
snprintf(t_buf, 255, "%" PRIu32, fsize);
if(send(f_sockd, t_buf, sizeof(t_buf), 0) < 0){
perror("error on sending file size\n");
onexit(f_sockd, m_sockd, 0, 2);
}
2: The client receives the file size and put it into fsize:
if(recv(f_sockd, t_buf, sizeof(t_buf), 0) < 0){
perror("error on receiving file size");
onexit(f_sockd, 0 ,0 ,1);
}
fsize = atoi(t_buf);
----------------- The code above makes my program working perfectly!
The problem happens if i write this code instead of the previous one:
1: The server send fsize:
if(send(f_sockd, &fsize, sizeof(fsize), 0) < 0){
perror("error on sending file size\n");
onexit(f_sockd, m_sockd, 0, 2);
}
2: The client receives fsize:
if(recv(f_sockd, &fsize, sizeof(fsize), 0) < 0){
perror("error on receiving file size");
onexit(f_sockd, 0, 0, 1);
}
Where uint32_t fsize; and char t_buf[256];.
The problem is that with the first method all work but with the second method the client doesn't receive all file but only a piece of it. What is wrong with this code?Thanks!
recv(2) doesn't necessarily fill the complete output buffer - it might return fewer bytes depending on how much data is available:
The receive calls normally return any data available, up to the requested amount, rather than waiting for receipt of the full amount requested.
The return value (when > 0) will be the number of bytes received, so you can call it in a loop if you want to be sure to receive everything.
Alternatively, you could pass the MSG_WAITALL flag:
This flag requests that the operation block until the full request is satisfied. However, the call may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned.
So in your case, you could do something like:
ssize_t bytes = recv(f_sockd, &fsize, sizeof(fsize), MSG_WAITALL);
if(bytes == sizeof(fsize))
{
/* received everything */
}
else
{
/* something went wrong */
}
Hard to tell without more data.
Could be endianness. You should print out the value of fsize.
Could also be partial return from recv(), as was mentioned in a comment.
Related
EDIT: Changed the while loop condition from > 0 to !=0. There are also 2 or more newlines being output on the terminal after printing the content, but in the file there aren't.
I'm doing a file transfer from the client to the server. The contents are transferred successfully, but when receiving it doesn't exit the while loop. Is my while condition somehow wrong? After I force stop the program, i can see that the file has actually been copied successfully, it just does not exit this while loop after writing everything to the file.
This is the code where I'm having trouble:
ssize_t bytes_read = 0;
int error;
while((bytes_read = read(sd, buf, sizeof(buf))) != 0){
printf("writing: %s\n", buf);
if((error = fwrite(buf, 1, bytes_read, fp)) < 0)
{
printf("error");
}
if((bytes_read = read(sd, buf, sizeof(buf))) < 0){
break;
}
}
Tried to keep it as minimal as possible, as I'm sure there's something very simplistically wrong here.
EDIT: I tried it with a larger file, and it seems that its missing some content here and there but not in a consistent pattern.
EDIT: Here is the client side.
else if(strcmp(shortCommand, "put") == 0){
char *tmp = buf + 4;
char filename[MAX_BLOCK_SIZE];
strcpy(filename, "filename ");
strcat(filename, tmp);
FILE *fp;
printf("File name: %s\n", tmp);
fp = fopen(tmp, "rb");
if(fp == NULL){
printf("ERROR: Requested file does not exist.\n");
}
else{
printf("Client sending filename...\n");
if ((nw = write(sd, buf, sizeof(buf))) < 0){ //sending the file name to the client first
printf("Error sending client's filename.\n");
}
printf("Client sending file...\n");
size_t bytes_read = 0;
ssize_t bytes_written = 0;
while((bytes_read = fread(buf, 1, sizeof(buf), fp)) != 0){ //sending the file contents
if ((bytes_written = write(sd, buf, bytes_read)) < 0){
printf("Error sending client file.\n");
}
}
fclose(fp);
}
}
The server code assumes that it will observe read() to return 0 after it reads all the data that the client transfers, but that will not happen just because the client stops sending data. read() returns 0 to indicate that the end of the file has been reached, but "file" in that sense means the read side of its socket. Until the client closes the peer socket, the server will not perceive EOF on its socket. And that's perfectly reasonable because until then, the client might send more data, proving that indeed EOF had not previously been reached. Therefore, if the server tries to perform a (blocking) read on the socket after receiving all the data then it will block until the client sends more or closes the connection, or until it detects an error, is interrupted by a signal, etc..
If the client doesn't actually intend to send any further data after the file content then it should close() that socket after sending all the bytes of the file content, or at least shutdown() the write side.
If, on the other hand, the client is keeping the socket open because it intends to send more data later, then you need a new strategy. In that case the end of the file content does not coincide with the logical end of the stream, and you therefore need a different mechanism to communicate to the server where the end of the file data is. This is where communication protocols such as HTTP come in, but for your purpose, you don't need a protocol nearly as complicated as HTTP.
It might suffice, say, for the client to send data to the server as one or more chunks, each comprising a byte count followed by the specified number of bytes of content. You could insist that the whole file content always go into one chunk, but if you want to allow splitting the file into multiple chunks then you could use a chunk length of zero to signal the end of the data.
So i need to recv an html file from the server to the client, the file is bigger than the buffer so i make several sends. Thats why i have this loop when i recv
while (i = recv(s, buf, TAM_BUFFER, 0)) {
if (i == -1) {
perror(argv[0]);
fprintf(stderr, "%s: error reading result\n", argv[0]);
exit(1);
}
while (i < TAM_BUFFER) {
j = recv(s, &buf[i], TAM_BUFFER - i, 0);
if (j == -1) {
perror(argv[0]);
fprintf(stderr, "%s: error reading result\n", argv[0]);
exit(1);
}
i += j;
}
/* Print out the file line by line. */
printf("%s", buf);
}
the send looks something like this:
while (fgets(buf, sizeof(buf), fp)){
if (send(s, buf, TAM_BUFFER, 0) != TAM_BUFFER) errout(hostname);
}
The problem is the loop never ends, becase it doesnt recv the eof and i is never 0, its just remain blocked there.
I cant do the close to send the eof because after he recv the whole file, the client will ask for another file.
I tryed to send a SIGALRM if the loop stays blocked for longer than 5 seconds but it doesnt work as expected, because the loop wont stop, and it will throw an error.
Also how can i do to be able to recv less than TAM_BUFFER?(in the send, change the TAM_BUFFER -> strlen(buf)) I know i need to change the interior loop, but then ill have the same problem, j will not be 0 never, so i dont know how could i end it.(or maybe i dont need the second loop in this case).
EDIT: i cant send the lenght of the file beucause of the protocol im following
TCP is a protocol used to transport a single unstructured octet stream in each direction. Shutdown of the connection (i.e. EOF) is the only way in TCP to signal to the peer that no more data will be sent in this connection. If you need a different way because you need to distinguish between multiple messages inside the same TCP connection then you need to use an application level protocol which can specify such message boundaries. This is usually done by fixed message size, prefixing the message with a length or by special boundary markers.
If you can't embed payload size in your protocol, you have to identify EOF by closing socket or checking for timeout. You can use select function and set timeout for it, see here Using select and recv to obtain a file from a web server through a socket and https://stackoverflow.com/a/30395738/4490542
Not sure how well I worded the title. I've written a linux domain socket server and client. The client sets a timeout value on the write. If the client can't send all of its data I don't want the server to accept the data that it has sent. Is there way the client can indicate that it didn't send all of the data? Maybe somehow cause the server's read() to fail? The sockets are setup as stream sockets.
So basically I want to know what to do in this case:
ssize_t bytes_written = write(fd, buffer, length);
if (bytes_written == -1)
{
result = -1;
goto done;
}
// I think the only case where we can have write return
// a successful code but not all bytes written is when the
// timeout value has elapsed and some number of bytes have
// been written.
if (bytes_written != length)
{
result = -1;
errno = ETIMEDOUT;
}
.
.
.
done:
if (result == -1)
result = errno;
if (fd != -1)
{
shutdown(fd, SHUT_RDWR);
close(fd);
}
return result;
}
I realize an obvious solution is for the client to send a byte count first and then send the bytes. I was wondering whether there was another way. Also, each message could be a different size.
You can packet your data with a length in the head. If the data doesn't match the length, the server can drop the data.
so I have two processes, one client-process one server-process. The user can issue a command to the client, when a user enters a command the client will send the command length to the server, and after that it will send the actual command.
The server then sends back first the length of the response and then a response.
I can do 5-30 commands or so with no problem at all, but at some point it fails to read enough bytes, despite the correct response size being received.
The server sends the response in the following way:
str[0] = '\0';
unsigned long int totalSize = 0;
while ((fgets(outBuf, MAXOUTPUT, myFile)) != NULL)
{
strcat(str, outBuf);
}
uint32_t *un = 0;
totalSize = strlen(str);
*un = htonl(totalSize);
result= send(clientFD, un, sizeof(uint32_t), 0);
if(result < 1)
{
printf("Failed sending message size to client");
exit(-1);
}
while(token != NULL)
{
size_t length = strlen(token);
token[length] = '\n';
write(clientFD, token, length + 1);
token = strtok(NULL, "\n");
}
The client has received the message length correctly(verified with prints) and reads the response this way:
result = read(socketFD, recvBuf, bufferlen); //bufferlen is response size
if(result < bufferlen)
{
perror("read()");
exit(-1);
}
I have verified that the client receives the correct message length every time, including the last one where it fails to read.
So my question is: What are likely reasons that my read sometimes fail to retrieve the full response? It happens after doing about 5-30 commands or so usually, and the perror returned is Error 0 (aka no error to be found).
As an additional note, the commands tested are
ls -la, ls -l, ls.
I have not found a pattern in which commands cause the crash, but I have combined them a lot.
Also: Both the client and server are 32 bit and being run on the same machine locally.
read() (especially on sockets) returns as soon as some data is available, it may always return less bytes than you asked for. In this case, you will need to repeat the read until you have read enough data:
size_t bytes_read = 0;
while (bytes_read < bufferlen) {
result = read(socketFD, recvBuf + bytes_read, bufferlen - bytes_read);
if (result < 0) {
perror("read()");
exit(-1);
}
bytes_read += result;
}
The TCP socket works on a byte stream concept. The server is adding bytes to the byte stream, and the client is consuming them. The socket need not send all of the bytes at once; it will eventually send them and they will be read in order at the other end. Messages are not guaranteed to be kept whole. You encounter a problem when you can read the bufferlen field but the whole corresponding message has not arrived yet.
Your client needs to continue reading from the socket until bufferlen bytes have been read.
Also be aware that the act of reading the bufferlen field may also need to be completed with multiple reads.
I'm writing a C program to transfer a file of fixed size, a little over 2Mb, from a server to a client. I'm using TCP sockets on Linux and the code I wrote is the following:
Server (sender)
while (1) {
int nread = read(file, buffer, bufsize);
if (nread == 0) // EOF
break;
if (nread < 0) {
// handle errors
}
char* partial = buffer;
while (nread > 0) {
int nwrite = write(socket, partial, nread);
if (nwrite <= 0) {
// handle errors
}
nread -= nwrite;
partial += nwrite;
}
}
// file sent
shutdown(socket, SHUT_WR);
Client (receiver)
while (filesize > 0) {
nread = read(socket, buffer, bufsize);
if (nread == 0) {
// EOF - if we reach this point filesize is still > 0
// so the transfer was incomplete
break;
}
if (nread < 0) {
// handle errors
}
char* partial = buffer;
while (nread > 0) {
nwrite = write(file, partial, nread);
if (nwrite <= 0) {
// handle errors
}
nread -= nwrite;
partial += nwrite;
filesize -= nwrite;
}
}
if (filesize > 0) {
// incomplete transfer
// handle error
}
close(socket);
When testing the code on my laptop (both client and server "are" on localhost and the communication happen on the loopback interface), sometimes the client exits because read received an EOF, and not because it received all filesize bytes. Since i use a shutdown on the server, this should mean that there is no other data to read.
(Note that the server sent all the bytes and executed the shutdown correctly)
Can you explain me why this is happening?
Where are the missing bytes gone?
-----
EDIT 1 - Clarifications
Some users asked a couple of clarifications so i am posting the answers here:
The program is using TCP blocking sockets
The filesize is a fixed value and is hardcoded in both client and server.
No special socket options as, for example, SO_LINGER are enabled/used.
When the error occur, the server (sender) has already sent all the data and executed the shutdown correctly.
The error, as of today, never happened when testing the application with the client and the server on different machines (transfer over a real network interface and not a loopback interface)
EDIT 2
User Cornstalks pointed me to a really interesting article about the, non always reliable, behaviours of TCP.
The article, which is well worth a read, describe a few tricks useful when sending an unknown amount of data between TCP sockets. The tricks described are the followings:
Take advantage of the SO_LINGER option on the sender. This will help to keep the socket open, upon a call to close(2) or shutdown(2), until all the data has successfully been sent.
On the receiver, beware of pending readable data before the actual receiving loop. It could lead to an immediate reset being sent.
Take advantage of shutdown(2) to signal the receiver the the sender has done sending data.
Let the receiver know the size of the file that will be sent before actually sending the file.
Let the receiver acknowledge the sender that the receiving loop is over. This will help to prevent the sender from closing the socket too soon.
After reading the article, i upgraded my code to implement the tricks number 1 and 5.
This is how i implemented trick number 5:
Server (sender)
// sending loop ...
// file sent
shutdown(socket, SHUT_WR);
// wait acknowledgement from the client
ack = read(socket, buffer, bufsize);
if (ack < 0) {
// handle errors
}
Client (receiver)
// receiving loop..
if (filesize > 0) {
// incomplete transfer
// handle error
}
// send acknowledgement to the server
// this will send a FIN and trigger a read = 0 on the server
shutdown(socket, SHUT_WR);
close(socket);
What about tricks number 2, 3 and 4?
Trick number 2 is not needed because as soon as the server accepts the connection the application proceed to the file transfer. NO extra messages are exchanged.
Trick number 3 is already implemented
Trick number 4 is also already implemented. As mentioned earlier the file size is hardcoded, so there is no need to exchange it.
Did this solve my original problem?
NO my problem was not solved. The error is still happening, and as of today, it only happened when testing the application with both client and server on localhost.
What do you think?
Is there a way to prevent this?
You're:
assuming that read fills the buffer, even though
you're defending magnificently against write() not writing the entire buffer.
You need to do (1), and you don't need to do (2) because you're in blocking mode and POSIX assures that write() doesn't return until all the data is written.
A simple version of both loops:
while ((nread = read(inFD, buffer, 0, sizeof buffer)) > 0)
{
write(outFD, buffer, 0, nread);
}
if (nread == -1)
; // error
A more correct version would check the result of write() for errors of course.