This is my receive file function for a non overlapped socket.
HANDLE recvfile = CreateFile(fileinfo[0], FILE_APPEND_DATA, 0, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (recvfile == INVALID_HANDLE_VALUE) {
sockprintf(sockfd, "[Error Creating File] : %ld", GetLastError());
}
else {
memset(recvbuf, '\0', BUFFER); // Clear main buffer
int total = 0; // Total bytes received
// set_blocking_mode(sockfd, FALSE);
do{ // IF Total is equal to expected bytes. Break the loop, And stop receiving.
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = FALSE;
printf("[X] Connection interrupted while receiving file %s for %s size.", fileinfo[0], fileinfo[1]);
}
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL); // Write file data to file
total += fsize; // Add number of bytes received to total.
} while(total != expected);
// set_blocking_mode(sockfd, TRUE);
if (write == FALSE)
{
sockprintf(sockfd, "[Error Writing file %s of %s size] Error : %ld.", fileinfo[0], fileinfo[1], GetLastError());
}
else {
sockprintf(sockfd, "\n[ Received File : %s ]\n[ File Size : %s bytes ]\n[ Bytes written : %ld ]\n", fileinfo[0], fileinfo[1], dwBytesWritten);
}
CloseHandle(recvfile);
}
This is working excellent! File Transfer is almost instant and with no errors or corrupted buffers.
I'm on Visual studio 2019. The Problem is, When I compile and move the Executable to a test Virtual machine. The program crashes. Tested on Windows 10 and also Windows 7.
Crash occurs on the do while loop. The program does not break out of it. I have tried compiling with msvc (the latest) and also mingw-w64 and also tdm gcc. The bug is same.
The code only works on my own machine, It's not working on any other.
You need to check for more than just WSAECONNRESET. Pretty much every error (save for "would block" in non-blocking) is fatal and indicates the socket should be aborted. Don't forget that recv can and will return 0 if the remote side closes its end gracefully.
This whole loop:
do{ // IF Total is equal to expected bytes. Break the loop, And stop receiving.
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = FALSE;
printf("[X] Connection interrupted while receiving file %s for %s size.", fileinfo[0], fileinfo[1]);
}
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL); // Write file data to file
total += fsize; // Add number of bytes received to total.
} while(total != expected);
Should be:
do{
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR) {
int err = WSAGetLastError();
if (err == WSAEWOULDBLOCK) {
// ISSUE SELECT CALL HERE TO WAIT FOR MORE DATA
}
else {
// FATAL ERROR - abort
break;
}
}
else if (fsize == 0) {
break; // remote connection terminated cleanly.
}
else {
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL);
total += fsize;
}
} while(total != expected);
I have resolved the issue, IT seems the problem was my own ignorance, The server side was supplying the file information too quickly, And due to that, fileinfo[] was mixed up. And thus the loop not breaking. Still don't understand why it worked on the development machine tho. Anyway, As expected the problem was not in the Code that I have posted in this Question. Thanks ya'll!
The solution was to send the data with a small time sleep. :)
Related
I am studying Multithread while making a file transfer program.
Client Code(The code that sends the file)
// Opened a file using fopen function.
int SendPercent = 0; // Value that shows the current progress
while (1)
{
len = fread(buf, 1, BUFSIZE, fp);
if (len)
{
retval = send(sock, buf, len, 0);
if (retval == SOCKET_ERROR) {
err_display("send()");
break;
}
SendPercent += len;
printf("percent : %2.2f%%\r", (float)SendPercent / (float)AllLen * 100.0f);
}
else if (SendPercent == AllLen)
{
printf("\nFinished.\n");
break;
}
else {
printf("\nIt did not end successfully.\n");
break;
}
}
fclose(fp);
closesocket(sock);
WSACleanup();
return 0;
Server Code(The code that receives the file)
FILE* fp = fopen(filename, "wb");
while (1)
{
retval = recv(client_sock, buf, sizeof(buf), MSG_WAITALL);
if (retval == SOCKET_ERROR) {
err_display("recv()");
break;
}
else if (retval == 0) {
printf("\ncurLen : %f, allLen : %f\n", (float)curLen, (float)allLen);
break;
}
curLen += fwrite(buf, 1, retval, fp);
memset(&buf, 0, sizeof(buf));
printf("percent : %2.2f%%\r", (float)curLen / (float)allLen * 100.0f);
if (curLen == allLen)
{
printf("\nSuccessfully Received.\n");
break;
}
}
fclose(fp);
}
The server's receive function acts as multi-threaded through the CreateThread function.
It works fine when sending small files of 4KB size, but problems arise when sending large files of more than 77000KB.
When a problem occurs(server)
percent : 99.92%
curLen : 79141464.000000, allLen : 79202368.000000
[TCP Server] Disconnected. IP Address=127.0.0.1, Port Number=....
When a problem occurs(client)
percent : 100.00%
Finished.
The client disconnects before the curLen reaches allLen.
What's even more strange is that when I open a folder in the server program and look at the received files, the files have been copied successfully.
Is the file sent correctly?
Why isn't the server's transmission rate going up properly?
Obviously, this program worked normally when it was single thread.
Sadly, I have not even identified the cause of the problem before solving this problem. Please help me.
I am writing a program that a client can ask for files to a server. Then the server will send them in chunks of 512 bytes. The problem is that when the client read the file:
*Sometimes the first 512 bytes are different from the original file. The total read file also has a different size (and obviously it also ends different from the original file) and therefore the client loop that writes to the new file does never end.
*Sometimes it works perfectly and i don't know why.
Server:
/* Check if file exists */
if(access(retrFileName, F_OK) == 0){
/* Open file */
fd = open(retrFileName, O_RDONLY);
lseek(fd, 0, SEEK_SET);
if (fd == -1){
fprintf(stderr, "Error opening file --> %s", strerror(errno));
exit(EXIT_FAILURE);
}
/* Get file stats */
if (fstat(fd, &fileStat) < 0){
fprintf(stderr, "Error fstat --> %s", strerror(errno));
exit(EXIT_FAILURE);
}
sprintf(fileSize, "%li", fileStat.st_size);
/* Sending file data */
offset = 0;
remainData = fileStat.st_size;
while (((sentBytes = sendfile(clientSock, fd, &offset, 512)) == 512) && (remainData > 0)) {
remainData -= sentBytes;
fprintf(stdout, "Server envio %d bytes del file, offset ahora vale: %li y quedan = %d bytes\n", sentBytes, offset, remainData);
}
remainData -= sentBytes;
fprintf(stdout, "Server envio %d bytes del file, offset ahora vale: %li y quedan = %d bytes\n", sentBytes, offset, remainData);//do while
close(fd);////////////////////////
send(clientSock, NICETRANSFER, sizeof(NICETRANSFER), 0); //LO METE AL ARCHIVO
printf("send\n");
//close(clientSock);///////////
}
else{
send(clientSock, FILEERROR, sizeof(FILEERROR), 0);
printf("send\n");
}
}
Client:
/* Open file */
receivedFile = fopen("r.txt", "wb");
if (receivedFile == NULL){
fprintf(stderr, "Failed to open file --> %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
/* Write to the file */
int contador = 0;
int remainData = fileSize;
do{
if(remainData < 512)
bytesLeidos = recv(clientSock, readingBuffer, remainData, 0);
else
bytesLeidos = recv(clientSock, readingBuffer, 512, 0);
fwrite(readingBuffer, bytesLeidos, 1, receivedFile);
remainData -= 512;
contador += 512;
printf("bytesleidos: %li, contador: %d:\n%s\n\n", bytesLeidos, contador, readingBuffer);
}while(contador < fileSize);
fclose(receivedFile);
Golden rule of socket programming: Always check the return value from recv. It's not always what you think it will be.
Even though you "send" 512 bytes at a time, you are in no way guaranteed that TCP will deliver the same number of bytes at a time to the receiver. TCP segmentation, IP fragmentation, and general Internet weirdness will cause the recv side to get an arbitrary number of bytes at a time.
Hence, your hardcoded assumption that recv will always return 512 is incorrect:
remainData -= 512;
contador += 512;
Instead, you should be saying:
remainData -= bytesLeidos;
contador += bytesLeidos;
An you need to check for errors and socket closing too.
This is an improved main loop for your client code:
while (remainData > 0)
{
size_t recvSize = (remainData >= 512) ? 512 : remainData;
bytesLeidos = recv(clientSock, readingBuffer, recvSize, 0);
if (bytesLeidos > 0)
{
fwrite(readingBuffer, bytesLeidos, 1, receivedFile);
remainData -= bytesLeidos;
contador += bytesLeidos;
/* null terminate readingBuffer so garbage isn't printed.*/
/* Make sure readingBuffer is allocated to be at least */
/* N+1 bytes (513) to account for this character being appended. */
readingBuffer[bytesLeidos] = '\0';
printf("bytesleidos: %li, contador: %d:\n%s\n\n", bytesLeidos, contador, readingBuffer);
}
else if (bytesLeidos == 0)
{
/* remote side closed connection */
printf("Remote side exited connection\n");
break;
}
else if (bytesLeidos < 0)
{
/* connection error */
printf("Connection error\n");
break;
}
}
I solved my problem!! I needed to sync both client and server. To do so, the server send the size of the file and waits for an answer for the client with recv. When the client recieve the file size, it send a "" message.
I don't know if this is the correct solution, but this way you can sync server and client.
After sync, the server send the respective file normally with sendfile
I write program and it works fine, but i want to rewrite it using sendfile() and now i got stuck in a loop.
Server side:
send name = ok
send md5 checksum = ok
send size = ok
send file = ko
Client side:
recv name = ok
recv md5 cecksum = ok
recv size = ok
create dir and create file = ok
write data to created file = ko
P.S In previous version of program i stuck some time to, but it depend how much i use printf why? for e.x i add one line with printf program stuck, delete it, works fine.
UPDT: rewrite code client/server
client
/* Received file name */
int rc_byte = 0;
rc_byte = recv(fd, rx_tx_file->out_name, sizeof(rx_tx_file->out_name),0);
if (rc_byte < 0){
perror("Failed to receive file name: ");
exit(-1);
} else
printf("Recv out name %s\n", rx_tx_file->out_name);
//printf("file name rc %s\n", rx_tx_file->out_name);
trimm_path_name(rx_tx_file);
/* Received md5sum */
rc_byte = recv(fd, rx_tx_file->md5sum, sizeof(rx_tx_file->md5sum), 0);
if (rc_byte < 0) {
perror("Failed to receive check sum: ");
exit(-1);
} else
printf("recv md5s %s\n", rx_tx_file->md5sum);
/* Received file size */
rc_byte = recv(fd, &size, sizeof(size), 0);
if(rc_byte < 0) {
perror("Recevid size of file: ");
exit(-1);
}
printf("%d recv size\n", size);
to_read = size;
if (stat(dir, &st) == -1){
mkdir(dir, 0777);
}
send_data: (add func to server)
void send_data(int client_fd, m_file *rx_tx_file, int option, int size) {
int send_byte = 0;
int total_send = 0;
if (option == SEND_NAME) {
while (total_send < strlen(rx_tx_file->in_name)) {
send_byte = send(client_fd, rx_tx_file->in_name, sizeof(rx_tx_file->in_name),0);
if(send_byte == -1) {
perror("Failed to send file name to client: ");
exit(SEND_TO_CLIENT_ERROR);
}
total_send += send_byte;
}
}
else if (option == SEND_MD5) {
total_send = 0;
send_byte = 0;
while (total_send < strlen(rx_tx_file->md5sum)) {
send_byte = send(client_fd, rx_tx_file->md5sum, sizeof(rx_tx_file->md5sum),0);
if(send_byte == -1){
perror("Failed to send file md5sum to client: ");
exit(-1);
}
total_send += send_byte;
}
}
else if (option == SEND_SIZE) {
send_byte = send(client_fd, &size, sizeof(size),0);
if (send_byte == -1) {
perror("Failed to send size: ");
}
}
}
server:
client_fd = accept(server_fd, (struct sockaddr*) &client_addr, &length)
/*send name of file*/
send_data(client_fd, rx_tx_file, SEND_NAME, 0);
/*send md5 sum*/
take_check_sum(rx_tx_file,rx_tx_file->file_in, 0);
send_data(client_fd, rx_tx_file, SEND_MD5, 0);
/*send size of file*/
size = stats.st_size;
send_data(client_fd, rx_tx_file, SEND_SIZE, size);
remain_data = stats.st_size;
printf("File [%s] ready to send\ncheck sum [%s]\n", rx_tx_file->in_name,rx_tx_file->md5sum);
while (((send_byte = sendfile(client_fd, file_fd, &offset, size)) > 0) && (remain_data > 0))
{
remain_data -= send_byte;
printf("remain %d", remain_data);
}
printf("Succesfully");
Since i work with one client and pass file which should send on server side through command line args, i dont need to wait in while (client_fd = accpet) i just work with one connection and close server. Now its work good. But one question is open, how i should rewrite client side to recv data in a loop. I don't know which size i should recv and because of that i cant write right condition to my while loop. THX all for helping.
TCP is a stream. It has no message boundaries. Your code won't work because of that.
First, you send the name of the file:
send(client_fd, rx_tx_file->in_name, strlen(rx_tx_file->in_name)+1,0)
then you immediately send the md5 sum and then the file size:
send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0)
send(client_fd, &size, sizeof(int),0)
Since the first two strings don't have a fixed number of bytes, it's quite likely that when you try to read the file size or md5 sum from the server you also read the size of the file and maybe even some of the file data.
First, stop trying to put as much of your send and read code as you can into the conditional clause of your if and while statements.
What exactly does
if (send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0) == -1) {
perror("Failed to send file md5sum to client: ");
exit(-1);
}
gain you over
ssize_t bytes_sent = send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0);
if ( bytes_sent < 0 )
{
perror("Failed to send file md5sum to client: ");
exit(-1);
}
Putting all that code into the if clause gains you nothing on the send. And what if strlen(rx_tx_file->md5sum)+1 is 87 and the send() call returns 15? That's a possible return value that your code can't handle because it stuffs everything into the if clause.
ssize_t bytes_sent = send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0);
if ( bytes_sent < 0 )
{
perror("Failed to send file md5sum to client: ");
exit(-1);
}
else if ( bytes_sent < strlen(rx_tx_file->md5sum)+1 )
{
// partial send...
}
That's actually better coded as a loop.
You didn't post your receive code, but if it's in the same style you not only don't gain anything, by putting everything into the if clause you again can't do any decent error detection or correction.
If your file name recv code is similar to
char filename[1024];
if (recv(fd, &filename, sizeof(filename), 0) < 0) {
perror("Failed to read file name: ");
exit(-1);
}
you can't tell what you just received. How many bytes did you just receive? You may have received the file name. You may have received only part of the file name. You may have received the file name, the md5 sum, and some of the file contents itself.
You don't know what you received, and with your code you can't tell. If you zero out the file name and md5 receive buffers and only recv up to one byte less than the size of the buffer, you at least avoid undefined behavior. But if you don't zero out the buffer, or if you read up the the last byte of the buffer, you can also wind up without a nul-terminated string for your filename or md5 sum. And when you then try to treat it as a nul-terminated string you get undefined behavior.
And if you did get extra bytes in the recv calls you make before trying to read the file data, that explains why your code gets stuck - it already read some of the file contents before getting to the loop, so the loop will never see all the content - some of it is gone.
You should avoid using strlen here in your server:
if(send(client_fd, rx_tx_file->in_name, strlen(rx_tx_file->in_name)+1,0) == -1)
Rather just send fixed length string of size sizeof(rx_tx_file->out_name) as you expect in your client
If the filename is smaller just pad it with spaces to make it of length sizeof(rx_tx_file->out_name)
You should also put each receive call in while loop, and add checks that it actually received expected number of bytes, at times recv will just return partial data, you need to post another recv to receive rest of the expected data
I have client server architecture wherein I transfer a binary file from server to client.
Server Code:
FILE *outFile = fopen("/tmp/localpkg.deb","wb");
if( outFile == NULL )
{
RPTLOG(1, "Error opening file for writing.\n");
return -1;
}
FILE *readFilefp = fopen(file_name, "rb");
if ( readFilefp == NULL)
{
RPTLOG(1, "Error opening file.\n");
return -1;
}
char fileBuffer[1024];
bzero(fileBuffer, 1024);
while(1)
{
size_t readSize = fread(fileBuffer, sizeof(char), sizeof(fileBuffer) -1, readFilefp);
if(readSize == 0)
{
RPTLOG(1, "No more contents are left in file.\n");
n = write(sockfd, "\r\n", 2);
break;
}
n = write(sockfd, fileBuffer,readSize);
fwrite(&fileBuffer, sizeof(char), readSize, outFile);
RPTLOG(1, "Data[%d] written to sock=%s.\n", n, fileBuffer);
bzero(fileBuffer, 1024);
}
fclose(readFilefp);
fclose(outFile);
char *endfile_var =(char*) malloc(2048);
bzero(endfile_var,100);
strcpy(endfile_var,"ENDFILE\r\n");
n = write(sockfd, endfile_var, 9 );
RPTLOG(1, "ENDFILE text sent to client[%s] NumBytes sent=%d\n", endfile_var, n );
Client code:
FILE *fp = fopen(localfile, "wb");
if( fp == NULL )
{
RPTLOG(1, "Not enough permissions to write on disk, exiting....\n");
break;
}
memset(buf, 0, 2048);
//Started receiving installation package from server
while ((ret = read(sock, buf, 2047)) > 0) //Stcuk point: read blocks over here when it is about to receive last few lines of
{ //binary file from server
buf[ret] = '\0';
if ( ret == 1 )
{
RPTLOG(1, "Caught character where ret = %d\n", ret);
if ( buf[0] == '\n' )
continue;
}
if (strstr(buf, "ENDFILE") != 0 )
{
RPTLOG(1, "Endfile detected\n");
break;
}
else
{
fwrite(buf, 1, ret, fp);
}
memset(buf, 0, 2048);
}
if( ret == 0)
{
RPTLOG(4, "Connection closed from server = %d \n", ret );
}
else if( ret < 0)
{
RPTLOG(4, "Read error on client socket= %d \n", ret );
}
fclose(fp);
My Problem:
When server is about to send last few lines of binary file, client stuck in read call [stuck point in client code posted above]. Here client program is designed in such a way that when it receives "ENDFILE" line from server it will assume that file contents have been ended from server side and come out of while loop.
So kindly suggest the solution to receive binary file from server successfully.
[Special Note:] Client code has been wrapped up in a build and installed at customer end. So I have to make changes in server side code only. Also as indicated in server code above, for debugging purpose I have written content sent on socket to '/tmp/localpkg.deb' file as well. And that file contains all the contents which were written on socket at server side.
[Special Note 2:] When I try to send plain text file, I can send it successfully. Problem comes in only while sending binary file.
Thanks in advance.
The reason why your code fails is simple enough. You are reading a chunk of binary data into your buf, and than you call strstr on this buf. strstr goes until first \0 byte when looking for given token. Obviously, binary data will have a lot of those, so search stops after first one encountered and returns 'nothing was found'.
As a result, your code never exits the read loop, and keeps waiting for more data to arrive from the server - which will never send it.
The proper solution is, of course, implement a proper protocol for sending binary data. However, if this is impossible due to the listed constraints, a semi-workable solution would be to to replace strstr function with a custom one, which will go over provided buffer ignoring the the nul-terminators (using size of the buffer instead) and looks for the token provided. It will still break if your binary data has the token in it, but there is nothing you can do about it.
What im trying to do is connect to a remote server , read contents from a file on the local machine and send it over to the server. Then capture the server response and save it. I put the GET command in a text file and am trying get the results of the same. Here is some part of the code. Im doing this using sockets and C.
if ( inet_pton(AF_INET,ip, &(nc_args->destaddr.sin_addr.s_addr)) <= 0 )
printf("\n\t\t inet_pton error");
if (connect(sockfd, (struct sockaddr *) &nc_args->destaddr, sizeof(&nc_args->destaddr)) < 0)
{
printf("\n\t\t Connection error");
exit(1);
}
puts("\n\t\t Connection successful to ...");
// file parameter is taken from command line and passéd to this function
fp = fopen(file,"rb");
if ( fp == NULL)
{
printf("\n\t\t File not found");
exit(3);
}
else
{
printf("\n\t\t Found file %s\n", file);
fseek(fp, 0, SEEK_END);
file_size = ftell(fp);
rewind(fp);
//allocate memory to the buffer dynamically
buffer = (char*) malloc (sizeof(char)*file_size);
if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}
for (i=0 ; i<sizeof(buffer); i++)
{
printf("\n\t\t %s", buffer);
}
printf("\n\t\t File contains %ld bytes!\n", file_size);
printf("\n\t\t Sending the file now");
}
while (1)
{
bytes_read = fread(buffer,1, file_size, fp);
printf("\n\t\t The bytes read is %zd", bytes_read);
if (bytes_read == 0) // We're done reading from the file
{
printf("\n\t\t The bytes read is %zd", bytes_read);
break;
}
if (bytes_read < 0)
{
printf("\n\t\t ERROR reading from file");
}
void *p = buffer;
while (bytes_read > 0)
{
ssize_t bytes_written = send(sockfd, buffer, bytes_read,0);
if (bytes_written <= 0)
{
printf("\n\t\t ERROR writing to socket\n");
}
bytes_read -= bytes_written;
p += bytes_written;
printf("\n\t\t Bytes %zd written", bytes_written);
}
}
printf("\n\n\t\t Sending complete.");
What is happening here is that i get the message "connection successful", then it displays "sending the file now" and then the program quits unexpectedly. if i do echo $? i get 141 as the exit code. I am trying to connect from my server to a different server at work and get the results. These two can communicate correctly, and i can run the GET command from command line without issues. Its just not working from the code. Can someone let me know what the issue could be ?
On Linux, and probably other Unixes, the return code encodes a signal that the process received. Here it is 141 - 128 so 13 which corresponds to SIGPIPE.
If you don't want that signal to be raised because you capture the error return of send, anyhow, on Linux you can use MSG_NOSIGNAL in the flags argument to send to inhibit that signal. On other platforms you might have to program more complicated signal handlers to deal with that situation.
sizeof(&nc_args->destaddr) is the wrong thing to pass to connect. It wants the size of the address, not the size of a pointer to the address.
And this loop:
for (i=0 ; i<sizeof(buffer); i++)
{
printf("\n\t\t %s", buffer);
}
is baffling. buffer is a pointer, as we can see from when it was assigned a vlue returned by malloc. So its size is going to be 4 or 8 bytes on 32-bit and 64-bit architectures respectively; not related to the size of the malloc'ed object it points to. The loop runs 4 or 8 times, and prints... the same thing each time. Why?