I am studying Multithread while making a file transfer program.
Client Code(The code that sends the file)
// Opened a file using fopen function.
int SendPercent = 0; // Value that shows the current progress
while (1)
{
len = fread(buf, 1, BUFSIZE, fp);
if (len)
{
retval = send(sock, buf, len, 0);
if (retval == SOCKET_ERROR) {
err_display("send()");
break;
}
SendPercent += len;
printf("percent : %2.2f%%\r", (float)SendPercent / (float)AllLen * 100.0f);
}
else if (SendPercent == AllLen)
{
printf("\nFinished.\n");
break;
}
else {
printf("\nIt did not end successfully.\n");
break;
}
}
fclose(fp);
closesocket(sock);
WSACleanup();
return 0;
Server Code(The code that receives the file)
FILE* fp = fopen(filename, "wb");
while (1)
{
retval = recv(client_sock, buf, sizeof(buf), MSG_WAITALL);
if (retval == SOCKET_ERROR) {
err_display("recv()");
break;
}
else if (retval == 0) {
printf("\ncurLen : %f, allLen : %f\n", (float)curLen, (float)allLen);
break;
}
curLen += fwrite(buf, 1, retval, fp);
memset(&buf, 0, sizeof(buf));
printf("percent : %2.2f%%\r", (float)curLen / (float)allLen * 100.0f);
if (curLen == allLen)
{
printf("\nSuccessfully Received.\n");
break;
}
}
fclose(fp);
}
The server's receive function acts as multi-threaded through the CreateThread function.
It works fine when sending small files of 4KB size, but problems arise when sending large files of more than 77000KB.
When a problem occurs(server)
percent : 99.92%
curLen : 79141464.000000, allLen : 79202368.000000
[TCP Server] Disconnected. IP Address=127.0.0.1, Port Number=....
When a problem occurs(client)
percent : 100.00%
Finished.
The client disconnects before the curLen reaches allLen.
What's even more strange is that when I open a folder in the server program and look at the received files, the files have been copied successfully.
Is the file sent correctly?
Why isn't the server's transmission rate going up properly?
Obviously, this program worked normally when it was single thread.
Sadly, I have not even identified the cause of the problem before solving this problem. Please help me.
Related
This is my receive file function for a non overlapped socket.
HANDLE recvfile = CreateFile(fileinfo[0], FILE_APPEND_DATA, 0, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (recvfile == INVALID_HANDLE_VALUE) {
sockprintf(sockfd, "[Error Creating File] : %ld", GetLastError());
}
else {
memset(recvbuf, '\0', BUFFER); // Clear main buffer
int total = 0; // Total bytes received
// set_blocking_mode(sockfd, FALSE);
do{ // IF Total is equal to expected bytes. Break the loop, And stop receiving.
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = FALSE;
printf("[X] Connection interrupted while receiving file %s for %s size.", fileinfo[0], fileinfo[1]);
}
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL); // Write file data to file
total += fsize; // Add number of bytes received to total.
} while(total != expected);
// set_blocking_mode(sockfd, TRUE);
if (write == FALSE)
{
sockprintf(sockfd, "[Error Writing file %s of %s size] Error : %ld.", fileinfo[0], fileinfo[1], GetLastError());
}
else {
sockprintf(sockfd, "\n[ Received File : %s ]\n[ File Size : %s bytes ]\n[ Bytes written : %ld ]\n", fileinfo[0], fileinfo[1], dwBytesWritten);
}
CloseHandle(recvfile);
}
This is working excellent! File Transfer is almost instant and with no errors or corrupted buffers.
I'm on Visual studio 2019. The Problem is, When I compile and move the Executable to a test Virtual machine. The program crashes. Tested on Windows 10 and also Windows 7.
Crash occurs on the do while loop. The program does not break out of it. I have tried compiling with msvc (the latest) and also mingw-w64 and also tdm gcc. The bug is same.
The code only works on my own machine, It's not working on any other.
You need to check for more than just WSAECONNRESET. Pretty much every error (save for "would block" in non-blocking) is fatal and indicates the socket should be aborted. Don't forget that recv can and will return 0 if the remote side closes its end gracefully.
This whole loop:
do{ // IF Total is equal to expected bytes. Break the loop, And stop receiving.
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR && WSAGetLastError() == WSAECONNRESET)
{
connected = FALSE;
printf("[X] Connection interrupted while receiving file %s for %s size.", fileinfo[0], fileinfo[1]);
}
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL); // Write file data to file
total += fsize; // Add number of bytes received to total.
} while(total != expected);
Should be:
do{
fsize = recv(sockfd, recvbuf, BUFFER, 0); // Receive file
if (fsize == SOCKET_ERROR) {
int err = WSAGetLastError();
if (err == WSAEWOULDBLOCK) {
// ISSUE SELECT CALL HERE TO WAIT FOR MORE DATA
}
else {
// FATAL ERROR - abort
break;
}
}
else if (fsize == 0) {
break; // remote connection terminated cleanly.
}
else {
write = WriteFile(recvfile, recvbuf, fsize, &dwBytesWritten, NULL);
total += fsize;
}
} while(total != expected);
I have resolved the issue, IT seems the problem was my own ignorance, The server side was supplying the file information too quickly, And due to that, fileinfo[] was mixed up. And thus the loop not breaking. Still don't understand why it worked on the development machine tho. Anyway, As expected the problem was not in the Code that I have posted in this Question. Thanks ya'll!
The solution was to send the data with a small time sleep. :)
I am writing a program that a client can ask for files to a server. Then the server will send them in chunks of 512 bytes. The problem is that when the client read the file:
*Sometimes the first 512 bytes are different from the original file. The total read file also has a different size (and obviously it also ends different from the original file) and therefore the client loop that writes to the new file does never end.
*Sometimes it works perfectly and i don't know why.
Server:
/* Check if file exists */
if(access(retrFileName, F_OK) == 0){
/* Open file */
fd = open(retrFileName, O_RDONLY);
lseek(fd, 0, SEEK_SET);
if (fd == -1){
fprintf(stderr, "Error opening file --> %s", strerror(errno));
exit(EXIT_FAILURE);
}
/* Get file stats */
if (fstat(fd, &fileStat) < 0){
fprintf(stderr, "Error fstat --> %s", strerror(errno));
exit(EXIT_FAILURE);
}
sprintf(fileSize, "%li", fileStat.st_size);
/* Sending file data */
offset = 0;
remainData = fileStat.st_size;
while (((sentBytes = sendfile(clientSock, fd, &offset, 512)) == 512) && (remainData > 0)) {
remainData -= sentBytes;
fprintf(stdout, "Server envio %d bytes del file, offset ahora vale: %li y quedan = %d bytes\n", sentBytes, offset, remainData);
}
remainData -= sentBytes;
fprintf(stdout, "Server envio %d bytes del file, offset ahora vale: %li y quedan = %d bytes\n", sentBytes, offset, remainData);//do while
close(fd);////////////////////////
send(clientSock, NICETRANSFER, sizeof(NICETRANSFER), 0); //LO METE AL ARCHIVO
printf("send\n");
//close(clientSock);///////////
}
else{
send(clientSock, FILEERROR, sizeof(FILEERROR), 0);
printf("send\n");
}
}
Client:
/* Open file */
receivedFile = fopen("r.txt", "wb");
if (receivedFile == NULL){
fprintf(stderr, "Failed to open file --> %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
/* Write to the file */
int contador = 0;
int remainData = fileSize;
do{
if(remainData < 512)
bytesLeidos = recv(clientSock, readingBuffer, remainData, 0);
else
bytesLeidos = recv(clientSock, readingBuffer, 512, 0);
fwrite(readingBuffer, bytesLeidos, 1, receivedFile);
remainData -= 512;
contador += 512;
printf("bytesleidos: %li, contador: %d:\n%s\n\n", bytesLeidos, contador, readingBuffer);
}while(contador < fileSize);
fclose(receivedFile);
Golden rule of socket programming: Always check the return value from recv. It's not always what you think it will be.
Even though you "send" 512 bytes at a time, you are in no way guaranteed that TCP will deliver the same number of bytes at a time to the receiver. TCP segmentation, IP fragmentation, and general Internet weirdness will cause the recv side to get an arbitrary number of bytes at a time.
Hence, your hardcoded assumption that recv will always return 512 is incorrect:
remainData -= 512;
contador += 512;
Instead, you should be saying:
remainData -= bytesLeidos;
contador += bytesLeidos;
An you need to check for errors and socket closing too.
This is an improved main loop for your client code:
while (remainData > 0)
{
size_t recvSize = (remainData >= 512) ? 512 : remainData;
bytesLeidos = recv(clientSock, readingBuffer, recvSize, 0);
if (bytesLeidos > 0)
{
fwrite(readingBuffer, bytesLeidos, 1, receivedFile);
remainData -= bytesLeidos;
contador += bytesLeidos;
/* null terminate readingBuffer so garbage isn't printed.*/
/* Make sure readingBuffer is allocated to be at least */
/* N+1 bytes (513) to account for this character being appended. */
readingBuffer[bytesLeidos] = '\0';
printf("bytesleidos: %li, contador: %d:\n%s\n\n", bytesLeidos, contador, readingBuffer);
}
else if (bytesLeidos == 0)
{
/* remote side closed connection */
printf("Remote side exited connection\n");
break;
}
else if (bytesLeidos < 0)
{
/* connection error */
printf("Connection error\n");
break;
}
}
I solved my problem!! I needed to sync both client and server. To do so, the server send the size of the file and waits for an answer for the client with recv. When the client recieve the file size, it send a "" message.
I don't know if this is the correct solution, but this way you can sync server and client.
After sync, the server send the respective file normally with sendfile
I write program and it works fine, but i want to rewrite it using sendfile() and now i got stuck in a loop.
Server side:
send name = ok
send md5 checksum = ok
send size = ok
send file = ko
Client side:
recv name = ok
recv md5 cecksum = ok
recv size = ok
create dir and create file = ok
write data to created file = ko
P.S In previous version of program i stuck some time to, but it depend how much i use printf why? for e.x i add one line with printf program stuck, delete it, works fine.
UPDT: rewrite code client/server
client
/* Received file name */
int rc_byte = 0;
rc_byte = recv(fd, rx_tx_file->out_name, sizeof(rx_tx_file->out_name),0);
if (rc_byte < 0){
perror("Failed to receive file name: ");
exit(-1);
} else
printf("Recv out name %s\n", rx_tx_file->out_name);
//printf("file name rc %s\n", rx_tx_file->out_name);
trimm_path_name(rx_tx_file);
/* Received md5sum */
rc_byte = recv(fd, rx_tx_file->md5sum, sizeof(rx_tx_file->md5sum), 0);
if (rc_byte < 0) {
perror("Failed to receive check sum: ");
exit(-1);
} else
printf("recv md5s %s\n", rx_tx_file->md5sum);
/* Received file size */
rc_byte = recv(fd, &size, sizeof(size), 0);
if(rc_byte < 0) {
perror("Recevid size of file: ");
exit(-1);
}
printf("%d recv size\n", size);
to_read = size;
if (stat(dir, &st) == -1){
mkdir(dir, 0777);
}
send_data: (add func to server)
void send_data(int client_fd, m_file *rx_tx_file, int option, int size) {
int send_byte = 0;
int total_send = 0;
if (option == SEND_NAME) {
while (total_send < strlen(rx_tx_file->in_name)) {
send_byte = send(client_fd, rx_tx_file->in_name, sizeof(rx_tx_file->in_name),0);
if(send_byte == -1) {
perror("Failed to send file name to client: ");
exit(SEND_TO_CLIENT_ERROR);
}
total_send += send_byte;
}
}
else if (option == SEND_MD5) {
total_send = 0;
send_byte = 0;
while (total_send < strlen(rx_tx_file->md5sum)) {
send_byte = send(client_fd, rx_tx_file->md5sum, sizeof(rx_tx_file->md5sum),0);
if(send_byte == -1){
perror("Failed to send file md5sum to client: ");
exit(-1);
}
total_send += send_byte;
}
}
else if (option == SEND_SIZE) {
send_byte = send(client_fd, &size, sizeof(size),0);
if (send_byte == -1) {
perror("Failed to send size: ");
}
}
}
server:
client_fd = accept(server_fd, (struct sockaddr*) &client_addr, &length)
/*send name of file*/
send_data(client_fd, rx_tx_file, SEND_NAME, 0);
/*send md5 sum*/
take_check_sum(rx_tx_file,rx_tx_file->file_in, 0);
send_data(client_fd, rx_tx_file, SEND_MD5, 0);
/*send size of file*/
size = stats.st_size;
send_data(client_fd, rx_tx_file, SEND_SIZE, size);
remain_data = stats.st_size;
printf("File [%s] ready to send\ncheck sum [%s]\n", rx_tx_file->in_name,rx_tx_file->md5sum);
while (((send_byte = sendfile(client_fd, file_fd, &offset, size)) > 0) && (remain_data > 0))
{
remain_data -= send_byte;
printf("remain %d", remain_data);
}
printf("Succesfully");
Since i work with one client and pass file which should send on server side through command line args, i dont need to wait in while (client_fd = accpet) i just work with one connection and close server. Now its work good. But one question is open, how i should rewrite client side to recv data in a loop. I don't know which size i should recv and because of that i cant write right condition to my while loop. THX all for helping.
TCP is a stream. It has no message boundaries. Your code won't work because of that.
First, you send the name of the file:
send(client_fd, rx_tx_file->in_name, strlen(rx_tx_file->in_name)+1,0)
then you immediately send the md5 sum and then the file size:
send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0)
send(client_fd, &size, sizeof(int),0)
Since the first two strings don't have a fixed number of bytes, it's quite likely that when you try to read the file size or md5 sum from the server you also read the size of the file and maybe even some of the file data.
First, stop trying to put as much of your send and read code as you can into the conditional clause of your if and while statements.
What exactly does
if (send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0) == -1) {
perror("Failed to send file md5sum to client: ");
exit(-1);
}
gain you over
ssize_t bytes_sent = send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0);
if ( bytes_sent < 0 )
{
perror("Failed to send file md5sum to client: ");
exit(-1);
}
Putting all that code into the if clause gains you nothing on the send. And what if strlen(rx_tx_file->md5sum)+1 is 87 and the send() call returns 15? That's a possible return value that your code can't handle because it stuffs everything into the if clause.
ssize_t bytes_sent = send(client_fd, rx_tx_file->md5sum, strlen(rx_tx_file->md5sum)+1, 0);
if ( bytes_sent < 0 )
{
perror("Failed to send file md5sum to client: ");
exit(-1);
}
else if ( bytes_sent < strlen(rx_tx_file->md5sum)+1 )
{
// partial send...
}
That's actually better coded as a loop.
You didn't post your receive code, but if it's in the same style you not only don't gain anything, by putting everything into the if clause you again can't do any decent error detection or correction.
If your file name recv code is similar to
char filename[1024];
if (recv(fd, &filename, sizeof(filename), 0) < 0) {
perror("Failed to read file name: ");
exit(-1);
}
you can't tell what you just received. How many bytes did you just receive? You may have received the file name. You may have received only part of the file name. You may have received the file name, the md5 sum, and some of the file contents itself.
You don't know what you received, and with your code you can't tell. If you zero out the file name and md5 receive buffers and only recv up to one byte less than the size of the buffer, you at least avoid undefined behavior. But if you don't zero out the buffer, or if you read up the the last byte of the buffer, you can also wind up without a nul-terminated string for your filename or md5 sum. And when you then try to treat it as a nul-terminated string you get undefined behavior.
And if you did get extra bytes in the recv calls you make before trying to read the file data, that explains why your code gets stuck - it already read some of the file contents before getting to the loop, so the loop will never see all the content - some of it is gone.
You should avoid using strlen here in your server:
if(send(client_fd, rx_tx_file->in_name, strlen(rx_tx_file->in_name)+1,0) == -1)
Rather just send fixed length string of size sizeof(rx_tx_file->out_name) as you expect in your client
If the filename is smaller just pad it with spaces to make it of length sizeof(rx_tx_file->out_name)
You should also put each receive call in while loop, and add checks that it actually received expected number of bytes, at times recv will just return partial data, you need to post another recv to receive rest of the expected data
I have written the following code for sending file from a client:
FILE *fp = fopen("video.mp4","rb");
if(fp==NULL)
{
printf("File opern error");
return 1;
}
int bytesToWrite=84440670;
int bytesWritten=0;
while(bytesWritten<bytesToWrite)
{
int m=minimum(256,bytesToWrite-bytesWritten);
/* First read file in chunks of 256 bytes or the remaining bytes*/
unsigned char *buff=malloc(sizeof(char)*m);
bzero(buff, m);
int nread = fread(buff,1,m,fp);
printf("Bytes read %d \n", nread);
bytesWritten+=m;
/* If read was success, send data. */
if(nread > 0)
{
printf("Sending \n");
write(sockfd, buff, nread);
}
/*
* There is something tricky going on with read ..
* Either there was error, or we reached end of file.
*/
if (nread < 256)
{
if (feof(fp))
printf("End of file\n");
if (ferror(fp))
printf("Error reading\n");
break;
}
}
and the following code for receiving by the server :
FILE *fp;
fp = fopen("receive.mp4", "wb");
if(NULL == fp)
{
printf("Error opening file");
return 1;
}
int bytesToRead=84440670;
/* Receive data in chunks of 256 or remaining bytes */
int totalbytesread=0;
while(totalbytesread < bytesToRead)
{
int m = minimum(256,bytesToRead-totalbytesread);
char *recvBuff=malloc(sizeof(char)*m);
bzero(recvBuff,m);
bytesReceived = read(newsocfd, recvBuff,m );
printf("Bytes received %d\n",bytesReceived);
totalbytesread += bytesReceived;
fwrite(recvBuff, 1,bytesReceived,fp);
}
if(bytesReceived < 0)
{
printf("\n Read Error \n");
}
The sending is working correctly for .txt files ,some data is being lost in sending other formats(for e.g some ending seconds of .mp4 files is not being sent).
I am new to socket programming. Any help would be deeply appreciated.
You are missing a fclose(fp); in the server after you have finished receiving. You can also use fflush(fp); to just flush the contents.
Besides, there are several memory leaks and improper error handling.
I have a simple server program written in C, and the program is running on an Ubuntu Linux distribution. The program is intended to listen for messages sent from the client, write those messages to a file (each message goes into a separate file), and send an acknowledgement back to the client once the message has been received and stored.
I have noticed that, as the server continues to receive and store messages, the amount of available system memory quickly decreases and continues to decrease until messages have stopped. Memory remains constant when no messages are being sent. However, I have also noticed that I can free up the memory again by deleting the written files from disk (I can do this even while the server is still running). I am therefore led to believe that the memory issue has something to do with my file operations, though I can't see any issue with the code that writes the files.
Can someone help?
NOTE: I am observing the memory usage with "top".
I have included an excerpt from the program. The below function handles input from the client and writes that information to file. This is where I currently believe the problem to be:
void handleinput (int sock)
{
char filename[strlen(tempfolder) + 27];
generatefilename(filename);
int rv;
int n = 1;
int received = 0;
char buffer[BUFFER_SIZE];
FILE *p = NULL;
fd_set set;
char response[768];
struct timeval timeout;
timeout.tv_sec = 360;
timeout.tv_usec = 0;
FD_ZERO(&set);
FD_SET(sock, &set);
bzero(buffer, BUFFER_SIZE);
bzero(response, sizeof response);
rv = select(sock + 1, &set, NULL, NULL, &timeout);
if (rv == -1)
{
error("error on select in handleinput");
close(sock);
exit(1);
}
else if (rv == 0)
{
close(sock);
exit(0);
}
else
{
n = read(sock, buffer, BUFFER_SIZE-1);
if (n <= 0)
{
close(sock);
exit(0);
}
}
// open file
if (n != 0)
{
p = fopen(filename, "a");
if (p == NULL)
{
error("ERROR writing message to file");
close(sock);
exit(1);
}
}
// loop until full message is received
while (n != 0)
{
if (n < 0)
{
error("ERROR reading from socket");
close(sock);
exit(1);
}
received = 1;
// write content to file
fwrite(buffer, strlen(buffer), 1, p);
if (buffer[strlen(buffer)-1] == 0x1c)
{
break;
}
bzero(buffer, BUFFER_SIZE);
rv = select(sock + 1, &set, NULL, NULL, &timeout);
if (rv == -1)
{
error("ERROR select in loop in handleinput");
close(sock);
exit(1);
}
else if (rv == 0)
{
close(sock);
exit(0);
}
else
{
n = read(sock, buffer, BUFFER_SIZE-1);
}
}
// close file if we opened it earlier
if (p != NULL)
{
fclose(p);
}
// send acknowledgement back to client
if (received == 1)
{
generateResponse(response, filename);
n = write(sock, response, strlen(response));
if (n < 0)
{
error("ERROR writing to socket");
close(sock);
exit(1);
}
}
}
Its because of the caching mechanism of writing. IF you have a lot of clients trying to write files, the IO buffer fills in kernel memory, but doesn't actually write to that file until you close the socket or the buffer fills. You can fix this just by flushing the buffer. What some other people have suggested is to get rid of the write and read wrappers in stdio, and just use the kernel call write, since this will help performance and will probably flush the buffer automatically.
You can flush with fsync() btw.