I have been working on how to transfer an image using UDP in C, I have created a code that sometimes works, sometimes it doesn't. In what I think the issue is that sometimes the server receives more packages than writes. I know that I am trying to create the TCP, but that is what I am kind looking for, but not sure how to do it.
I think to fix it the client should send the buff of the img and only sends the second part when the server reply back to the client.
Here is the code:
Client:
while (!feof(p))
{
fread(*&c, 1, BLEN, p);
sprintf(buf, "%s", *&c);
temp=sendto(s,buf,BLEN, 0, (struct sockaddr *) &si_other, slen);
//sleep(3);
//printf("%d ",temp);
if(temp < 0)
{
fprintf(stderr,"sendto error.\n");
printf("erro");
exit(1);
}
i++;
}
Server:
while(1){
if(recvfrom(s, buf, BLEN, 0, (struct sockaddr *) &si_other, (unsigned int *) &slen)==-1){
perror("recvfrom error.\n");
exit(1);
}
//printf("%s ", &si_other);
flagr[0] = buf[0];
flagr[1] = buf[1];
flagr[2] = buf[2];
if (strcmp(flagr, flag) == 0 ){
break;
}
fwrite(buf, 1, BLEN, pp);
i++;
}
UDP is a datagram protocol, meaning that each call to sendto sends one message. If that message is larger than an IP packet can hold, it will be fragmented across multiple IP datagrams. If any one of those fragments fails to arrive, the whole thing is dropped at the OS level.
The data needs to be sent in chunks of no more than about 1450 bytes. Then the receiving side will need to read each packet and, because UDP does not guarantee that data will arrive in order, you will need to reassemble them in the proper order.
That means each packet has to have a user-defined header which contains the sequence number so that the receiver knows what order to put them in.
You also need to worry about retransmissions, since UDP doesn't guarantee that a packet that is send is actually received.
There's a program I wrote called UFTP which does all of this. Take a look at the documentation and code to get an idea of what you need to do to implement reliable data transfer over UDP.
Related
When I send() and recv() data from my program locally it works fine.
However, on my remote server, the same program, which usually receives data in chunks of 4096, will receive in buffers capped at 1428, which rarely jump above this number.
Worse of all, after a minute or so of transferring data the socket just freezes and stops execution, and the program perpetually stays in this frozen state, like so:
Received: 4096
Received: 4096
Received: 3416
The server is simple, it accepts a connection from a client and receives data in chunks of 4096, which works absolutely fine locally, but on my remote server it is failing consistently, unless I only send a small chunk of data (sending 1000 byte files worked fine).
int main()
{
while(1){
int servSock = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);
if(servSock < 0){
fprintf(stderr, "Socket error.\n");
continue;
}
struct sockaddr_in servAddr;
memset(&servAddr, 0, sizeof(servAddr));
servAddr.sin_family = AF_INET;
servAddr.sin_addr.s_addr = htonl(INADDR_ANY);
servAddr.sin_port = htons(atoi(STANDARD_PORT));
if(bind(servSock, (struct sockaddr*) &servAddr, sizeof(servAddr)) < 0){
fprintf(stderr, "Bind error.\n");
close(servSock);
continue;
}
if(listen(servSock, BACKLOG) < 0){
fprintf(stderr, "Listen error.\n");
close(servSock);
continue;
}
printf("%s", "Listening on socket for incoming connections.\n");
struct sockaddr_in clntAddr;
socklen_t clntAddrLen = sizeof(clntAddr);
while(1) {
int newsock = accept(servSock, (struct sockaddr*) &clntAddr, &clntAddrLen);
if(newsock < 0){
fprintf(stderr, "Accept connection error");
return 1;
continue;
}
char clntName[INET_ADDRSTRLEN];
if (inet_ntop(AF_INET, &clntAddr.sin_addr.s_addr, clntName, sizeof(clntName)) != NULL)
printf("Handling client %s:%d\n", clntName, ntohs(clntAddr.sin_port));
char file[17];
memset(file, 0, 17);
int recvd = recv(newsock, file, 16, 0);
file[17] = '\0';
char local_file_path[200];
memset(local_file_path, 0, 200);
strcat(local_file_path, "/home/");
strcat(local_file_path, file);
printf(local_file_path);
FILE* fp = fopen(local_file_path, "wb");
char buffer[4096];
while(1)
{
memset(buffer, 0, 4096);
recvd = recv(newsock, buffer, 4096, 0);
printf("Received: %d\n", recvd);
fwrite(buffer, sizeof(char), recvd, fp);
if(recvd == -1 || recvd == 0) {
fclose(fp);
break;
} else
}
close(newsock);
}
close(servSock);
}
return 1;
}
EDIT: For more context, this is a Windows server I am adapting to linux. Perhaps the recv() call is blocking when it shouldn't be, I'm going to test with flags.
However, on my remote server, the same program, which usually receives data in chunks of 4096, will receive in buffers capped at 1428, which rarely jump above this number.
Insufficient context has been presented for confidence, but that looks like a plausible difference between a socket whose peer is on the same machine (one connected to localhost, for example) and one whose peer is physically separated from it by an ethernet network. The 1428 is pretty close to the typical MTU for such a network, and you have to allow space for protocol headers.
Additionally, you might be seeing that one system coallesces the payloads from multiple transport-layer packets more or differently than the other does, for any of a variety of reasons.
In any case, at the userspace level, the difference in transfer sizes for a stream socket is not semantically meaningful. In particular, you cannot rely upon one end of the connection to read data in the same size chunks that the other sends it. Nor can you necessarily rely on receiving data in full-buffer units, regardless of the total amount being transferred or the progress of the transfer.
Worse of all, after a minute or so of transferring data the socket just freezes and stops execution, and the program perpetually stays in this frozen state, like so:
"Worst" suggests other "bad", which you have not described. But yes, your code is susceptible to freezing. You will not see EOF on the socket until the remote peer closes their side, cleanly. The closure part is what EOF means for a network socket. The cleanness part is required, at the protocol level, for the local side to recognize the closure. If the other end holds the connection open but doesn't write anything else to it then just such a freeze will occur. If the other side is abruptly terminated, or physically or logically cut off from the network without a chance to close their socket, then just such a freeze will occur.
And indeed, you remarked in comments that ...
Both the client and the server are hanging. The client program just stops sending data, and the server freezes as well.
If the client hangs mid-transfer, then, following from the above, there is every reason to expect that the server will freeze, too. Thus, it sounds like you may be troubleshooting the wrong component.
Perhaps the recv() call is blocking when it shouldn't be, I'm going to test with flags.
There is every reason to think the recv() call is indeed blocking when you don't expect it to do. It's highly unlikely that it is blocking when it shouldn't.
It is possible to set timeouts for socket operations, so that they eventually will fail instead of hanging indefinitely when the remote side fails. Doing so would allow your server to recover, but it would not resolve the client-side issue. You'll need to look into that more deeply.*
*You might see the client unfreeze after the server times out and closes the connection on its end. Don't take that as a resolution.
Just for the purpose of learning raw sockets in C I am writing a simple server that uses raw sockets to receive and send messages.
I create the socket
if ((r_sock = socket(AF_INET, SOCK_RAW, IPPROTO_UDP))< 0){
perror("socket");
exit(-1);
}
Then I create an infinite loop and start receiving, processing, and replying
while(1){
if((n = recvfrom(r_sock, buffer, BUFLEN, 0, (struct sockaddr *) &client, &client_len))<0){
perror("recvfrom");
exit(-1);
}
// Discard messages not intended to the server
if(htons(udp->uh_dport) != my_port){
continue;
}
//Do whatever with the data received and then send reply to client
// ....
if((n = sendto(r_sock, udp, ntohs(udp->uh_len), 0, (struct sockaddr *) &client, client_len))<0){
perror("sendto");
exit(-1);
}
}
I am not showing here the definition of every single variable but for the sake of completeness, buffer is a char array of size BUFLEN (big enough) and udp is a struct udphdr pointer to the right position in the buffer.
The point is that I have another program that serves as client using standard UDP sockets (SOCK_DGRAM) which is proved to be working properly (I also tried with netcat just in case). When I send a message with the client, it never receives the reply back. It seems that when the server sends the reply to the client, the server itself gets the message and the client gets nothing.
So, my question is: is there a way of solving this with raw sockets? That is, to make the server not receive its own messages and preventing others from receiving them?
Thanks in advance!
I have just realised that it was a problem with the checksum... Once I had a correct checksum in UDP the packet was correctly received by the client.
Wireshark gave me the lead to the solution. I saw that the checksum was not validated so I went to Edit > Preferences > Protocols > UDP > Validate the UDP checksum if possible and checked it.
Hope it helps
This is more of a conceptual confusion. I am making a multicast server which just echoes back the datagram received.Here's the code
while (1) {
cnt = recvfrom(sock, message, sizeof(message), 0,
(struct sockaddr *) &addr, &addrlen);
//printf("%d \n",cnt);
if (cnt < 0) {
perror("recvfrom");
exit(1);
} else if (cnt == 0) {
break;
}
printf("%s: message = \"%s\"\n", inet_ntoa(addr.sin_addr), message);
addr.sin_addr.s_addr = inet_addr(EXAMPLE_GROUP);
cnt = sendto(sock, message, sizeof(message), 0,
(struct sockaddr *) &addr, addrlen);
if (cnt < 0) {
perror("sendto");
exit(1);
}
}
The problem with this is as a multicast server will also receive the datagram. So, after it recieves a datagram, it sends, it again recieves the same datagram, and so on entering an infinite loop. Any pointers on how to implement such type of server?
You need to disable multicast loopback via setsockopt().
One option is what EJP said, to disable multicast loopback so that the sending machine doesn't receive a copy of its own multicast packet back. However, be aware that if you do that, you won't be able to test with clients and server all running on the same machine anymore, since IP_MULTICAST_LOOP is implemented at the IP routing level.
A second possible option to avoid infinite packet loops would be to have the echo-server send its response packets to a different multicast group than the one it listens on (or even have it send its responses via unicast rather than multicast; the server could call recvfrom() to find out the unicast source address of any packet it receives, so it's easy for it to know where to send the reply packet back to)
A third option would be to modify the contents of the packet somehow to mark it as already-seen, so that your server knows not to echo it a second time. For example, you could specify that your server will only echo packets whose first byte is set to zero, and when your server echoes a packet, it makes sure to set the packet's first byte to one before send()-ing it out. (Your clients would need to be aware of this convention, of course)
I want to send a byte array X times until I've reached the buffer size. My client send the byte array until the buffer size is full. But the problem is packet loss and the server only recieves like 8/10 of the buffer size.
Client:
while(writeSize < bufferSize)
{
bytes_sent += sendto(servSocket, package, sizeof(package), 0, (struct sockaddr *)&server, sizeof(server));
writeSize+= sizeof(package);
}
Server:
while(packetSize < bufferSize)
{
bytes_recv += recvfrom(servSocket, package, sizeof(package), 0, (struct sockaddr*)&client, &size);
packetSize += sizeof(package);
printf("Bytes recieved: %d\n", bytes_recv);
}
I can't really come up with a solution with this problem. When the client has sent all the packages, and the server suffers from package loss the while loop won't end, any ideas how I can solve this?
Many thanks
Hmpf. You are using UDP. For this protocol its perfectly ok to throw away packets if there is need to do so. So it's not reliable in terms of "what is sent will arrive". What you have to do in your client is to check wether all packets you need arrived, and if not, talk politely to your server to resend those packets you did not receive. To implement this stuff is not that easy, and you might finally end up with a solution wich might be less performant than just using plain old tcp-sockets, so my recomendation would be in the first instance to switch to a tcp connection.
If you have to use UDP to transfer a largish chunk of data, then design a small application-level protocol that would handle possible packet loss and re-ordering (that's part of what TCP does for you). I would go with something like this:
Datagrams less then MTU (plus IP and UDP headers) in size (say 1024 bytes) to avoid IP fragmentation.
Fixed-length header for each datagram that includes data length and a sequence number, so you can stitch data back together, and detect missed, duplicate, and re-ordered parts.
Acknowledgements from the receiving side of what has been successfully received and put together.
Timeout and retransmission on the sending side when these acks don't come within appropriate time.
Here you go. We started building TCP again ...
Of course TCP would be the better choice but if you only have UDP option i'd wrap the data into a structure with sequence and buf_length field.
It is absolutely NOT an optimal solution... It just take care of sequence number in order to assure the data is totally arrived and with the right order. If not, just stop to receive data (It could do something smarter like ask to the send to resend pieces and rebuild the data despite thw wrong order but it will become really complex)
struct udp_pkt_s {
int seq;
size_t buf_len;
char buf[BUFSIZE];
};
sending side:
struct udp_pkt_s udp_pkt;
udp_pkt.seq = 0;
while(writeSize < bufferSize)
{
// switch to the next package block
// ...
memcpy(udp_pkt.buf, package);
udp_pkt.buf_len = sizeof(package);
udp_pkt.seq++;
bytes_sent += sendto(servSocket, &udp_pkt, sizeof(udp_pkt_s), 0, (struct sockaddr *)&server, sizeof(server));
writeSize += sizeof(package);
}
receiving side:
last_seq = 0;
while(packetSize < bufferSize)
{
bytes_recv += recvfrom(servSocket, &udp_pkt, sizeof(udp_pkt_s), 0, (struct sockaddr*)&client, &size);
// break it if there's a hole
if (udp_pkt.seq != (last_seq + 1))
break;
last_seq = udp_pkt.seq;
packetSize += udp_pkt.buf_len;
printf("Bytes recieved: %d\n", udp_pkt.buf_len);
}
after sendto you can wait for sendto buffer to reach zero, you can use the following api after each sendto call.
void waitForSendBuffer(void)
{
int outstanding = 0;
while(1)
{
ioctl(socketFd_, SIOCOUTQ, &outstanding);
if(outstanding == 0)
break;
printf("going for sleep\n");
usleep(1000);
}
}
I have created an echo server under UDP, and am working on a client that splits up a given file into packets and transfers it to a server, which then returns the packets to be reassembled by the client.
Sending the individual packets works just fine. However, I am trying to use the UDP_CORK socket option to transfer as many parts of the file as possible in one packet. I am working with the following code:
#define LINE_SIZE 80
...
// s = socket descriptor.
...
int one = 1;
int zero = 0;
setsockopt(s, IPPROTO_UDP, UDP_CORK, &one, sizeof(one)); // cork
/* Send to server, receive from server, write to new file. */
FILE *orig, *copy;
char line[LINE_SIZE];
if ((orig = fopen(input + 3, "rb")) == NULL) // original file
print_error(input);
if ((copy = fopen(filename, "wb")) == NULL) // copy of file
print_error("fopen");
while (fread(line, sizeof(char), LINE_SIZE, orig) > 0) {
if (sendto(s, line, LINE_SIZE, 0, (struct sockaddr *)&srv, len) == -1)
print_error("sendto");
}
setsockopt(s, IPPROTO_UDP, UDP_CORK, &zero, sizeof(zero)); // uncork
if (recvfrom(s, line, LINE_SIZE, 0, (struct sockaddr *)&srv, &len) == -1)
print_error("recvfrom");
fwrite(line, sizeof(char), LINE_SIZE, copy);
The server indicates that it is only receiving the one "normal-sized" packet, and not the corked packet that I would like. It follows that it is only sending back this one normal packet, received by the client via the recvfrom() call.
I am not sure that I set up the UDP_CORK correctly. Is the second argument correct? I am unfamiliar with what the second option truly means, as the man page was not entirely clear. It is also possible that I am misunderstanding how the cork works.
A few things to note:
I consider myself an intermediate-level programmer, but I have little network programming experience.
I realize that UDP is not the best to transfer files. This is eventually going to be adapted to a different protocol that I am helping to develop.
Thank you!
How big is the file? You know that UDP datagrams are limited to 64K in size, right? And then anything over 1472 bytes (1500 bytes of available ethernet payload less minimum of 20 bytes of IP header, less 8 bytes of UDP header) is IP-fragmented.
Then you never check for the return value of the setsockopt(2). How do you know it's succeeding?
Then fread(3) tells you how much it read, but you still try to send LINE_SIZE bytes. This is wrong.