This question already has answers here:
Is there complete isolation between two sendalls in the same TCP connection?
(1 answer)
Can't receive multiple messages on server using TCP socket (Python)
(1 answer)
Python TCP Server receives single TCP packet split into multiple packets
(2 answers)
Closed 8 months ago.
I'm working on a C project that implements a TCP client-server. The sockets and the send() functions i'm using are the one defined in the libraries sys/socket.h and winsock2.h.
My problem is that when i try to send multiple strings one after the other, some messages aren't transmitted correctly, with some data (sometimes all the message) that goes missing. The following code, for example, works without a problem when i'm running server and client on the same machine, but if I try to run it with a remote server, then the third message isn't properly received.
Client Side
char message[1024];
memset(message, 0, 1024);
fill_message(message, msg1); //A function that prints something in the message string.
//It may fill less than 1024 characters.
send(clientSocket, message, 1024,0);
fill_message(message, msg2);
send(clientSocket, message, 1024,0);
fill_message(message, msg3);
send(clientSocket, message, 1024,0);
Server Side
char message[1024];
memset(message, 0, 1024);
recv(clientSocket, message, 1024,0);
print_and_do_stuff(message);
recv(clientSocket, message, 1024,0);
print_and_do_stuff(message);
recv(clientSocket, message, 1024,0);
print_and_do_stuff(message);
Note: the string message may not be exactly of length 1024.
My solution has been to make the client wait for 1 second by calling sleep(1) after each message is sent. Is this the proper way to address the issue? Or am i missing something about how send() and recv() work?
More in general: what is the "proper" way to program with sockets? Should I maybe be sending the message byte-by-byte and specifying the length as the first thing? If someone could point me toward a good tutorial/guide on what the best practices are when working with sockets, I'd be happy to read it.
Socket functions may or may not read/send the entire data in one call, which means that you have to verify the correct reception server side, and maybe create a custom protocol on top of TCP to keep track of the size you sent and received.
TCP, contrary to UDP, guarantees the integrity of data, meaning that you won't lose anything when sending, but you may need to use multiple function calls to ensure all of the data has been sent and red.
As for good tutorial and guides, as someone already said in comments, you can find loads of examples and guides about it.
Related
I want to write basic chat program. I don't release that, I just want to learn socket programming. The chat program will be between client and server.
server code:
//bla bla code
new_socket = accept(server_fd, (struct sockaddr*)&address,(socklen_t*)&addrlen);
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(server_msg, 64, stdin);
send(new_socket, server_msg, strlen(server_msg), 0);
recv(new_socket, client_msg, 128, 0);
}
client code:
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(client_msg, 64, stdin);
send(server_socket, client_msg, strlen(client_msg), 0);
recv(server_socket, server_msg, 128, 0);
}
Questions:
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
My English is not good, I'm sorry.
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
you open a TCP socket connection, so, once in the connected state (the ESTABLISHED state) it remains there until one side closes it (and the other side will notice it by reading 0 bytes as result, this explanation is oversimplified to make it simpler to your case, normally you should detect the end of file in the socket, so you can close it, and accept(2) another connection)
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
Your program is not a chat program, as the server uses the SAME socket to write what it has read, so it will return the message sent to the same client that issued it (acting more as an echo server, than a chat)
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
Normally, a chat server should wait for a message to arrive at several sockets (by means of the select(2) system call) and will determine which socket needs attention. As a result, the server will read(2) from that socket (probably in a non blocking way, to avoid being blocked on a read to a socket, that will not be interrupted if data enters on another socket) The chat server should write what it has read over all the other sockets it has connections from, so every other user of your chat system receives a copy of the message (sending back the message to the sender is optional, but not very elegant, the issuer of the message should do local echo of it, although, to ensure the user sees his messages interspersed between the ones from other users)
I recommend you to read the book "Unix Network Programming" from Richard Stevens. It describes perfectly how to use the sockets API and IMHO it is the best reference you can get to learn how to use the socket interface.
One final note:
Don't use plain numbers hardwired in your code, as it is error prone (you declare a buffer string to be 128 bytes, but then read only 64, making 64 unused bytes in the array, you can use sizeof buffer as to make the compiler to use the amount declared in the variable, or you can #define BUFFSIZE 128 and then declare char buffer[BUFFSIZE]; and later do a fgets(buffer, BUFFSIZE, socket_FILE_descriptor); if you use a constant to size a buffer, use a constant name to name it :) and use it everywhere, in case you decide to give it a different value, all the occurences of it will change, instead of having to reviste all places in your code where you have used the number 64.
I am doing some test with TCP client application in a Raspberry Pi (server in the PC), with PPP (Point to Point Protocol) using a LTE Modem. I have used C program with sockets, checking system call's response. I wanted to test how socket works in a bad coverage area so I did some test removing the antenna.
I have followed the next steps:
Connect to server --> OK
Start sending data (write system call) --> OK (I also check in the server)
I removed the LTE modem's antenna (There is no network, it can't do ping)
Continue sending data (write system call) --> OK (server does not receive anything!!!)
It finished sending data and closed socket --> OK (connection is still opened and there is no data since the antenna was removed)
Program was finished
I put the antenna again
Some time later, the data has been uploaded and the connection closed. But I did another test following this steps but with more data, and it did not upload this data...
I do not know if there any way to ensure that the data written to TCP server is received by the server (I thought that TCP layer ensured this..). I could do it manually using an ACK but I guess that it has to be a better way to do.
Sending part code:
while(i<100)
{
sprintf(buf, "Message %d\n", i);
Return = write(Sock_Fd, buf, strlen(buf));
if(Return!=strlen(buf))
{
printf("Error sending data to TCP server. \n");
printf("Error str: %s \n", strerror(errno));
}
else
{
printf("write successful %d\n", i);
i++;
}
sleep(2);
}
Many thanks for your help.
The write()-syscall returns true, since the kernel buffers the data and puts it in the out-queue of the socket. It is removed from this queue when the data was sent and acked from the peer. When the OutQueue is full, the write-syscall will block.
To determine, if data has not been acked by the peer, you have to look at the size of the outqueue. With linux, you can use an ioctl() for this:
ioctl(fd, SIOCOUTQ, &outqlen);
However, it would be more clean and portable to use an inband method for determining if the data has been received.
TCP/IP is rather primitive technology. Internet may sound newish, but this is really antique stuff. TCP is needed because IP gives almost no guarantees, but TCP doesn't actually add that many guarantees. Its chief function is to turn a packet protocol into a stream protocol. That means TCP guarantees a byte order; no bytes will arrive out of order. Don't count on more than that.
You see that protocols on top of TCP add extra checks. E.g. HTTP has the famous HTTP error codes, precisely because it can't rely on the error state from TCP. You probably have to do the same - or you can consider implementing your service as a HTTP service. "RESTful" refers to an API design methodology which closely follows the HTTP philosophy; this might be relevant to you.
The short answer to your 4th and 5th topics was taken as a shortcut from this answer (read the whole answer to get more info)
A socket has a send buffer and if a call to the send() function succeeds, it does not mean that the requested data has actually really been sent out, it only means the data has been added to the send buffer. For UDP sockets, the data is usually sent pretty soon, if not immediately, but for TCP sockets, there can be a relatively long delay between adding data to the send buffer and having the TCP implementation really send that data. As a result, when you close a TCP socket, there may still be pending data in the send buffer, which has not been sent yet but your code considers it as sent, since the send() call succeeded. If the TCP implementation was closing the socket immediately on your request, all of this data would be lost and your code wouldn't even know about that. TCP is said to be a reliable protocol and losing data just like that is not very reliable. That's why a socket that still has data to send will go into a state called TIME_WAIT when you close it. In that state it will wait until all pending data has been successfully sent or until a timeout is hit, in which case the socket is closed forcefully.
The amount of time the kernel will wait before it closes the socket,
regardless if it still has pending send data or not, is called the
Linger Time.
BTW: that answer also refers to the docs where you can see more detailed info
In C, to receive/send data you usually do(roughly):
Server:
Create socket
Bind socket to port
listen
Accept
Receive Send data
On client side:
Create socket
Connect
Receive send
My question comes after server has done accept.
Imagine after accept on the server side there are three separate lines
to send data:
connfd = accept(listenfd, (struct sockaddr*)NULL ,NULL);
write(connfd, var1, var1Size);
write(connfd, var2, var2Size);
write(connfd, var3, var3Size);
Does this mean on the client side I need to have three reads?
Like this:
read(sockfd, &x, size1);
read(sockfd, &y, size2);
read(sockfd, &z, size3);
In other words how should send and receive calls correspond
on server and client side? Should for each send be a corresponding receive on the client side?
What if on client side, after 3 read calls(like above), I want to send data to server?
Shall I just add one new send and one new receive on client and server side respectively?
Should all these send/receives be happening within a single accept call context?
Here is a image to better illustrate what kind of scenario I could be interested in:
Pseudo code explaining how to handle this kind of connections would be welcome.
Unless you are working with a protocol which has a concept of "messages", e.g. UDP, all you have is a stream of bytes. You can send and receive them any way you wish.
You could, for example, send two 16-bit integers and receive them as one 32-bit integer. This is probably not what you intended but it's perfectly legal and used all the time in situations where it is needed. You can compose data structures on either side (sending and receiving) independandly, as long as it makes sense to your application.
Your bytes are sent in the order of your write()'s and you WILL receive them in the same order. I.e.
send(var1) ---> recv(var1)
send(var2) ---> recv(var2)
There is no way in normal TCP (barring unused edge cases which I'll not even specify because nobody should use them) that you will receive var2 before var1.
TCP communication is bi-directional: each end-point (client and server) can send at the same time. It is up to you and your application to decide when to send and when to receive. The sending and receiving buffers are independant: you can send a few bytes, receive a few, send some more... and there will be no interference between them (i.e. you will not "overwrite" the receive buffer by sending some data nor vice versa).
I'll repeat it again: ALL you have in TCP is a stream of bytes. TCP doesn't know and doesn't care how these bytes are structured, neither on the sending nor on the receiving side. It's ALL up to you. When you send an integer or a data structure, you are sending a memory dump of those, as bytes.
For example, there's a common error where you attempt to send() a data structure and because the sending buffers are full, the system will make a partial write. If you do not check the return status of the send() call to detect this situation and then send the remainder of bytes by yourself, in another send() call, your client WILL be stuck in recv() when it expects the full structure and receives only a part of it, if you specify MSG_WAITALL.
TCP is a stream protocol, In the receiver side you cannot determine how many times the send has been called. Whenever recv is called it will give the number of bytes asked to read, if the requested number of bytes are not available then it will return the number of bytes currently in the socket buffer.
In case of UDP it will work as you mentioned, It is a datagram protocol. (use recvfrom to recv the data)
I am programming TCP server client. I sending the three string seperately using seperate send system call.
But receiving end i getting only single string that is first string which i send. remaining two string missed.
Below i given the part of my server client program.
client.c
char *info = "infolog";
char *size = "filesize";
char *end = "fileend";
send(client, info, strlen(info)+1, 0);
send(client, size, strlen(size)+1, 0);
send(client, end, strlen(end)+1, 0);
server.c
while ((read_size = recv(client, msg, sizeof(msg), 0))) {
printf("Data: %s\n", msg);
memset(msg, 0, sizeof(msg));
}
Actual output:
Data: infolog
Expected output
Data: infolog
Data: filesize
Data: fileend
Thanks.
Try printing out read_size. You probably have received all the messages already.
Due to Nagle's Algorithm, the sender probably batched up your three send() calls and sent a single packet to the server. While you can disable Nagle's algorithm, I don't think it's a good idea in this case. Your server needs to be able to handle receiving of partial data, and handle receiving more data than it expects.
You might want to look into using an upper-layer protocol for your messages, such as Google Protocol Buffers. Take a look at the techniques page, where they describe how they might do it: build up a protocol buffer, and write its length to the stream before writing the buffer itself. That way the receive side can read the length and then determine how many bytes it needs to read before it has a complete message.
TCP is not a message protocol but a byte stream protocol.
The three send-s could be recv-ed as a single input (or something else, e.g. in two or five recv etc....)
The application should analyze the input and buffer it to be able to splice it in meaningful messages.
the transmission may split or merge the messages, e.g. intermediate routers can and will split or merge the "packets".
In practice you'll better have some good conventions about your messages. Either decide that each message is e.g. newline terminated, or decide that it starts with some header giving its size.
Look at HTTP, SMTP, IMAP, SCGI or ONC/XDR (documented in RFC5531) as concrete examples. And document quite well your protocol (a minima, in long descriptive comments for some homework toy project, and more seriously, in a separate public document).
So I've created a DNS proxy in C. I'm using DIG as the client program; it sends a query packet to my server, my server forwards it to a DNS, and my server receives the answers, then sends them back to the client.
My server is bound to a UDP socket; I'm transmitting the DNS packet via TCP. However, my recv() call (from the TCP socket) is always returning 0. I'll get the original query back, but without answers.
Code:
fromlen=sizeof(client);
recvfrom(UDPSock,buffer,sizeof(buffer),0,(struct sockaddr *)&client,&fromlen); //receive from client
int msglen=strlen(buffer);
connect(TCPSock,(struct sockaddr*) &dest, sizeof(dest)); //connect to DNS
int m=send(TCPSock,buffer,msglen,0); //send packet to dns
recv(TCPSock,buffer,sizeof(buffer),0); //this returns 0
//send back
sendto(UDPSock,buffer,sizeof(buffer),0,(struct sockaddr *)&client,fromlen); //send message back`
The buffer is 300 bytes.
Your immediate problem is that you're not speaking the correct protocol. The DNS/TCP/IP protocol is not identical to the DNS/UDP/IP protocol. Read the RFCs describing the protocols and follow them.
Your more fundamental problem is a design problem with your code as given. It doesn't really make sense, not least in terms of network overhead, to have one-connection-per-query TCP/IP on the back end when there's only UDP/IP on the front end. Moreover: A properly written forwarding proxy DNS server has to cope with large UDP/IP packets, truncated responses, truncated and otherwise malformed queries, TCP/IP connection timeouts and resets, loop detection, and DNS/UDP/IP retry. But this is beyond the scope of the question.
Aside from not checking for errors from any of the system calls one big problem is the strlen(3) use. The recvfrom(2) tells you how many bytes it placed in your buffer (or -1 for error). Some of these bytes could have zero values so the strlen(2) is not at all applicable in this case. Fix that and see if it helps. Otherwise you'd have to explain why you forward over TCP and connect on every packet.
recv() returns zero when the other end disconnects. So you are making the other end disconnect rather than send any data. Probably it doesn't understand what you are sending.