So I've created a DNS proxy in C. I'm using DIG as the client program; it sends a query packet to my server, my server forwards it to a DNS, and my server receives the answers, then sends them back to the client.
My server is bound to a UDP socket; I'm transmitting the DNS packet via TCP. However, my recv() call (from the TCP socket) is always returning 0. I'll get the original query back, but without answers.
Code:
fromlen=sizeof(client);
recvfrom(UDPSock,buffer,sizeof(buffer),0,(struct sockaddr *)&client,&fromlen); //receive from client
int msglen=strlen(buffer);
connect(TCPSock,(struct sockaddr*) &dest, sizeof(dest)); //connect to DNS
int m=send(TCPSock,buffer,msglen,0); //send packet to dns
recv(TCPSock,buffer,sizeof(buffer),0); //this returns 0
//send back
sendto(UDPSock,buffer,sizeof(buffer),0,(struct sockaddr *)&client,fromlen); //send message back`
The buffer is 300 bytes.
Your immediate problem is that you're not speaking the correct protocol. The DNS/TCP/IP protocol is not identical to the DNS/UDP/IP protocol. Read the RFCs describing the protocols and follow them.
Your more fundamental problem is a design problem with your code as given. It doesn't really make sense, not least in terms of network overhead, to have one-connection-per-query TCP/IP on the back end when there's only UDP/IP on the front end. Moreover: A properly written forwarding proxy DNS server has to cope with large UDP/IP packets, truncated responses, truncated and otherwise malformed queries, TCP/IP connection timeouts and resets, loop detection, and DNS/UDP/IP retry. But this is beyond the scope of the question.
Aside from not checking for errors from any of the system calls one big problem is the strlen(3) use. The recvfrom(2) tells you how many bytes it placed in your buffer (or -1 for error). Some of these bytes could have zero values so the strlen(2) is not at all applicable in this case. Fix that and see if it helps. Otherwise you'd have to explain why you forward over TCP and connect on every packet.
recv() returns zero when the other end disconnects. So you are making the other end disconnect rather than send any data. Probably it doesn't understand what you are sending.
Related
In recv()'s man page I found that return value can be zero in such case:
Datagram sockets in various domains (e.g., the UNIX and Internet
domains) permit zero-length datagrams. When such a datagram is
received, the return value is 0.
When or why should one use zero-length datagrams in UNIX socket intercommunication? What's its purpose?
One such use is to unblock a recvfrom() call when you wish to close a UDP service thread - set a 'terminate' flag and send the zero-length datagram on the localhost stack.
One example I stumbled upon yesterday while researching the answer to another question.
In the old RFC 868 protocol for getting the current time of a remote server, the workflow for using UDP looks like:
When used via UDP the time service works as follows:
S: Listen on port 37 (45 octal).
U: Send an empty datagram to port 37.
S: Receive the empty datagram.
S: Send a datagram containing the time as a 32 bit binary number.
U: Receive the time datagram.
The server listens for a datagram on port 37. When a datagram
arrives, the server returns a datagram containing the 32-bit time
value. If the server is unable to determine the time at its site, it
should discard the arriving datagram and make no reply.
In this case, the server has to receive a datagram to be alerted that a user is requesting the time (And to know what address to send a reply to), due to UDP's connectionless nature (The TCP version just needs to connect to the server). The contents of that datagram are ignored, so might as well specify that it should be empty.
How to receive message from only desired client if I know that this client has been connected to my server?
recvfrom(sockfd, buf, BUFFSIZE, MSG_PEEK,
(struct sockaddr *)&addr, &caddrlen);
addr will be replaced by corresponding data, but I want receive data from only one client
How to receive message from only desired client if I know that this client has been connected to my server?
You can't really do this with a datagram socket in the AF_INET or AF_INET6 family, unless undesired clients are blocked at a lower level, such as via a firewall. At least, not if you want to be able to continue to receive messages from desired clients after one from an undesired client arrives. The network driver queues datagrams for you, and you need to handle them in order, where the C API for "handling" a datagram is to receive it via one of the several system calls that serve that purpose, such as recvfrom().
You can discriminate between messages after receiving them, such as by discarding those from undesired clients. However, it is of limited, special-purpose use to retrieve message data without dequeueing it, as the MSG_PEEK flag to recvfrom() provides. In particular, that does not serve your stated purpose -- you will still need to receive every message, via a subsequent call that does not use MSG_PEEK. Instead, I suggest simply reading the data via recvfrom(), checking the address to determine whether it's from the client you're interested in, and handling it appropriately.
If you want to handle multiple clients at the same time, then you have some alternatives. A relatively simple one is to have a function, perhaps running in a dedicated thread, that receives all incoming messages and dispatches them appropriately according to their source addresses. Another alternative is to open a new (datagram) socket for each client, each on its own port, and to set up a protocol by which you tell each client which port to use after their first contact. Datagrams from unexpected clients on these additional ports would be erroneous, and therefore safe to reject.
Of course, the latter of those is an approximation to a connection-oriented protocol. If it seems attractive to you then perhaps you should be looking at stream sockets instead of datagram sockets. Not only would that get you no-fuss client specificity, it would also provide for reliable and guaranteed in-order communication.
recvfrom(sockfd, buf, BUFFSIZE, MSG_PEEK,
(struct sockaddr *)&addr, &caddrlen);
if(addr.sin_addr.s_addr == "ADDRESS_OF_DESIRED_CLIENT")
{
//ALLOW USER
}
For IPV4 "addr.sin_addr.s_addr" is an int, but you can also get the Address in a string from.
I am doing some test with TCP client application in a Raspberry Pi (server in the PC), with PPP (Point to Point Protocol) using a LTE Modem. I have used C program with sockets, checking system call's response. I wanted to test how socket works in a bad coverage area so I did some test removing the antenna.
I have followed the next steps:
Connect to server --> OK
Start sending data (write system call) --> OK (I also check in the server)
I removed the LTE modem's antenna (There is no network, it can't do ping)
Continue sending data (write system call) --> OK (server does not receive anything!!!)
It finished sending data and closed socket --> OK (connection is still opened and there is no data since the antenna was removed)
Program was finished
I put the antenna again
Some time later, the data has been uploaded and the connection closed. But I did another test following this steps but with more data, and it did not upload this data...
I do not know if there any way to ensure that the data written to TCP server is received by the server (I thought that TCP layer ensured this..). I could do it manually using an ACK but I guess that it has to be a better way to do.
Sending part code:
while(i<100)
{
sprintf(buf, "Message %d\n", i);
Return = write(Sock_Fd, buf, strlen(buf));
if(Return!=strlen(buf))
{
printf("Error sending data to TCP server. \n");
printf("Error str: %s \n", strerror(errno));
}
else
{
printf("write successful %d\n", i);
i++;
}
sleep(2);
}
Many thanks for your help.
The write()-syscall returns true, since the kernel buffers the data and puts it in the out-queue of the socket. It is removed from this queue when the data was sent and acked from the peer. When the OutQueue is full, the write-syscall will block.
To determine, if data has not been acked by the peer, you have to look at the size of the outqueue. With linux, you can use an ioctl() for this:
ioctl(fd, SIOCOUTQ, &outqlen);
However, it would be more clean and portable to use an inband method for determining if the data has been received.
TCP/IP is rather primitive technology. Internet may sound newish, but this is really antique stuff. TCP is needed because IP gives almost no guarantees, but TCP doesn't actually add that many guarantees. Its chief function is to turn a packet protocol into a stream protocol. That means TCP guarantees a byte order; no bytes will arrive out of order. Don't count on more than that.
You see that protocols on top of TCP add extra checks. E.g. HTTP has the famous HTTP error codes, precisely because it can't rely on the error state from TCP. You probably have to do the same - or you can consider implementing your service as a HTTP service. "RESTful" refers to an API design methodology which closely follows the HTTP philosophy; this might be relevant to you.
The short answer to your 4th and 5th topics was taken as a shortcut from this answer (read the whole answer to get more info)
A socket has a send buffer and if a call to the send() function succeeds, it does not mean that the requested data has actually really been sent out, it only means the data has been added to the send buffer. For UDP sockets, the data is usually sent pretty soon, if not immediately, but for TCP sockets, there can be a relatively long delay between adding data to the send buffer and having the TCP implementation really send that data. As a result, when you close a TCP socket, there may still be pending data in the send buffer, which has not been sent yet but your code considers it as sent, since the send() call succeeded. If the TCP implementation was closing the socket immediately on your request, all of this data would be lost and your code wouldn't even know about that. TCP is said to be a reliable protocol and losing data just like that is not very reliable. That's why a socket that still has data to send will go into a state called TIME_WAIT when you close it. In that state it will wait until all pending data has been successfully sent or until a timeout is hit, in which case the socket is closed forcefully.
The amount of time the kernel will wait before it closes the socket,
regardless if it still has pending send data or not, is called the
Linger Time.
BTW: that answer also refers to the docs where you can see more detailed info
Good day.
Intro.
Recently I've started to study some 'low-level' network programming as well as networking protocols in Linux. For this purpose I decided to create a small library for networking.
And now I wonder on some questions. I will ask one of them now.
As you know there are at least two protocols built on top of IP. I talk about TCP and UDP. Their implementation may differ in OS due to connection-orientation property of those.
According to man 7 udp all receive operations on UDP socket return only one packet. It is rational as different datagrams may come from different sources.
On the other hand TCP connection packets sequence may be considered as continuous byte flow.
Now, about the problem itself.
Say, I have an API for TCP connection socket and for UDP socket like:
void tcp_connection_recv(endpoint_t *ep, buffer_t *b);
void udp_recv(endpoint_t *ep, buffer_t *b);
endpoint_t type will describe the endpoint (remote for TCP connection and local for UDP). buffer_t type will describe some kind of vector-based or array-based buffer.
It is quite possible that buffer is already allocated by user and I'm not sure that this will be right for UDP to not change size of the buffer. And thus, to abstract code for TCP and UDP operations I think it will need to allocate as much buffer as needed to contain whole received data.
Also, to prevent from resizeing user buffer each socket may be maped to its own buffer (although it will be userspace buffer, but it will be hidden from user). And then on user's request data will be copied from that 'inner' buffer to user's one or read from socket if there is not enough amount.
Any suggestions or opinions?
If you want to create such API, it will depend on the service you want to provide. In TCP it will be different than UDP as TCP is stream oriented.
For TCP, tcp_connection_recv instead of reallocating a buffer, if the buffer passed by the user is not big enough, you can fill the whole buffer and then return, maybe with an output parameter, and indication that there is more data waiting to be read. Basically you can use the receive buffer that TCP connection already provides in the kernel, no need to create other buffer.
For, udp, you can request the user a number indicating the maximum datagram size it is waiting for. When you read from a UDP socket with recvfrom, if you read less data than what came in the arrived datagram, the rest of the datagram data is lost. You can read first with MSG_PEEK flag in order to find out how much data is available.
In general I wouldn't handle the buffer for the application as the application, actually the application layer protocol, is the one that knows how it expects to receive the data.
In C, to receive/send data you usually do(roughly):
Server:
Create socket
Bind socket to port
listen
Accept
Receive Send data
On client side:
Create socket
Connect
Receive send
My question comes after server has done accept.
Imagine after accept on the server side there are three separate lines
to send data:
connfd = accept(listenfd, (struct sockaddr*)NULL ,NULL);
write(connfd, var1, var1Size);
write(connfd, var2, var2Size);
write(connfd, var3, var3Size);
Does this mean on the client side I need to have three reads?
Like this:
read(sockfd, &x, size1);
read(sockfd, &y, size2);
read(sockfd, &z, size3);
In other words how should send and receive calls correspond
on server and client side? Should for each send be a corresponding receive on the client side?
What if on client side, after 3 read calls(like above), I want to send data to server?
Shall I just add one new send and one new receive on client and server side respectively?
Should all these send/receives be happening within a single accept call context?
Here is a image to better illustrate what kind of scenario I could be interested in:
Pseudo code explaining how to handle this kind of connections would be welcome.
Unless you are working with a protocol which has a concept of "messages", e.g. UDP, all you have is a stream of bytes. You can send and receive them any way you wish.
You could, for example, send two 16-bit integers and receive them as one 32-bit integer. This is probably not what you intended but it's perfectly legal and used all the time in situations where it is needed. You can compose data structures on either side (sending and receiving) independandly, as long as it makes sense to your application.
Your bytes are sent in the order of your write()'s and you WILL receive them in the same order. I.e.
send(var1) ---> recv(var1)
send(var2) ---> recv(var2)
There is no way in normal TCP (barring unused edge cases which I'll not even specify because nobody should use them) that you will receive var2 before var1.
TCP communication is bi-directional: each end-point (client and server) can send at the same time. It is up to you and your application to decide when to send and when to receive. The sending and receiving buffers are independant: you can send a few bytes, receive a few, send some more... and there will be no interference between them (i.e. you will not "overwrite" the receive buffer by sending some data nor vice versa).
I'll repeat it again: ALL you have in TCP is a stream of bytes. TCP doesn't know and doesn't care how these bytes are structured, neither on the sending nor on the receiving side. It's ALL up to you. When you send an integer or a data structure, you are sending a memory dump of those, as bytes.
For example, there's a common error where you attempt to send() a data structure and because the sending buffers are full, the system will make a partial write. If you do not check the return status of the send() call to detect this situation and then send the remainder of bytes by yourself, in another send() call, your client WILL be stuck in recv() when it expects the full structure and receives only a part of it, if you specify MSG_WAITALL.
TCP is a stream protocol, In the receiver side you cannot determine how many times the send has been called. Whenever recv is called it will give the number of bytes asked to read, if the requested number of bytes are not available then it will return the number of bytes currently in the socket buffer.
In case of UDP it will work as you mentioned, It is a datagram protocol. (use recvfrom to recv the data)