Multiple recv() until timeout? - c

So I'm trying to do the following:
I have two participants (let's call them A and B) communicating via TCP socket (send() and recv()). A is sending a counter and a random Nonce, B is just responding with that same message it gets. A then checks if the response matches the sent packet and if yes, it increments the counter and repeats.
This is a code snippet illustrating what A does at the moment:
send(sock, payload, strlen(payload), 0);
struct timeval t_out;
t_out.tv_sec = 0;
t_out.tv_usec = 5000;
setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,&t_out,sizeof(t_out)) <0)
int len = recv(sock, rx_buffer, sizeof(rx_buffer) - 1, 0);
if (len < 0)
{
print("Timeout reached, recv failed: errno %d", errno);
}
else
{
rx_buffer[len] = 0;
if(strncmp(rx_buffer, payload, payload_len) == 0)
{
pack_nr++;
}
}
Now I'm encountering one problem.
Let's say B, for some reason, has a delay in responding. This causes something like that:
A sends something like "1xyz"
B has a delay ......
A times out and resends something like "1abc"
B's first response ("1xyz") reaches A, A decides that this is the wrong payload
B's second response ("1abc") reaches A too, but A is only executing one recv() and it's unseen for now
A resends something like "1uvw"
A reads "1abc" from recv() and again decides that this is the wrong payload
B's third response ("1uvw") reaches A, and so on and on
So what I'd like to do is to put a recv() in a loop, so that in step 5, A would first look for another response from B until the timeout is reached.
So is there clever way to do this? I was thinking about something like
send(sock, payload, strlen(payload), 0);
int flag = 0;
gettimeofday(&start_time, NULL);
while((tx_time < start_time + timeout) && flag = 0)
{
gettimeofday(&tx_time, NULL);
recv(sock, rx_buffer, sizeof(rx_buffer) - 1, 0);
if(rx_buffer is okay)
{
flag = 1;
}
wait_a_bit();
}
if(flag == 1) pack_nr++;

"... B is just responding with that same message it gets. A then checks if the response matches the sent packet ..."
You have a code problem and a terminology problem.
First, the terminology problem: Don't say "matches the sent packet". The data can be sent in one packet or ten packets, TCP doesn't care. You don't receive packets, you receive data that may be split or combined across packets as TCP wishes. It really helps (trust me) to be very precise in your use of words. If you mean a message, say "message". If you mean data, say "data". If you mean a datagram, say "datagram".
Unfortunately, your code problem is enormous. You want B to respond to A with the same message it received. That means you need a protocol that sends and receives messages. TCP is not a message protocol. So you need to implement a message protocol and write code that actually sends and receives messages.
If A write "foobar", B might receive "foobar" or it might first receive "foo" and then later "bar". If A writes "foo" then "bar", B might receive "foobar" or "f" and then "oobar". That's TCP. If you need a message protocol, you need to implement one.

First off, you are not checking for a timeout correctly. recv() could fail for any number of reasons. You need to check errno (or WSAGetLastError() on Windows) to find out WHY it failed. But even if it did actually fail due to timeout, TCP is a byte stream, the delayed data may still show up (especially since 5000 microseconds (0.005 seconds) is way too short a timeout to reliably use for TCP network traffic), but your sender would have moved on. The only sensible thing to do if a timeout occurs in TCP is to close the connection, since you don't know the state of the stream anymore.
In your situation, you are basically implementing an ECHO protocol. Whatever the sender sends just gets echoed back as-is. As such, if you send 4 bytes (which you are not verifying, BTW), then you should keep reading until 4 bytes are received, THEN compare them. If any failure occurs in that process, immediately close the connection.
int sendAll(int sock, void *data, int len)
{
char *ptr = (char*) data;
while (len > 0) {
int sent = send(sock, ptr, len, 0);
if (sent < 0) {
if (errno != EINTR)
return -1;
}
else {
ptr += sent;
len -= sent;
}
}
return 0;
}
int recvAll(int sock, void *data, int len)
{
char *ptr = (char*) data;
while (len > 0) {
int recvd = recv(sock, ptr, len, 0);
if (recvd < 0) {
if (errno != EINTR)
return -1;
}
else if (recvd == 0) {
return 0;
}
else {
ptr += recvd;
len -= recvd;
}
}
return 1;
}
...
int payload_len = strlen(payload);
if (sendAll(sock, payload, payload_len) < 0)
{
// error handling
close(sock);
}
else
{
struct timeval t_out;
t_out.tv_sec = 5;
t_out.tv_usec = 0;
if (setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO, &t_out, sizeof(t_out)) < 0)
{
// error handling
close(sock);
}
else
{
int res = recvAll(sock, rx_buffer, payload_len);
if (res < 0)
{
if (errno == EAGAIN || errno == EWOULDBLOCK)
print("Timeout reached");
else
print("recv failed: errno %d", errno);
close(sock);
}
else if (res == 0)
{
print("disconnected");
close(sock);
}
else
{
if (memcmp(rx_buffer, payload, payload_len) == 0)
{
print("data matches");
pack_nr++;
}
else
print("data mismatch!");
}
}
}

Related

Winsock: recv() on Server is blocking, but client has already moved past send()

I am working on a project for school and have run into the following problem. My server is blocking out on recv() despite my client already sending its full message.
This is what I want to happen:
Server Client
recv() <---- send()
send() ----> recv()
This is what is happening:
Server Client
recv() <---- send()
recv() ----- recv()
Some Background
2 Weeks ago I created the client by itself with an already coded server application. When I coded the client it functioned properly with the provided server, so I want to say that the client is wrong, but I don't know how to get the server that I coded to recognize that no more data will be coming in.
Code
Here is the code that I believe is relevant:
Client:
bytesSent = 0;
retVal = send(sock, phrase, msgLen, 0);
bytesSent = retVal;
while (bytesSent < msgLen) {
retVal = send(sock, phrase + bytesSent, msgLen - bytesSent, 0);
if (retVal == SOCKET_ERROR) {
DisplayFatalErr("send() function failed.");
exit(1);
}
bytesSent += retVal;
// May need to re-call send in order to keep sending the data.
}
...
bytesRead = 0;
while (bytesRead < msgLen) {
retVal = recv(sock, rcvBuffer, RCVBUFSIZ - 1, 0);
if (retVal <= 0) {
DisplayFatalErr("recv() function failed.");
exit(1);
}
bytesRead += retVal;
for (int i = 0; i < retVal; i++) {
printf("%c", rcvBuffer[i]);
}
}
Server:
char* rcvBuffer[RCVBUFSIZ]; // RCVBUFSIZ = 50
char* msg = "";
int bytesRead = 0;
do {
if ((bytesRead = recv(clientSock, rcvBuffer, RCVBUFSIZ - 1, 0)) == 0) {
break;
}
if (bytesRead < 0) {
return -1;
}
char* msgConcatenated;
int msgLen = strlen(msg);
msgConcatenated = malloc(msgLen + bytesRead);
if (msgConcatenated != NULL) {
int newMsgLen = strlen(msgConcatenated);
strncpy_s(msgConcatenated, newMsgLen, msg, msgLen);
strncat_s(msgConcatenated, newMsgLen, rcvBuffer, bytesRead);
msg = msgConcatenated;
}
} while (bytesRead != 0);
Let me know if I need to provide extra information.
When using TCP, to signal the other end of the socket that no more data will be sent, a packet with the FIN flag set must be sent. This is accomplished in Winsock by calling the function shutdown with SD_SEND as the second parameter. This will cause the program on the other end of the socket to no longer block when calling recv. Instead, recv will return 0 indicating that the connection has been gracefully closed (unless there is data left that has not been read yet). See the Microsoft documentation on the shutdown function for further information. This documentation page also contains some helpful information about graceful socket closure.
Also, as has been pointed out in the comments, your code contains a memory leak in the following line:
msg = msgConcatenated
In that line, you reassign msg without first freeing the memory that msg is pointing to. Unfortunately, fixing that memory leak is not easy, because you can't simply call free on msg before reassigning it. This is because, in the first iteration of the loop, msg can also be pointing to something else than dynamically allocated memory. Therefore, to fix the leak, you would also have to keep track of what type of memory msg is pointing to, or make it always point to dynamically allocated memory, even when the string is empty (i.e. when it only contains the terminating null character).

ntohs() issue : Write Integer in C socket

I am trying to write and read Integer value into/from C socket. Sometimes ntohs() return very big values like 55000 , 32000 etc...Though client is always sending value <1500. If I run the program it happens after 10-15 minutes...Sometimes after 20-30 minutes.
Can you please check below code and tell me
Why this line getting printed ?
printf("Garbage value - ntohs problem ..Exiting... ");
// write exactly n byte
inline int write_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = write(fd, buf, left)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
// send exactly n byte
inline int send_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = send(fd, buf, left, MSG_NOSIGNAL)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
uint16_t nread, len, plength, nsend;
int MTU = 1500;
char buffer[2000];
// Server receive ( Linux 64 bit)
while (1) {
// read packet length
nread = read_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nread <=0) {
break;
}
len = ntohs(plength);
if (len <=0 || len > 1500 ) {
**printf("Garbage value - ntohs problem ..Exiting... "); // WHY ?**
break;
}
// read packat data
nread = read_n(SOCKFD, buffer, len);
if (nread != len) {
break;
}
}
//---------------------
// CLIENT send ( Android 5 )
while (1) {
nread = read(tunfd, buffer, MTU);
if (nread <= 0 || nread > 1500) { // always <=1500
break;
}
plength = htons(nread);
// send packet lenght
nsend = send_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nsend != sizeof(plength)) {
break;
}
// send packet data
nsend = send_n(TCP_SOCKFD, buffer, nread);
if (nsend != nread) {
break;
}
}
Thank you
We cannot tell you with certainty what's happening because you cannot provide a verifiable example. Additionally, you've not presented the implementation of read_n(), but supposing that it follows the same model as write_n() and send_n(), we can nevertheless perform some analysis.
Each of the data transfer functions returns a short count in the event that data transfer is interrupted by an error. The client code watches for this, and breaks out of its loop if it detects it. Well and good. The server code does not do this when reading plength, however. Since plength, as a uint16_t, is two bytes in size, a partial read is possible and would go unnoticed by your server code.
In your example, plength is modified only via the one read_n() call presented. Network byte order is big-endian, so the most-significant byte is read first. It is possible that the combination of that byte with the stale one left over from the previous read would represent a number exceeding 1500. For example, if a 221(0x00dd)-byte packet is followed by a 1280(0x0500)-byte packet, and a partial read occurs on the second packet size, then the combined result will be 1501(0x05dd).
I don't presently see any reason to think that the client sends data different in nature than you think it does, and I don't presently see any other way that your server code could give the appearance of receiving different data than the client sends, especially since client and server each abort at the first recognized sign of trouble.
Do note, however, that this code could still be made more robust. In particular, consider that read(), write(), and send() can fail even when there is no problem with the underlying socket or data transfer request. In particular, they can fail with EINTR if the call is interrupted by a signal, and if the socket is in non-blocking mode then they can fail with EAGAIN. There may be others. It does not seem useful to operate your socket in non-blocking mode, but you might indeed want to watch for EINTR and resume reading after receiving it.
I would also suggest that, at least during development, you emit more data about the nature of the error. Call perror(), for example, and afterward print the bad data. You might even consider logging data sent and received.

TCP Socket Multiplexing Send Large Data

Got some trouble with TCP socket multiplexing.
//socket is non-blocking
const int MAX = 4096;
char *buff[MAX];
char *p = buff;
int fd, rvalue;
rvalue = 0;
if ( (fd = open(path, O_RDONLY)) < 0 ) {
return errno;
} else {
int didsend, didread;
int shouldsend;
while ((didread = read(fd, buff, MAX)) > 0) {
p = buff;
shouldsend = didread;
while ( 1 ) {
didsend = send(sockfd, p, shouldsend, 0);
//if send succeeds and returns the number of bytes fewer than asked for then try to send rest part in next time.
if (didsend < shouldsend) {
p += didsent;
shouldsend -= didsend;
continue;
}
//if there is no place for new data to send, then wait a brief time and try again.
if ( didsend < 0 && (errno == EWOULDBLOCK || errno == EAGAIN) ) {
usleep(1000);
continue;
}
//if all data has been sent then sending loop is over.
if (didsend == shouldsend) {
break;
}
//send error
if ( didsend < 0 ) {
rvalue = errno;
break;
}
}
}
close(fd);
if (didread == -1) {
return errno;
}
return rvalue;
}
Assume I use an I/O Multiplexing function poll() or kqueue(), and non-blocking socket, then if there are only some small data like send a short message, it works fine.
But if it comes to large data, I mean larger than send()'s buffer size, since using non-blocking socket, send() will just send a portion of data, and return how much data it sends, the rest part of data can only be sent in another call of send(), but it takes time, and can't tell how long it will takes. So the second while() is actually a blocking send which using non-blocking socket.
Equivalent to:
//socket is blocking
const int MAX = 4096;
char *buff[MAX];
int fd, n;
if ( (fd = open(path, O_RDONLY)) < 0 ) {
return errno;
} else {
while ((n = read(fd, buff, MAX)) > 0) {
if (send(sockfd, buff, n, 0) < 0) {
return errno;
}
}
close(fd);
return 0;
}
So, what is the solution to this, multithreading might work but that's kind of wasting resource maybe.
This is the general pattern for a single-threaded server that works with multiple connections and non-blocking sockets.
It's primarily pseudo-code in C and doesn't do the necessary error checking. But it gives you an idea that for each accepted connection, you keep a struct instance that maintains the socket handle, request parsing state, response stream, and any other "state" members of that connection. Then you just loop using "select" to wait or having multiple threads doing this same thing.
Again this is only pseudo-code and uses select/poll as an example. You can get even more scalability with epoll.
while (1)
{
fd_set readset = {};
fd_set writeset = {};
for (int i = 0; i < number_of_client_connections; i++)
{
if (client_connections[i].reading_request)
FD_SET(client_connection.sock, &readset);
else
FD_SET(client_connection.sock, &writeset);
}
// add the listen socket to the read set
FD_SET(listen_socket, &readset);
select(n + 1, &readset, &writeset, &timeout); // wait for a socket to be ready (not shown - check for errors and return value)
if (FD_ISSET(listen_socket, &readset))
{
int new_client_socket = accept(listen_socket, &addr, &addrlength);
// create a struct that keeps track of the connection state data
struct ConnectionData client_connection = {};
client_connection.sock = new_client_socket;
client_connection.reading_request = 1; // awaiting for all the request bytes to come in
client_connections[number_of_client_connections++] = client_connection; // pseudo code, add the client_connection to the list
}
for (int i = 0; i < number_of_client_connections; i++)
{
if (client_connections[i].reading_request)
{
if (FD_ISSET(client_connections[i], &readset))
{
char buffer[2000];
int len = recv(client_connections[i].sock, buffer, 2000, 0);
// not shown - handle error case when (recv < 0)
// not shown - handle case when (recv == 0)
ProcessIncomingData(client_connections[i], buffer, len); // do all the request parsing here. Flip the client_connections[i].reading_request to 0 if ready to respond
}
}
else if (client_connections[i].reading_request == 0)
{
if (FD_ISSET(client_connections[i], &writeset))
{
client_connection* conn = &client_connections[i];
int len = send(conn->sock, conn->response_buffer + conn->txCount, conn->response_size - conn->txCount, 0);
conn->txCount += len;
if (conn->txCount == conn->response_size)
{
// done sending response - we can close this connection or change it to back to the reading state
}
}
}
}

How to use nanomsg survey architecture without while loop?

I was going through nanomsg usage for IPC and wanted to use SURVEY-archetype architecture described here.In this, processes run as client and server and exchange data. Now server has following code (also listed in the provided link):
int server (const char *url)
{
int sock = nn_socket (AF_SP, NN_SURVEYOR);
assert (sock >= 0);
assert (nn_bind (sock, url) >= 0);
sleep(1); // wait for connections
int sz_d = strlen(DATE) + 1; // '\0' too
printf ("SERVER: SENDING DATE SURVEY REQUEST\n");
int bytes = nn_send (sock, DATE, sz_d, 0);
assert (bytes == sz_d);
while (1)
{
char *buf = NULL;
int bytes = nn_recv (sock, &buf, NN_MSG, 0);
if (bytes == ETIMEDOUT) break;
if (bytes >= 0)
{
printf ("SERVER: RECEIVED \"%s\" SURVEY RESPONSE\n", buf);
nn_freemsg (buf);
}
}
return nn_shutdown (sock, 0);
}
Since socket type is NN_SURVEYOR, the while-loop doesn't wait at nn_recv. For client, socket type is NN_RESPONDENT and so, while loop in client waits at nn_recv.
Now since this while loop runs continuously and infinitely, the CPU usage shoots up to 99%. Can you please tell me that is there any other way to make survey architecture using nanomsg.
Try adding yield() call in the end of loop if nothing was received.

Trying to pipe data from a child-process server to its parent process

I'm working on an assignment for my Distributed Systems class. I'm a master's student in C.S., but my specialty in programming is .NET and I'm working on a project that requires some fairly involved Unix knowledge, which is tripping me up.
The assignment is implementing a flush channel protocol API. So I'm coding a small function library that other apps can implement to use flush channel communication. I've set it up so that when the init function is called, it forks a child process to act as the server for incoming messages. The child communicates with the parent process by sending incoming data to the parent through a pipe.
This works OK if messages are sent and received one at a time; e.g.,
send -> receive -> send -> receive -> etc.
However, if multiple messages are sent before doing any receives; e.g.,
send -> send -> send -> receive
then it gets messed up. Specifically, the first message is received correctly, but when I go to receive the second message, the program hangs and needs to be killed. I've done a lot of searching online and been plugging away at this for hours but haven't made much progress.
The program as a whole is far too large to show here, but here are the most relevant bits. Here's the part where I get the server going and receive messages. Note the line
write(fd[1], buffer, (strlen(buffer)+1));
-- I think that's a good candidate for being the source of the problem here, but not sure what to do differently. (Tried fwrite() and that didn't work at all.)
fd = malloc(2 * sizeof(int));
int nbytes;
if (pipe(fd) < 0) {
perror("Could not create pipe");
return -1;
}
pID = fork();
if (pID < 0) {
perror("Failed to fork");
return -1;
} else if (pID == 0) { // child
close(fd[0]); // close input side of pipe
int cc;
int fsize;
struct sockaddr_in from;
int serials[500];
int i;
for (i = 0; i < 500; i++) serials[i] = 0;
char buffer[2048];
while (1) {
fsize = sizeof(from);
cc = recvfrom(socketNo, buffer, 2048, 0, (struct sockaddr*)&from, &fsize);
if (cc < 0) perror("Receive error");
datagram data = decodeDatagram(buffer);
if (serials[data.serial] == 0) {
write(fd[1], buffer, (strlen(buffer)+1));
serials[data.serial] = 1;
}
}
} else { // parent
close(fd[1]); // close output side of pipe
return 0;
}
(The "serials" array is for not forwarding repeated messages, as messages are sent multiple times to improve reliability. I know a fixed size for this array is not good practice, but the tests for this assignment don't send that many messages so it's OK in this context.)
The beginning of the receive function looks like this:
int fRecv(int* id, char* buf, int nbytes) {
checkDatagramTable(*id);
char* tbuf = malloc((nbytes + 9) * sizeof(char));
int rbytes = read(fd[0], tbuf, nbytes + 9);
The "+9" is to accommodate additional information that gets packaged along with the message to be sent, for flush channel ordering. This is also a pretty sketchy area, but allocating more space to be extra sure has not helped the issue.
I know there's quite a bit of extraneous stuff in here, references to other functions etc. But the problem surely lies in how I'm piping the data through, so the source of my issue should lie there somewhere.
Thanks in advance for your assistance; it is truly appreciated.
This looks suspicious. (what is in the packets? They could be binary) Where is the typedefinition for datagram ?
fsize = sizeof(from);
cc = recvfrom(socketNo, buffer, 2048, 0, (struct sockaddr*)&from, &fsize);
if (cc < 0) perror("Receive error");
datagram data = decodeDatagram(buffer);
if (serials[data.serial] == 0) {
write(fd[1], buffer, (strlen(buffer)+1)); // <-- ????
serials[data.serial] = 1;
}
I'd try instead:
write(fd[1], buffer, cc);
UPDATE:
If the message is not null terminated, you'll have to terminate it explicitly:
(if cc == 2048) cc -= 1;
buffer [cc] = '\0'; // <<--
datagram data = decodedatagram(buffer);
...
Also, it is advisable to use "sizeof buffer" instead of "2048".
UPDATE2:
You could test if the strings in the packets are really null-terminated by:
unsigned pos;
cc = recvfrom(socketNo, buffer, 2048, 0, (struct sockaddr*)&from, &fsize);
if (cc < 0) perror("Receive error");
for pos=0; pos < cc; pos++) {
if (buff[pos] == 0) break;
}
switch (cc-pos) {
case 0: fprintf (stderr, "No nul byte found in packet: I lose!\n" ); break;
default: fprintf (stderr, "Spurious nul byte found in the middle of packet\n" );
case 1: break;
}
datagram data = decodeDatagram(buffer);
if (serials[data.serial] == 0) {
write(fd[1], buffer, cc);
serials[data.serial] = 1;
}

Resources