I want to make a non-blocking send() and recv() with select() and FD_ISSET(). I run into this behavior in which FD_ISSET() will be true for the first socket, and all other sockets are always not ready. I am confused why that is happening and I am wondering if I am using select() properly.
If I sent 100 requests, eventually one of the socket other than the first will be ready to recv(), but I am not getting that behavior.
for(int i = 0; i < P - 1; i++) {
sockArr[i] = GetSocket(server, port);
if (sockArr[i] < 0) {
// handle error
}
fcntl(sockArr[i], F_SETFL, O_NONBLOCK);
if(sockArr[i] > maxfd) {
maxfd = sockArr[i];
}
}
fd_set sockSet;
for(int i = 0; i < P - 1; i++) {
numBytes = send(sockArr[i], request, requestLen, 0);
if (numBytes < 0 || numBytes != requestLen) {
// handle error
}
// read from other stackoverflow post you need to rest
// per select() call
FD_ZERO(&sockSet);
for(int i = 0; i < P - 1; i++) {
FD_SET(sockArr[i], &sockSet);
}
if(select(maxfd+1, &sockSet, NULL, NULL, NULL)) {
for(int j = 0; j < i; j++) {
if(FD_ISSET(sockArr[j], &sockSet)) { // <------ only true for j = 0
numBytes = recv(sockArr[j], buffer, MAXBUFLEN - 1, 0);
if (numBytes < 0) {
// handle error
}
}
}
}
}
Your first for loop needs to terminate after the send() calls, and you should then start another to handle the selects and receives.
You need to check for recv() returning zero as well as -1, and close the socket and remove it from the FD set in either case.
You also need to loop while j <= P.
Related
I need to make the code more readable, but it has a lot of cycle, can I fix this somehow?
The variable i constantly jumps there and there, it is very inconvenient to watch in debugging.
for(;;)
{
//
// For connection orientated protocols, we will handle the
// packets received from a connection collectively. For datagram
// protocols, we have to handle each datagram individually.
//
//
// Check to see if we have any sockets remaining to be served
// from previous time through this loop. If not, call select()
// to wait for a connection request or a datagram to arrive.
//
for (i = 0; i < numSocket; i++)
{
if (FD_ISSET(servSock[i], &SockSet))
break;
}
if (i == numSocket)
{
for (i = 0; i < numSocket; i++)
{
FD_SET(servSock[i], &SockSet);
}
if (select(numSocket, &SockSet, 0, 0, 0) == SOCKET_ERROR)
continue;
}
for (i = 0; i < numSocket; i++)
{
if (FD_ISSET(servSock[i], &SockSet))
{
FD_CLR(servSock[i], &SockSet);
break;
}
}
...
}
To make the code more readable (and testable and maintainable) you can encapsulate some of the logic into separate functions with meaningful names. Consider this version of your loop
for(;;)
{
int i = find_socket_to_be_served(numSocket, servSock, &SockSet);
if (i == numSocket)
{
set_all_sockets(numSocket, servSock, &SockSet);
if (select(numSocket, &SockSet, 0, 0, 0) == SOCKET_ERROR)
continue;
}
// ...
}
Where the two functions used are defined as
int find_socket_to_be_served(int n_sockets, int *fds, fd_set *fdset)
{
int i = 0;
for (; i < n_sockets; i++)
{
if (FD_ISSET(fds[i], fdset))
{
FD_CLR(fds[i], fdset);
break;
}
}
return i;
}
void set_all_sockets(int n_sockets, int *fds, fd_set *fdset)
{
for (int i = 0; i < n_sockets; i++)
{
FD_SET(fds[i], fdset);
}
}
As far as I understand your code, you don't need FD_CLEAR() your sockets one by one.
Just FD_ZERO() all of them at once, then use a single loop to FD_SET() them again.
Note that after select(), many sockets could be ready to use; you don't have to break when one has been used.
Note also that the first parameter of select() is unused on Windows but should be 1+ the max of your socket file descriptors (not the number of them) everywhere else.
for(;;)
{
FD_ZERO(&SockSet);
for (int i = 0; i < numSocket; i++)
{
FD_SET(servSock[i], &SockSet);
}
if (select(numSocket, &SockSet, 0, 0, 0) == SOCKET_ERROR)
continue; // or probably break...
}
for (int i = 0; i < numSocket; i++)
{
if (FD_ISSET(servSock[i], &SockSet))
{
// use servSock[i]
}
}
}
It's my first post so ask for remotely anything if it can help and I didn't provide it.
My application requires multiple sockets being opened at once from Master, then the slaves connect to WiFi, and then to the sockets
Problem is: I have to make it "bulletproof" against constant reconnecting from slaves and i get Accept error:
E (23817) TCP SOCKET: accept error: -1 Too many open files in system
It appears when I reconnect client for 5th time, when Max Number of Open Sockets = 5 in menuconfig,
I disconnect clients from the server when they don't send anything in 1second -> then i assume they got DC-d.
I do it with close() procedure.
void closeOvertimedTask(void * ignore)
{
while(1)
{
for(int i = 0; i < openedSockets;)
{
if(needsRestart[i] == 1)
{
ESP_LOGI("RESTARTING", " task#%d",i);
//lwip_close_r(clientSock[i]);
//closesocket(clientSock[i]);
//ESP_LOGI("closing result", "%d", close(clientSock[i]));
stopSocketHandler(i);
needsRestart[i] = 0;
//if(isSocketOpened[i])
{
}
ESP_LOGI("close", "%d", lwip_close_r(clientSock[i]));
isSocketOpened[i] = 0;
xTaskCreate( handleNthSocket, "TCP_HANDLER", 10*1024, &(sockNums[i]) , tskIDLE_PRIORITY, &socketHandlerHandle[i]);
configASSERT(socketHandlerHandle[i]);
needsRestart[i] = 0;
}
if(isSocketOpened[i])
{
int diff = ((int)((uint64_t)esp_timer_get_time()) - lastWDT[i]) - 2*TCPWDT;
if(diff > 0)
{
if(isSocketOpened[i])
{
ESP_LOGI("I FOUND OUT HE DC-d","");
//closesocket(clientSock[i]);
}
ESP_LOGI("close", "%d", close(clientSock[i]));
stopSocketHandler(i);
isSocketOpened[i] = 0;
xTaskCreate( handleNthSocket, "TCP_HANDLER", 10*1024, &(sockNums[i]) , tskIDLE_PRIORITY, &socketHandlerHandle[i]);
configASSERT(socketHandlerHandle[i]);
}
}
}
}
}
For each socket I run 1 task that is supposed to receive from that socket and act further.
For all of them I have an other task that checks last time a message arrived and restarts tasks when time has exceeded (it's 2 seconds)
I need around 16 sockets opened in the final version so there is no room to have sockets that are still closing after Slave has restarted whole connection
How to properly close a Task with running recv() procedure in it to properly close Socket.
Is there a way to read from Server side that socket has been closed if WiFi hasn't realized STA DC-d
Is this about TIME_WAIT from tcp stack ?
Socket read code:
void handleNthSocket(void * param) // 0 <= whichSocket < openedSockets
{
int whichSocket = *((int *) param);
ESP_LOGI("TCP SOCKET", "%s #%d", getSpaces(whichSocket), whichSocket);
struct sockaddr_in clientAddress;
while (1)
{
if(needsRestart [whichSocket] == 0)
{
socklen_t clientAddressLength = sizeof(clientAddress);
clientSock[whichSocket] = accept(sock[whichSocket], (struct sockaddr *)&clientAddress, &clientAddressLength);
if (clientSock[whichSocket] < 0)
{
ESP_LOGE("TCP SOCKET", "accept error: %d %s", clientSock[whichSocket], strerror(errno)); //HERE IT FLIPS
//E (232189) TCP SOCKET: accept error: -1 Too many open files in system
isSocketOpened[whichSocket] = 0;
needsRestart[whichSocket] = 1;
continue;
}
//isSocketOpened[whichSocket] = 1;
// We now have a new client ...
int total = 1000;
char dataNP[1000];
char *data;
data = &dataNP[0];
for(int z = 0; z < total; z++)
{
dataNP[z] = 0;
}
ESP_LOGI("TCP SOCKET", "%snew client",getSpaces(whichSocket));
ESP_LOGI(" ", "%s#%d connected",getSpaces(whichSocket), whichSocket);
lastWDT[whichSocket] = (uint64_t)esp_timer_get_time() + 1000000;
isSocketOpened[whichSocket] = 1;
// Loop reading data.
while(isSocketOpened[whichSocket])
{
/*
if (sizeRead < 0)
{
ESP_LOGE(tag, "recv: %d %s", sizeRead, strerror(errno));
goto END;
}
if (sizeRead == 0)
{
break;
}
sizeUsed += sizeRead;
*/
ssize_t sizeRead = recv(clientSock[whichSocket], data, total, 0);
/*for (int k = 0; k < sizeRead; k++)
{
if(*(data+k) == '\n')
{
ESP_LOGI("TCP DATA ", "%sthere was enter", getSpaces(whichSocket));
//ESP_LOGI("TIME ", "%d", (int)esp_timer_get_time());
}
//ESP_LOGI("last wdt", "%d", (int)lastWDT[whichSocket]);
}*/
lastWDT[whichSocket] = (uint64_t)esp_timer_get_time();
int diff = ((int)((uint64_t)esp_timer_get_time()) - lastWDT[whichSocket]) - 2*TCPWDT;
ESP_LOGI("last wdt", "%d, data = %s", (int)lastWDT[whichSocket], data);
if(diff > 0)
{
ESP_LOGI("last wdt", "too long - %d", diff);
isSocketOpened[whichSocket] = 0;
}
if (sizeRead < 0)
{
isSocketOpened[whichSocket] = 0;
}
//TODO: all RX from slave routine
for(int k = 0; k < sizeRead; k++)
{
*(data+k) = 0;
}
// ESP_LOGI("lol data", "clientSock[whichSocket]=%d,
/*if(sizeRead > -1)
{
ESP_LOGI("TCP DATA: ", "%c", *(data + sizeRead-1));
}
else
{
ESP_LOGI("TCP DC ", "");
goto END;
}*/
}
if(isSocketOpened[whichSocket])
{
ESP_LOGI("closing result", "%d", close(clientSock[whichSocket]));
}
}
}
}
I don't see you closing your sockets anywhere?
Sockets, no matter the platform, is usually a limited resource, and a resource that will be reused. If you don't close the sockets then the system will think that you still use then, and can't reuse those sockets for new connections (and on POSIX systems even opening files will be affected).
So close connections immediately when they are not needed any more.
Usually this is done by checking what recv and send returns: If they return a value less than zero an error occured and in most cases it's a non-recoverable errors, so connection should be closed. Even if it is a recoverable error, it's easier to close the connection and let the client reconnect.
For recv there's also the special case when it returns zero. That means the other end has closed the connection. That of course you need to close your end as well.
this post solved all my problems
https://www.esp32.com/viewtopic.php?t=911
Got some trouble with TCP socket multiplexing.
//socket is non-blocking
const int MAX = 4096;
char *buff[MAX];
char *p = buff;
int fd, rvalue;
rvalue = 0;
if ( (fd = open(path, O_RDONLY)) < 0 ) {
return errno;
} else {
int didsend, didread;
int shouldsend;
while ((didread = read(fd, buff, MAX)) > 0) {
p = buff;
shouldsend = didread;
while ( 1 ) {
didsend = send(sockfd, p, shouldsend, 0);
//if send succeeds and returns the number of bytes fewer than asked for then try to send rest part in next time.
if (didsend < shouldsend) {
p += didsent;
shouldsend -= didsend;
continue;
}
//if there is no place for new data to send, then wait a brief time and try again.
if ( didsend < 0 && (errno == EWOULDBLOCK || errno == EAGAIN) ) {
usleep(1000);
continue;
}
//if all data has been sent then sending loop is over.
if (didsend == shouldsend) {
break;
}
//send error
if ( didsend < 0 ) {
rvalue = errno;
break;
}
}
}
close(fd);
if (didread == -1) {
return errno;
}
return rvalue;
}
Assume I use an I/O Multiplexing function poll() or kqueue(), and non-blocking socket, then if there are only some small data like send a short message, it works fine.
But if it comes to large data, I mean larger than send()'s buffer size, since using non-blocking socket, send() will just send a portion of data, and return how much data it sends, the rest part of data can only be sent in another call of send(), but it takes time, and can't tell how long it will takes. So the second while() is actually a blocking send which using non-blocking socket.
Equivalent to：
//socket is blocking
const int MAX = 4096;
char *buff[MAX];
int fd, n;
if ( (fd = open(path, O_RDONLY)) < 0 ) {
return errno;
} else {
while ((n = read(fd, buff, MAX)) > 0) {
if (send(sockfd, buff, n, 0) < 0) {
return errno;
}
}
close(fd);
return 0;
}
So, what is the solution to this, multithreading might work but that's kind of wasting resource maybe.
This is the general pattern for a single-threaded server that works with multiple connections and non-blocking sockets.
It's primarily pseudo-code in C and doesn't do the necessary error checking. But it gives you an idea that for each accepted connection, you keep a struct instance that maintains the socket handle, request parsing state, response stream, and any other "state" members of that connection. Then you just loop using "select" to wait or having multiple threads doing this same thing.
Again this is only pseudo-code and uses select/poll as an example. You can get even more scalability with epoll.
while (1)
{
fd_set readset = {};
fd_set writeset = {};
for (int i = 0; i < number_of_client_connections; i++)
{
if (client_connections[i].reading_request)
FD_SET(client_connection.sock, &readset);
else
FD_SET(client_connection.sock, &writeset);
}
// add the listen socket to the read set
FD_SET(listen_socket, &readset);
select(n + 1, &readset, &writeset, &timeout); // wait for a socket to be ready (not shown - check for errors and return value)
if (FD_ISSET(listen_socket, &readset))
{
int new_client_socket = accept(listen_socket, &addr, &addrlength);
// create a struct that keeps track of the connection state data
struct ConnectionData client_connection = {};
client_connection.sock = new_client_socket;
client_connection.reading_request = 1; // awaiting for all the request bytes to come in
client_connections[number_of_client_connections++] = client_connection; // pseudo code, add the client_connection to the list
}
for (int i = 0; i < number_of_client_connections; i++)
{
if (client_connections[i].reading_request)
{
if (FD_ISSET(client_connections[i], &readset))
{
char buffer[2000];
int len = recv(client_connections[i].sock, buffer, 2000, 0);
// not shown - handle error case when (recv < 0)
// not shown - handle case when (recv == 0)
ProcessIncomingData(client_connections[i], buffer, len); // do all the request parsing here. Flip the client_connections[i].reading_request to 0 if ready to respond
}
}
else if (client_connections[i].reading_request == 0)
{
if (FD_ISSET(client_connections[i], &writeset))
{
client_connection* conn = &client_connections[i];
int len = send(conn->sock, conn->response_buffer + conn->txCount, conn->response_size - conn->txCount, 0);
conn->txCount += len;
if (conn->txCount == conn->response_size)
{
// done sending response - we can close this connection or change it to back to the reading state
}
}
}
}
I have select with N pipe, my problem:
After catch ,,one" pipe - select don't work. Probably I don't remove i
descriptor. And here is my questions:
1) Do I really have to make dynamic array like vector in C and insert and remove i
2) How to make my code better? How Can I solve problem with select?
My code:
while(1)
{
sel = select(val+1,&set,NULL,NULL,&timeout);
if(sel < 0)
perror("Blad funkcji select");
else if(sel == 0)
printf("Brak komunikatow \n");
else{
for(i = 0; i < val; i++)
{
if(FD_ISSET(fd[i][0],&set))
{
while(read(fd[i][0],&buf,rozmiar) > 0)
write(1,&buf,rozmiar);
} // check if exist and write to stdout
} // end SELECT
for(i = 0, j =0; i< val; i++)
{
FD_SET(fd[i][0], &set);
}
timeout.tv_sec = 4;
timeout.tv_usec = 0;
}
RETURN VALUE top
On success, select() and pselect() return the number of file
descriptors contained in the three returned descriptor sets (that is,
the total number of bits that are set in readfds, writefds,
exceptfds) which may be zero if the timeout expires before anything
interesting happens. On error, -1 is returned, and errno is set to
indicate the error; the file descriptor sets are unmodified, and
timeout becomes undefined.
if(sel ==-1)
perror("Blad funkcji select");
I am facing some trouble dealing with zombie processes. I wrote a simple server which creates tic tac toe matches between players. I am using select() to multiplex between multiple connected clients. Whenever there are two clients, the server will fork another process which execs a match arbiter program.
The problem is that select() blocks. So therefore, say if there is a match arbiter program running as a child process and it exits, the parent will never wait for the child if there are no incoming connections because select() is blocking.
I have my code here, apologies since it is quite messy.
while(1) {
if (terminate)
terminate_program();
FD_ZERO(&rset);
FD_SET(tcp_listenfd, &rset);
FD_SET(udpfd, &rset);
maxfd = max(tcp_listenfd, udpfd);
/* add child connections to set */
for (i = 0; i < MAXCLIENTS; i++) {
sd = tcp_confd_lst[i];
if (sd > 0)
FD_SET(sd, &rset);
if (sd > maxfd)
maxfd = sd;
}
/* Here select blocks */
if ((nready = select(maxfd + 1, &rset, NULL, NULL, NULL)) < 0) {
if (errno == EINTR)
continue;
else
perror("select error");
}
/* Handles incoming TCP connections */
if (FD_ISSET(tcp_listenfd, &rset)) {
len = sizeof(cliaddr);
if ((new_confd = accept(tcp_listenfd, (struct sockaddr *) &cliaddr, &len)) < 0) {
perror("accept");
exit(1);
}
/* Send connection message asking for handle */
writen(new_confd, handle_msg, strlen(handle_msg));
/* adds new_confd to array of connected fd's */
for (i = 0; i < MAXCLIENTS; i++) {
if (tcp_confd_lst[i] == 0) {
tcp_confd_lst[i] = new_confd;
break;
}
}
}
/* Handles incoming UDP connections */
if (FD_ISSET(udpfd, &rset)) {
}
/* Handles receiving client handles */
/* If client disconnects without entering their handle, their values in the arrays will be set to 0 and can be reused. */
for (i = 0; i < MAXCLIENTS; i++) {
sd = tcp_confd_lst[i];
if (FD_ISSET(sd, &rset)) {
if ((valread = read(sd, confd_handle, MAXHANDLESZ)) == 0) {
printf("Someone disconnected: %s\n", usr_handles[i]);
close(sd);
tcp_confd_lst[i] = 0;
usr_in_game[i] = 0;
} else {
confd_handle[valread] = '\0';
printf("%s\n", confd_handle); /* For testing */
fflush(stdout);
strncpy(usr_handles[i], confd_handle, sizeof(usr_handles[i]));
for (j = i - 1; j >= 0; j--) {
if (tcp_confd_lst[j] != 0 && usr_in_game[j] == 0) {
usr_in_game[i] = 1; usr_in_game[j] = 1;
if ((child_pid = fork()) == 0) {
close(tcp_listenfd);
snprintf(fd_args[0], sizeof(fd_args[0]), "%d", tcp_confd_lst[i]);
snprintf(fd_args[1], sizeof(fd_args[1]), "%d", tcp_confd_lst[j]);
execl("nim_match_server", "nim_match_server", usr_handles[i], fd_args[0], usr_handles[j], fd_args[1], (char *) 0);
}
close(tcp_confd_lst[i]); close(tcp_confd_lst[j]);
tcp_confd_lst[i] = 0; tcp_confd_lst[j] = 0;
usr_in_game[i] = 0; usr_in_game[j] = 0;
}
}
}
}
}
}
Is there a method which allows wait to run even when select() is blocking? Preferably without signal handling since they are asynchronous.
EDIT: Actually, I found out that select has a timeval data structure which we can specify the timeout. Would using that be a good idea?
I think your options are:
Save all your child descriptors in a global array and call wait() from a signal handler. If you don't need the exit status of your children in your main loop, I think this is the easiest.
Instead of select, use pselect -- it will return upon receiving a specified (set of) signal(s), in your case, SIGCHLD. Then call wait/WNOHANG on all child PIDs. You will need to block/unblock SIGCHLD at the right moments before/after pselect(), see here: http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html
Wait on/cleanup child PIDs from a secondary thread. I think this is the most complicated solution (re. synchronization between threads), but since you asked, it's technically possible.
If you just want to prevent zombie processes, you could set up a SIGCHLD signal handler. If you want to actually wait for the return status, you could write bytes into a pipe (non-blocking, just in case) from the signal handler and then read those bytes in the select loop.
For how to handle SIGCHLD, see http://www.microhowto.info/howto/reap_zombie_processes_using_a_sigchld_handler.html -- you want to do something like while (waitpid((pid_t)(-1), 0, WNOHANG) > 0) {}
Perhaps the best approach is sending a single byte from the SIGCHLD signal handler to the main select loop (non-blocking, just in case) and doing the waitpid loop in the select loop when bytes can be read from the pipe.
You could also use a signalfd file descriptor to read the SIGCHLD signal, although that works only on Linux.