I have a blocking accept() call in a thread that is waiting for connection requests. When the application is about to close, I want to signal the thread that is waiting on accept() to exit gracefully. I have found in the documentation for Winsock that I can set a timeout value for send() and recv(), but I can't do that for accept().
I have read that I can make the socket non-blocking and use select(), and pass a timeout value for select(), but I am looking for a solution for blocking sockets.
I have read that I can make the socket non-blocking and use select(), and pass a timeout value for select(), but I am looking for a solution for blocking sockets.
You can do this on blocking socket:
sock = socket(...);
bind(sock, ...);
listen(sock, ...);
while (program_running())
{
timeval timeout = {1, 0};
fd_set fds;
FD_ZERO(&fds);
FD_SET(sock, &fds);
select(sock+1, &fds, NULL, NULL, &timeout);
if (FD_ISSET(sock, &fds))
{
client = accept(sock, ...);
// do things with client
}
From MSDN accept function documentation:
The parameter readfds identifies the sockets that are to be checked for readability. If the socket is currently in the listen state, it will be marked as readable if an incoming connection request has been received such that an accept is guaranteed to complete without blocking.
Related
I wonder about the behaviour of socket on blocking and nonblocking actions. What happens to threads blocking on socket when the socket blocking mode changes ? Here is the scenario;
thread1(T1) creates a UDP socket and
fd = socket(AF_INET , SOCK_DGRAM, 0);
T1 waiting(sleeping) for receive
recv(fd, buf , sizeof(buf) , 0);
and thread2(T2) changes socket mode to non-blocking before socket receive any data
fcntl(fd, F_SETFL, O_NONBLOCK);
what happens to T1 ? Is it signalled/waked because the socket is no more blocking ?
The behavior is literally unspecified: fcntl is not required to unblock any threads.
Linux just sets the flag in the file description struct file and returns without unblocking any blocked threads.
A thread already blocked in recv can be scheduled to run only if:
data to read becomes available;
or an error condition on the file descriptor is detected (FIN/RST, socket read timeout, TCP keep-alive failure, the file descriptor is closed by another thread);
or a signal is received and the signal disposition does not include SA_RESTART;
or it is pthread_cancelled.
The fact that you are trying to change flags of a file descriptor of another thread suggests that your design requires a review. Ideally, threads must not share any data and not poke at each other's state, rather they should use message passing to communicate with each other.
You made me curious. First, it's quite obvious that since there is no standard that specifies that the socket should wake up, it will not be awoken because it would be quite a pain to implement this (since the non-blocking flag is in a different layer than where the socket is blocking). So we can say quite confidently that the socket will not wake up until we receive a packet. Or will it?
#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <err.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <string.h>
#include <time.h>
#include <fcntl.h>
int sock;
static void
sighand(int s)
{
write(1, "sig\n", 4);
}
static void *
rcv(void *v)
{
struct sigaction sa;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = sighand;
sa.sa_flags = SA_RESTART;
if (sigaction(SIGUSR1, &sa, NULL) == -1)
err(1, "sigaction");
char buf[64];
ssize_t sz = recv(sock, buf, sizeof(buf), 0);
printf("recv %d\n", (int)sz);
return NULL;
}
pthread_t t1, t2;
static void *
beeper(void *v)
{
for (;;) {
nanosleep(&((struct timespec){.tv_sec = 1}), NULL);
if (pthread_kill(t1, SIGUSR1))
errx(1, "pthread_kill");
printf("beep\n");
}
}
int
main(int argc, char **argv)
{
if ((sock = socket(PF_INET, SOCK_DGRAM, 0)) == -1)
err(1, "socket");
struct sockaddr_in sin;
memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_port = htons(4711);
sin.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(sock, (struct sockaddr *)&sin, sizeof(sin)) == -1)
err(1, "bind");
if (pthread_create(&t1, NULL, rcv, NULL))
errx(1, "pthread_create");
if (pthread_create(&t2, NULL, beeper, NULL))
errx(1, "pthread_create");
/* pretend that this is correct synchronization. */
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
printf("setting non-block\n");
if (fcntl(sock, F_SETFL, O_NONBLOCK) == -1)
err(1, "fcntl");
printf("set\n");
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
return 0;
}
The code above (sorry, couldn't make it shorter). Blocks the thread in recv, waits a bit and then sets non-blocking on the file descriptor. As expected nothing happens. But then I added a twist. The receiving thread has a signal handler that wakes up once a second with SA_RESTART. Of course, since we have SA_RESTART recv should not wake up.
On OSX this will "misbehave" if we consider any behavior correct. I'm pretty sure this will behave the same way on all BSDs. Same on Linux. In fact, the way SA_RESTART is usually implemented I'm pretty sure this will "misbehave" pretty much everywhere.
Funny. This of course doesn't mean that the code above is useful for anything, but it's an interesting curiosity. To answer your question, this is unspecified, but in most cases it won't wake up, unless it will. Please don't do weird things like this.
I think POSIX specification is pretty clear about this:
If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives. If no messages are available at the socket and O_NONBLOCK is set on the socket's file descriptor, recv() shall fail and set errno to [EAGAIN] or [EWOULDBLOCK].
You call recv when O_NONBLOCK is not yet set, to it should block until message arrives (not until mode changes).
Unblocked thread (didn't block yet due to reading when it was in blocking mode) will behave as if it was always a non-blocking mode.
Reading recv(2) manual page
If no messages are available at the socket, and if the socket is nonblocking, the value -1 is returned and the external
variable errno is set to EAGAIN or EWOULDBLOCK.
Blocked threads (blocked while reading when in blocking mode) before changing to non-blocking.
As pointed by #Maxim sharing the code of the function which doesn't awaken threads, The blocked threads will only be awaken after a write is done (data is available).
I am aware that select() will be triggered whenever there is a data in the registered socket buffer.
what will happen if there is a delay between these two statements.
FD_SET(listener, &read_fds); // &
(select(fdmax+1, &read_fds, NULL, NULL, NULL) == -1)
what will happen if packet arrives between these two statements?
//create socket and listen for packets &
FD_SET(listener, &read_fds);
Assume that recv() is done once select is triggered.
What will happen if a packet arrives before the select() call is
made.?
does FD_ISSET still detects the packet which is already in
socket buffer or it will be detected only if new packet arrives and
select gets triggered?
Sample code:
// add the listener to the master set
FD_SET(listener, &master);
// keep track of the biggest file descriptor
fdmax = listener; // so far, it's this one
// main loop
for(;;) {
read_fds = master; // copy it
if (select(fdmax+1, &read_fds, NULL, NULL, NULL) == -1) {
perror("select");
exit(4);
}
// run through the existing connections looking for data to read
for(i = 0; i <= fdmax; i++) {
if (FD_ISSET(i, &read_fds)) { // we got one!!
Understanding gap between socket creation and select() system call
There is no gap between socket creation and select() in your question.
I am aware that select() will be triggered whenever there is a data in the registered socket buffer.
That's true for read events and it applies to the socket receive buffer of connected sockets. It also triggers when there is an inbound connection on a listening socket, or room in the socket send buffer for send events.
what will happen if there is a delay between these two statements.
FD_SET(listener, &read_fds); // &
(select(fdmax+1, &read_fds, NULL, NULL, NULL) == -1)
Nothing bad. Any event that occurs between them will still be signalled. But the first statement isn't a socket creation, contrary to your title.
what will happen if packet arrives between these two statements?
//create socket and listen for packets &
FD_SET(listener, &read_fds);
The socket send buffer exists from the moment the socket is created, so the data will go into the buffer, so when select() runs it will see that and trigger a read event.
Assume that recv() is done once select is triggered.
What will happen if a packet arrives before the select() call is made.?
The socket send buffer exists from the moment the socket is created, so the data will go into the buffer, so when select() runs it will see that and trigger a read event.
does FD_ISSET still detects the packet which is already in socket buffer
Yes.
or it will be detected only if new packet arrives and select gets triggered?
It will always be detected.
If data is waiting to be read, select will return immediately, and FD_ISSET will return true for the file descriptor that the data arrived on. It doesn't matter if data arrived before or after select was called.
select() completes immediately if one or more of the watched conditions is already active; otherwise it blocks until one or more of the watched conditions becomes active (or the timeout, if specified, expires).
I need to perform some operations only after the time a TCP connection is fully closed, that's to say - all the data segments, as well as the finishing routine (FIN-ACK or RST) have been performed and done, and no packets will be sent on the wires.
Since closesocket() is not synchronous and could return before a full close of the connection and socket, I've used the SO_LINGER option to get the moment of closing.
According to the instructions in the MSDN for closesocket, and to the fact that my socket is non-blocking (and thus asynchronous), I wrote this code:
int ret;
/* config 2 secs of linger */
struct linger lng = {1, 2};
setsockopt(s, SOL_SOCKET, SO_LINGER, (const char*)&lng, sizeof(lng));
/* graceful close of TCP (FIN-ACK for both sides) */
shutdown(s, SD_BOTH);
/* linger routine for asynchronous sockets according to msdn */
do {
ret = closesocket(s);
} while (ret == SOCKET_ERROR && WSAGetLastError() == WSAEWOULDBLOCK);
/* my code to be run immediately after all the traffic */
printf("code to run after closing");
However, the closesocket call returns zero (success; instead of getting into the loop) and I see in Wireshark that my final printing is called before all the packets were sent, so - it looks like the linger isn't working.
By the way, the functions I used to open and connect the asynchronous socket were socket(), WSAIoctl() and its lpfnConnectEx() callback.
What's the reason that the lingered closesocket return before a full finish of the TCP connection? Is there a solution?
Is there any way to check if there are no pending connection requests to the server?
In my case, I have this code, from C:
listen(mysocket, 3);
//while(no pending connections) do this
int consocket = accept(mysocket, (struct sockaddr *)&dest, &socksize);
And what I need, is that while there are no pending connections, do something else, instead of getting stuck waiting, by calling accept().
You can set a socket in non-blocking mode with fcntl.
fcntl(sockfd, F_SETFD, O_NONBLOCK);
After that, a call to accept(sockfd) will either immediately return a newly accepted connection or immediately fail with EWOULDBLOCK which you can use to decide to do “something else”.
Another option would be to use select, maybe with a finite timeout, to gain finer grained control over blocking.
You can spawn a new thread and do your alternate work in that thread while the main thread waits for clients. Once a client is connected, accept() gets unblocked and then you can cancel the newly spawned thread.
The code should be on the lines of:
while(1) {
pthread_create(&thread, NULL, do_something_else, NULL);
consocket = accept(mysocket, (struct sockaddr *)&dest, &socksize);
pthread_cancel(thread);
}
Is it possible to bind and listen to multiple ports in Linux in one application?
For each port that you want to listen to, you:
Create a separate socket with socket.
Bind it to the appropriate port with bind.
Call listen on the socket so that it's set up with a listen queue.
At that point, your program is listening on multiple sockets. In order to accept connections on those sockets, you need to know which socket a client is connecting to. That's where select comes in. As it happens, I have code that does exactly this sitting around, so here's a complete tested example of waiting for connections on multiple sockets and returning the file descriptor of a connection. The remote address is returned in additional parameters (the buffer must be provided by the caller, just like accept).
(socket_type here is a typedef for int on Linux systems, and INVALID_SOCKET is -1. Those are there because this code has been ported to Windows as well.)
socket_type
network_accept_any(socket_type fds[], unsigned int count,
struct sockaddr *addr, socklen_t *addrlen)
{
fd_set readfds;
socket_type maxfd, fd;
unsigned int i;
int status;
FD_ZERO(&readfds);
maxfd = -1;
for (i = 0; i < count; i++) {
FD_SET(fds[i], &readfds);
if (fds[i] > maxfd)
maxfd = fds[i];
}
status = select(maxfd + 1, &readfds, NULL, NULL, NULL);
if (status < 0)
return INVALID_SOCKET;
fd = INVALID_SOCKET;
for (i = 0; i < count; i++)
if (FD_ISSET(fds[i], &readfds)) {
fd = fds[i];
break;
}
if (fd == INVALID_SOCKET)
return INVALID_SOCKET;
else
return accept(fd, addr, addrlen);
}
This code doesn't tell the caller which port the client connected to, but you could easily add an int * parameter that would get the file descriptor that saw the incoming connection.
You only bind() to a single socket, then listen() and accept() -- the socket for the bind is for the server, the fd from the accept() is for the client. You do your select on the latter looking for any client socket that has data pending on the input.
In such a situation, you may be interested by libevent. It will do the work of the select() for you, probably using a much better interface such as epoll().
The huge drawback with select() is the use of the FD_... macros that limit the socket number to the maximum number of bits in the fd_set variable (from about 100 to 256). If you have a small server with 2 or 3 connections, you'll be fine. If you intend to work on a much larger server, then the fd_set could easily get overflown.
Also, the use of the select() or poll() allows you to avoid threads in the server (i.e. you can poll() your socket and know whether you can accept(), read(), or write() to them.)
But if you really want to do it Unix like, then you want to consider fork()-ing before you call accept(). In this case you do not absolutely need the select() or poll() (unless you are listening on many IPs/ports and want all children to be capable of answering any incoming connections, but you have drawbacks with those... the kernel may send you another request while you are already handling a request, whereas, with just an accept(), the kernel knows that you are busy if not in the accept() call itself—well, it does not work exactly like that, but as a user, that's the way it works for you.)
With the fork() you prepare the socket in the main process and then call handle_request() in a child process to call the accept() function. That way you may have any number of ports and one or more children to listen on each. That's the best way to really very quickly respond to any incoming connection under Linux (i.e. as a user and as long as you have child processes wait for a client, this is instantaneous.)
void init_server(int port)
{
int server_socket = socket();
bind(server_socket, ...port...);
listen(server_socket);
for(int c = 0; c < 10; ++c)
{
pid_t child_pid = fork();
if(child_pid == 0)
{
// here we are in a child
handle_request(server_socket);
}
}
// WARNING: this loop cannot be here, since it is blocking...
// you will want to wait and see which child died and
// create a new child for the same `server_socket`...
// but this loop should get you started
for(;;)
{
// wait on children death (you'll need to do things with SIGCHLD too)
// and create a new children as they die...
wait(...);
pid_t child_pid = fork();
if(child_pid == 0)
{
handle_request(server_socket);
}
}
}
void handle_request(int server_socket)
{
// here child blocks until a connection arrives on 'server_socket'
int client_socket = accept(server_socket, ...);
...handle the request...
exit(0);
}
int create_servers()
{
init_server(80); // create a connection on port 80
init_server(443); // create a connection on port 443
}
Note that the handle_request() function is shown here as handling one request. The advantage of handling a single request is that you can do it the Unix way: allocate resources as required and once the request is answered, exit(0). The exit(0) will call the necessary close(), free(), etc. for you.
In contrast, if you want to handle multiple requests in a row, you want to make sure that resources get deallocated before you loop back to the accept() call. Also, the sbrk() function is pretty much never going to be called to reduce the memory footprint of your child. This means it will tend to grow a little bit every now and then. This is why a server such as Apache2 is setup to answer a certain number of requests per child before starting a new child (by default it is between 100 and 1,000 these days.)