As described in Beej's Guide to network Programming, select() monitors a set of file descriptors for reading (using recv()), a set of file descriptor for writing (using send()) and the last one, I don't know. When the server socket receives message from client sockets, read_fds set will be modified and select() return from blocking status. It is the same for sending message to client sockets. For example:
for(;;) {
read_fds = master; // copy it
if (select(fdmax+1, &read_fds, NULL, NULL, NULL) == -1) {
perror("select");
exit(4);
}
//the rest is code for processing ready socket
I guess the read_fds set will contain the only ready socket descriptor at this point (the others are removed), and the ready socket descriptor is the new connected socket or message sent from connected socket. Is my understanding correct?
It seems the ready socket must be handled one by one. When I tried to run it on gdb to understand the behavior, while the program was processing the ready socket (the code after select() return), I tried to send some message and connect to the server by some new clients. How can it recognize the new clients or newly sent message, even if select() is not called?
As described in Beej's Guide to network Programming, select() monitors a set of file descriptors for reading (using recv()), a set of file descriptor for writing (using send())
Yes
and the last one, I don't know.
The last one no longer has any useful meaning.
I guess the read_fds set will contain the only ready socket descriptor at this point (the others are removed), and the ready socket descriptor is the new connected socket or message sent from connected socket. Is my understanding correct?
That's correct.
It seems the ready socket must be handled one by one. When I tried to run it on gdb to understand the behavior, while the program was processing the ready socket (the code after select() return), I tried to send some message and connect to the server by some new clients. How can it recognize the new clients or newly sent message, even if select() is not called?
Normally when you create a polling loop such as this, you'd add new sockets to the loop. That is, you'd add them to the appropriate fd_sets before your next call to select.
When the new socket becomes writable, you'd send on it.
When you're dealing with multiple sockets that may potentially block (in your case reading sockets), you need to determine which sockets have data in them waiting to be read. You can do this by calling select() and adding the sockets to your read_set.
For your listening socket, if you call accept() and there is no pending connection, then your accept will block until a new connection arrives. So you also want to select() this socket. Once you accept that client, you will want to add that to your read_set.
e.g. Pseudo-code
for (;;) {
struct timeval tv = { timeout, 0 };
fd_set read_set;
FD_ZERO(&read_set);
FD_SET(listen_sock, &read_set);
max_fd = max(max_fd, listen_sock);
/* add all your other other client sockets to thread read_set */
n = select(max_fd, &read_set, NULL, NULL, tv);
if (n > 0) {
if (FD_ISSET(listen_sock, &read_set)) {
cli = accept(listen_sock);
/* add to list of clients */
}
else {
for (int i = 0; i < max_clients; i++) {
if (FD_ISSET(clients[i], &read_set)) {
/* data is waiting. recv */
bytes = recv(clients[i], ..)
if (bytes <= 0) {
/* error or EOF, remove client list, so we don't select on this anymore */
}
}
}
}
}
Note that sends can also block, if the other end is not actively reading, and the send buffer is full. So if you're sending, you might want to check if it's "sendable".
Related
I'm working on this calendar/reminders server-client thing. I've stumbled across a bug where I may have misunderstood the behaviour of the accept() function while trying to handle new connections.
The client basically specifies a day to view and sends it to the server, then gets some text back and closes the connection.(I have been using telnet and netcat to test this so far though.)
After I hit ctrl+d on netcat after I send the command and receive the message, the server gets and infinite output loop of "New connection\n".
The way I understood accept() was that when it is called, it sets the left hand side to a socket descriptor for a connection on the listen() backlog, or waits until there is a connection before returning. So either I am mistaken or I am doing something wrong:
bind(client_socket, (struct sockaddr*)&server_address, sizeof(server_address));
listen(client_socket, 5);
//start main loop. first level checks for commands
while (1)
{
client_socket = accept(client_socket, NULL, NULL);
printf("New connection.\n");
recv(client_socket, buffer, sizeof(buffer), 0);
/*Bunch of code here that interprets what was sent with some string manipulation
and serves back parts of a text file. No socket functions other than send() twice here*/
recv(client_socket, buffer, sizeof(buffer), 0);
}
The idea I had in mind was, once the job is done, wait until the client closes the connection, which sends a message of length 0(hence the recv() at the end), then loop back to the accept() which accepts or waits for the next connection. What am I missing here?
In your second call to accept, you are passing as the listening socket the descriptor for the first connection rather than the listening socket. You really need to check for errors on all these calls. Otherwise, your code will be impossible to debug.
client_socket = accept(client_socket, NULL, NULL);
This is fine the first time. But it leaks the descriptor for the listening socket. So you can't accept another connection.
I need to know whether a client connected/disconnected and handle it.
This was my only idea:
while(!serverStop)
{
fd_set rfds, wfdsBefore, wfdsAfter;
FD_ZERO(&rfds);
FD_SET(serverFd, &rfds);
FD_ZERO(&wfdsBefore);
fillWithClientFds(&wfdsBefore); // clients only listen for messages
wfdsAfter = wfdsBefore;
while(1)
{
select(notimportant, &rfds, &wfdsAfter, NULL, NULL);
if (FD_ISSET(serverFd, &rfds)) // new client appeared
break;
if (doSetsDiffer(&wfdsBefore, &wfdsAfter)) // some client disconnected (doesn't work)
break;
}
// inform connected clients about disconnected ones
}
Not only busy waiting would occur but also this approach doesn't even work (wfdsAfter doesn't change despite the fact that client closed the socket).
Is there any way to do it? The only requirement is to not use multithreading.
serverFd was made with PF_UNIX and SOCK_STREAM flags.
You should place each client file descriptor in the read descriptors (rfds) set after it connects, and, when the file descriptor is subsequently returned as readable, attempt to read from the socket.
First, if your client is really sending nothing (and isn't yet disconnected), its socket will never be marked as readable. That seems like it would solve your issue since you say the client never actually sends anything: it won't be marked readable then until the client disconnects.
But even if the client sends data, the file descriptor would only be marked readable if there were data available OR the client had disconnected. You can easily then distinguish by attempting to read the socket. The return value would be either number of bytes read (if there are data), or zero if the client has disconnected.
(Servers often add the O_NONBLOCK option to sockets to ensure they get notified when the client has data to send but want to ensure they don't block waiting for data from a client. With that option, read still returns 0 when the client has disconnected. With the option, if the client is still around, but there is no data available, the read call would return -1 with errno set to EAGAIN/EWOULDBLOCK.)
One other nuance I haven't explained is that it is possible to close data delivery in one direction while allowing it to continue in the other (see shutdown(2) if you care about this).
You are putting the client sockets in the write descriptor set. You need to put them in the read descriptor set instead.
When a server socket has at least 1 pending client request, it is readable. You can call accept() to accept a client.
When a socket has data in its inbound buffer, or its connected peer has disconnected, it is readable, not writable. You can call read() to differentiate. read() returns > 0 on inbound data, 0 on graceful disconnect, and -1 on error.
A socket is writable when it has available space in its outbound buffer. If write() fails with an EWOULDBLOCK error, the outbound buffer has filled up, and the socket is no longer writable. When the buffer clears up some space, the socket will become writable again.
Also, select() modifies the fdsets you pass to it, so you need to reset rfds on every loop iteration. To avoid that, you can use (e)poll() instead.
So, you need something more like this instead:
fd_set rfds;
while (!serverStop)
{
FD_ZERO(&rfds);
FD_SET(serverFd, &rfds);
fillWithClientFds(&rfds); // clients only listen for messages
if (select(notimportant, &rfds, NULL, NULL, NULL) < 0)
break;
if (FD_ISSET(serverFd, &rfds)) // new client appeared
{
// call accept(), add client to connected list...
}
// clear disconnected list...
for (each client in connected list)
{
if (FD_ISSET(clientFd, &rfds))
{
int nBytes = read(clientFd, ...);
if (nBytes > 0)
{
// handle client data as needed ...
}
else if (nBytes == 0)
{
// add client to disconnected list
}
else
{
// handle error...
// possibly add client to disconnected list...
}
}
}
for (each client in disconnected list)
{
// remove client from connected list...
}
for (each client in disconnected list)
{
// inform connected clients
}
}
I have a program which works as a simple tcp server, it will accept and serve as much as 4 clients. I wrote the following code snippet to archive this goal, for more clearly, I replace some MACRO with "magic number".
while(1) {
int ret = 0;
///
// sock_ is the listening socket, rset_conn_ only contains sock_
//
if(select(sock_ + 1, &rset_conn_, NULL, NULL, &timeo_) <= 0) {
ret = 0;
}
if(FD_ISSET(sock_, &rset_conn_)) {
ret = 1;
}
///
// because the rule select() clears file descriptor and updates timeout is
// complex, for simplicity, let's reset the sock_ and timeout
//
FD_SET(sock_, &rset_conn_);
timeo_.tv_sec = 0;
timeo_.tv_usec = 500000;
if(! ret) {
continue ;
}
///
// csocks_ is an 4 integer array, used for storing clients' socket
//
for(int i = 0 ; i < 4 ; ++i) {
if((-1 == csocks_[i]) && (-1 != (csocks_[i] = accept(sock_, NULL, NULL)))) {
break ;
}
}
}
now, when less than 4 clients connect, this code snippet works normally, but when the 5th client connects, as you can see, it will not be accepted in the for loop, this results in such a problem: in the following loops of while, select() call keeps returning with sock_ bit in rset_conn_ set --- just because I don't accept it!!
I realize that I should not just ignoring it(not accept) in the for loop, but how can I handle it properly by telling kernel "I don't want to accept this client, so not tell me this client want to connect me any more!"?
Any help will be appreciated..
Update:
I can not stop calling select() call, because the clients already connected may disconnect, and then my program will have room for new clients. I need to keep listening on a socket to accept connection, not stop listening when reaches one point.
Why are you asking a question you don't want the answer to? Stop calling select on the listening socket if you don't want to be told that there's an unaccepted connection.
I can not stop calling select() call, because the clients already connected may disconnect, and then my program will have room for new clients. I need to keep listening on a socket to accept connection, not stop listening when reaches one point.
You can stop calling select now. When a client disconnects, then start calling select again. Only ask questions you want the answers to.
There is no socket API to tell the kernel to ignore pending connections (well, on Windows, there is a callback in WSAAccept() for that, but that is a Microsoft-specific extension).
On most platforms, you have to either:
close() the listening socket when your array fills up, then reopen it when a client disconnects.
accept() the client but immediately close() it if the array is full.
stop calling select() and accept() while your array is full. Any clients that are in the backlog will time out eventually, unless your array fills up and you accept() a pending client before it times out.
You can remove the listening socket from the readFDs while there are four existing clients, and put it back when one drops off, but I don't see the point. Why only four clients? Why not remove the limit and eliminate the whole problem? Real servers are not built this way.
[I asked something similar before. This is a more focused version.]
What can cause a server's select() call on a TCP socket to consistently time-out rather than "see" the client's close() of the socket? On the client's side, the socket is a regular socket()-created blocking socket that successfully connects to the server and successfully transmits a round-trip transaction. On the server's side, the socket is created via an accept() call, is blocking, is passed to a child server process via fork(), is closed by the top-level server, and is successfully used by the child server process in the initial transaction. When the client subsequently closes the socket, the select() call of the child server process consistently times-out (after 1 minute) rather than indicating a read-ready condition on the socket. The select() call looks for read-ready conditions only: the write-ready and exception arguments are NULL.
Here's the simplified but logically equivalent select()-using code in the child server process:
int one_svc_run(
const int sock,
const unsigned timeout)
{
struct timeval timeo;
fd_set fds;
timeo.tv_sec = timeout;
timeo.tv_usec = 0;
FD_ZERO(&fds);
FD_SET(sock, &fds);
for (;;) {
fd_set readFds = fds;
int status = select(sock+1, &readFds, 0, 0, &timeo);
if (status < 0)
return errno;
if (status == 0)
return ETIMEDOUT;
/* This code not reached when client closes socket */
/* The time-out structure, "timeo", is appropriately reset here */
...
}
...
}
Here's the logical equivalent of the sequence of events on the client-side (error-handling not shown):
struct sockaddr_in *raddr = ...;
int sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
(void)bindresvport(sock, (struct sockaddr_in *)0);
connect(sock, (struct sockaddr *)raddr, sizeof(*raddr));
/* Send a message to the server and receive a reply */
(void)close(sock);
fork(), exec(), and system() are never called. The code is considerably more complex than this, but this is the sequence of relevant calls.
Could Nagel's algorithm cause the FIN packet to not be sent upon close()?
Most likely explanation is that you're not actually closing the client end of the connection when you think you are. Probably because you have some other file descriptor that references the client socket somewhere that is not being closed.
If your client program ever does a fork (or related calls that fork, such as system or popen), the forked child might well have a copy of the file descriptor which would cause the behavior you're seeing.
One way to test/workaround the problem is to have the client do an explicit shutdown(2) prior to closing the socket:
shutdown(sock, SHUT_RDWR);
close(sock);
If this causes the problem to go away then that is the problem -- you have another copy of the client socket file descriptor somewhere hanging around.
If the problem is due to children getting the socket, the best fix is probably to set the close-on-exec flag on the socket immediately after creating it:
fcntl(sock, F_SETFD, fcntl(sock, F_GETFD) | FD_CLOEXEC);
or on some systems, use the SOCK_CLOEXEC flag to the socket creation call.
Mystery solved.
#nos was correct in the first comment: it's a firewall problem. A shutdown() by the client isn't needed; the client does close the socket; the server does use the right timeout; and there's no bug in the code.
The problem was caused by the firewall rules on our Linux Virtual Server (LVS). A client connects to the LVS and the connection is passed to the least-loaded of several backend servers. All packets from the client pass through the LVS; all packets from the backend server go directly to the client. The firewall rules on the LVS caused the FIN packet from the client to be discarded. Thus, the backend server never saw the close() by the client.
The solution was to remove the "-m state --state NEW" options from the iptables(8) rule on the LVS system. This allows the FIN packets from the client to be forwarded to the backend server. This article has more information.
Thanks to all of you who suggested using wireshark(1).
select() call of Linux will modify value of timeout argument. From man page:
On Linux, select() modifies timeout to reflect the amount of time not
slept
So your timeo will runs to zero. And when it is zero select will return immediately (mostly with return value zero).
The following change may help:
for (;;) {
struct timeval timo = timeo;
fd_set readFds = fds;
int status = select(sock+1, &readFds, 0, 0, &timo);
In my server I check if any socket is ready to read using select() to determine it. As a result in main loop select() is executed every time it iterates.
To test the server I wrote a simple client that sends only one message and then quits. BTW. I use protocol buffers to send information - message means an object of type class Message in this library.
The test session looks like:
select()
server's socket ready to read
accept() client's socket
read message from client's socket
select()
server's socket not ready to read, client's one ready
read message from client's socket
The last step is wrong because client has already closed connection. As a result protobuf library gets Segmentation fault. I wonder why FD_ISSET says the socket is ready in step 6 when it is closed. How can I check if a socket is closed?
EDIT:
I've found how to check if the socket is open
int error = 0;
socklen_t len = sizeof (error);
int retval = getsockopt (socket_fd, SOL_SOCKET, SO_ERROR, &error, &len );
the socket is "readable" if the remote peer closes it, you need to call recv and handle both the case where it returns an error, and the case where it returns 0, which indicates that the peer shut down the connection in an orderly fashion.
Reading the SO_ERROR sockopt is not the correct way, as it returns the current pending error (from, eg. a non-blocking connect)
The socket used for communication between a client and your server will be flagged as readable (i.e. select() will return) when there is data to read, or when there's an EOF to read (i.e. the peer closed the connection).
Just read() when select() returns and your fd is flagged. If read() returns a positive number, you got data. If it returns 0, you got EOF. If it returns -1, you have a problem (unless errno is EAGAIN).