I am practicing POSIX programming sockets and I have following problem with my server, when i terminate one of its clients, the function pselect gets error bad filedescriptor (I think its errno=EBADF), could you help me and write how can I avoid it in my code?
Here is what i wrote:
void doServer(int fd) {
//fd is the listening socket (there is only one)
int maxfd = fd;
//base, and reading fd set
fd_set base_rfds, rfds;
int i;
sigset_t mask, oldmask;
//zero out base fd set
FD_ZERO(&base_rfds);
//add listening socket to base fd set
FD_SET(fd, &base_rfds);
//add SIGINT to blocking signal mask
sigemptyset (&mask);
sigaddset (&mask, SIGINT);
sigprocmask (SIG_BLOCK, &mask, &oldmask);
//main server loop (block on pselect untill fd changes or we get signal)
while(!stop){
rfds=base_rfds;
//call pselect with oldmask to not block SIG_INT
if(pselect(maxfd+1,&rfds,NULL,NULL,NULL,&oldmask)>0){
if(FD_ISSET(fd,&rfds)){
//connect client
maxfd+=add_new_client(fd, &base_rfds);
//remove listening socket from reading fd set
FD_CLR(fd,&rfds);
}
//remove listening socket from base fd set
FD_CLR(fd,&base_rfds);
handle_connection(rfds,&base_rfds,maxfd);
//add listening socket back into base fd set
FD_SET(fd,&base_rfds);
}
else{
if(EINTR==errno) continue;
ERR("select");
}
}
//close all of fd's from connected clients
for(i=0;i<maxfd;i++)
if(FD_ISSET(i,&base_rfds) && TEMP_FAILURE_RETRY(close(i))<0)ERR("close");
sigprocmask (SIG_UNBLOCK, &mask, NULL);
}
EDIT:
here is add_new_client function:
int add_new_client(int sfd, fd_set *base_rfds){
int nfd,new_flags;
//accept now if we can (non-blocking)
if((nfd=TEMP_FAILURE_RETRY(accept(sfd,NULL,NULL)))<0) {
if(EAGAIN==errno||EWOULDBLOCK==errno) return 0;
ERR("accept");
}
//remember to make the fd non-blocking
//get current flags, add O_NONBLOCK
new_flags = fcntl(nfd, F_GETFL) | O_NONBLOCK;
//apply new flags
fcntl(nfd, F_SETFL, new_flags);
//add discriptor to base fd set
FD_SET(nfd,base_rfds);
return 1;
}
You are selecting on a socket that you have closed. When you close a socket you need to remove it from the FD set. Your handling of this set seems pretty optimistic. For example, you can't be sure that adding a new client is really going to increment the maximum FD. I also don't understand why you're removing the listening socket from the set once, let alone twice, and then reinstating it. You need to look at some sample code for select(). This is all very strange.
Related
I've this problem with my server/client on C. If I close the server socket after a SIGINT, and then I try to write on this closed connection from the client, I've to do write two times before than client generates SIGPIPE. Shouldn't it generate it immediately? Is this a normal behaviour or something I need to fix? This is my code. I'm testing things on ubuntu, same PC, connecting via 127.0.0.1.
server.c
sigset_t set;
struct sigaction sign;
int sock_acc;
int sock;
void closeSig(){
close(sock_acc);
close(sock);
exit(1);
}
int main(){
sigemptyset(&set);
sigaddset(&set, SIGINT);
sig.sa_sigaction = &closeSig;
sig.sa_flags = SA_SIGINFO;
sig.sa_mask = set;
sigaction(SIGINT, &sig, NULL);
//other code to accept the connection from the client
sigprocmask(SIG_UNBLOCK, &set, NULL);
//write/read calls
}
client.c
void closeSigPipe(){
close(ds_sock);
printf("Stop...");
exit(1);
}
int main(){
sigpipe.sa_sigaction = &closeSigPipe;
sigpipe.sa_flags = SA_SIGINFO;
sigaction(SIGPIPE, &sigpipe, NULL);
//other code to connect the server, and write/read calls
}
The problem is that when I close the server terminal with CTRL+C, the first write to the connection from the client works without any problem... perror("Error:"); prints "success"...
The TCP protocol doesn't provide a way for the receiver to tell the sender that it's closing the connection. When it closes the connection, it sends a FIN segment, but this just means that it's done sending, not that it can no longer receive.
The way the sender detects that the connection has been closed is that it tries to send data, and the receiver sends back a RST segment in response. But writing to a socket doesn't wait for the response, it just queues the data and returns immediately. The signal occurs when the RST is received, which will be a short time later.
The reason you have to do two writes may be because of Nagle's Algorithm. To avoid excessive network overhead, TCP tries to combine short messages into a single segment. The Wikipedia page includes some workarounds for this.
I wonder about the behaviour of socket on blocking and nonblocking actions. What happens to threads blocking on socket when the socket blocking mode changes ? Here is the scenario;
thread1(T1) creates a UDP socket and
fd = socket(AF_INET , SOCK_DGRAM, 0);
T1 waiting(sleeping) for receive
recv(fd, buf , sizeof(buf) , 0);
and thread2(T2) changes socket mode to non-blocking before socket receive any data
fcntl(fd, F_SETFL, O_NONBLOCK);
what happens to T1 ? Is it signalled/waked because the socket is no more blocking ?
The behavior is literally unspecified: fcntl is not required to unblock any threads.
Linux just sets the flag in the file description struct file and returns without unblocking any blocked threads.
A thread already blocked in recv can be scheduled to run only if:
data to read becomes available;
or an error condition on the file descriptor is detected (FIN/RST, socket read timeout, TCP keep-alive failure, the file descriptor is closed by another thread);
or a signal is received and the signal disposition does not include SA_RESTART;
or it is pthread_cancelled.
The fact that you are trying to change flags of a file descriptor of another thread suggests that your design requires a review. Ideally, threads must not share any data and not poke at each other's state, rather they should use message passing to communicate with each other.
You made me curious. First, it's quite obvious that since there is no standard that specifies that the socket should wake up, it will not be awoken because it would be quite a pain to implement this (since the non-blocking flag is in a different layer than where the socket is blocking). So we can say quite confidently that the socket will not wake up until we receive a packet. Or will it?
#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <err.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <string.h>
#include <time.h>
#include <fcntl.h>
int sock;
static void
sighand(int s)
{
write(1, "sig\n", 4);
}
static void *
rcv(void *v)
{
struct sigaction sa;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = sighand;
sa.sa_flags = SA_RESTART;
if (sigaction(SIGUSR1, &sa, NULL) == -1)
err(1, "sigaction");
char buf[64];
ssize_t sz = recv(sock, buf, sizeof(buf), 0);
printf("recv %d\n", (int)sz);
return NULL;
}
pthread_t t1, t2;
static void *
beeper(void *v)
{
for (;;) {
nanosleep(&((struct timespec){.tv_sec = 1}), NULL);
if (pthread_kill(t1, SIGUSR1))
errx(1, "pthread_kill");
printf("beep\n");
}
}
int
main(int argc, char **argv)
{
if ((sock = socket(PF_INET, SOCK_DGRAM, 0)) == -1)
err(1, "socket");
struct sockaddr_in sin;
memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_port = htons(4711);
sin.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(sock, (struct sockaddr *)&sin, sizeof(sin)) == -1)
err(1, "bind");
if (pthread_create(&t1, NULL, rcv, NULL))
errx(1, "pthread_create");
if (pthread_create(&t2, NULL, beeper, NULL))
errx(1, "pthread_create");
/* pretend that this is correct synchronization. */
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
printf("setting non-block\n");
if (fcntl(sock, F_SETFL, O_NONBLOCK) == -1)
err(1, "fcntl");
printf("set\n");
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
return 0;
}
The code above (sorry, couldn't make it shorter). Blocks the thread in recv, waits a bit and then sets non-blocking on the file descriptor. As expected nothing happens. But then I added a twist. The receiving thread has a signal handler that wakes up once a second with SA_RESTART. Of course, since we have SA_RESTART recv should not wake up.
On OSX this will "misbehave" if we consider any behavior correct. I'm pretty sure this will behave the same way on all BSDs. Same on Linux. In fact, the way SA_RESTART is usually implemented I'm pretty sure this will "misbehave" pretty much everywhere.
Funny. This of course doesn't mean that the code above is useful for anything, but it's an interesting curiosity. To answer your question, this is unspecified, but in most cases it won't wake up, unless it will. Please don't do weird things like this.
I think POSIX specification is pretty clear about this:
If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives. If no messages are available at the socket and O_NONBLOCK is set on the socket's file descriptor, recv() shall fail and set errno to [EAGAIN] or [EWOULDBLOCK].
You call recv when O_NONBLOCK is not yet set, to it should block until message arrives (not until mode changes).
Unblocked thread (didn't block yet due to reading when it was in blocking mode) will behave as if it was always a non-blocking mode.
Reading recv(2) manual page
If no messages are available at the socket, and if the socket is nonblocking, the value -1 is returned and the external
variable errno is set to EAGAIN or EWOULDBLOCK.
Blocked threads (blocked while reading when in blocking mode) before changing to non-blocking.
As pointed by #Maxim sharing the code of the function which doesn't awaken threads, The blocked threads will only be awaken after a write is done (data is available).
I have no problems with running the TCP Server and I like the fact that it's in blocking to avoid useless loops and sleeping code and useless cpu cycles.
The problem happens when shutting it down in Linux environment, it stays on, until the connected user sends something, then it turns off.
I figured it's because it's blocking even though the endless while loop is set to exit. But when it's blocking changing the socket id's to NON_BLOCKING doesn't help at all, most likely has to be set to NON_BLOCKING before the block occurs.
#include <pthread.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <errno.h>
#include <fcntl.h> /* Added for the nonblocking socket */
#define LISTEN_MAX 10 /* Maximum clients that can queue up */
#define LISTEN_PORT 32000
#define MAX_COMMANDS_AT_ONCE 4000
#define NANO_SECOND_MULTIPLIER 1000000 // 1 millisecond = 1,000,000 Nanoseconds
//Global so I can access these where I shut the threads off.
int listenfd, connfd; //sockets that must be set to non-blocking before exiting
int needQuit(pthread_mutex_t *mtx)
{
switch(pthread_mutex_trylock(mtx)) {
case 0: /* if we got the lock, unlock and return 1 (true) */
pthread_mutex_unlock(mtx);
return 1;
case EBUSY: /* return 0 (false) if the mutex was locked */
return 0;
}
return 1;
}
/* this is run on it's own thread */
void *tcplistener(void *arg)
{
pthread_mutex_t *mx = arg;
//keyboard event.
SDLKey key_used;
struct timespec ts;
//int listenfd,connfd,
int n,i, ans;
struct sockaddr_in servaddr,cliaddr;
socklen_t clilen;
pid_t childpid;
char mesg[MAX_COMMANDS_AT_ONCE];
listenfd=socket(AF_INET,SOCK_STREAM,0);
bzero(&servaddr,sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr=htonl(INADDR_ANY);
servaddr.sin_port=htons(LISTEN_PORT);
int option = 1;
if(setsockopt(listenfd, SOL_SOCKET,SO_REUSEADDR,(char*)&option,sizeof(option)) < 0)
{
printf("setsockopt failed\n");
close(listenfd);
}
bind(listenfd,(struct sockaddr *)&servaddr,sizeof(servaddr));
listen(listenfd,LISTEN_MAX);
while( !needQuit(mx) )
{
clilen=sizeof(cliaddr);
connfd = accept(listenfd,(struct sockaddr *)&cliaddr,&clilen);
while( !needQuit(mx) )
{
n = recv(connfd,mesg,MAX_COMMANDS_AT_ONCE,0);
if(n == 0 || n == -1) break;
//...Do Stuff here with mesg...
}
}
close(connfd);
}
close(connfd);
close(listenfd);
return NULL;
}
int main(int argc, char *argv[])
{
/* this variable is our reference to the thread */
pthread_t tcp_listener_thread;
pthread_mutex_t mxq; /* mutex used as quit flag */
/* init and lock the mutex before creating the thread. As long as the
mutex stays locked, the thread should keep running. A pointer to the
mutex is passed as the argument to the thread function. */
pthread_mutex_init(&mxq,NULL);
pthread_mutex_lock(&mxq);
/* create a hread which executes tcplistener(&x) */
if(pthread_create(&tcp_listener_thread, NULL, tcplistener, &mxq)) {
fprintf(stderr, "Error creating TCP Listener thread\n");
//clear thread for tcp listener on exit.
/* unlock mxq to tell the thread to terminate, then join the thread */
fcntl(listenfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
fcntl(connfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
pthread_mutex_unlock(&mxq);
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
return 0;
}
//End of the TCP Listener thread.
// Waits 500 milliseconds before shutting down
struct timespec ts;
ts.tv_sec = 0;
ts.tv_nsec = 500 * NANO_SECOND_MULTIPLIER;
nanosleep((&ts, NULL);
//Forces a shutdown of the program and thread.
//clear thread for tcp listener on exit.
/* unlock mxq to tell the thread to terminate, then join the thread */
fcntl(listenfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
fcntl(connfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
pthread_mutex_unlock(&mxq);
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
return 0;
}
I tried the fix EJP suggested like so, still hanging..
I've made connfd and listernfd both global scope
pthread_mutex_unlock(&mxq);
close(connfd); //<- this
close(listenfd); //<-- this
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
To unblock accept(), just close the listening socket. Make sure the code around accept() handles is correctly.
To unblock recv(), shutdown the receiving socket for input. That will cause recv() to return zero, which again must be handled correctly. Or else just close the socket as above, which might be better if you want the receive code to know that you're closing the application.
Your listener thread will indeed block in the accept().
The nasty way to fix this (almost) is to send a signal to the listener thread with pthread_kill(). This causes accept() to return with errno == EINTR, which you test for and then return.
However, that has a race condition: if the signal is received between testing the while (!needQuit(mx)) condition and entering the accept() then it'll be lost and the accept() will block again.
One correct way to solve this is to use something like select() and a pipe. You select for read over the pipe and the socket. When the main thread wants the listener thread to exit it writes a byte to the pipe. The listener thread's select() call returns either when a byte is readable from the pipe (in which case it exits) and/or when a client can be accepted.
Non-blocking sockets are primarily used to multiplex lots of sockets into one event loop (i.e. thread). That's a good idea for server scalability, but not necessary here.
I have the following code in C.
void setNonBlocking(SOCKET fd){
int flags;
if (-1 == (flags = fcntl(fd, F_GETFL, 0)))
flags = 0;
fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}
int main(){
int sock;
connect(sock, .....);
setNonBlocking(sock);
....
close(sock);
//we will do something here but the application exits in/after the close operation
}
I am using the socket in non-blocking mode with setNonBlocking function. When i close socket, the application immediately exits without segfault or anything else. I don't see this issue if i don't use setNonBlocking function.
How can i close the non-blocking socket without this issue?
Perhaps your application is getting SIGPIPE. You should normally handle or ignore the SIGPIPE signal when programming with sockets.
You are ignoring any error-result from fcntl. If fcntl returns -1, you should at least print out an error message (using perror, for example).
What is the difference between read() and recv(), and between send() and write() in socket programming in terms of performances, speed and other behaviors?
The difference is that recv()/send() work only on socket descriptors and let you specify certain options for the actual operation. Those functions are slightly more specialized (for instance, you can set a flag to ignore SIGPIPE, or to send out-of-band messages...).
Functions read()/write() are the universal file descriptor functions working on all descriptors.
Per the first hit on Google
read() is equivalent to recv() with a flags parameter of 0. Other values for the flags parameter change the behaviour of recv(). Similarly, write() is equivalent to send() with flags == 0.
read() and write() are more generic, they work with any file descriptor.
However, they won't work on Windows.
You can pass additional options to send() and recv(), so you may have to used them in some cases.
I just noticed recently that when I used write() on a socket in Windows, it almost works (the FD passed to write() isn't the same as the one passed to send(); I used _open_osfhandle() to get the FD to pass to write()). However, it didn't work when I tried to send binary data that included character 10. write() somewhere inserted character 13 before this. Changing it to send() with a flags parameter of 0 fixed that problem. read() could have the reverse problem if 13-10 are consecutive in the binary data, but I haven't tested it. But that appears to be another possible difference between send() and write().
Another thing on linux is:
send does not allow to operate on non-socket fd. Thus, for example to write on usb port, write is necessary.
On Linux I also notice that :
Interruption of system calls and library functions by signal handlers
If a signal handler is invoked while a system call or library function call is blocked, then either:
the call is automatically restarted after the signal handler returns; or
the call fails with the error EINTR.
... The details vary across UNIX systems; below, the details for Linux.
If a blocked call to one of the following interfaces is interrupted
by a signal handler, then the call is automatically restarted after
the signal handler returns if the SA_RESTART flag was used; otherwise the call fails with the error EINTR:
read(2), readv(2), write(2), writev(2), and ioctl(2) calls on "slow" devices.
.....
The following interfaces are never restarted after being interrupted by a signal handler, regardless of the use of SA_RESTART; they
always fail with the error EINTR when interrupted by a signal handler:
"Input" socket interfaces, when a timeout (SO_RCVTIMEO) has been set on the socket using setsockopt(2): accept(2), recv(2),
recvfrom(2), recvmmsg(2) (also with a non-NULL timeout argument), and recvmsg(2).
"Output" socket interfaces, when a timeout (SO_RCVTIMEO) has been set on the socket using setsockopt(2): connect(2), send(2),
sendto(2), and sendmsg(2).
Check man 7 signal for more details.
A simple usage would be use signal to avoid recvfrom blocking indefinitely.
An example from APUE:
#include "apue.h"
#include <netdb.h>
#include <errno.h>
#include <sys/socket.h>
#define BUFLEN 128
#define TIMEOUT 20
void
sigalrm(int signo)
{
}
void
print_uptime(int sockfd, struct addrinfo *aip)
{
int n;
char buf[BUFLEN];
buf[0] = 0;
if (sendto(sockfd, buf, 1, 0, aip->ai_addr, aip->ai_addrlen) < 0)
err_sys("sendto error");
alarm(TIMEOUT);
//here
if ((n = recvfrom(sockfd, buf, BUFLEN, 0, NULL, NULL)) < 0) {
if (errno != EINTR)
alarm(0);
err_sys("recv error");
}
alarm(0);
write(STDOUT_FILENO, buf, n);
}
int
main(int argc, char *argv[])
{
struct addrinfo *ailist, *aip;
struct addrinfo hint;
int sockfd, err;
struct sigaction sa;
if (argc != 2)
err_quit("usage: ruptime hostname");
sa.sa_handler = sigalrm;
sa.sa_flags = 0;
sigemptyset(&sa.sa_mask);
if (sigaction(SIGALRM, &sa, NULL) < 0)
err_sys("sigaction error");
memset(&hint, 0, sizeof(hint));
hint.ai_socktype = SOCK_DGRAM;
hint.ai_canonname = NULL;
hint.ai_addr = NULL;
hint.ai_next = NULL;
if ((err = getaddrinfo(argv[1], "ruptime", &hint, &ailist)) != 0)
err_quit("getaddrinfo error: %s", gai_strerror(err));
for (aip = ailist; aip != NULL; aip = aip->ai_next) {
if ((sockfd = socket(aip->ai_family, SOCK_DGRAM, 0)) < 0) {
err = errno;
} else {
print_uptime(sockfd, aip);
exit(0);
}
}
fprintf(stderr, "can't contact %s: %s\n", argv[1], strerror(err));
exit(1);
}
"Performance and speed"? Aren't those kind of ... synonyms, here?
Anyway, the recv() call takes flags that read() doesn't, which makes it more powerful, or at least more convenient. That is one difference. I don't think there is a significant performance difference, but haven't tested for it.
The only difference between recv() and read() is the presence of flags. With a zero flags argument, recv() is generally equivalent to read()
you can use write() and read() instead send() and recv() but send() and recv() offer much greater control over your data transmission