Related
I have a main program that generates a few threads (using a while loop with accept() to get clients), and one that all it has to do is "listen to the keyboard" and when the user enters the word exit it will close the entire program.
first, the main program create the listening thread, then it enters a while loop that accept the clients. even if the listening thread get the exit input the loop is still stuck on accept.
i don't have to use a seperate thread to listen to the keyboard but i could'nt find a none blocking way that would work.
the listening thread:
DWORD WINAPI ListenService(LPVOID lpParam)
{
char buffer[5];
if (EOF == scanf("%s", buffer))
{
printf("faile get word from keyboard\n");
}
if (buffer[4] != '\0')
strcat(buffer, "\0");
if (STRINGS_ARE_EQUAL(buffer, "exit"))
{
return 999;
}
return -1;
}
in the main code:
ThreadListen = CreateThread(NULL,0,ListenService,NULL,0,&(ThreadId));
while(1)
{
SOCKET AcceptSocket = accept(MainSocket, NULL, NULL);
if (AcceptSocket == INVALID_SOCKET)
{
printf("Accepting connection with client failed, error %ld\n", WSAGetLastError());
CleanupWorkerThreads();
WSACleanup();
}
printf("Client Connected.\n");
}
There are many different ways you can handle this.
You can abort a blocked accept() by simply closing the listening socket.
Or, you can use select() with a short timeout to detect when a new client is waiting before then calling accept(). You can check your exit condition in between calls to select(). Just be aware that there is a small race condition where a client may disconnect between the time select() and accect() are called, so accept() may still block, if there are no more clients waiting.
Or, you can get rid of your threads and just use non-blocking sockets in a single thread, checking your exit condition periodically in between socket operations.
Or, you can use asynchronous sockets, using WSACreateEvent(), WSAEventSelect(), and WSAWaitForMultipleEvents() to detect socket activity. Then you can create an addition event to wait on for when the exit condition happens.
Or, you can use an I/O Completion Port to handle socket activity, and then you can post a custom exit packet into the IOCP queue using PostQueuedCompletionStatus() to "wake up" any waiting threads.
I'm writing a multithreaded server program in C that works with AF_UNIX sockets.
The basic structure of the server is:
Main thread initialize data structures and spears a pool of "worker" threads.
Worker threads start waiting for new requests on an empty thread-safe queue
Main thread listen on various sockets (new connection and already connected clients) with a select() call.
select() reveals possible read on connection socket: main thread calls accept() and puts the returned file descriptor in the fd_set (read set).
select() reveal possible read on already connected sockets: main thread removes the ready file descriptors from the fd_set (read set) and puts them in the thread-safe queue.
Worker thread extracts a file descriptor from the queue and starts to communicate with the linked client for serve the request. At the end of the service worker thread puts socket file descriptor back to the fd_set (i worte a function to make this operation thread-safe) and it returns waiting again on the queue for a new request.
This routine is repeated in a infinite cycle until a SIGINT is raised.
Another function has to be performed on SIGUSR1 without exiting from the cycle.
My doubt is about this because if I raise a SIGINT my program exit with EINTR = Interrupted system call.
I know about the pselect() call and the "self pipe" trick but i can't figure out how to make the things work in a multithreaded situation.
I'm looking for a (POSIX compatible) signal management that that prevent the EINTR error while main thread is waiting on pselect().
I post some pieces of code for clarification:
Here i set up signal handlers (ignore errorConsolePrint function)
if(signal(SIGINT, &on_SIGINT) == SIG_ERR)
{
errorConsolePrint("File: %s; Line: %d; ", "Setting SIGINT handler", __FILE__, __LINE__);
exit(EXIT_FAILURE);
}
if(signal(SIGTERM, &on_SIGINT) == SIG_ERR)
{
errorConsolePrint("File: %s; Line: %d; ", "Setting SIGINT handler", __FILE__, __LINE__);
exit(EXIT_FAILURE);
}
if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
{
errorConsolePrint("File: %s; Line: %d; ", "Setting to SIGUSR1 handler", __FILE__, __LINE__);
exit(EXIT_FAILURE);
}
if(signal(SIGPIPE, SIG_IGN) == SIG_ERR)
{
errorConsolePrint("File: %s; Line: %d; ", "Setting to ignore SIGPIPE", __FILE__, __LINE__);
exit(EXIT_FAILURE);
}
Here i set up signal mask for pselect
sigemptyset(&mask);
sigemptyset(&saveMask);
sigaddset(&mask, SIGINT);
sigaddset(&mask, SIGUSR1);
sigaddset(&mask, SIGPIPE);
Here i call pselect
test = saveSet(masterSet, &backUpSet, &saveMaxFd);
CHECK_MINUS1(test, "Server: creating master set's backup ");
int test = pselect(saveMaxFd+1, &backUpSet, NULL, NULL, &waiting, &mask);
if(test == -1 && errno != EINTR)
{
...error handling...
continue;
}
Hope in some help!
Thank you all in advance.
What you should probably do is dedicate a thread to signal handling. Here's a sketch:
In main, before spawning any threads, block all signals (using pthread_sigmask) except for SIGILL, SIGABRT, SIGFPE, SIGSEGV, and SIGBUS.
Then, spawn your signal handler thread. This thread loops calling sigwaitinfo for the signals you care about. It takes whatever action is appropriate for each; this could include sending a message to the main thread to trigger a clean shutdown (SIGINT), queuing the "another function" to be processed in the worker pool (SIGUSR1), etc. You do not install handlers for these signals.
Then you spawn your thread pool, which doesn't have to care about signals at all.
I would suggest the following strategy:
During initialization, set up your signal handlers, as you do.
During initialization, block all (blockable) signals. See for example Is it possible to ignore all signals?.
Use pselect in your main thread to unblock threads for the duration of the call, again as you do.
This has the advantage that all of your system calls, including all those in all your worker threads, will never return EINTR, except for the single pselect in the main thread. See for example the answers to Am I over-engineering per-thread signal blocking? and pselect does not return on signal when called from a separate thread but works fine in single thread program.
This strategy would also work with select: just unblock the signals in your main thread immediately before calling select, and re-block them afterwards. You only really need pselect to prevent hanging if your select timeout is long or infinite, and if your file descriptors are mostly inactive. (I've never used pselect myself, having worked mostly with older Unix's which did not have it.)
I am presuming that your signal handlers as suitable: for example, they just atomically set a global variable.
BTW, in your sample code, do you need sigaddset(&mask, SIGPIPE), as SIGPIPE is already ignored?
Ok, finally I got a solution.
The heart of my problem was about the multithreading nature of my server.
After long search I found out that in the case we have signals raised from other process (in an asyncronous way), it doens't matter which thread capture signal because the behaviour remains the same: The signal is catched and the previously registered handler is executed.
Maybe this could be obvious for others but this was driving me crazy because I did not know how to interpret errors that came out during execution.
After that i found another problem that I solved, is about the obsolete signal() call.
During execution, the first time i rise SIGUSR1, the program catch and manage it as expected but the second time it exit with User defined signal 1.
I figured out that signal() call set "one time" handler for a specific signal, after the first time that the signal is handled the behaviour for that signal return the default one.
So here's what I did:
Here the signal handlers:
N.B.: I reset handler for SIGUSR1 inside the handler itself
static void on_SIGINT(int signum)
{
if(signum == SIGINT || signum == SIGTERM)
serverStop = TRUE;
}
static void on_SIGUSR1(int signum)
{
if(signum == SIGUSR1)
pendingSIGUSR1 = TRUE;
if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
exit(EXIT_FAILURE);
}
Here I set handlers during server's initialization:
if(signal(SIGINT, &on_SIGINT) == SIG_ERR)
exit(EXIT_FAILURE);
if(signal(SIGTERM, &on_SIGINT) == SIG_ERR)
exit(EXIT_FAILURE);
if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
exit(EXIT_FAILURE);
if(signal(SIGPIPE, SIG_IGN) == SIG_ERR)
exit(EXIT_FAILURE);
And here the server's listening cycle:
while(!serverStop)
{
if (pendingSIGUSR1)
{
... things i have to do on SIGUSR1...
pendingSIGUSR1 = FALSE;
}
test = saveSet(masterSet, &backUpSet, &saveMaxFd);
CHECK_MINUS1(test, "Server: creating master set's backup ");
int test = select(saveMaxFd+1, &backUpSet, NULL, NULL, &waiting);
if((test == -1 && errno == EINTR) || test == 0)
continue;
if (test == -1 && errno != EINTR)
{
perror("Server: Monitoring sockets: ");
exit(EXIT_FAILURE);
}
for(int sock=3; sock <= saveMaxFd; sock++)
{
if (FD_ISSET(sock, &backUpSet))
{
if(sock == ConnectionSocket)
{
ClientSocket = accept(ConnectionSocket, NULL, 0);
CHECK_MINUS1(ClientSocket, "Server: Accepting connection");
test = INset(masterSet, ClientSocket);
CHECK_MINUS1(test, "Server: Inserting new connection in master set: ");
}
else
{
test = OUTset(masterSet, sock);
CHECK_MINUS1(test, "Server: Removing file descriptor from select ");
test = insertRequest(chain, sock);
CHECK_MINUS1(test, "Server: Inserting request in chain");
}
}
}
}
Read first signal(7) and signal-safety(7); you might want to use the Linux specific signalfd(2) since it fits nicely (for SIGTERM & SIGQUIT and SIGINT) into event loops around poll(2) or the old select(2) (or the newer pselect or ppoll)
See also this answer (and the pipe(7) to self trick mentioned there, which is POSIX-compatible) to a very similar question.
Also, signal(2) documents:
The effects of signal() in a multithreaded process are unspecified.
so you really should use sigaction(2) (which is POSIX).
I am writing a multithreaded server application which echoes back whatever client sends. I am spawning one thread per new client. I have used a while(1) loop to handle successive clients indefinitely (not exactly indefinite, I have forcefully limited it to 64, after which it rejects any new connection). Now I need to handle Ctrl+C signal in my server application.
One problem is that its not recommended to use signal() in multithreaded application.
However, even if I use it to catch the SIGINT signal (from Ctrl+C) using the method already discussed at SO this is what I need to do:
1) The server should no longer accept more clients.
2) The present connections to server should continue, except when client choses to disconnect.
My signal handler function is:
void Ctrl_C_handler(int sig)
{
if(sig == SIGINT) // Ctrl+C
{
printf("Ctrl+C detected by server !!\n");
printf("No more connections will be accepted!!");
pthread_cancel(main_thread_id);
}
}
The methods I propose to use inside main() to cancel the thread are:
// Method 1
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
while(1)
{
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
// ..
// client handling
// ..
}
// Method 2
while(1)
{
pthread_testcancel(); // Create Cancellation point
// ..
// client handling
// ..
}
// Method 3
while(1)
{
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_testcancel();
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
// ..
// client handling
// ..
}
Reason for why I am using pthread_cancel():
According to this link the thread will be finally terminated using pthread_exit() (Requirement 1 fulfilled). Since the NOTES section of pthread_exit() explains that it allows other threads to continue execution (Requirement 2 fulfilled).
Now if I am right till here,
Which method is best for cancellation of thread.
I chose method 3 because:
(Demerits) Method 1: Asynchronous thread cancel is not safe (as I saw in an answer in SO) if the thread allocates memory, and few other reasons
(Demerits) Method 2: According to this link there are so many other functions which can also work as cancellation points, some of those I will be using in my code.
(Merits) Method 3: Similar to method 2 but safer because other cancellation points will not be active since thread is cancel disabled.
Please tell me if my way is correct or not? And its wrong, is there any other better method to handle Ctrl+C similarly other than using pthread_cancel()?
I'd suggest avoiding cancellation.
Instead, register a SIGINT handler that close()s the server's listening socket. (Use sigaction for this.) The main thread can quit gracefully after the socket is closed, leaving any client threads running.
E.g. in pseudocode:
static int listen_fd = -1 // server socket
void handle_SIGINT(int s): // Requirement 1
if listen_fd >= 0:
close(listen_fd)
listen_fd = -1
int main(...):
listen_fd = new_listening_socket(...)
block_signal(SIGINT)
trap_signal(SIGINT, handle_SIGINT) // again, calls sigaction
while 1:
unblock_signal(SIGINT)
new_client = accept(listen_fd)
block_signal(SIGINT)
if new_client < 0: // Requirement 1, also general error handling
break
spawn_worker_thread(new_client) // probably attr PTHREAD_CREATE_DETACHED
pthread_exit(NULL) // Requirement 2
return 0 // not reached
You can of course include more nuanced error handling, cleanup routines, etc., as needed.
I am trying to make a simple client-server chat program. On the client side I spin off another thread to read any incomming data from the server. The problem is, I want to gracefully terminate that second thread when a person logs out from the main thread. I was trying to use a shared variable 'running' to terminate, problem is, the socket read() command is a blocking command, so if I do while(running == 1), the server has to send something before the read returns and the while condition can be checked again. I am looking for a method (with common unix sockets only) to do a non-blocking read, basically some form of peek() would work, for I can continually check the loop to see if I'm done.
The reading thread loop is below, right now it does not have any mutex's for the shared variables, but I plan to add that later don't worry! ;)
void *serverlisten(void *vargp)
{
while(running == 1)
{
read(socket, readbuffer, sizeof(readbuffer));
printf("CLIENT RECIEVED: %s\n", readbuffer);
}
pthread_exit(NULL);
}
You can make socket not blockable, as suggested in another post plus use select to wait input with timeout, like this:
fd_set input;
FD_ZERO(&input);
FD_SET(sd, &input);
struct timeval timeout;
timeout.tv_sec = sec;
timeout.tv_usec = msec * 1000;
int n = select(sd + 1, &input, NULL, NULL, &timeout);
if (n == -1) {
//something wrong
} else if (n == 0)
continue;//timeout
if (!FD_ISSET(sd, &input))
;//again something wrong
//here we can call not blockable read
fcntl(socket, F_SETFL, O_NONBLOCK);
or, if you have other flags:
int x;
x=fcntl(socket ,F_GETFL, 0);
fcntl(socket, F_SETFL, x | O_NONBLOCK);
then check the return value of read to see whether there was data available.
note: a bit of googling will yield you lots of full examples.
You can also use blocking sockets, and "peek" with select with a timeout. It seems more appropriate here so you don't do busy wait.
The best thing is likely to get rid of the extra thread and use select() or poll() to handle everything in one thread.
If you want to keep the thread, one thing you can do is call shutdown() on the socket with SHUT_RDWR, which will shut down the connection, wake up all threads blocked on it but keep the file descriptor valid. After you have joined the reader thread, you can then close the socket. Note that this only works on sockets, not on other types of file descriptor.
Look for function setsockopt with option SO_RCVTIMEO.
I discovered an issue with thread implementation, that is strange to me. Maybe some of you can explain it to me, would be great.
I am working on something like a proxy, a program (running on different machines) that receives packets over eth0 and sends it through ath0 (wireless) to another machine which is doing the exactly same thing. Actually I am not at all sure what is causing my problem, that's because I am new to everything, linux and c programming.
I start two threads,
one is listening (socket) on eth0 for incoming packets and sends it out through ath0 (also socket)
and the other thread is listening on ath0 and sends through eth0.
If I use threads, I get an error like that:
sh-2.05b# ./socketex
Failed to send network header packet.
: Interrupted system call
If I use fork(), the program works as expected.
Can someone explain that behaviour to me?
Just to show the sender implementation here comes its code snippet:
while(keep_going) {
memset(&buffer[0], '\0', sizeof(buffer));
recvlen = recvfrom(sockfd_in, buffer, BUFLEN, 0, (struct sockaddr *) &incoming, &ilen);
if(recvlen < 0) {
perror("something went wrong / incoming\n");
exit(-1);
}
strcpy(msg, buffer);
buflen = strlen(msg);
sentlen = ath_sendto(sfd, &btpinfo, &addrnwh, &nwh, buflen, msg, &selpv2, &depv);
if(sentlen == E_ERR) {
perror("Failed to send network header packet.\n");
exit(-1);
}
}
UPDATE: my main file, starting either threads or processes (fork)
int main(void) {
port_config pConfig;
memset(&pConfig, 0, sizeof(pConfig));
pConfig.inPort = 2002;
pConfig.outPort = 2003;
pid_t retval = fork();
if(retval == 0) {
// child process
pc2wsuThread((void *) &pConfig);
} else if (retval < 0) {
perror("fork not successful\n");
} else {
// parent process
wsu2pcThread((void *) &pConfig);
}
/*
wint8 rc1, rc2 = 0;
pthread_t pc2wsu;
pthread_t wsu2pc;
rc1 = pthread_create(&pc2wsu, NULL, pc2wsuThread, (void *) &pConfig);
rc2 = pthread_create(&wsu2pc, NULL, wsu2pcThread, (void *) &pConfig);
if(rc1) {
printf("error: pthread_create() is %d\n", rc1);
return(-1);
}
if(rc2) {
printf("error: pthread_create() is %d\n", rc2);
return(-1);
}
pthread_join(pc2wsu, NULL);
pthread_join(wsu2pc, NULL);
*/
return 0;
}
Does it help?
update 05/30/2011
-sh-2.05b# ./wsuproxy 192.168.1.100
mgmtsrvc
mgmtsrvc
Failed to send network header packet.
: Interrupted system call
13.254158,75.165482,DATAAAAAAmgmtsrvc
mgmtsrvc
mgmtsrvc
Still get the interrupted system call, as you can see above.
I blocked all signals as followed:
sigset_t signal_mask;
sigfillset(&signal_mask);
sigprocmask(SIG_BLOCK, &signal_mask, NULL);
The two threads are working on the same interfaces, but on different ports. The problem seems to appear still in the same place (please find it in the first code snippet). I can't go further and have not enough knowledge of how to solve that problem. Maybe some of you can help me here again.
Thanks in advance.
EINTR does not itself indicate an error. It means that your process received a signal while it was in the sendto syscall, and that syscall hadn't sent any data yet (that's important).
You could retry the send in this case, but a good thing would be to figure out what signal caused the interruption. If this is reproducible, try using strace.
If you're the one sending the signal, well, you know what to do :-)
Note that on linux, you can receive EINTR on sendto (and some other functions) even if you haven't installed a handler yourself. This can happen if:
the process is stopped (via SIGSTOP for example) and restarted (with SIGCONT)
you have set a send timeout on the socket (via SO_SNDTIMEO)
See the signal(7) man page (at the very bottom) for more details.
So if you're "suspending" your service (or something else is), that EINTR is expected and you should restart the call.
Keep in mind if you are using threads with signals that a given signal, when delivered to the process, could be delivered to any thread whose signal mask is not blocking the signal. That means if you have blocked incoming signals in one thread, and not in another, the non-blocking thread will receive the signal, and if there is no signal handler setup for the signal, you will end-up with the default behavior of that signal for the entire process (i.e., all the threads, both signal-blocking threads and non-signal-blocking threads). For instance, if the default behavior of a signal was to terminate a process, one thread catching that signal and executing it's default behavior will terminate the entire process, for all the threads, even though some threads may have been masking the signal. Also if you have two threads that are not blocking a signal, it is not deterministic which thread will handle the signal. Therefore it's typically the case that mixing signals and threads is not a good idea, but there are exceptions to the rule.
One thing you can try, is since the signal mask for a spawned thread is inherited from the generating thread, is to create a daemon thread for handling signals, where at the start of your program, you block all incoming signals (or at least all non-important signals), and then spawn your threads. Now those spawned threads will ignore any incoming signals in the parent-thread's blocked signal mask. If you need to handle some specific signals, you can still make those signals part of the blocked signal mask for the main process, and then spawn your threads. But when you're spawning the threads, leave one thread (could even be the main process thread after it's spawned all the worker threads) as a "daemon" thread waiting for those specific incoming (and now blocked) signals using sigwait(). That thread will then dispatch whatever functions are necessary when a given signal is received by the process. This will avoid signals from interrupting system calls in your other worker-threads, yet still allow you to handle signals.
The reason your forked version may not be having issues is because if a signal arrives at one parent process, it is not propagated to any child processes. So I would try, if you can, to see what signal it is that is terminating your system call, and in your threaded version, block that signal, and if you need to handle it, create a daemon-thread that will handle that signal's arrival, with the rest of the threads blocking that signal.
Finally, if you don't have access to any external libraries or debuggers, etc. to see what signals are arriving, you can setup a simple procedure for seeing what signals might be arriving. You can try this code:
#include <signal.h>
#include <stdio.h>
int main()
{
//block all incoming signals
sigset_t signal_mask;
sigfillset(&signal_mask);
sigprocmask(SIG_BLOCK, &signal_mask, NULL);
//... spawn your threads here ...
//... now wait for signals to arrive and see what comes in ...
int arrived_signal;
while(1) //you can change this condition to whatever to exit the loop
{
sigwait(&signal_mask, &arrived_signal);
switch(arrived_signal)
{
case SIGABRT: fprintf(stderr, "SIGABRT signal arrived\n"); break;
case SIGALRM: fprintf(stderr, "SIGALRM signal arrived\n"); break;
//continue for the rest of the signals defined in signal.h ...
default: fprintf(stderr, "Unrecognized signal arrived\n");
}
}
//clean-up your threads and anything else needing clean-up
return 0;
}