This is not any error in code, but to get an idea how to accomplish the following criteria. I have seen similar questions here and here, but did not get any particular way to do it.
Consider the following parts of code:
....
pthread_create(&Receiver,NULL,ReceiveFromServer,(void*)&ClientSoc);
sending =1;
while (sending)
{
SHOW_PROMPT;
gets(message);
memcpy(Packet.data,message,MAX_DATA_LENGTH);
if (SetCommand(&Packet,message) == CMD_EXT)
sending = 0;
send(ClientSoc,&Packet,PACKET_SIZE,0);
}
close(ClientSoc);
pthread_join(Receiver,NULL);
return 0;
....
And the ReceiveFromServer function:
void* ReceiveFromServer(void* ClientSoc)
{
int receiving =1;
int Status;
strPacket Packet;
while(receiving)
{
if(recv(*(int*)ClientSoc,&Packet,PACKET_SIZE,0)>0)
{
ParseReply(Packet);
SHOW_PROMPT;
}
if(GET_COMMAND(Packet.Header) == CMD_EXT)
receiving = 0;
}
return NULL;
}
Assume everything is declared or defined correctly.
So the thread terminates depending upon the received data from server, but the sender still loops as it do not know that the receiver is terminated. The sender only comes out of loop depending upon the user input (a particular word such as 'exit').How to notify the parent that the thread is terminating? I tried to make sending global and change it from inside the thread, but it didn't work.
Close the socket in the 'ReceiveFromServer' thread when you get the CMD_EXT. The sending thread should get the message soon enough - the send() call will return with an error.
In the main loop of the parent, replace:
while (sending)
with:
while (sending && pthread_kill(Receiver, 0) != ESRCH)
This will check the existence of the Receiver thread. See How do you query a pthread to see if it is still running? for more info.
Note, however, that How do I determine if a pthread is alive? emphasizes that you cannot do this for a detached thread. And http://man7.org/linux/man-pages/man3/pthread_kill.3.html states:
POSIX.1-2008 recommends that if an implementation detects the use of a thread ID after the end of its lifetime, pthread_kill() should return the error ESRCH. The glibc implementation returns this error in the cases where an invalid thread ID can be detected. But note also that POSIX says that an attempt to use a thread ID whose lifetime has ended produces undefined behavior, and an attempt to use an invalid thread ID in a call to pthread_kill() can, for example, cause a segmentation fault.
Closing the socket in the ReceiveFromServer thread when you get the CMD_EXT, would probably also work, as Are parallel calls to send/recv on the same socket valid? seems to indicate POSIX sockets are thread safe. But R. strongly recommends against it in Is closesocket thread safe?. And the RHEL5 Linux man 2 close states:
It is probably unwise to close file descriptors while they may be in use by system calls in other threads in the same process. Since a file descriptor may be re-used, there are some obscure race conditions that may cause unintended side effects.
When dealing with sockets, you have to be sure that there is no recv(2) still blocking on it on another thread, otherwise it might block forever, since no more messages will be sent via the socket. Be sure to use shutdown(2) to shut down all parts the connection before closing the socket.
Also, you'll probably also want to replace gets(message) with a non-blocking equivalent, to handle the case where the peer asynchronously sends a CMD_EXT while you are waiting for human input.
And of course you'll need to add socket error-handling, but that is beyond the scope of this question.
Maybe you could try to use pthread_tryjoin_np (I never used it) in the parent loop to detect if the thread has finished. It is only available for linux as I understood.
Related
I have a thread I made to accept incoming connections with:
pthread_t thread;
pthread_create(&thread, NULL, worker_func, NULL);
pthread_detach(thread);
Then worker_func does:
<snip>
fd = accept(GlobalFD, NULL, NULL);
<snip>
However, If I try to close GlobalFD in my original thread, accept will still block (I want it to fail if GlobalFD is closed). I've looked online and at other questions and can't seem to find the answer to my particular problem.
Any ideas?
Thanks
Different threads of the same program share memory, including file descriptor tables. If one thread closes an FD then it is closed for all other threads, too. This is one of the differences between using multiple threads and using multiple processes. Therefore, do not allow one thread to close a file descriptor that another is relying upon to remain open.
More generally, however, you must take great care about modifying shared data. Generally speaking, you must synchronize access via a semaphore, a condition variable, or some other construct or action with significance for synchronization. Program behavior is otherwise not well defined. In particular, you cannot expect that closing a file descriptor will cause an active I/O function running in a different thread to terminate. Send the target thread a signal instead.
In general, closing a file-descriptor in one thread isn't guaranteed to cause a function that's waiting on that file-descriptor in another thread to return.
Your options are 1) install a signal handler and use pthread_kill() (don't forget to check the return-code of the blocked function) and 2) create a "termination file-descriptor" that is also passed to the select() or poll() function and either close it or write to it in the other thread.
Good luck.
This really is two questions, but I suppose it's better they be combined.
We're working on a client that uses asynchronous TCP connection. The idea is that the program will block until certain message is received from the server, which will invoke a SIGPOLL handler. We are using a busy waiting loop, basically:
var = 1
while (var) usleep(100);
//...and somewhere else
void sigpoll_handler(int signum){
......
var = 0;
......
}
We would like to use something more reliable instead, like a semaphore. The thing is, when a thread is blocked on a semaphore, will the signal get through still? Especially considering that signals get delivered when it switches back to user level; if the process is off the runqueue, how will it happen?
Side question (just out of curiosity):
Without the "usleep(100)" the program never progresses past the while loop, although I can verify the variable was set in the handler. Why is that? Printing changes its behaviour too.
Cheers!
[too long for a comment]
Accessing var from inside the signal handler invokes undefined behaviour (at least for a POSIX conforming system).
From the related POSIX specification:
[...] if the process is single-threaded and a signal handler is executed [...] the behavior is undefined if the signal handler refers to any object [...] with static storage duration other than by assigning a value to an object declared as volatile sig_atomic_t [...]
So var shall be defined:
volatile sig_atomic_t var;
The busy waiting while-loop, can be replaced by a single call to a blocking pause(), as it will return on reception of the signal.
From the related POSIX specification:
The pause() function shall suspend the calling thread until delivery of a signal whose action is either to execute a signal-catching function or to terminate the process.
Using pause(), btw, will make the use of any global flag like var redundant, to not say needless.
Short answer: yes, the signal will get through fine with a good implementation.
If you're going to be using a semaphore to control the flow of the program, you'll want to have the listening be on one child with the actual data processing be on another. This will then put the concurrency fairness in the hands of the OS which will make sure your signal listening thread gets a chance to check for a signal with some regularity. It shouldn't ever be really "off the runqueue," but cycling through positions on the runqueue instead.
If it helps you to think about it, what you have right now seems to basically be a a very rough implementation of a semaphore on its own -- a shared variable whose value will stop one block of code from executing until another code block clears it. There isn't anything inherently paralyzing about a semaphore on a system level.
I kind of wonder why whatever function you're using to listen for the SIGPOLL isn't doing its own blocking, though. Most of those utilities that I've seen will stop their calling thread until they return a value. Basically they handle the concurrency for you and you can code as if you were dealing with a normal synchronous program.
With regards to the usleep loop: I'd have to look at what the optimizer's doing, but I think there are basically two possibilities. I think it's unlikely, but it could be that the no-body loop is compiling into something that isn't actually checking for a value change and is instead just looping. More likely to me would be that the lack of any body steps is messing up the underlying concurrency handling, and the loop is executing so quickly that nothing else is getting a chance to run -- the queue is being flooded by loop iterations and your signal processsing can't get a word in edgewise. You could try just watching it for a few hours to see if anything changes; theoretically if it's just a concurrency problem then the random factor involved could clear the block on its own with a few billion chances.
Say I have a flag to indicate an exit condition that I with to enable with a signal. Then I can attach the following handler to SIGUSR1 for instance.
volatile sig_atomic_t finished = 0;
void catch_signal(int sig)
{
finished = 1;
}
I then use the flag to determine when a particular loop should end. In this particular case I have a thread running (but I believe my problem applies without threads also, so don't focus on that part).
void *thread_routine(void *arg)
{
while (!finished) {
/* What if the signal happens here? */
if ((clientfd = accept(sockfd, &remote_addr, &addr_size)) == -1) {
if (errno == EINTR)
continue;
/* Error handling */
}
handle_client(clientfd);
}
}
This loop is supposed to continue to run until I raise my SIGUSR1 signal. When it receives the signal I want it to stop gracefully as soon as possible. Since I have a blocking accept call I don't have the loop spinning around wasting CPU cycles, which is good, and the signal can at any moment interrupt the blocking accept and cause the loop to terminate.
The problem is, as shown in the comment in the code, that the signal could be delivered right after the while condition but before the accept call. Then the signal handler will set finished to true, but after the execution resumes, accept will be called and block indefinitely. How can I avoid this condition and make sure that I always will be able to terminate the loop with my signal?
Assuming I still want to use a signal to control this, I can think of two possible solutions. The first one is to turn on some alarm that re-raises a signal after a while if the signal was missed the first time. The second one is to put a timeout on the socket so that accept returns after some amount time so that the flag can be examined again. But these solutions are more like workarounds (especially since I change the blocking behaviour of accept in my second solution) and if there is some cleaner and more straightforward solution I'd like to use that instead.
The Self-Pipe Trick can be used in such cases.
You open a pipe and use select to wait both on the pipefd and sockfd. The handler writes a char to the pipe. After the select, checking fd set helps you determine if you can go for accept or not.
I realize this question is over a year old, now, but pselect() was designed exactly for this type of situation. You can provide pselect() (and select() generally) with file descriptors of listening sockets, and those functions will return when there is an accept()able connection available.
The general approach is you block all relevant signals, and then call pselect() with a signal mask to unblock them. pselect() will atomically:
Unblock the signal(s)
Call accept()
Block the signal(s) again when accept() returns
so you can essentially guarantee that the only time that signal will actually be delivered and handled is when pselect() is running, and you don't have to worry about it being caught after you check finished but before you call accept(). In other words, you make sure that whenever that signal is delivered, it'll always interrupt pselect() and set errno to EINTR, so that's the only place you have to check for it.
This is a question similar to Proper way to close a blocking UDP socket. I have a thread in C which is reading from a UDP socket. The read is blocking. I would like to know if it is possible to be able to exit the thread, without relying on the recv() returning? For example can I close the socket from another thread and safely expect the socket read thread to exit? Didn't see any high voted answer on that thread, thats why I am asking it again.
This really depends on what system you're running under. For example, if you're running under a POSIX-compliant system and your thread is cancelable, the recv() call will be interrupted when you cancel the thread since it's a cancel point.
If you're using an older socket implementation, you could set a signal handler for your thread for something like SIGUSR1 and hope nobody else wanted it and signal, since recv() will interrupt on a signal. Your best option is not to block, if at all possible.
I don't think closing a socket involved in a blocking operation is a safe guaranteed way of terminating the operation. For instance, kernel.org warns darkly:
It is probably unwise to close file descriptors while they may be in
use by system calls in other threads in the same process. Since a
file descriptor may be reused, there are some obscure race conditions
that may cause unintended side effects.
Instead you could use a signal and make recv fail with EINTR
(make sure SA_RESTART is not enabled). You can send a signal to a
specific thread with pthread_kill
You could enable SO_RCVTIMEO on the socket before starting the recv
call
Personally I usually try to stay clear of all the signal nastiness but it's a viable option.
You've got a couple of options for that. A signal will interrupt the read operation, so all you need to do is make sure a signal goes off. The recv operation should fail with error number EINTR.
The simplest option is to set up a timer to interrupt your own process after some timeout e.g. 30 seconds:
itimerval timer
timeval time;
time.tv_sec = 30;
time.tv_usec = 0;
timer.it_value = time;
if( setitimer( ITIMER_REAL, &timer, NULL ) != 0 )
printf( "failed to start timer\n" );
You'll get a SIGALRM after the specified time, which will interrupt your blocking operation, and give you the chance to repeat the operation or quit.
You cannot deallocate a shared resource while another thread is or might be using it. In practice, you will find that you cannot even write code to do what you suggest.
Think about it. When you go to call close, how can you possibly know that the other thread is actually blocked in recv? What if it's about to call recv, but then another thread calls socket and gets the descriptor you just closed? Now, not only will that thread not detect any error, but it will be calling recv on the wrong socket!
There is probably a good way to solve your outer problem, the reason you need to exit from a blocking UDP socket read. There are also several ugly hacks available. The basic approach is to make the socket non-blocking and instead of making a blocking UDP socket read, fake a blocking read with select or poll. You can then abort this loop several ways:
One way is to have select time out and check an 'abort' flag when select returns.
Another way is to also select on the read end of a pipe. Send a single byte to the pipe to abort the select.
If posix complient system, you can try to monitor your thread:
pthread_create with a function that makes your recv and pthread_cond_signal just after, then returns.
The calling thread makes a pthread_cond_timedwait with the desired timeout and terminates the called thread if timed_out.
I'm running a multi-threaded C program (process?) , making use of semaphores & pthreads. The threads keep interacting, blocking, waking & printing prompts on stdout continuously, without any human intervention. I want to be able to exit this process (gracefully after printing a message & putting down all threads, not via a crude CTRL+C SIGINT) by pressing a keyboard character like #.
What are my options for getting such an input from the user?
What more relevant information could I provide that will help to solve this problem?
Edit:
All your answers sound interesting, but my primary question remains. How do I get user input, when I don't know which thread is currently executing? Also, semaphore blocking using sem_wait() breaks if signalled via SIGINT, which may cause a deadlock.
There is no difference in reading standard input from threads except if more than one thread is trying to read it at the same time. Most likely your threads are not all calling functions to read standard input all the time, though.
If you regularly need to read input from the user you might want to have one thread that just reads this input and then sets flags or posts events to other threads based on this input.
If the kill character is the only thing you want or if this is just going to be used for debugging then what you probably want to do is occasionally poll for new data on standard input. You can do this either by setting up standard input as non-blocking and try to read from it occasionally. If reads return 0 characters read then no keys were pressed. This method has some problems, though. I've never used stdio.h functions on a FILE * after having set the underlying file descriptor (an int) to non-blocking, but suspect that they may act odd. You could avoid the use of the stdio functions and use read to avoid this. There is still an issue I read about once where the block/non-block flag could be changed by another process if you forked and exec-ed a new program that had access to a version of that file descriptor. I'm not sure if this is a problem on all systems. Nonblocking mode can be set or cleared with a 'fcntl' call.
But you could use one of the polling functions with a very small (0) timeout to see if there is data ready. The poll system call is probably the simplest, but there is also select. Various operating systems have other polling functions.
#include <poll.h>
...
/* return 0 if no data is available on stdin.
> 0 if there is data ready
< 0 if there is an error
*/
int poll_stdin(void) {
struct pollfd pfd = { .fd = 0, .events = POLLIN };
/* Since we only ask for POLLIN we assume that that was the only thing that
* the kernel would have put in pfd.revents */
return = poll(&pfd, 1, 0);
}
You can call this function within one of your threads until and as long as it retuns 0 you just keep on going. When it returns a positive number then you need to read a character from stdin to see what that was. Note that if you are using the stdio functions on stdin elsewhere there could actually be other characters already buffered up in front of the new character. poll tells you that the operating system has something new for you, not what C's stdio has.
If you are regularly reading from standard input in other threads then things just get messy. I'm assuming you aren't doing that (because if you are and it works correctly you probably wouldn't be asking this question).
You would have a thread listening for keyboard input, and then it would join() the other threads when receiving # as input.
Another way is to trap SIGINT and use it to handle the shutdown of your application.
The way I would do it is to keep a global int "should_die" or something, whose range is 0 or 1, and another global int "died," which keeps track of the number of threads terminated. should_die and died are both initially zero. You'll also need two semaphores to provide mutex around the globals.
At a certain point, a thread checks the should_die variable (after acquiring the mutex, of course). If it should die, it acquires the died_mutex, ups the died count, releases the died_mutex, and dies.
The main initial thread periodically wakes up, checks that the number of threads that have died is less than the number of threads, and goes back to sleep. The main thread dies when all the other threads have checked in.
If the main thread doesn't spawn all the threads itself, a small modification would be to have "threads_alive" instead of "died". threads_alive is incremented when a thread forks, and decremented when the thread dies.
In general, terminating a multithreaded operation cleanly is a pain in the butt, and besides special cases where you can use things like the semaphore barrier design pattern, this is the best I've heard of. I'd love to hear it if you find a better, cleaner one.
~anjruu
In general, I have threads waiting on a set of events and one of those events is the termination event.
In the main thread, when I have triggered the termination event, I then wait on all the threads having exited.
SIGINT is actually not that difficult to handle and is often used for graceful termination. You need a signal handler and a way to tell all the threads that it's time to stop. One global flag that threads check in their loops and the signal handler sets might do. Same approach works for "on user command" termination, though you need a way to get the input from the terminal - either poll in a dedicated thread, or again, set the terminal to generate a signal for you.
The tricky part is to unblock waiting threads. You have to carefully design the notification protocol of who tells who to stop and what they need to do - put dummy message into a queue, set a flag and signal a cv, etc.