I have multiple workers forked from parent process and need to inform them all at exactly the same time to run / stop;
Parent:
while(1)
{
workers_start!
usleep(work_time);
workers_stop!
usleep(sleep_time);
}
What should I use to inform them ALL quick(!)? Semaphores (counting N), mutexes, signals (after execve forked program)? I heard that semaphores/mutexes are slow. I'm not sure about signals but I don't know if sending so many signals is a good idea (N workers * 1 or 2 signals / per 0.2sec, where N is cpu cores).
Sorry, it's about linux.
You might want to assign all your worker processed to a process group. You can do this by calling
setpgid(pid_t pid, pid_t pgid)
Where "pid is the the return of the "fork()" call for each of your workers, and pgid is the project group ID of all your workers, and should be the same for all workers. The manual page is here:
http://man7.org/linux/man-pages/man2/getpgrp.2.html
Then you can send a signal to all your processes with one single call to
killpg(int pgrp, int sig)
Where pgrp is the group that you assigned to all your workers in the step above. As for the values of "sig" in the call of killpg, if you have a POSIX platform, you might want to user the user defined siganls, for instance SIGUSR1, SIGUSR2 for signalling start and stop respectively. If not, you can use standard signals and capture them (override their behaviour). The manual page to killpg is here:
http://man7.org/linux/man-pages/man2/killpg.2.html
I hope this is clear enough and is similar to what you are looking for.
This depends on the Operating system you're using. Generally information of the quality "I heard semaphores are slow" are always to be taken with a grain of salt. Some quick testing will probably reveal the best solution for your case :)
Related
The waiting works fine with pidfd_open and poll.
The problem I’m facing, after the process quits, apparently the poll() API removes the information about the now dead process, so the waitid with P_PIDFD argument fails at once saying code 22 “Invalid argument”
I don’t think I can afford launching a thread for every child process to sleep on the blocking waitpid, I have multiple processes, and another handles which aren’t processes I need to poll efficiently.
Any workarounds?
If it matters, I only need to support Linux 5.13.12 and newer running on ARM64 and ARMv7 CPUs.
The approximate sequence of kernel calls is following:
fork
In the child: setresuid, setresgid, execvpe
In the new child: printf, sleep, _exit
Meanwhile in the parent: pidfd_open, poll, once completed waitid with P_PIDFD first argument.
Expected result: waitid should give me the exit code of the child.
Actual result: it does nothing and sets errno to EINVAL
There is one crucial bit. From man waitid:
Applications shall specify at least one of the flags WEXITED, WSTOPPED, or WCONTINUED to be OR'ed in with the options argument.
I was passing was WNOHANG
And you want to pass WNOHAND | WEXITED ;)
You can use a single reaper thread, looping on waitpid(-1, &status, 0). Whenever it reaps a child process, it looks it up in the set of current child processes, handles possible notifications (semaphore or callback), and stores the exit status.
There is one notable situation that needs special consideration: the child process may exit before fork() returns in the parent process. This means it is possible for the reaper to see a child process exiting before the code that did the fork() manages to register the child process ID in any data structure. Thus, both the reaper and the fork() registering functions must be ready to look up or create the record in the data store keeping track of child processes; including calling the callback or posting the semaphore. It is not complicated at all, but unless you are used to thinking in asynchronous terms, it is easy to miss these corner cases.
Because wait(...)/waitpid(-1,...) returns immediately when there are no child processes to wait for (with -1 and errno set to ECHILD), the reaper thread should probably wait on a condition variable when there are no child processes to wait for, with the code that registers the child process ID signaling on that condition variable to minimize resource use in the no-child-processes case. (Also, do remember to minimize the reaper thread stack size, as it is unreasonably large (order of 8 MiB) by default, and wastes resources. I often use 2*PTHREAD_STACK_MIN, myself.)
I have a shared memory X (array), 2 type of process and I have 3 semaphores, one for modifier X and the other one is for wait until I want to wake up one of these waited process.
Step by step:
1. I'm the process 1 and I ask for semaphore 1 (mutex).
2. OK I got it so I can add me in the array, now I wait (semaphore 2).
3. REPITE this for all the process
Now I have an array with 20 PID's for example, and I want wake up the process 5 (x[4]).
The question is: how can I wake up this process? How can I send a signal to a concret PID?
Excuse my english...
Thank you guys!!
You might use the kill(2) syscall to send a signal to some given process.
I am not sure it is the best way to synchronize (I believe it is not; read signal(7)). You could use Posix semaphores sem_overview(7) (or even old SysV semaphores svipc(7)) or pipes pipe(7) (or unix(7) sockets) with poll(2)
Read Advanced Linux Programming
I have a multi-thread application where each thread has a helper thread that helps the first one to accomplish a task. I would like that when a thread is terminated (likely calling exit) the helper thread is terminated as well.
I know that there is the possibility to use exit_group, but this system call kills all threads in the same group of the calling thread. For example, if my application has 10 threads (and therefore 10 additional helper threads) I would like that only the thread and the helper thread associated is terminated, while the other threads keep on running.
My application works exclusively on Linux.
How can I have this behavior?
Reading around about multithreading I got a bit confused about the concept of thread group and process group in Linux. Are these terms referring to the same thing?
Precisely, the process group (and perhaps the thread group) is the pid retrieved by one of the following calls :
pid_t getpgid(pid_t pid);
pid_t getpgrp(void); /* POSIX.1 version */
pid_t getpgrp(pid_t pid); /* BSD version */
You are a bit adrift here. Forget exit_group, which these days is the same as exit on linux is not what you are looking for. Similarly the various get-pid calls aren't really what you want either.
The simplest (and usually best) way to handle this is have each primary thread signal its helper thread to shut down and then pthread_join it - or not if it is detached.
So something like:
(a) primary work thread knows - however it knows - its work is done.
(b) signals helper thread via a shared switch or similar mechanism
(c) helper thread periodically checks flag, cleans up and calls pthread_exit
(d) primary worker thread calls pthread_join (or not) on dead helper thread
(e) primary worker cleans up and calls pthread_exit on itself.
There are a lot of variations on that but that's the basic idea. Beyond that you get into things like pthread_cancel and areas you may want to avoid if you don't absolutely require them (and the potential headaches).
While running a thread program and repeatedly killing the main program using Ctrl + C, i see unexpected results in the program in second run. However, if i let the program run and voluntarily exit, there are no issues.
So, my doubt is, does Ctrl + C, kill threads also along with the main process?
Thanks in advance.
In multithreaded programming, signals are delivered to a single thread (usually chosen unpredictably among the threads that don't have that particular signal blocked). However, this does not mean that a signal whose default action is to kill the process only terminates one thread. In fact, there is no way to kill a single thread without killing the whole process.
As long as you leave SIGINT with its default action of terminating the process, it will do so as long as at least one thread leaves SIGINT unblocked. It doesn't matter which thread has it unblocked as long as at least one does, so library code creating threads behind the application's back should always block all signals before calling pthread_create and restore the signal mask in the calling thread afterwards.
Well, the only thing that Ctrl + C does is sending SIGINT to one thread in the process that is not masking the signal. Signals can be handled or ignored.
If the program does handle Ctrl+C, the usual behavior is self-termination, but once again, it could be used for anything else.
In your case, SIGINT is being received by one thread, which probably does kill itself, but does not kill the others.
Under Linux 2.6 using NPTL threads: I am assuming that the process uses the default signal handler, or calls exit() in it: Yes it does. The C library exit() call maps to the exit_group system call which exits all the threads immediately; the default signal handler calls this or something similar.
Under Linux 2.4 using Linuxthreads (or using 2.6 if your app still uses Linuxthreads for some weird reason): Not necessarily.
The Linuxthreads library implements threads using clone(), creating a new process which happens to share its address-space with the parent. This does not necessarily die when the parent dies. To fix this, there is a "master thread" which pthreads creates. This master thread does various things, one of them is to try to ensure that all the threads get killed when the process exits (for whatever reason).
It does not necessarily succeed
If it does succeed, it is not necessarily immediate, particularly if there are a large number of threads.
So if you're using Linuxthreads, possibly not.
The other threads might not exit immediately, or indeed at all.
However, no matter what thread library you use, forked child processes will continue (they might receive the signal if they are still in the same process-group, but can freely ignore it)
people. For a academic exercise i have to implement a program in c for nix platform which is to synchronize multiple processes through signal handling, using only signal,pause,kill and fork basic functions.
I searched google and have not found any clear example: I hope the wisdom of one of you will light my way.
Thanks!
pause doesn't return until a signal is received. The basic design is thus:
fork to create the necessary workers
catch SIGINT in each worker. The handler sets a flag meaning the process should exit after finishing it's current job.
each process does work while it can, then pauses. Repeat unless SIGINT is received (test before & after pause).
when one process has work available for another process, it signals the other process with SIGCONT
when there is no more work for a process, signal it with SIGINT.
This doesn't quite include synchronized access to shared data. To do that, you could add an additional rule:
when a process signals another that work is available, it should pause
Of course, this rather defeats the purpose of concurrent programming.
Since most system calls are interrupted by signals (causing them to return -1, with errno set to EINTR), you'll have to handle this contingency, repeating each affected system call until it's successful. For example:
while ((readCount = read(...)) < 0 && errno == EINTR) {}
One important thing to be aware of is that Linux (at least, and possibly many other Unices) can collapse multiple signals of the same type into a single instance. So if you send a process one signal of value x, that process is guaranteed to receive it; but if you send 2 or more signals of value x, the process is only guaranteed to receive at least one of those signals.
Also, signals are not guaranteed to be received in the order they are sent.
(Why? Under the hood, Linux maintains a bitmask for each process recording which outstanding signals have been sent. Whenever a process is woken up by the scheduler, signal handlers for all outstanding signals are run, in some arbitrary order.)
What all this means is that signals are generally inappropriate for synchronising processes. They only work reliably when time intervals between signals are large with respect to the interval between wake-up times of the receiving process. And if a process spends a lot of time blocked, wake-up events can be arbitrarily far apart.
Conclusion: don't using signals for IPC.