This question already has an answer here:
Synchronising N sibling processes after fork
(1 answer)
Closed 3 years ago.
I have a shared variable set to 0 and then fork N processes. These processes then busy wait for the shared variable to become 1. The parent is responsible for setting this variable to 1. What's the best option here to eliminate busy waiting?
I was looking into semaphores (sem_open, etc) but my understanding is that parent will need sem_post N times so that all presses can proceed.
One way would be to create a pipe in the parent process. Each child process closes the write end of the pipe, then calls read (which blocks). When the parent process is ready, it closes both ends of its pipe. This makes all the children return from read and they can now close their read end of the pipe and proceed with their code.
For situations like this, I sometimes use atomic operation. You can use atomic flags to notify sub-process, but it creates busy-waiting. So you can use it on a very special cases.
Other method is to create an event with something like pthread_cond_broadcast() and pthread_cond_timedwait(). All sub-process wait on your condition. When you are ready to go, unlock it on parent process.
Related
I am just going to post pseudo code,
but my question is I have a loop like such
for(i<n){
createfork();
if(child)
/*
Exit so I can control exact amount of forks
without children creating more children
*/
exit
}
void createfork(){
fork
//execute other methods
}
Does my fork create a process do what it is suppose to do and exit then create another process and repeat? And if so what are some ways around this, to get the processes running concurrently?
Your pseudocode is correct as written and does not need to be modified.
The processes are already executing in parallel, all six of them or however many you spawn. As written, the parent process does not wait for the children to finish before spawning more children. It calls fork(), checks if (child) (which is skipped), then immediately proceeds to the next for loop iteration and forks again.
Notably, there's no wait() call. If the parent were to call wait() or waitpid() to wait for each child to finish then that would introduce the serialism you're trying to avoid. But there is no such call, so you're good.
When a process successfully performs a POSIX fork(), that process and the new child process are initially both eligible to run. In that sense, they will run concurrently until one or the other blocks. Whether there will be any periods of time when both are executing machine instructions (on different processing units) depends at least on details of hardware capabilities, OS scheduling, the work each process is performing, and what other processes there are in the system and what they are doing.
The parent certainly does not, in general, automatically wait for the child to terminate before it proceeds with its own work (there is a family of functions to make it wait when you want that), nor does the child process automatically wait for any kind of signal from the parent. If the next thing the parent does is fork another child, then that will under many circumstances result in the parent running concurrently with both (all) children, in the sense described above.
I cannot speak to specifics of the behavior of your pseudocode, because it's pseudocode.
I know there's another thread with the same name, but this is actually a different question.
When a process forks multiple times, does the parent finish executing before the children? Vice versa? Concurrently?
Here's an example. Lets say I have a for loop that forks 1 parent process into 4 children. At the end of that for loop, I want the parent process to feed some data to the children via pipes. The data is written to each child process' respective stdin.
Will the parent send the data first, before any of the children execute their code? This is important, because we don't want it to start working from an invalid stdin.
The order of the execution is determined by the specific OS scheduling policy and not guaranteed by anything. In order to synchronize the processes there are special facilities for the inter-process communication (IPC) which are designed for this purpose. The mentioned pipes are one example. They make the reading process to actually wait for the other process to write it, creating a (one-way) synchronization point. The other examples would be FIFOs and sockets. For simpler tasks the wait() family of functions or signals can be used.
When a process forks multiple times, does the parent finish executing before the children? Vice versa? Concurrently? -
Concurrently and depends on the scheduler and its unpredictable.
Using pipe to pass integer values between parent and child
This link explains in detail about sharing data between parent process and child.
Since you have four child process you may need to create different individual pipes between each child process.
Each byte of data written to a pipe will be read exactly once. It isn't duplicated to every process with the read end of the pipe open.
Multiple child processes reading/writing on the same pipe
Alternatively you can try shared memory for the data transfer.
They will execute concurrently. This is basically the point of processes.
Look into mutexes or other ways to deal with concurrency.
Linux supports the POSIX wait mechanism defined in "sys/wait.h". The methods wait, waitid, waitpid might be used to exchange status information between parent and child processes that have been created using fork.
Windows neither does provide (native) support for fork nor the POSIX wait mechanism. Instead there are other means available to spwan child processes i.e. CreateProcess.
When porting linux applications written in C or C++ using fork/wait to Windows what would the most proper native* way to monitor state changes (namely WEXITED, WSTOPPED, WCONTINUED) of child processes in the parent process?
*native meaning using no additional libraries, frameworks, programs (like cygwin, minGW) that do not ship with windows or are provided directly by MS in form of runtime environments.
Edit: As requested in the comments I did provide some more information about what problem should be solved in form of pseudo code:
//creates a new child process that is a copy of the parent (compare
//POSIX fork()) and returns some sort of handle to it.
function spawnChild()
// returns TRUE if called from the master process FALSE otherwise
function master()
// return TRUE if called from a child process FALSE otherwise
function child()
// returns TRUE if child process has finished its work entirely,
// FALSE otherwise.
function completelyFinished()
//sends signal/message "sig" to receive where receiver is a single
//handle or a set of handles to processes that shall receive sig
function sendSignal(sig, receiver)
// terminates the calling process
function exit()
// returns a handle to the sender of signal "sig"
function senderOf(sig)
function masterprocess()
master //contains handle to the master process
children = {} //this is an empty set of handles to child processes
buf[SIZE] //some memory area of SIZE bytes available to master process and all children
FOR i = 0 TO n - 1
//spawn new child process and at its handle to the list of running
//child processes.
children <- children UNION spawnChild()
IF(master())
<logic here>
sendSignal(STARTWORKING, children) //send notification to children
WHILE(signal = wait()) // wait for any child to respond (wait is blocking)
IF signal == IMDONE
<logic here (involving reads/writes to buf)>
sendSignal(STARTWORKING, senderOf(signal))
ELSEIF signal == EXITED
children <- children \ signal.sender //remove sender from list of children
ELSEIF(child())
WHILE(wait() != STARTWORKING);
<logic here (involving reads/writes to buf)>
IF completelyFinished()
sendSignal(EXITED, master)
exit()
ELSE
sendSignal(IMDONE, master)
Before I answer the actual question, I'm going to recommend a better solution: you should consider simplifying the relationship between the parent and children.
Based on the pseudocode, the signals between parent and children are serving as a crude form of cross-process mutex, i.e., all they do is to prevent the code here:
IF signal == IMDONE
<logic here (involving reads/writes to buf)>
sendSignal(STARTWORKING, senderOf(signal))
from running multiple instances simultaneously. Instead, <logic here> should be moved into the corresponding child process, protected by a mutex so that only one child can run it at a time.
At that point, all the parent needs to do is to launch the children and wait for them all to exit. That is easily done in Windows by waiting on the process handle.
(I would imagine that modern POSIX also supports some sort of cross-process mutex somewhat more sophisticated than signals.)
It would also be worth reconsidering whether you really really need multiple processes. Multiple threads would be more efficient, and if the code is properly written, it should not be difficult to adapt it.
Be that as it may, if for some reason you absolutely must retain as much of the original program structure as possible, pipes are probably going to be your best bet.
Sending a signal becomes writing a single byte.
In a child, waiting for a signal from the parent becomes reading a single byte.
Waiting in the parent for a message from any of the children is a little trickier. It is still a single-byte read (for each child) but you'll need to use overlapped I/O and, if you need to support more than 64 children, IOCP.
(Alternatively, you could use multiple threads, but that might involve too much of a structural change.)
If the pipes are implemented correctly, when a child exits or dies the corresponding read operation in the parent will terminate with the ERROR_BROKEN_PIPE error. So there is no need for a separate mechanism to monitor the health of the children.
In this context, I think anonymous pipes would be the most appropriate choice. These are simplex, so you'll need two pipes for each child. You can pass the child's end of the pipe handles as the standard input and output for the child process.
For anonymous pipes, you will need to make sure that you close the parent's copy of the handles once each child has been started, and also that each child only inherits the handles corresponding to its own pipe. If there are any additional handles left open to the child's end of its pipe, the parent will not receive any notification when the child exits.
None of this is particularly complicated, but be aware that named pipe I/O has a bit of a learning curve. Asynchronous I/O even more so, particularly if you are coming from a UNIX background. Note in particular that to use asynchronous I/O, you issue an operation and then wait for it to complete, as opposed to the UNIX model where you wait for the I/O to be ready and then issue the operation.
If you want to signal boolean conditions to other processes you probably should use shared events for that. You can share them by name or by handle duplication. You can have as many of these signals as you like. For example, you could have one for each of WEXITED, WSTOPPED, WCONTINUED.
Seeing your edit: Events are great for that. Create named events in the parent and pass their names on the command like to the children. That way parent and child can signal each other.
You also need to share a memory section, for example though a memory mapped file. That would correspond to buf in your code.
What you have there appears to be a work queue arrangement, where you have a producer process and a bunch of worker processes. It's unclear whether you're using the shared memory merely as a work queue, or whether your workers are operating on the shared memory (maybe it's a massive matrix or vector problem).
In Win32, you probably wouldn't implement this as separate processes.
You'd use a collection of producer/consumer threads, which are already sharing memory (same address space), and you'd implement a work queue using semaphores or condition variables.
In fact, you'd probably use a higher-level abstraction, such as QueueUserWorkItem. This uses the default Windows thread pool, but you can create your own thread pool, using CreateThreadpool.
I am writing a C program in which the parent forks n child processes. A child process once created invokes a SIGSTOP to allow other child processes to be created. The parent after creating all the n child processes sends a SIGCONT signal to all the child.
All the child processes execute an infinite loop and share a common resource using semaphores. Now I want that whenever the user presses ctrl-c, the parent and all the child processes terminate together. However before terminating the child processes should update in a file how many times each has used the resource.
eg:
Process 1 - 5 times
Process 2 - 3 times
and so on.
Need help in this implementation please...
The formal signal handler function should do as little as feasible. The C standard says it can write to a volatile sig_atomic_t variable, or call abort() or _Exit() (or, with restrictions, signal()). POSIX allows more to happen, and you're probably working on Linux (though you didn't say so). So, your signal handler will change the value of a sig_atomic_t variable from 0 to 1 to indicate that the signal occurred.
So, your child processes will be looping. As part of the loop condition, you should check a sig_atomic_t variable to see whether the child should terminate. When it detects that a signal occurred, it will stop looping, open the log file for append, write its information to that file, and exit. You could check the sig_atomic_t variable at other points in the processing than just the main loop condition; that's your decision.
Note that you should use sigaction() rather than signal() to control the signal handling, and you should block interrupts while processing a signal.
So, to summarize:
Your signal handler does as little as possible.
Your code detects in the main loop when the signal handler has been called and arranges to exit.
Your code can also detect when the signal handler has been called at other convenient points.
You can call a function to do the logging and exit.
In order to write the file, you need to add the code to write to said file in your signal handler for SIGINT. If you want each process to write to that file, you're going to need to make sure that the SIGINT gets sent to the entire process group.
You can probably get by without sending the SIGQUIT to each, as you could have each process simply exit itself after processing SIGINT. If you want to optimize a little, you could keep a shared data structure of which processes have already received the SIGINT so that you don't send each process several SIGINTs.
So I have an application which uses threads. Now when the program first starts up, I want it to go through setting up database connections and whatnot before it backgrounds itself so that whatever/whoever starts the program can know if there was an error starting up.
I did some looking around and have found some resources that say 'do not mix fork and threads', while others say that forking in linux will only duplicate the main thread and leave the others alone.
In the case of the latter (where it just duplicates the main thread), how then do the threads access file level (global) variables? Will the threads not be able to access the variables that are now in the forked process's address space?
Ultimately the goal is to have the application background itself after threads have been created. If this is not possible, I can put the fork before the thread creation, just would like to do it as late as possible.
Note: at the time of the fork, the threads will be doing a sleep() loop until the main thread puts data into a shared variable for them to process. So if the sleep gets interrupted, they wont be harmed.
There is no way to duplicate threads as part of the fork, and the parent's threads will all terminate when the parent exits, so even if they could access the child's memory, it wouldn't help you. You need to either create your threads after forking, or use pthread_atfork to register handlers that will recreate them in the child process. I would recommend just waiting until after forking to create your threads since it's a lot simpler and more efficient.
Why is it that you want to delay forking as long as possible? If you want to maintain connection to a terminal or something until initialization is finished, you can just have the parent process wait to terminate until the child process (with its threads) is done initializing and ready to be "in the background". Various synchronization tools could be used to accomplish this. One simple one would be opening a pipe through which the child sends its output back to the parent to display; the parent could simply exit when it receives EOF on this pipe.
Forking a process creates two different processes and threads in one process will not be able to access memory in the second process. If you want different processes to access the same memory, you want something called shared memory.
When a thread in a process calls fork(), a new process is created by copying, among other things, (1) the full address space of the process and (2) the (one) thread that called fork. If there are other threads in the process, they don't get copied. This will almost certainly lead to bugs in your program. Hence the advice not to mix threads and forks.
If you want to create a background process with many threads, you must fork it before spawning any other thread. Then, the two processes behave normally, like any two isolated processes: threads within one process share the same memory, but your background threads and your foreground process won't share any memory (by default).