I've searched and searched to no avail, so I've finally come to ask for some help.
My assignment involves (in C running on RedHat Linux) creating two child processes, each of which write characters for an arbitrary amount of iterations to a pipe. This pipe is shared with the parent process that created them. The parent reads from the pipe while the children still have characters to write to it. Once the children have exited and the pipe is empty, the parent(main) can then terminate.
Here is a quick & dirty example of the logic of the code I have written.
main()
{
//create pipe
fork(); //childA
//writes to pipe
fork(); //childB
//writes to pipe
//parent reading
while(condition) {
//read from pipe + print chars to terminal
}
}
Now, my question is regarding the condition of the while loop.
I need to read when the children are blocked from writing due to a full pipe, but I cannot figure out what type of condition would allow me to do this. Any help would be absolutely amazing.
Is it a requirement that the pipe needs to be full when you read? Or is it simply a requirement that you keep reading from both children, even if one of the pipes is full so the child is blocked on writing?
I don't know of any standard way to tall if a pipe is full. You could try using the FIONREAD ioctl to determine how much data there is to read on the pipe, and compare that against PIPE_BUF, but that may not work properly if there is less than PIPE_BUF data in the pipe, but the child is doing a write that would put it over PIPE_BUF; that call may block without filling up the pipe. I would not rely on this technique working.
The usual way to keep reading from two file descriptors, regardless of which one is ready, is to use the select system call. This allows you to wait until one or the other file descriptor has data available. This means that the parent process won't block trying to read from one child which doesn't have any data available, while the other child is blocking because its buffer is full.
edit: After re-reading your question, it sounds like there is actually only one pipe, that both children are writing to. Is that correct? Again, the question comes up about whether you are required to wait until the children are blocking. If the parent simply reads from the pipe, it will block until one of the children has written to the pipe; you don't need to do anything special.
If your assignment requires you to wait until the pipe is actually full, I'd be interested in seeing the exact wording, because I'm not sure why you would want to do that.
edit 2: In response to your questions in the comment:
Does the code for my parent process need to follow the code for my two child processes within the program?
No, there is no requirement about the order of the code for the parent or child processes. In order to distinguish the code the parent runs from the code the children run, you check the return value of fork(). If the return value is 0, you are in the child process. If it is not, you are in the parent; if the return value is -1, then there was an error, if it's positive, it is the PID of the child process.
So, you can write code like this:
int pid = fork();
if (pid) {
// in the parent, check if pid is -1 for errors
} else {
// in the child
}
Or:
int pid = fork();
if (pid == 0) {
// in the child, do whatever you need to do and...
exit(0);
}
// in the parent; since the child calls exit() above, control will never
// reach here in the child. Or you could do an execl() in the child, which
// replaces the current program with another one, so again, control will
// never reach here within the child process.
How can I keep reading from the pipe until the two children terminate AND the pipe is empty?
Just keep reading until read returns 0. read on the read end of the pipe will not return 0 until all processes have closed the write end of the pipe, and all data has been read out of the pipe. If something still has it open, but there is no data in the pipe, read will block until data is written to the pipe.
One gotcha is to remember to close the write end of the pipe in the parent process before trying to read; otherwise, when the parent tries to do a read after the children have finished, it will block waiting for it to close its own pipe.
Related
I need to create a multi-process program that:
Creates 5 processes using fork();
Sends stuff to do the the child processes using pipes
When a child process completes its stuff, it should receive new stuff to do from parent process until all stuff are completed.
Right now my idea is to wait() on completed child tasks (and it exits) and then to create a new child process so I always have a maximum of 5 processes.
Is there a way to "re-use" the already existing process? Maybe "signaling" something? Cannot find it on Google.
Using C.
I solved my problem in this way:
Children write the result of their calculation on a pipe A (this pipe is not blocking). Then they wait their next input on a pipe B (this pipe is blocking).
Parent loops on all children trying to read something from pipe A (this pipe is not blocking so parent goes on if child didn't send anything). If pipe A contains a result, the parent sends another task to that child on pipe B.
There is a pipe A and a pipe B for each child.
I am attempting to write a program which forks and waits for his child to finish, then the child does some work on an input and then forks the same way it's parent does and so on.
Now, I know that forking copies to the child the array of file descriptors and that I should close the ones associated with the parent, but I can't figure out which are the parents. Do I need to give to my child it's parents pid?
I've been trying to wrap my head around it for the better part of an hour and I think I have some kind of a mind block because I can't come to a conclusion.
TL;DR: As a child process how do I know which file descriptors belong to my parent?
Just after the fork (and before any exec function) your child process has the same state as its parent process (except for the result of the fork, which is 0 only in the child). So you know what are the file descriptors, since you have coded the program running in parent&child. On Linux you might also read the /proc/self/fd/ directory, see proc(5).
You might close most file descriptors after the fork and before the exec; you could code something like
for (int fd=3; fd<64; fd++) (void) close(fd);
we are starting from 3 which is after STDERR_FILENO which is 2, and we are stopping arbitrarily at 64, and the cast to (void) on the close call means to the reader that we don't care about failing close.... Of course, if you have e.g. some pipe(7)-s to communicate between parent and child you'll be careful to avoid closing their relevant file descriptor(s).
(However, doing a closing loop like above is poor taste and old fashion)
In general, you'll be careful in your program to set the close-on-exec flag on most file descriptors (e.g. fcntl(2) on F_SETFD operation and FD_CLOEXEC flag, or directly open(2) with O_CLOEXEC), then the execve(2) (done in most child processes after the fork) would close them.
I do not know why the parent process needs to close both the file descriptors of a pipe before calling wait()?
I have a C program which does:
Parent creates child_a, which executes ls -l using execvp, and writes to the pipe (after closing read end of pipe).
Parent creates another child (without closing any file descriptor for pipe), called child_b, which executes 'wc' by reading from pipe.(after closing write end of pipe).
Parent waits for both children to complete by calling wait() twice.
I noticed that program is blocked if parent does not close both file descriptors of the pipe before calling the wait() syscall. Also after reading few questions already posted online it looks like this is the general rule and needs to be done. But I could not find the reason why this has to be done?
Why does wait() not return if the parent does not close the file descriptors of the pipe?
I was thinking that, in the worst case, if the parent does not close the file descriptor of pipe, then the only consequence would be that the pipe would keep existing (which is a waste of resource). But I never thought this would block the execution of child process (as can be seen because wait() does not return).
Also remember, parent is not using the pipe at all. It is child_a writing in the pipe, and child_b reading from the pipe.
If the parent process doesn't close the write ends of the pipes, the child processes never get EOF (zero bytes read) because there's a process that might (but won't) write to the pipe. The child process must also close the write end of the pipe for the same reason — if it doesn't, there's a process (itself) that might (but won't) write to the pipe, so the read won't return EOF.
If you duplicate one end of a pipe to standard output or standard error, you should close both ends of that pipe. It is a common mistake not to have enough calls to close() in multiprocess code using pipes. Occasionally, you get away with being sloppy, but the details vary by case and usually you don't.
So I've been working on code for my OS class, and in my project, I have to create a new child process for each file on the command line, and pipe information from the child to the parent. We're supposed to be re-creating Unix's wc utility (I've handled this part already).
What I've tried thus far is:
for(i=0; i<argcount; i++){
int pid;
pid = fork();
if(pid == 0){
/* Child Process */
/* Close read pipes */
close(l_pipe[0]);
close(w_pipe[0]);
close(c_pipe[0]);
wc(lflag, wflag, cflag, filenames[i]);
} else {
/* Parent Process for piping */
/* Close write pipes */
close(l_pipe[1]);
close(w_pipe[1]);
close(c_pipe[1]);
/* Read from pipes */
read(l_pipe[0], &buffer, sizeof(count_t));
lines+=buffer;
read(w_pipe[0], &buffer, sizeof(count_t));
words+=buffer;
read(c_pipe[0], &buffer, sizeof(count_t));
bytes+=buffer;
}
}
However, this creates as many parents as there are children, which is obviously wrong. I'm not exactly sure where I should be forking. I have to use a pipeline between the child and parent processes, and I'm certain that the parent needs to read() as many times as the child write()s.
Thank you for any suggestions you can provide.
As well as the issues raised by Greg Hewgill in his answer, I observe the following issues:
You could have the loop fork and run the child processes, while the parent part simply goes back to the next iteration of the loop.
You would then have a loop to read from the pipes.
Before that loop, the parent process would close the write ends of all three pipes (otherwise it will never see EOF on the pipes).
The loop to read from the pipes should read from each pipe in turn, rather than draining each pipe in turn.
I'm assuming that you have count_t buffer; — it helps to show variable declarations. If you have some sort of char buffer[sizeof(count_t)] instead, then you have all sorts of problems, large and small.
When the second child is created in your current scheme, the write ends of the pipes are all closed, so the second and subsequent children will be unable to send anything to the parent. You must move the three calls to close the write ends of the pipes so it is outside the loop.
Your pipes must be in global variables so that the wc function can use them. This isn't the end of the world, but it is often neater to avoid global variables. This is a second-order problem; you have other more major problems to fix first.
If you need to associate sizes with individual files, you have more bookkeeping to do. At the moment, you only aggregate grand totals. In that case, your current synchronous design is likely appropriate. Failing that, you'd have the children write a PID or another ID number plus the count on the pipe in one operation (to ensure the operation is atomic). Individual write operations are not interleaved unless they're too big for the pipe's internal buffer, which won't be a problem for a couple of integers.
There's a couple of things I notice straight away.
In your child process branch, you're not calling _exit() after wc(). This means that your child process will loop around and start forking itself for further children.
In your parent process branch, you are waiting for the response from the child you just spawned before proceeding to create the next child. So essentially you've serialised the process and won't take advantage of multiple processes.
I've got a little C server that needs to accept a connection and fork a child process. I need the stderr of the child process to go to an already existing named pipe, the stdout of the child to go to the stdout of the parent, and the stdin of the child tp come from the same place as the stdin of the parent.
My initial attempts involved popen() but I could never seem to get quite what I wanted.
Finally, this particular solution only needs to work in Solaris. Thanks.
EDIT: Updated the question in hopes of more accurately portraying what I'm trying to accomplish. Thanks for being patient with me.
EDIT2: I also need the parent to get the return value of the child process and then do something with it if that makes any difference.
You might be using the wrong function - popen() is used when you want the invoking program either to write to the forked process's standard input or read from its standard output. It seems you want neither. It also takes two arguments.
Your requirements are also somewhat contradictory:
I want it to (ideally) inherit stdin and stdout from the parent
any input to the parent goes to the child and any output from the child goes back to the parent
but at a minimum, I'd like it to inherit stdin and write stdout to a named pipe
The first option is easy - it requires no special coding. Any data supplied to the stdin of the parent will also be available on the stdin of the child (but only one of the two processes will get to read it). The child's stdout will normally go to the same place as the parent's stdout. If you want the parent to read the child's stdout, then you do need a pipe - and popen() is then appropriate, but the 'at minimum' stuff is confusing.
So, let's define what you really want?
Option 1
The standard error of the child should go to a named pipe.
The standard output of the child should be read by the invoking process.
The standard input of the child should come from the same place as the standard input of the parent.
The named pipe already exists.
Hence:
FILE *fp = popen("/run/my/command -with arguments 2>/my/other/pipe", "r");
Note that the child will be hung until a process opens '/my/other/pipe' for reading; that in turn means that if the parent process reads from fp, it too will be hung until some other process opens '/my/other/pipe' for reading.
Option 2
The standard error of the child should go to a named pipe.
The standard output of the child should go to the standard output of the parent.
The standard input of the child should come from the same place as the standard input of the parent.
The named pipe already exists.
Now popen() is not appropriate, and we get into naked `fork & exec' code. What follows is more pseudo-code than operational C.
if ((pid = fork() < 0)
error
else if (pid > 0)
{
/* Parent - might wait for child to complete */
}
else
{
int fd = open("/my/other/pipe", O_WRONLY|O_NONBLOCK);
if (fd < 0)
error
dup2(fd, 2); /* There is a symbolic name for stderr too */
close(fd); /* Do not want this open any more */
char *cmd[4] = { "/bin/sh", "-c", "/run/my/command -with arguments", 0 };
execv(cmd[0], cmd);
error - if execv returns, it failed!
}
If you're totally confident no-one has pulled any stunts on you like closing stdout, you can avoid using dup2() by closing stderr (fd = 2) before calling open(). However, if you do that, you can't report any errors any more - because you closed stderr. So, I would do it as shown.
If you have a different requirement, state what you want to achieve.
As noted by p2vb, if you want the parent to wait for the child to finish, then simply using system() may be sufficient. If the parent should continue while the child is running, you might try system() where the command string ends with an ampersand (&) to put the child into the background, or you might use the code outlined in Option 2 above.
Using system(), the parent will have little chance to read the /my/other/pipe which gets the standard error from the child. You could easily deadlock if the child produces a lot.
Also, be careful with your FD_CLOEXEC flag - set it on files that you don't want the child modifying. On Linux, you can use the O_CLOEXEC flag on the open() call; with Solaris, you have to set it via fcntl() - carefully.