I've seen this question before, but still a bit confused: how would I create communication between child processes of the same parent? All I'm trying to do at the moment is passing a message from the first child process to the nth child process. My idea is to create n-1 pipes in the parent process and then redirect the parent's ends to the next child process. What I can't figure out is, how would we redirect the ends from the parent if the next child process hasn't been created? I feel there's an issue in the way I'm approaching this.
EDIT: My goal is to print the message that was passed from the first child process to the last one. It's a simple program.
You don't need to create the processes first. The solution is as follows: you first create all the needed pipes, save them in some array. Then you do a fork, redirect input and output streams (for the child process) accordingly, close unused pipe ends and perform an exec. Pipes can exist without the corresponding processes, they have buffers, so you can write to a pipe while nobody's still reading, it will be ok.
You should just be aware of closing unused id's before doing exec. And be careful writing to a pipe which input endpoints (all the input endpoints) could become closed: it could result in a SIGPIPE.
The code to setup a pipe and have stadin/stdout redirected to the pipe is.
In parent (before fork)
int p[2];
pipe(p);
In the child (after fork) to get the pipe as stdin
close(0);
dup(p[0]);
In the child (after fork) to get the pipe as stdout
close(1);
dup(p[1]);
You can start writing to the pipe as soon as it is created. However this child process which is writing to the pipe will pause as soon as the pipe-buffer is filled and until the other process is starting reading from the pipe.
Also look at the popen call as that may actually be a simpler version of what you need.
Rather than using un-named pipes, you may want to look into an alternate mechanism for inter-process communication such as using a FIFO, message-queue, or a socket using datagrams (i.e., UDP) that would allow you to write a "message" into the communication buffer, and then have another child read the message, see if it's for them and if it's not, have them place it back in the communicatons buffer for another child to read. Otherwise if the message is for them, they then accept it.
Your message can be a struct that contains a sender ID, receiver ID, and then some buffer to hold a message, whether that's a string-type, etc.
The communications buffer can be setup by the parent, and inherited by the children.
In a simplified non-exec case, you should do something like.
#define nproc 17
int pipes[nproc - 1][2];
int pids[nproc];
int i;
for( i = 0; i < nproc - 1; i++ ) {
pipe( pipes[i] );
}
int rank;
for( rank = 0; rank < nproc; rank++ ) {
pids[rank] = fork();
if( !pids[rank] ) {
if( rank == 0 ) {
write( pipe[rank][1], /* your message */ );
}
read( pipe[rank-1][0], /* read the message somewhere*/ );
if( rank < nproc - 1 ) {
write( pipe[rank][1], /* write the message to the next child process*/ );
} else {
// Here print the received message (it's already received above)
}
}
}
for( rank = 0; rank < nproc; ++rank ) {
wait( pids[rank] );
}
Related
I am confused as to how to properly use close to close pipes in C. I am fairly new to C so I apologize if this is too elementary but I cannot find any explanations elsewhere.
#include <stdio.h>
int main()
{
int fd[2];
pipe(fd);
if(fork() == 0) {
close(0);
dup(fd[0]);
close(fd[0]);
close(fd[1]);
} else {
close(fd[0]);
write(fd[1], "hi", 2);
close(fd[1]);
}
wait((int *) 0);
exit(0);
}
My first question is: In the above code, the child process will close the write side of fd. If we first reach close(fd[1]), then the parent process reach write(fd[1], "hi", 2), wouldn't fd[1] already been closed?
int main()
{
char *receive;
int[] fd;
pipe(fd);
if(fork() == 0) {
while(read(fd[0], receive, 2) != 0){
printf("got u!\n");
}
} else {
for(int i = 0; i < 2; i++){
write(fd[1], 'hi', 2);
}
close(fd[1]);
}
wait((int *) 0);
exit(0);
}
The second question is: In the above code, would it be possible for us to reach close(fd[1]) in the parent process before the child process finish receiving all the contents? If yes, then what is the correct way to communicate between parent and child. My understanding here is that if we do not close fd[1] in the parent, then read will keep being blocked, and the program won't exit either.
First of all note that, after fork(), the file descriptors fd would also get copied over to the child process. So basically, a pipe acts like a file with each process having its own references to the read and write end of the pipe. Essentially there are 2 read and 2 write file descriptors, one for each process.
My first question is: In the above code, the child process will close
the write side of fd. If we first reach close(fd[1]), then the parent
process reach write(fd[1], "hi", 2), wouldn't fd[1] already been
closed?
Answer: No. The fd[1] in parent process is the parent's write end. The child has forsaken its right to write on the pipe by closing its fd[1], which does not stop the parent from writing into it.
Before answering the second question, I fixed your code to actually run it and produce some results.
int main()
{
char receive[10];
int fd[2];
pipe(fd);
if(fork() == 0) {
close(fd[1]); <-- Close UNUSED write end
while(read(fd[0], receive, 2) != 0){
printf("got u!\n");
receive[2] = '\0';
printf("%s\n", receive);
}
close(fd[0]); <-- Close read end after reading
} else {
close(fd[0]); <-- Close UNUSED read end
for(int i = 0; i < 2; i++){
write(fd[1], "hi", 2);
}
close(fd[1]); <-- Close write end after writing
wait((int *) 0);
}
exit(0);
}
Result:
got u!
hi
got u!
hi
Note: We (seemingly) lost one hi because we are reading it into same array receive which essentially overrides the first hi. You can use 2D char arrays to retain both the messages.
The second question is: In the above code, would it be possible for us
to reach close(fd[1]) in the parent process before the child process
finish receiving all the contents?
Answer: Yes. Writing to a pipe() is non-blocking (unless otherwise specified) until the pipe buffer is full.
If yes, then what is the correct
way to communicate between parent and child. My understanding here is
that if we do not close fd[1] in the parent, then read will keep being
blocked, and the program won't exit either.
If we close fd[1] in parent, it will signal that parent has closed its write end. However, if the child did not close its fd[1] earlier, it will block on read() as the pipe will not send EOF until all the write ends are closed. So the child will be left expecting itself to write to the pipe, while reading from it simultaneously!
Now what happens if the parent does not close its unused read end? If the file had only one read descriptor (say the one with the child), then once the child closes it, the parent will receive some signal or error while trying to write further to the pipe as there are no readers.
However in this situation, parent also has a read descriptor open and it will be able to write to the buffer until it gets filled, which may cause problems to the next write call, if any.
This probably won't make much sense now, but if you write a program where you need to pass values through pipe again and again, then not closing unused ends will fetch you frustrating bugs often.
what is the correct way to communicate between parent and child[?]
The parent creates the pipe before forking. After the the fork, parent and child each close the pipe end they are not using (pipes should be considered unidirectional; create two if you want bidirectional communication). The processes each have their own copy of each pipe-end file descriptor, so these closures do not affect the other process's ability to use the pipe. Each process then uses the end it holds open appropriately for its directionality -- writing to the write end or reading from the read end.
When the writer finishes writing everything it intends ever to write to the pipe, it closes its end. This is important, and sometimes essential, because the reader will not perceive end-of-file on the read end of the pipe as long as any process has the write end open. This is also one reason why it is important for each process to close the end it is not using, because if the reader also has the write end open then it can block indefinitely trying to read from the pipe, regardless of what any other process does.
Of course, the reader should also close the read end when it is done with it (or terminate, letting the system handle that). Failing to do so constitutes excess resource consumption, but whether that is a serious problem depends on the circumstances.
I want to communicate with a child process like the following:
int main(int argc, char *argv[])
{
int bak, temp;
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
dup2(STDOUT_FILENO, fd[1]);
fflush(stdout);
bak = dup(1);
temp = open("/dev/null", O_WRONLY);
dup2(temp, 1);
close(temp );
Mat frame;
std::vector<uchar> buf;
namedWindow( "Camera", WINDOW_AUTOSIZE );
VideoCapture cam(0 + CAP_V4L);
sleep(1);
if (!cam.isOpened())
{
cout << "\nCould not open reference " << 0 << endl;
return -1;
}
for (int i=0; i<30; i++)
{
cam>>frame;
}
//cout<<"\nCamera initialized\n";
/*Set the normal STDOUT back*/
fflush(stdout);
dup2(bak, 1);
close(bak);
imencode(".png",frame, buf);
cout<<buf.size()<<endl;
ssize_t written= 0;
size_t s = 128;
while (written<buf.size())
{
written += write(fd[1], buf.size()+written, s);
}
cout<<'\0';
return 0;
}
The process corresponding to the compilation of the source code above is called from the parent with popen.
Note that I am writing to the std out that has been duplicated with a pipe.
The parent will read the data and resend them to UDP socket.
If I do something like this:
#define BUFLEN 128
FILE *fp;
char buf[BUFLEN];
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((fgets(buf, BUFLEN, fp)!=NULL))
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
the program is working i.e. the receiver of sendto will receive the data.
I tried to use a pipe as done in the child process:
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
dup2(STDIN_FILENO, fd[0]);
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((read(fd[0], buf, BUFLEN) > 0)
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
but with this are not sent.
So how to use pipe in this case to achieve the same behaviour of the first case? Should I do dup2(STDIN_FILENO, fd[0]); or dup2(STDOUT_FILENO, fd[0]);?
I am using the sandard(s) since the file descriptors are inherited by the child process so should not require any other effort. That is why I thought I can use pipe but is that so?
In the parent:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
you get a pipe, and then immediately close one end of it. This pipe is now useless, because no-one will ever be able to recover the closed end, and so no data can flow through it. You have converted a pipe into a hollow cylinder sealed at one end.
Then in the child:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
you create another unrelated pipe, and seal this at the other end. The two pipes are not connected, and now you have two separate hollow cyclinders, each sealed at one end. Nothing can flow through either of them.
If putting something in the first cylinder made it appear in the other, that'd be a pretty good magic trick. Without sleight of hand or cleverly arranged mirrors, the solution is to create one pipe, keep both ends open and push data through it.
The usual way to manually set up a pipe from which a parent process can read a child process's standard output has these general steps:
parent creates a pipe by calling pipe()
parent fork()s
parent closes (clarification: its copy of) the write end of the pipe
child dupes the write end of the pipe onto its standard output via dup2()
child closes the original file descriptor for the write end of the pipe
(optional) child closes (clarification: its copy of) the read end of the pipe
child execs the desired command, or else performs the wanted work directly
The parent can then read the child's output from the read end of the pipe.
The popen() function does all of that for you, plus wraps the parent's pipe end in a FILE. Of course, it can and will set up a pipe going in the opposite direction instead if that's what the caller requests.
You need to understand and appreciate that in the procedural scheme presented above, it is important which actions are performed by which process, and in what order relative to other actions in the same process. In particular, the parent must not close the write end of the pipe before the child is launched, because that renders the pipe useless. The child inherits the one-end-closed pipe, through which no data can be conveyed.
With respect to your latter example, note also that redirecting the standard input to the read end of the pipe is not part of the process for either parent or child. The fact that your pipe is half-closed, so that nothing can ever be read from it anyway, is just icing on the cake. Moreover, the parent clobbers its own standard input this way. That's not necessarily wrong, but the parent does not even rely on it.
Overall, however, there is a bigger picture that you seem not to appreciate. Even if you performed the redirection you seem to want in the parent, so that it could be inherited by the child, popen() performs its own redirection to a pipe of its own creation. The FILE * it returns is the means by which you can read the child's output. No previous output redirection you may have performed is relevant (clarification: of the child's standard output).
In principle, an approach similar to yours could be used to create a second redirection going the other way, but at that point the convenience factor of popen() is totally lost. It would be better go take the direct pipe / fork / dup2 / exec route all the way through if you want to redirect the child's input and output.
Applying all that to your first example, you have to appreciate that although a process can redirect its own standard streams, it cannot establish a pipe to its parent process that way. The parent needs to provide the pipe, else it has no knowledge of it. And when a process dupes one file descriptor onto another, that replaces the original with the new, closing the original if it is open. It does not redefine the original. And of course, in this case, too, a pipe is useless once either end is no longer open anywhere.
I have to implement a program in which a process sends data it has received from parent process to its child process, waits until the child sends him processed data back, and then return processed data to child process (so e.g. in case of 4 processes the data flow would look like this P1->P2->P3->P4->P3->P2->P1). For means of interprocess communication I need to use pipes. Here's an approach I planned to take:
./child
// Assert argv contains 2 pipe descriptors - for reading
// from parent and for writing to parent, both of type char[]
// I'm not handling system errors currently
int main(int argc, char *argv[]) {
int read_dsc, write_dsc;
read_dsc = atoi(argv[1]);
write_dsc = atoi(argv[2]);
char data[DATA_SIZE];
read (read_dsc, data, DATA_SIZE - 1);
close (read_dsc);
// Process data...
(...)
// Pass processed data further
int pipeRead[2]; // Child process will read from this pipe
int pipeWrite[2]; // Child process will write into this pipe
pipe(pipeRead);
pipe(pipeWrite);
switch(fork()) {
case 0:
close (pipeRead[1]);
close (pipeWrite[0]);
char pipeReadDsc[DSC_SIZE];
char pipeWriteDsc[DSC_SIZE];
printf (pipeReadDsc, "%d", pipeRead[0]);
printf (pipeWriteDsc, "%d", pipeWrite[1]);
execl ("./child", "child", pipeReadDsc, pipeWriteDsc, (char *) 0);
default:
close(pipeRead[0]);
close(pipeWrite[1]);
wait(0);
read (pipeWrite[0], data, DATA_SIZE - 1);
close (pipeWrite[0]);
// Pass data to parent process
write (write_dsc, data, DATA_SIZE - 1);
close (write_dsc);
}
}
High level description of my solution is as follows: make 2 pipes, one for writing to child process, one for reading from child process. Wait until child process finishes and then read from read pipe and pass data to parent.
The problem is I don't know whether this approach is correct. I've read somewhere that not closing unused pipes is an error as it clutters OS file descriptors and there shouldn't be many opened pipes at once. Here however we're keping unclosed pipe for reading from a child and potentially if there are n processes, there are n opened pipes when process number n processes it's data (all parent processes are waiting for data to come back). However I can't see any other way to solve this problem...
So - is my solution correct? If it isn't, how should I approach this problem?
Yes your solution is correct. But there is problems in your code:
case 0 is the child, you will benefit in redirecting pipe ends onto standard input and output (use dup or dup2); passing descriptor ids to the child is weird.
default is the parent, so you need to write before reading.
"not closing unused pipes is an error" : it is not an error but may cause problems (detecting the end of a communication would be difficult or impossible), but it seems that you correctly close all non useful pipe ends in your code, so ok. In general the number of open pipes is not really an issue, as open files...
I need a little help with my plumbing.
I'm trying to create a program that begins with a single process, spawns child processes based on a user-defined number, and then flow back into another (single) child process. The data flow looks something like this:
/-->-█->-\
█-->--█->--█
\-->-█->-/
I've got the first part of the process creation finished. The fork works well - I run it through a loop limited to the number specified by the user. It's the piping that's throwing me off.
For simplicity, I'm focusing on the first part (from parent to multiple children). So I create the pipe before forking the process - that much is given. Then I close the write-end of the child process, close stdin, dup so the child's 0 redirects to stdin, then close the child's 0. That should take care of the plumbing on the child-side, right?
So then back in the parent process, I need to plumb for the pipes going to the children. For whatever reason, this is the harder part for me. Would someone mind walking me through it a little?
Here's what I've got for this part of the code:
for (i = 0; i < numberOfChildren; ++i) {
(void) pipe(workFDs[i]); /* Creates a pipe before the fork */
if ((workPIDs[i] = fork()) < 0) {
perror("Error: failure when forking workPID #: \n");
exit(-1);
}
if (workPIDs[i] == 0) {
/* ************************* WORKER PROCESS *********************** */
close(workFDs[i][1]); /* Closes the write-end for worker proc */
close(0); /* Closes stdin */
dup(workFDs[i][0]); /* Redirects workFDs 0 to stdin */
close(workFDs[i][0]);
//Fgets to get from the pipe
//Exec sort stuff here
}
} else {
/* *********************** PARENT PROCESS *********************** */
assert(inputPID == getpid()); /* Just to be sure */
close(workFDs[i][0]);
//Fputs to put into the pipe
}
}
I've been messing around in C trying to figure out how to do this. Let's say I have my main program, the parent process. The parent creates three child processes, each of which will eventually run programs (but that's not important right now). What I'd like to do is make it so that the first child's stdout will be received by the second child's stdin. The second child's stdout will then be received by the third child's stdin.
The parent process's stdin/stdout aren't messed with at all.
So far, what I've got is
pipe(procpipe);
parentPid = getpid();
for(i = 0; i < 3; i++)
{
if(getpid() == parentPid)
{
child[i] = fork();
if(child[i] == 0)
{
mynumber = i+1;
}
}
}
But from there I'm kind of stuck as to how to use dup2 to properly assign my pipes, and in which section of my code to insert it. There are lots of examples on Google and this website of how to pipe from a parent to a child, but I'm yet to see one that will tell me exactly how to connect a child's stdout to another child's stdin.
Edit:
Forgot to mention: assume all my variables are properly initialised. The int 'mynumber' is so a child process knows upon creation which number it is, so I can give it instructions via
if(mynumber == whatever)
So you have a loop that creates several child processes. Each of these child processes will be using two pipes: read from previous and write to the next. To set up a pipe for the reading end you need to close the write end of the pipe, and dup2 the read end into the stdin. Similar for the pipe where the process will be writing.
void set_read(int* lpipe)
{
dup2(lpipe[0], STDIN_FILENO);
close(lpipe[0]); // we have a copy already, so close it
close(lpipe[1]); // not using this end
}
void set_write(int* rpipe)
{
dup2(rpipe[1], STDOUT_FILENO);
close(rpipe[0]); // not using this end
close(rpipe[1]); // we have a copy already, so close it
}
When you fork each children you need to attach the pipes to it.
void fork_and_chain(int* lpipe, int* rpipe)
{
if(!fork())
{
if(lpipe) // there's a pipe from the previous process
set_read(lpipe);
// else you may want to redirect input from somewhere else for the start
if(rpipe) // there's a pipe to the next process
set_write(rpipe);
// else you may want to redirect out to somewhere else for the end
// blah do your stuff
// and make sure the child process terminates in here
// so it won't continue running the chaining code
}
}
With this in hand you can now write a loop that continuously forks, attaches the pipes, and then reuses the output pipe as the input pipe for the next one. Of course, once both ends of a pipe have been connected to child processes, the parent should not leave it open for itself.
// This assumes there are at least two processes to be chained :)
// two pipes: one from the previous in the chain, one to the next in the chain
int lpipe[2], rpipe[2];
// create the first output pipe
pipe(rpipe);
// first child takes input from somewhere else
fork_and_chain(NULL, rpipe);
// output pipe becomes input for the next process.
lpipe[0] = rpipe[0];
lpipe[1] = rpipe[1];
// chain all but the first and last children
for(i = 1; i < N - 1; i++)
{
pipe(rpipe); // make the next output pipe
fork_and_chain(lpipe, rpipe);
close(lpipe[0]); // both ends are attached, close them on parent
close(lpipe[1]);
lpipe[0] = rpipe[0]; // output pipe becomes input pipe
lpipe[1] = rpipe[1];
}
// fork the last one, its output goes somewhere else
fork_and_chain(lpipe, NULL);
close(lpipe[0]);
close(lpipe[1]);
The closing bits are very important! When you fork with an open pipe, there will be four open file descriptors: two on the parent process, and two others on the child process. You have to close all of those you won't be using. That's why the code above always closes the irrelevant ends of the pipes in the child processes, and both ends on the parent.
Also note that I am giving special treatment to the first and the last processes, because I don't know where the input for the chain will come from, and where the output will go to.