Understanding Pipes, Redirection and IPC - c

I'm new to piping and have been trying to create a pair of pipes which allow a child process to write to the parent process, and the parent process to communicate back. There is 1 parent with up to 4 children. The child becomes a different program with exec.
What I have working:
Writing from the parent to the child process. When I read in the child program's stdin, it will receive what I wrote from the parent.
The aim:
To create a card game where the parent talks to each individual client (the child processes) and gives all the moves and information to them, from its stdout to the children's stdin. The individual child processes give back their moves on their stdout, read by the main parent. The moves that the game makes is fully decided by a sequence, not players. So it's a bot game.
What I am stuck with:
I'm not sure how to get it so the parent can read the child's stdout through a file stream. When I try to setup the reading from child lines, the code seems to stop working. Not even the child can read from parent (it seems to stop at the now commented out liens for setting up child to parent).
I also am unsure how to "wait" until something appears. Like, at the start the players have to send a "ready" message back to the parent to let them know they are working. Once I send the "ready" from the child, how do I "wait" indefinitely until the next message appears?
I'm not sure if I'm setting up the pipes correctly. Can someone provide guidance on how to use communication pipes and confirm my logic below?
What I gather for getting parent to write to child is:
Create the pipe first
Fork off the parent process into another process (child)
Connect the pipe's in to the parent's stdout, and close off the reading side for the parent using dup2 and close
Connect the pipe's out to the child's stdin, and close off the writing part for the child using dup2 and close
Get a file stream using fdopen() from the file descriptor and then print to that.
The child process stdin is now whatever you print to stdout from the parent.
Is this correct? I tried applying this kind of logic for child to parent but reversing it.
Connect the in pipe to the read file stream, which is where the child program writes to from its stdout.
Connect the out pipe to the read stream, where the parent reads from.
void start_child_process(Game *game, int position) {
int child2Parent[2];
int parent2Child[2];
if (pipe(parent2Child)) {
printf("PIPE FAIL!\n");
}
if (pipe(child2Parent)) {
printf("PIPE FAIL!\n");
}
pid_t pid = fork();
game->readStream[position] = fdopen(child2Parent[0], "r");
game->writeStream[position] = fdopen(parent2Child[1], "w");
if (pid) { // Parent
// Write from parent to child
close(parent2Child[0]);
dup2(fileno(game->writeStream[position]), STDOUT_FILENO);
fprintf(game->writeStream[position], "%s", "test message");
fflush(game->writeStream[position]);
close(parent2Child[1]);
// Read from child -- not working
/*dup2(child2Parent[0], STDIN_FILENO);
close(child2Parent[0]);
close(child2Parent[1]);
*/
} else {
// Setup child to read from stdin from parent
dup2(parent2Child[0], STDIN_FILENO);
close(parent2Child[1]);
// Setup writing from child to parent
/*
if (dup2(child2Parent[1], STDOUT_FILENO) == -1) {
fprintf(stderr, "dup2 in child failed\n");
} else {
fprintf(stderr, "dup2 in child successful\n");
close(child2Parent[0]);
close(child2Parent[1]);
}
*/
if ((int)execl("child", "2", "A", NULL) == -1) {
printf("Failed child process\n");
}
}
}
My child main has the following which reads it:
char string[100];
printf("reading from pipe: %s\n", fgets(string, 100, stdin));
But I'm not sure how
Also, I'm not permitted to use popen() or write(). I'm also encouraged to use file streams apparently.

I speak principally to your main question of establishing two-way communication between parent and child processes. If you would like additional answers then please pose separate questions.
You seem to have a reasonable general approach, but you do have one serious misconception / design flaw: whereas it is reasonable for multiple clients to each connect their standard streams to pipes for communicating with the parent process, you cannot connect the parent's end of all those pipes to the parent's standard streams if you want to be able to handle more than one client at a time. The parent only has one set of standard streams, after all. To support multiple clients, then, the parent process must maintain a separate pair of file descriptors and/or streams for each one, and must communicate via those instead of via its standard streams.
I am uncertain why your parent / child communication is failing when you hook up the child-to-parent direction. The process is indeed analogous to setting up the other other endpoint. Here is a working example:
parent.c:
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
int main() {
int child2Parent[2];
int parent2Child[2];
char buffer[256];
FILE *p2cStream;
FILE *c2pStream;
pid_t pid;
if (pipe(parent2Child) || pipe(child2Parent)) {
perror("Failed to create pipes");
exit(EXIT_FAILURE);
}
switch (pid = fork()) {
case -1: /* error */
perror("Failed to fork");
break;
case 0: /* child */
// Setup child to read from stdin from parent
close(parent2Child[1]); /* ignoring any error */
close(child2Parent[0]); /* ignoring any error */
if ((dup2(parent2Child[0], STDIN_FILENO) < 0)
|| (dup2(child2Parent[1], STDOUT_FILENO) < 0)) {
perror("Failed to duplicate file descriptors");
} else {
/* conventionally, the first program argument is the program name */
/* also, execl() returns only if it fails */
execl("child", "child", "2", "A", NULL);
perror("Failed to exec child process");
}
exit(EXIT_FAILURE);
break;
default: /* parent */
close(parent2Child[0]); /* ignoring any error */
close(child2Parent[1]); /* ignoring any error */
if (!(p2cStream = fdopen(parent2Child[1], "w"))
|| !(c2pStream = fdopen(child2Parent[0], "r"))) {
perror("Failed to open streams");
exit(EXIT_FAILURE);
}
if ((fprintf(p2cStream, "test message from parent\n") < 0)
|| fclose(p2cStream)) {
perror("Failed to write to the child");
exit(EXIT_FAILURE);
}
if (fscanf(c2pStream, "%255[^\n]", buffer) < 1) {
perror("Failed to read the child's message");
exit(EXIT_FAILURE);
}
printf("The child responds: '%s'\n", buffer); /* ignoring any error */
break;
}
return EXIT_SUCCESS;
}
child.c:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
int main(int argc, char *argv[]) {
char buffer[256] = { 0 };
if (scanf("%255[^\n]", buffer) < 0) {
perror("Failed to reading input");
exit(EXIT_FAILURE);
}
/*
* If stdout is connected to the parent then we must avoid
* writing anything unexpected to it
*/
if (fprintf(stderr, "received: '%s'\n", buffer) < 0) {
perror("Failed to echo input");
exit(EXIT_FAILURE);
}
printf("Hi, Mom!\n"); /* ignoring any error */
fflush(stdout); /* ignoring any error */
return EXIT_SUCCESS;
}
In contrast to your code, do note
the attention to checking all return values that may signal an error that I care about;
the parent's use of streams other than the standard streams for communicating with the child (though with only one child, that's a convenience rather than a necessity);
the convention for execl() arguments.
Note also that as for waiting for something to "appear", I/O operations on the IPC streams you set up this way will automatically produce that effect. How you can or should make use of that, however, is certainly a different issue.

Related

Named pipe - run child process

I'm trying to use named pipe in C to run a child process in the background from a path in non-blocking mode and read the output of the child.
This is my code:
int fifo_in = open("fifo_1", O_RDONLY| O_NONBLOCK);
int fifo_out = open("fifo_2", O_WRONLY| O_NONBLOCK);
dup2(fifo_in, 0);
dup2(fifo_out, 1);
char app[] = "/usr/local/bin/probemB";
char * const argsv[] = { app, "1", NULL };
if (execv(app, argsv) < 0) {
printf("execv error\n");
exit(4);
}
I will later use read function to read the child process output.
But the problem is that execv is blocking while it is reading the output from the process instead of allowing me to read.
Can someone help me to correct the above problem please ?
You're wrong in that execv is blocking.
If execv works, it will never return. It replaces your program. You need to fork a new process for execv:
if (fork() == 0)
{
// In child process, first setup the file descriptors
dup2(fifo_out, STDOUT_FILENO); // Writes to standard output will be written to the pipe
close(fifo_out); // These are not needed anymore
close(fifo_in);
// Run the program with execv...
}
else
{
// Unless there was an error, this is in the parent process
close(fifo_out);
// TODO: Read from fifo_in, which will contain the standard output of the child process
}
Another thing, you seem have two different and unconnected named pipes. You should open only one pipe, for reading in the parent process, and for writing in the child process:
int fifo_in = open("fifo_1", O_RDONLY| O_NONBLOCK);
int fifo_out = open("fifo_1", O_WRONLY| O_NONBLOCK);
But if you only want to communicate internally, you don't need named pipes. Instead use anonymous pipes as created by the pipe function.

How to use pipe between parent and child process after call to popen?

I want to communicate with a child process like the following:
int main(int argc, char *argv[])
{
int bak, temp;
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
dup2(STDOUT_FILENO, fd[1]);
fflush(stdout);
bak = dup(1);
temp = open("/dev/null", O_WRONLY);
dup2(temp, 1);
close(temp );
Mat frame;
std::vector<uchar> buf;
namedWindow( "Camera", WINDOW_AUTOSIZE );
VideoCapture cam(0 + CAP_V4L);
sleep(1);
if (!cam.isOpened())
{
cout << "\nCould not open reference " << 0 << endl;
return -1;
}
for (int i=0; i<30; i++)
{
cam>>frame;
}
//cout<<"\nCamera initialized\n";
/*Set the normal STDOUT back*/
fflush(stdout);
dup2(bak, 1);
close(bak);
imencode(".png",frame, buf);
cout<<buf.size()<<endl;
ssize_t written= 0;
size_t s = 128;
while (written<buf.size())
{
written += write(fd[1], buf.size()+written, s);
}
cout<<'\0';
return 0;
}
The process corresponding to the compilation of the source code above is called from the parent with popen.
Note that I am writing to the std out that has been duplicated with a pipe.
The parent will read the data and resend them to UDP socket.
If I do something like this:
#define BUFLEN 128
FILE *fp;
char buf[BUFLEN];
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((fgets(buf, BUFLEN, fp)!=NULL))
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
the program is working i.e. the receiver of sendto will receive the data.
I tried to use a pipe as done in the child process:
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
dup2(STDIN_FILENO, fd[0]);
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((read(fd[0], buf, BUFLEN) > 0)
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
but with this are not sent.
So how to use pipe in this case to achieve the same behaviour of the first case? Should I do dup2(STDIN_FILENO, fd[0]); or dup2(STDOUT_FILENO, fd[0]);?
I am using the sandard(s) since the file descriptors are inherited by the child process so should not require any other effort. That is why I thought I can use pipe but is that so?
In the parent:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
you get a pipe, and then immediately close one end of it. This pipe is now useless, because no-one will ever be able to recover the closed end, and so no data can flow through it. You have converted a pipe into a hollow cylinder sealed at one end.
Then in the child:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
you create another unrelated pipe, and seal this at the other end. The two pipes are not connected, and now you have two separate hollow cyclinders, each sealed at one end. Nothing can flow through either of them.
If putting something in the first cylinder made it appear in the other, that'd be a pretty good magic trick. Without sleight of hand or cleverly arranged mirrors, the solution is to create one pipe, keep both ends open and push data through it.
The usual way to manually set up a pipe from which a parent process can read a child process's standard output has these general steps:
parent creates a pipe by calling pipe()
parent fork()s
parent closes (clarification: its copy of) the write end of the pipe
child dupes the write end of the pipe onto its standard output via dup2()
child closes the original file descriptor for the write end of the pipe
(optional) child closes (clarification: its copy of) the read end of the pipe
child execs the desired command, or else performs the wanted work directly
The parent can then read the child's output from the read end of the pipe.
The popen() function does all of that for you, plus wraps the parent's pipe end in a FILE. Of course, it can and will set up a pipe going in the opposite direction instead if that's what the caller requests.
You need to understand and appreciate that in the procedural scheme presented above, it is important which actions are performed by which process, and in what order relative to other actions in the same process. In particular, the parent must not close the write end of the pipe before the child is launched, because that renders the pipe useless. The child inherits the one-end-closed pipe, through which no data can be conveyed.
With respect to your latter example, note also that redirecting the standard input to the read end of the pipe is not part of the process for either parent or child. The fact that your pipe is half-closed, so that nothing can ever be read from it anyway, is just icing on the cake. Moreover, the parent clobbers its own standard input this way. That's not necessarily wrong, but the parent does not even rely on it.
Overall, however, there is a bigger picture that you seem not to appreciate. Even if you performed the redirection you seem to want in the parent, so that it could be inherited by the child, popen() performs its own redirection to a pipe of its own creation. The FILE * it returns is the means by which you can read the child's output. No previous output redirection you may have performed is relevant (clarification: of the child's standard output).
In principle, an approach similar to yours could be used to create a second redirection going the other way, but at that point the convenience factor of popen() is totally lost. It would be better go take the direct pipe / fork / dup2 / exec route all the way through if you want to redirect the child's input and output.
Applying all that to your first example, you have to appreciate that although a process can redirect its own standard streams, it cannot establish a pipe to its parent process that way. The parent needs to provide the pipe, else it has no knowledge of it. And when a process dupes one file descriptor onto another, that replaces the original with the new, closing the original if it is open. It does not redefine the original. And of course, in this case, too, a pipe is useless once either end is no longer open anywhere.

Feeding stdout to a child process which will execv() sort

I am trying to find out how I can send output of one process into a child process. I have gone down a journey learning of file descriptors and pipes. I think I am almost there but am missing a key component.
This is what I have so far:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <string.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int fd[2];
pid_t sort_pid;
/* Create the pipe */
if(pipe(fd) == -1) {
fprintf(stderr, "Pipe failed\n");
exit(EXIT_FAILURE);
}
/* create child process that will sort */
sort_pid = fork();
if(sort_pid < 0) { // failed to fork
fprintf(stderr, "Child Fork failed\n");
exit(EXIT_FAILURE);
}
else if(sort_pid == 0) { // child process
close(0); // close stdin
dup2(fd[0], 0); // make stdin same as fd[0]
close(fd[1]); // don't need this end of the pipe
execlp("D:/Cygwin/bin/sort", "sort", NULL);
}
else { // parent process
close(1); // close stdout
dup2(fd[1], 1); // make stdout same as fd[1]
close(fd[0]); // don't need this end of the pipe
printf("Hello\n");
printf("Bye\n");
printf("Hi\n");
printf("G'day\n");
printf("It Works!\n");
wait(NULL);
}
return EXIT_SUCCESS;
}
This doesn't work, as it seems to go into an endless loop or something. I tried combinations of the wait() but that doesnt help either.
I am doing this to learn how to apply this idea in my actual program. In my actual program I read files, parse them line by line and save the processed data to a static array of structs. I want to be able to then generate output based on these results and use the fork() and execv() syscalls to sort the output.
This is ultimately for a project in uni.
These are similar examples which I dissected to get to the stage I am at so far:
pipe() and fork() in c
How to call UNIX sort command on data in pipe
Using dup,pipe,fifo to communicate with the child process
Furthermore I read the manual pages on the relevant syscalls to try and understand them. I will admit my knowledge of pipes and using them is still basically nothing, as this is my first every try with them.
Any help is appreciated, even further sources of information I could look into myself. I seem to have exhausted most of the useful stuff a google search give me.
sort will read until it encounters end-of-file. You therefore have to close the write-end of the pipe if you want it to complete. Because of the dup2, you have two copies of the open file description, so you need
close(fd[1]); anytime after the call to dup2
close(1); after you're done writing to (the new) stdout
Make sure to fflush(stdout) before the second of these to ensure that all your data actually made it into the pipe.
(This is a simple example of a deadlock: sort is waiting on the pipe to close, which will happen when the parent exits. But the parent won't exit until it finishes waiting on the child to exit…)

Unix : What happens to a pipe(half-duplex) if the process dies

I am trying to prove one of my doubts, that two non-related processes can share the fd of half-duplex pipe and have communication.
I have created two programs for that. But then I had this another question, that what happens to the pipe if process dies?Because my reader got some garbage message, when I printed out the message.
Writer
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main()
{
int fd[2];
char str[] = "hello\n";
if(pipe(fd) < 0)
perror("Pipe creation failed\n");
//Since i am a writer, i should close the reading end as a best practice
close(fd[0]);
/*
The processes need not to be related processes, in order to use the half duplex pipes. fd is just a number/identifier
which can be shared across different processes
*/
printf("Hey there !!! use this file descriptor for reading : %d\n", fd[0]);
//writing message
write(fd[1],str,strlen(str)+1);
return 0;
}
Reader
#include <stdio.h>
#include <unistd.h>
int main()
{
int fd,bytesRead;
char buffer[1024];
printf("please enter the fd :");
scanf("%d",&fd);
bytesRead = read(fd,buffer,1024);
printf("Bytes Read : %d\nMessage : %s\n", bytesRead, buffer);
return 0;
}
You can't do this.
The table of file descriptors is per-process; every process has its own separate set of open file descriptors (note the distinction between open file descriptors, which are per-process, and open file descriptions, which are system-wide, discussed in open(2)). If you want to share a file descriptor between processes, you need to either inherit it over a fork(2) or pass it through a unix(7) domain socket via sendmesg(2) with SCM_RIGHTS in the cmesg(3) header.
(On Linux, you can also pass around paths to /proc/[PID]/fd/..., and other systems may have their own non-portable equivalents).
What is happening in your case is that the read is failing (you're giving it a file descriptor which is not open), leaving your buffer with uninitialized garbage. Since you don't check the return value, you never know that it failed.
In pipe man page,
pipe() creates a pipe, a unidirectional data channel that can be used for interprocess communication. The array pipefd is used to return two file descriptors referring to the ends of the pipe. pipefd[0] refers to the read end of the pipe. pipefd[1] refers to the write end of the pipe. Data written to the write end of the pipe is buffered by the kernel until it is read from the read end of the pipe.
The pipe is mainly used for the related process(parent and child). You are not able to use the pipe for non related process.
In related process, one end is closed. In other end process gets the SIGPIPE signal.
Example program using pipe :-
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int
main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (argc != 2) {
fprintf(stderr, "Usage: %s <string>\n", argv[0]);
exit(EXIT_FAILURE);
}
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */
close(pipefd[1]); /* Close unused write end */
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
write(STDOUT_FILENO, "\n", 1);
close(pipefd[0]);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */
close(pipefd[0]); /* Close unused read end */
write(pipefd[1], argv[1], strlen(argv[1]));
close(pipefd[1]); /* Reader will see EOF */
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
The above program creates a pipe, and then fork(2)s to create a child process; the child inherits a duplicate set of file descriptors that refer to the same pipe. After the fork(2), each process closes the descriptors that it doesn't need for the pipe (see pipe(7)). The parent then writes the string contained in the program's command-line argument to the pipe, and the child reads this string a byte at a time from the pipe and echoes it on standard output.

Why does closing a pipe take so long to terminate a child process?

I'm having trouble with my program waiting for a child process (gzip) to finish and taking a very long time in doing so.
Before it starts waiting it closes the input stream to gzip so this should trigger it to terminate pretty quickly. I've checked the system and gzip isn't consuming any CPU or waiting on IO (to write to disk).
The very odd thing is the timing on when it stops waiting...
The program us using pthreads internally. It's processing 4 pthreads side by side. Each thread processes many units of work and for each unit of work one it kicks off a new gzip process (using fork() and execve()) to write the result. Threads hang when gzip doesn't terminate, but it suddenly does terminate when other threads close their instance.
For clarity, I'm setting up a pipeline that goes: my program(pthread) --> gzip --> file.gz
I guess this could be explained in part by CPU load. But when processes are kicked off minutes apart and the whole system ends up using only 1 core of 4 because of this locking issue, that seems unlikely.
The code to kick off gzip is below. The execPipeProcess is called such that the child writes direct to file, but reads from my program. That is:
execPipeProcess(&process, "gzip", -1, gzFileFd)
Any suggestions?
typedef struct {
int processID;
const char * command;
int stdin;
int stdout;
} ChildProcess;
void closeAndWait(ChildProcess * process) {
if (process->stdin >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdin)) {
exitError(-1,errno, "Failed to close stdin for %s", process->command);
}
}
if (process->stdout >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdout)) {
exitError(-1,errno, "Failed to close stdout for %s", process->command);
}
}
int status;
stdLog("waiting on post process %d", process->processID);
if (waitpid(process->processID, &status, 0) == -1) {
exitError(-1, errno, "Could not wait for %s", process->command);
}
stdLog("post process finished");
if (!WIFEXITED(status)) exitError(-1, 0, "Command did not exit properly %s", process->command);
if (WEXITSTATUS(status)) exitError(-1, 0, "Command %s returned %d not 0", process->command, WEXITSTATUS(status));
process->processID = 0;
}
void execPipeProcess(ChildProcess * process, const char* szCommand, int in, int out) {
// Expand any args
wordexp_t words;
if (wordexp (szCommand, &words, 0)) exitError(-1, 0, "Could not expand command %s\n", szCommand);
// Runs the command
char nChar;
int nResult;
if (in < 0) {
int aStdinPipe[2];
if (pipe(aStdinPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdin = aStdinPipe[PIPE_WRITE];
in = aStdinPipe[PIPE_READ];
}
else {
process->stdin = -1;
}
if (out < 0) {
int aStdoutPipe[2];
if (pipe(aStdoutPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdout = aStdoutPipe[PIPE_READ];
out = aStdoutPipe[PIPE_WRITE];
}
else {
process->stdout = -1;
}
process->processID = fork();
if (0 == process->processID) {
// child continues here
// these are for use by parent only
if (process->stdin >= 0) close(process->stdin);
if (process->stdout >= 0) close(process->stdout);
// redirect stdin
if (STDIN_FILENO != in) {
if (dup2(in, STDIN_FILENO) == -1) {
exitError(-1, errno, "redirecting stdin failed");
}
close(in);
}
// redirect stdout
if (STDOUT_FILENO != out) {
if (dup2(out, STDOUT_FILENO) == -1) {
exitError(-1, errno, "redirecting stdout failed");
}
close(out);
}
// we're done with these; they've been duplicated to STDIN and STDOUT
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execvp(words.we_wordv[0], words.we_wordv);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exitError(-1, errno, "could not run %s", szCommand);
} else if (process->processID > 0) {
wordfree(&words);
// parent continues here
// close unused file descriptors, these are for child only
close(in);
close(out);
process->command = szCommand;
} else {
exitError(-1,errno, "Failed to fork");
}
}
Child process inherits open file descriptors.
Every subsequent gzip child process inherits not only pipe file descriptors intended for communication with that particular instance but also file descriptors for pipes connected to previous child process instances.
It means that stdin pipe is still open when the main process performs close since there are some other file descriptors for the same pipe in a few child processes. Once those ones terminate the pipe is finally closed.
A quick fix is to prevent child processes from inheriting pipe file descriptors intended for the master process by setting close-on-exec flag.
Since there are multiple threads involved spawning child processes should be serialized to prevent child process from inheriting pipe fds intended for another child process.
You have not given us enough information to be sure, as the answer depends on how you use the functions presented. However, your closeAndWait() function looks a bit suspicious. It may be reasonable to suppose that that the child process in question will exit when it reaches the end of its stdin, but what is supposed to happen to data it has written or even may still write to its stdout? It is possible that your child processes hang because their standard output is blocked, and it is slow for them to recognize it.
I think this reflects a design problem. If you are capturing the child processes' output, as you seem at least to support doing, then after you close the parent's end of a child's input stream you'll want the parent to continue reading the child's output to its end, and performing whatever processing it intends to do on it. Otherwise you may lose some of it (which for a child performing gzip would mean corrupted data). You cannot do that if you make closing both streams part of the process of terminating the child.
Instead, you should to close the parent's end of the child's stdin first, continue processing its output until you reach its end, and only then try to collect the child. You can make closing the parent's end of the child's output stream part of the process of collecting that child if you like. Alternatively, if you really do want to discard any remaining output from the child, then you should drain its output stream between closing the input and closing the output.

Resources