Interaction between pipes, execvpe, and shell scripts - c

Question

I have the following code that forks and execvpe's a shell script and redirects its STDERR and STDOUT to the parent process.
#define _GNU_SOURCE
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#define BUFLEN 65536
int main () {
int pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == -1) {return 1;}
else if (pid > 0) {
close(pipes[1]);
char buf[BUFLEN];
size_t n = read(pipes[0], buf, sizeof(buf));
int stat = 0;
waitpid(pid, &stat, 0);
printf("%.*s", n, buf);
} else {
dup2(pipes[1], STDOUT_FILENO);
dup2(pipes[1], STDERR_FILENO);
close(pipes[0]);
char *const argv[] = {"sh", "test", NULL};
execvpe(argv[0], argv, environ);
}
return 0;
}
As a minimal working example "test" is:
#!/bin/bash
cat file.txt
echo "Hello!"
echo "Goodbye!"
The output of the C program is the contents of file.txt and then the output from the echos are lost. If it's three echo statements then all of them get seen.
My best guess is that echo is a shell builtin and the shell will fork for cat and my pipes will be lost. In my project it seems the first command called in the script and the rest are lost.
If my assumption is correct how can I collect all the output from any and all children of what execvpe spawned?

Answer 1

I think the problem is simply a combination of timing and not checking for EOF before stopping reading from the pipe. If you wrap the read() call into a loop to read everything, all the data is read. When the cat completes, the read() returns everything that's available. The output from the echo commands is added to the pipe afterwards, but simply not read.
This code demonstrates, I believe. Note that execvpe() is not POSIX standard (and not available on macOS specifically) so I used my own surrogate header #include "execvpe.h" and implementation execvpe.c to obtain an implementation of it. Also, POSIX does not define a header that declares environ, so I declared it too. You're probably using Linux and the system headers there fix some gaps that POSIX leaves as holes.
Here's working code and data.
pipe17.c
/* SO 6412-3757 */
#define _GNU_SOURCE
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <stdio.h>
#define BUFLEN 65536
#include "execvpe.h" /* execvpe() is not in POSIX */
extern char **environ; /* No POSIX header declares environ */
int main(void)
{
int pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == -1)
{
return 1;
}
else if (pid > 0)
{
close(pipes[1]);
char buffer[BUFLEN];
char *b_str = buffer;
size_t b_len = sizeof(buffer);
size_t t_len = 0;
ssize_t n_len;
while (b_len > 0 && (n_len = read(pipes[0], b_str, b_len)) > 0)
{
b_str += n_len;
b_len -= n_len;
t_len += n_len;
}
close(pipes[0]);
int status = 0;
int corpse = waitpid(pid, &status, 0);
printf("%d 0x%.4X: [[%.*s]]\n", corpse, status, (int)t_len, buffer);
}
else
{
dup2(pipes[1], STDOUT_FILENO);
dup2(pipes[1], STDERR_FILENO);
close(pipes[0]);
close(pipes[1]);
char *argv[] = {"sh", "test", NULL};
execvpe(argv[0], argv, environ);
fprintf(stderr, "failed to execute '%s'\n", argv[0]);
exit(1);
}
return 0;
}
test
#!/bin/bash
cat file.txt
echo "Hello!"
echo "Goodbye!"
echo "Errors go to standard error" >&2
file.txt
Line 1 of file1.txt
The very last line of file1.txt
Sample output
14171 0x0000: [[Line 1 of file1.txt
The very last line of file1.txt
Hello!
Goodbye!
Errors go to standard error
]]
Note that the code closes both ends of the pipe before calling execvpe(). It isn't crucial here, but it often can be crucial to do so. Your original code passed a size_t value, n, to printf() for use by the * in the format. You might get away with that on a 64-bit machine where sizeof(int) == sizeof(size_t), but it yields compilation warnings on 64-bit machines where sizeof(int) < sizeof(size_t).
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.