UNIX Pipes Between Child Processes - c

I'm trying to write a program that will spawn an arbitrary number of child processes and pipe between them, similar to a command line pipeline. In my case I'm trying to do "ls -l | more" and output that to stdout, then have the parent continue executing more commands.
I have the following code as a minimal example:
int main (int argc, const char * argv[]) {
int fd[2];
pipe(fd);
chdir("/directory/with/lots/of/files");
// Create one child process for more
int pid = fork();
if (pid == 0) {
close(fd[1]);
int ret = dup2(fd[0],0);
if (ret < 0) perror("dup2");
char *argv[10];
argv[0] = "more"; argv[1] = NULL;
execvp("more", argv);
}
// Create another child process for ls
int pid2 = fork();
if (pid2 == 0) {
int ret = dup2(fd[1],1);
if (ret < 0) perror("dup2");
char *argv[10];
argv[0] = "ls"; argv[1] = "-l";
argv[2] = NULL;
execvp("ls", argv);
}
// wait for the more process to finish
int status;
waitpid(pid, &status, 0);
printf("Done!\n");
return 0;
}
Now, when I execute the program (enclosed in a main() function of course) what I end up with is more, which is expected. I'll hit "d" to page down more's output and "u" to go up, and it seems to work fine. But when I reach the bottom, instead of exiting like more does, it just leaves a blank line. Ctrl-C works to exit it but it exits the entire program, meaning the "Done!" line never gets printed. A movie is available here that illustrates what happens (note that at the very end I press Ctrl-C to get back to bash).
Any thoughts on this? I'm just trying to figure out how to change it to where instead of going to a blank line after more reaches the bottom, more quits and returns to the parent process so it can continue executing.

You need to close() at least the writing end of your pipe, otherwise more will never see EOF. For example:
...
// close parent's pipes
close(fd[0]);
close(fd[1]);
// wait for the more process to finish
int status;
waitpid(pid, &status, 0);
printf("Done!\n");
return 0;
}

I think it is because of the wait() function. Following the logic, your second child process outputs to the first child process, meaning it should end first than the second one.
In your wait function you are waiting for the first process to end, but you are not waiting for the second process. That means that if the second process does not send an EOF to the output ever, your first process won't end, i guess.
You can try to wait for the second process instead of the first and find out if that's the problem.

Related

Bug in forking and running processes

So I've been trying to fork a process two times, first fork is to let the main process continue working and the second is to let the child capture the output of the execution of the grand child process. The code is as follow:
void execute_run(char **parameters) {
task_t* task = create_task();
tasks[task_number] = *task;
int pipe_stdout[2];
ASSERT_SYS_OK(pipe(pipe_stdout)); // creating a pipe for stdout
pid_t pid = fork();
ASSERT_SYS_OK(pid);
if (pid == 0) {
// child process
char buffer[MAXLENGTH_OUTPUT];
pid = fork();
ASSERT_SYS_OK(pid);
if (pid == 0) {
// this process will redirect stdout to buffer
const char* program_name = parameters[0];
char** program_args = &parameters[1];
ASSERT_SYS_OK(close(pipe_stdout[0]));
ASSERT_SYS_OK(dup2(pipe_stdout[1], STDOUT_FILENO));
// redirecting stdout to pipe
ASSERT_SYS_OK(close(pipe_stdout[1]));
ASSERT_SYS_OK(execvp(program_name, program_args));
}
else {
ASSERT_SYS_OK(close(pipe_stdout[1]));
int status;
do
{
// this loop runs forever
} while (waitpid(pid, &status, WNOHANG) == 0);
ASSERT_SYS_OK(close(pipe_stdout[0]));
// this is never printed
fprintf(stderr, "Child process finished with status %d\n", status);
}
} else {
// parent process won't use any of the pipe ends
ASSERT_SYS_OK(close(pipe_stdout[0]));
ASSERT_SYS_OK(close(pipe_stdout[1]));
}
}
char** parameters are ['cat', 'in.txt'] in the example i'm trying to run (and file exists).
Thanks in advance!
I've tried debugging it and running with different commands like 'ls' for which it works fine, nevertheless I can't figure out why with 'cat' it doesn't work

Child Process not exiting in Piping [duplicate]

This question already has an answer here:
Strange behavior with Child process
(1 answer)
Closed 1 year ago.
I'm currently testing a program that executes the Linux command echo Hello | wc using piping.
The parent process in main() spawns two child processes, the first one which executes echo Hello, and the second one which executes wc. These two processes communicate via a pipe.
However, when I call waitpid() on the two child processes, only the second process exits. The first process successfully runs execvp(), but hangs there.
Here is the supposed output of the code:
command 0
command 0 should exit
command 1
1 1 6
If I uncommment the line waitpid(id[0],&status,0);
then the output is
command 0
command 0 should exit
command 1
int test(pid_t id[])
{
int i;
int pipefd[2];
char *cat_args[] = {"echo","Hello", NULL};
char *grep_args[] = {"wc", NULL};
// make a pipe
pipe(pipefd);
for(i = 0; i < 2; i++){
id[i] = fork();
if (id[i] == -1){
printf("Unable to create child process");
fprintf(stderr,"fork() failed to spawn child process");
exit(1);
}
else if (id[i] == 0){
printf("command ");
printf("%d\n",i);
if (i == 0){
dup2(pipefd[0],STDIN_FILENO);
}
else if (i == 1){
dup2(pipefd[1],STDOUT_FILENO);
}
// Close pipes
close(pipefd[0]);
close(pipefd[1]);
if (i == 1){
execvp(*cat_args,cat_args); //exit(0) normally
}
else if (i == 0){
printf("command 0 should exit\n");
execvp(*grep_args,grep_args);
printf("command 0 exited\n"); //If first child exits, should print it out
}
}
}
return 0;
}
Here is the main function:
int main(int argc, char **argv)
{
pid_t id[2];
test(id);
int status;
// If this is uncommented, the piped Linux command is never ran
// and main() never exits
//waitpid(id[0],&status,0);
waitpid(id[1],&status,0); // This works
return 0;
}
Thank you.
The sub-process running wc hangs because the pipe is still readable despite the sub-process running echo Hello died. wc still waits for something to read through the pipe.
Why is the pipe still readable? Because you didn't close it in the parent process, the one that does the two forks. Therefore, the parent can still read and write to the pipe and the wc sub-process stays blocked on read() and never receives an EOF.
BTW, your printf("command 0 exited\n") is useless since nothing is executed after a successful execvp().

Some pipe file descriptor stays open even though i close everything

static int pipefd[2];
static pid_t cpid = 0;
void sigtimeout(int num) {
kill(cpid, 9);
close(pipefd[1]);
close(pipefd[0]);
pipe(pipefd);
write(pipefd[1], "T/O\n", 5);
}
void settimer(float time) {
struct itimerval timer;
timer.it_value.tv_sec = (int)time;
timer.it_value.tv_usec = (timeout - (int)time) * 1000000;
timer.it_interval.tv_sec = 0;
timer.it_interval.tv_usec = 0;
setitimer(ITIMER_REAL, &timer, NULL);
}
pid_t popen2(char *cmd) {
if (pipe(pipefd) == -1)
return -1;
int pid;
if ((pid = fork()) == -1)
return -1;
if (pid == 0) {
close(STDIN_FILENO);
close(STDERR_FILENO);
dup2(pipefd[1], STDOUT_FILENO);
close(pipefd[0]);
close(pipefd[1]);
execlp("sh", "sh", "-c", cmd, NULL);
_exit(EXIT_SUCCESS);
} else
settimer(timeout);
return pid;
}
void getcmd(const Block *block, char *output)
{
if (block->signal)
{
output[0] = block->signal;
output++;
}
strcpy(output, block->icon);
char *cmd;
if (button)
{
cmd = strcat(exportstring, block->command);
cmd[14] = '0' + button;
cpid = popen2(cmd);
if (cpid == -1) {
close(pipefd[0]);
close(pipefd[1]);
return;
}
cmd[16] = '\0';
}
else
{
cmd = block->command;
cpid = popen2(cmd);
if (cpid == -1) {
close(pipefd[0]);
close(pipefd[1]);
return;
}
}
button = 0;
waitpid(cpid, 0, 0);
settimer(0);
kill(cpid, 9);
close(pipefd[1]);
int i = strlen(block->icon);
read(pipefd[0], output+i, CMDLENGTH-i-delimLen);
close(pipefd[0]);
for (char *c = output; *c; c++)
if (*c == '\n') {
c[1] = '\0';
break;
}
i = strlen(output);
if (delim[0] != '\0') {
//only chop off newline if one is present at the end
i = output[i-1] == '\n' ? i-1 : i;
strncpy(output+i, delim, delimLen);
}
else
output[i++] = '\0';
}
So I'm trying to modify dwmblocks to add a timeout functionality. That way, if a command hangs, the whole status bar doesn't freeze.
Everything seems to work fine, but there is one little catch.
My code makes my whole linux system crash, cause it leaves some file descriptor open every time it runs a command. As you probably know, linux has protections against that, so every other application on my system that tries to open a file descriptor crashes too.
Thing is, I'm literally closing every pipe I open in my code, even the ones that open automatically when you fork. I just can't figure out what the problem is.
I'd really appreciate any help.
BTW: I'm only putting the relevant code here, because the problem is with the file descriptors. This is the only place where I work with file descriptors in my code.
Feel free to ask for more parts of the code if you feel it's relevant in some way :)
Your popen2() function leaves both pipe ends in the parent process open. Parent should always close the write end (that the child process writes to). Perhaps that is the one that leaks.
If you were to print (int)getpid() to show the process ID, you could list the pseudofiles in /proc/PID/fd/ as those describe the file descriptor that process has currently open. (If you use ls -laF /proc/PID/fd/ with the process ID number instead of PID, you can even see what/where the descriptors are open.)
In this answer here, I recently showed how one can implement a safe_pipe(), run(), and wait_all_children() functions for exploring complicated piping schemes between child processes. safe_pipe() sets the close-on-exec flag for each pipe descriptor by default, and run() clears that flag only in the child process if it uses such a descriptor, so the forked child processes do not have the other pipes' descriptors open at all. The included example.c also shows how the parent process must close all pipe ends used by the child processes when the child processes have been started, so that the child processes correctly detect end of inputs. (If the parent process has write ends open, then it could write to the pipe, so any child reading from the read end of that pipe will never see end-of-input.)
If necessary or helpful, I'm willing to explain how safe_pipe() and run() and wait_all_children() do what they do, and also why. In particular, run() uses an extra pipe between the child process prior to exec, and the parent process, to detect problems exec'ing the specified binary (and other errors). While there may be typos in there, the two are very robust implementations, not leaking resources etc.
Well this is embarassing hahaha
I had a typo in my settimer function : timeout was supposed to be time.
So the real problem was due to a racing condition, even though my piping system is far from perfect.
That goes to show how hard bugs can be to debug sometimes.

Is pipe terminated when i have destroyed its writing end?

I have parent process that creates two child processes. First child will write to pipe and second child will read from the pipe. After 5 seconds parent will terminate first child.(so its write end should be automatically closed, isn't it?). I need second child to terminate automatically, because it uses pipe and after first child is terminated, the pipe should be terminated too. My problem: how can i force child 2 to die immediately when child 1 i killed?(i don't need child 2 to print something after child 1 is dead, even if he still has any information in the pipe buffer to read). Should i use pipe2 instead of simple pipe?
void do_close(int fd){
if(close(fd) != 0){
perror("close");
exit(2);
}
}
void signalhandler(int signum){
fprintf(stderr, "f1 terminated!\n");
exit(1);
}
int main(){
pid_t f1 = -1, f2 = -1;
int pipefd[2];
if(pipe(pipefd) == -1){
perror("pipe");
exit(2);
}
f1 = fork();
if(f1 > 0){
f2 = fork();
}
if(f1 > 0 && f2 > 0){
do_close(pipefd[0]);
do_close(pipefd[1]);
sleep(5);
kill(f1, SIGTERM);
waitpid(f1, NULL, 0);
waitpid(f2, NULL, 0);
}
else if(f1 == 0){
signal(SIGTERM, signalhandler);
do_close(pipefd[0]);
while(1){
fflush(stdout);
write(pipefd[1], "Hello world!\n", 13);
sleep(1);
}
}
else if(f2 == 0){
do_close(pipefd[1]);
char *buf = (char*)malloc(13);
while(read(pipefd[0], buf, 13) > 0){
for(int i = 0; i < 13; i++){
printf("%c", buf[i]);
}
sleep(3);
}
}
return 0;
}
"Terminated" isn't the usual terminology for a pipe, and it might be causing a slight misunderstanding. The termination of the writing process isn't immediately "felt" by the reader if there is still data in the buffer.
To summarize the program, you have one process that writes to a pipe at a rate of 1 line per second for 5 seconds, then dies. Another process reads from the pipe at a rate of 1 line every 3 seconds until EOF or error, then exits.
The lines are small and fixed-size so there's no chance of reading an incomplete line.
Nothing in the program should cause an error on the pipe, so the second child process will read until EOF.
EOF on a pipe occurs when 2 conditions are met: there are no writers, and the buffer is empty. The death of the first child process accomplishes the first condition. The second condition is not immediately true, because at the 5 second mark, there have been 5 lines written to the pipe, and only 2 of them have been read (one at the start, and one after 3 seconds).
The second child process keeps reading, pulling in the remaining lines, and eventually exits after about 5*3=15 seconds.
(The timing isn't infinitely precise, so you aren't guaranteed to get exactly 5 lines written to the pipe. When I ran it I got 6.)

Interrupt a process calling the function popen

I need to implement a child process that will execute a file and send the execution result, the 2 process will communicate with a shared memory segment.
My problem is that i want to kill the child process calling popen after 10 seconds but the function popen ignores signals.
Here is my code (shared memory segment not included) :
void kill_child(int sig)
{
kill(child_pid,SIGKILL);
printf("processus killed \n");
}
/*code....*/
signal(SIGALRM,(void (*)(int))kill_child);
if(fork()==0){
res.buffer=true;
FILE * fd;
char cmd[BUFFER_SIZE],output[BUFFER_SIZE];
strcpy(cmd,"./");
strcat(cmd,res.filepath);
system(cmd);
if((fd=popen(cmd,"r"))== NULL)
exit(1);
else
res.status=200;
strcpy(output,"");
while(fgets(buf,sizeof(buf)-1,fd))
strcat(output,buf);
if(pclose(fd))
exit(1);
strcat(res.customHTML,output);
res.buffer=true;
int err = sendResponse(res,args->client_fd);
if (err < 0) on_error("failed!\r\n");
exit(0);
}
else{
int status;
alarm(10);
waitpid(-1,&status,0);
printf("status %d _n);
}
How can make the child process interruptible?
thanks
First off, you need to actually store the child PID into child_pid. It's returned from fork for the parent process so changing your fork call to
child_pid = fork();
if(child_pid == 0)
{
...
otherwise your call to kill is being passed a random value. Luckily it seems to be defaulting to 0, which kill takes to mean kill all processes in the same process group so your child process is being killed.
Secondly, rather than calling popen() call the executable yourself with (for example) execvp() and have the parent read the output using a pipe you create yourself...
int fds[2];
pipe(fds);
child_pid = fork();
if(child_pid == 0)
{
char *cmd[]={"mycmd",NULL};
/* Replace stdout with the output of the pipe and close the original */
dup2(fds[1],1);
close(fds[0]);
close(fds[1]);
execvp(cmd[0],cmd);
}
else
{
close(fds[1]);
alarm(10);
while(...)
{
read(fds[0],....);
if(waitpid(child_pid,&status,WNOHANG))
{
....
}
}
}
This way you've only got the one child process which is running your executable and you've got visibility on when and how it exits.

Resources