I'm using the following code to create a child process. The idea is to filter (more generally speaking execute an arbitrary function on) stdin and stdout of the child.
For one shots this example should work, but for an interactive shell there are things missing to work. The question is basically: Is this the right way to achieve what I want? What am I missing? (e.g. I tried to catch all signals (execept STOP and KILL) and relay them to the child, but it didnt really help much)
I'm eager to understand the issue so I'd be glad for any help and insight on that matter
#include <errno.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>
#include <stdlib.h>
/* since pipes are unidirectional, we need two pipes.
one for data to flow from parent's stdout to child's
stdin and the other for child's stdout to flow to
parent's stdin */
#define NUM_PIPES 2
#define PARENT_WRITE_PIPE 0
#define PARENT_READ_PIPE 1
int pipes[NUM_PIPES][2];
/* always in a pipe[], pipe[0] is for read and
pipe[1] is for write */
#define READ_FD 0
#define WRITE_FD 1
#define PARENT_FROM_CHILD_FD ( pipes[PARENT_READ_PIPE][READ_FD] )
#define PARENT_TO_CHILD_FD ( pipes[PARENT_WRITE_PIPE][WRITE_FD] )
#define CHILD_FROM_PARENT_FD ( pipes[PARENT_WRITE_PIPE][READ_FD] )
#define CHILD_TO_PARENT_FD ( pipes[PARENT_READ_PIPE][WRITE_FD] )
pid_t pid;
void filterStream( char* buffer, int count){
printf("filter\n");
}
int main () {
// pipes for parent to write and read
pipe(pipes[PARENT_READ_PIPE]);
pipe(pipes[PARENT_WRITE_PIPE]);
if ((pid = fork()) < 0) {
exit(1);
}
if (pid==0){
/*child*/
char *argv[]={ "sh","-i",0};
dup2(CHILD_FROM_PARENT_FD, STDIN_FILENO);
dup2(CHILD_TO_PARENT_FD, STDOUT_FILENO);
//edit
close(CHILD_FROM_PARENT_FD);
close(CHILD_TO_PARENT_FD);
close(PARENT_FROM_CHILD_FD);
close(PARENT_TO_CHILD_FD);
execvp(argv[0], argv);
} else {
/*parent*/
char buffer[8192];
int count;
int buflen=sizeof(buffer)-1;
/* close fds not required by parent */
close(CHILD_FROM_PARENT_FD);
close(CHILD_TO_PARENT_FD);
while(1){
// read stdin
while((count = read(STDIN_FILENO, buffer, buflen))>0){
filterStream(buffer,count);
buffer[count] = 0;
write(PARENT_TO_CHILD_FD,buffer,count);
if (count<buflen){
break;
}
}
if (count<0){
printf("IOERR: %d\n",errno);
}
// Read from child’s stdout
while((count = read(PARENT_FROM_CHILD_FD, buffer, buflen))>0){
filterStream(buffer,count);
buffer[count] = 0;
write(STDOUT_FILENO,buffer,count);
if (count<buflen){
break;
}
}
if (count<0){
printf("IOERR2: %d\n",errno);
}
}
}
}
Problem 1:
After the first command no further commands are executed: Is this just a redirection problem from parent to child or is it deeper?
./a.out
sh-5.1$ touch test
filter
touch test
sh-5.1$ exit
# hangs
^C
sh-5.1$ exit
# hangs...
Problem 2: on some commands the shell just backgrounds: Who backgrounds it exactly?
./a.out
sh-5.1$ whoami
filter
whoami
filter
currentuser
[1] + 238242 suspended (tty input) ./a.out
Related
I have the following code that forks and execvpe's a shell script and redirects its STDERR and STDOUT to the parent process.
#define _GNU_SOURCE
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#define BUFLEN 65536
int main () {
int pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == -1) {return 1;}
else if (pid > 0) {
close(pipes[1]);
char buf[BUFLEN];
size_t n = read(pipes[0], buf, sizeof(buf));
int stat = 0;
waitpid(pid, &stat, 0);
printf("%.*s", n, buf);
} else {
dup2(pipes[1], STDOUT_FILENO);
dup2(pipes[1], STDERR_FILENO);
close(pipes[0]);
char *const argv[] = {"sh", "test", NULL};
execvpe(argv[0], argv, environ);
}
return 0;
}
As a minimal working example "test" is:
#!/bin/bash
cat file.txt
echo "Hello!"
echo "Goodbye!"
The output of the C program is the contents of file.txt and then the output from the echos are lost. If it's three echo statements then all of them get seen.
My best guess is that echo is a shell builtin and the shell will fork for cat and my pipes will be lost. In my project it seems the first command called in the script and the rest are lost.
If my assumption is correct how can I collect all the output from any and all children of what execvpe spawned?
I think the problem is simply a combination of timing and not checking for EOF before stopping reading from the pipe. If you wrap the read() call into a loop to read everything, all the data is read. When the cat completes, the read() returns everything that's available. The output from the echo commands is added to the pipe afterwards, but simply not read.
This code demonstrates, I believe. Note that execvpe() is not POSIX standard (and not available on macOS specifically) so I used my own surrogate header #include "execvpe.h" and implementation execvpe.c to obtain an implementation of it. Also, POSIX does not define a header that declares environ, so I declared it too. You're probably using Linux and the system headers there fix some gaps that POSIX leaves as holes.
Here's working code and data.
pipe17.c
/* SO 6412-3757 */
#define _GNU_SOURCE
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <stdio.h>
#define BUFLEN 65536
#include "execvpe.h" /* execvpe() is not in POSIX */
extern char **environ; /* No POSIX header declares environ */
int main(void)
{
int pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == -1)
{
return 1;
}
else if (pid > 0)
{
close(pipes[1]);
char buffer[BUFLEN];
char *b_str = buffer;
size_t b_len = sizeof(buffer);
size_t t_len = 0;
ssize_t n_len;
while (b_len > 0 && (n_len = read(pipes[0], b_str, b_len)) > 0)
{
b_str += n_len;
b_len -= n_len;
t_len += n_len;
}
close(pipes[0]);
int status = 0;
int corpse = waitpid(pid, &status, 0);
printf("%d 0x%.4X: [[%.*s]]\n", corpse, status, (int)t_len, buffer);
}
else
{
dup2(pipes[1], STDOUT_FILENO);
dup2(pipes[1], STDERR_FILENO);
close(pipes[0]);
close(pipes[1]);
char *argv[] = {"sh", "test", NULL};
execvpe(argv[0], argv, environ);
fprintf(stderr, "failed to execute '%s'\n", argv[0]);
exit(1);
}
return 0;
}
test
#!/bin/bash
cat file.txt
echo "Hello!"
echo "Goodbye!"
echo "Errors go to standard error" >&2
file.txt
Line 1 of file1.txt
The very last line of file1.txt
Sample output
14171 0x0000: [[Line 1 of file1.txt
The very last line of file1.txt
Hello!
Goodbye!
Errors go to standard error
]]
Note that the code closes both ends of the pipe before calling execvpe(). It isn't crucial here, but it often can be crucial to do so. Your original code passed a size_t value, n, to printf() for use by the * in the format. You might get away with that on a 64-bit machine where sizeof(int) == sizeof(size_t), but it yields compilation warnings on 64-bit machines where sizeof(int) < sizeof(size_t).
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.
Problem
I'm trying to pipe contents from the main routine to a execvp'd bash shell. I'm encountering a problem where when I write "exit" into the subshell, it doesn't tell me that the pipe is really broken. It should be though - right? The process died and thus the pipe fd should also return an EOF or a SIGPIPE. It doesn't, however, and just keeps on reading/writing like normal.
Code
The code is attached here:
/************************************************************
* Includes:
* ioctl - useless(?)
* termios, tcsetattr, tcgetattr - are for setting the
* noncanonical, character-at-a-time terminal.
* fork, exec - creating the child process for part 2.
* pthread, pipe - creating the pipe process to communicate
* with the child shell.
* kill - to exit the process
* atexit - does some cleanups. Used in termios, tcsetattr,
* tcgetattr.
************************************************************/
#include <sys/ioctl.h> // ioctl
#include <termios.h> // termios, tcsetattr, tcgetattr
#include <unistd.h> // fork, exec, pipe
#include <sys/wait.h> // waitpid
#include <pthread.h> // pthread
#include <signal.h> // kill
#include <stdlib.h> // atexit
#include <stdio.h> // fprintf and other utility functions
#include <getopt.h> // getopt
/**********************
* GLOBALS
**********************/
pid_t pid;
/**********************
* CONSTANTS
**********************/
static const int BUFFER_SIZE = 16;
static const int STDIN_FD = 0;
static const int STDOUT_FD = 1;
static const int STDERR_FD = 2;
// these attributes are reverted to later
struct termios saved_attributes;
// to revert the saved attributes
void
reset_input_mode (void) {
tcsetattr (STDIN_FILENO, TCSANOW, &saved_attributes);
}
// to set the input mode to correct non-canonical mode.
void
set_input_mode (void) {
struct termios tattr;
/* Make sure stdin is a terminal. */
if (!isatty (STDIN_FILENO))
{
fprintf (stderr, "Not a terminal.\n");
exit (EXIT_FAILURE);
}
/* Save the terminal attributes so we can restore them later. */
tcgetattr (STDIN_FILENO, &saved_attributes);
atexit (reset_input_mode);
/* Set the funny terminal modes. */
tcgetattr (STDIN_FILENO, &tattr);
tattr.c_lflag &= ~(ICANON|ECHO); /* Clear ICANON and ECHO. */
tattr.c_cc[VMIN] = 1;
tattr.c_cc[VTIME] = 0;
tcsetattr (STDIN_FILENO, TCSAFLUSH, &tattr);
}
// pthread 1 will read from pipe_fd[0], which
// is really the child's pipe_fd[1](stdout).
// It then prints out the contents.
void* thread_read(void* arg){
int* pipe_fd = ((int *) arg);
int read_fd = pipe_fd[0];
int write_fd = pipe_fd[1];
char c;
while(1){
int bytes_read = read(read_fd, &c, 1);
if(bytes_read > 0){
putchar(c);
}
else{
close(read_fd);
close(write_fd);
fprintf(stdout, "The read broke.");
fflush(stdout);
break;
}
}
}
// pthread 2 will write to child_pipe_fd[1], which
// is really the child's stdin.
// but in addition to writing to child_pipe_fd[1],
// we must also print to stdout what our
// argument was into the terminal. (so pthread 2
// does extra).
void* thread_write(void* arg){
set_input_mode();
int* pipe_args = ((int *) arg);
int child_read_fd = pipe_args[0];
int child_write_fd = pipe_args[1];
int parent_read_fd = pipe_args[2];
int parent_write_fd = pipe_args[3];
char c;
while(1) {
int bytes_read = read(STDIN_FD, &c, 1);
write(child_write_fd, &c, bytes_read);
putchar(c);
if(c == 0x04){
// If an EOF has been detected, then
// we need to close the pipes.
close(child_write_fd);
close(child_read_fd);
close(parent_write_fd);
close(parent_read_fd);
kill(pid, SIGHUP);
break;
}
}
}
int main(int argc, char* argv[]) {
/***************************
* Getopt process here for --shell
**************************/
int child_pipe_fd[2];
int parent_pipe_fd[2];
pipe(child_pipe_fd);
pipe(parent_pipe_fd);
// We need to spawn a subshell.
pid = fork();
if(pid < 0){
perror("Forking was unsuccessful. Exiting");
exit(EXIT_FAILURE);
}
else if(pid == 0){ // is the child.
// We dup the fd and close the pipe.
close(0); // close stdin. child's pipe should read.
dup(child_pipe_fd[0]); // pipe_fd[0] is the read. Make read the stdin.
close(child_pipe_fd[0]);
close(1); // close stdout
dup(parent_pipe_fd[1]); // pipe_fd[1] is the write. Make write the stdout.
close(parent_pipe_fd[1]);
char* BASH[] = {"/bin/bash", NULL};
execvp(BASH[0], BASH);
}
else{ // is the parent
// We dup the fd and close the pipe.
//
// create 2 pthreads.
// pthread 1 will read from pipe_fd[0], which
// is really the child's pipe_fd[1](stdout).
// It then prints out the contents.
//
// pthread 2 will write to pipe_fd[1], which
// is really the child's pipe_fd[0](stdin)
// but in addition to writing to pipe_fd[1],
// we must also print to stdout what our
// argument was into the terminal. (so pthread 2
// does extra).
//
// We also need to take care of signal handling:
signal(SIGINT, sigint_handler);
/*signal(SIGPIPE, sigpipe_handler);*/
int write_args[] = {child_pipe_fd[0], child_pipe_fd[1],
parent_pipe_fd[0], parent_pipe_fd[1]};
pthread_t t[2];
pthread_create(t, NULL, thread_read, parent_pipe_fd);
pthread_create(t+1, NULL, thread_write, write_args);
pthread_join(t[0], NULL);
pthread_join(t[1], NULL);
int status;
if (waitpid(pid, &status, 0) == -1) {
perror("Waiting for child failed.");
exit(EXIT_FAILURE);
}
printf("Subshell exited with the error code %d", status);
exit(0);
}
return 0;
}
The program basically pipes inputs from the terminal into the subshell and tries to execute them and return the outputs. To write to the pipe, I have a pthread that writes the stdin inputs into the subshell. To read to the pipe, I have a pthread that reads the pipe to the parent. To detect the broken pipe via the subshell dying(calling exit), I detect the EOF character from the read thread.
My attempts
I added a check for the 0x04 character(EOF), I checked for read_bytes == 0 or read_bytes < 0. It seems that it never gets the memo unless I explicitly close the pipes on the writing end. It only meets the EOF character if I send the character ^D(which, in my code, handles via closing all pipes of the child & parent).
Any comments would be appreciated! Thank you.
Your parent process is holding copies of the child's file descriptors. Thus, even after the child has exited, those FDs are still open -- so the other ends of those pipelines remain open as well, preventing any SIGPIPE.
Modify your code as follows:
else {
// pid >0; this is the parent
close(child_pipe_fd[0]); // ADD THIS LINE
close(parent_pipe_fd[1]); // ADD THIS LINE
Hello stackoverflow I tried to create a program which execute a son shell process and redirect his I/O to a pipe in order to communicate with his father process.
I can execute command via the write pipe (wpipefd) but I can't get the response from the shell process on the read pipe (rpipefd).
I had 3 errors so far according to Strace : First the read function was blocking the program so I made the read fd of the reading pipe non-blocking (rpipe[0]). Then I had an EAGAIN error with the read function... Finally I got an EPIPE error when I close the read fd from rpipe (close(rpipefd[0])) in the forked process just after the use of dup2() .
I don't understand what I did wrong. Here's what I did so far :
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#define BUF_SIZE 1024
int main(int argc, char **argv)
{
int rpipefd[2], wpipefd[2], pid;
pipe(rpipefd);
pipe(wpipefd);
char buffer[BUF_SIZE] = {0};
int flags = fcntl(rpipefd[0], F_GETFL, 0);
fcntl(rpipefd[0], F_SETFL, flags | O_NONBLOCK);
pid = fork();
if(pid == 0)
{
close(rpipefd[0]);
dup2(rpipefd[1],1);
dup2(rpipefd[1],2);
close(wpipefd[1]);
dup2(wpipefd[0],0);
close(rpipefd[1]);
close(wpipefd[0]);
execl("/bin/sh","/bin/sh",NULL);
}
close(wpipefd[0]);
write(wpipefd[1],"echo helloWorld",strlen("echo helloWorld"));
close(rpipefd[1]);
read(rpipefd[0],buffer,BUF_SIZE);
//perror("read()");
printf("%s",buffer);
exit(0);
}
Please help !
The main issue doesn't come from the code itself: the command passed to the shell is incomplete, you missed the final '\n' and thus the child process (your shell) is waiting for the rest of the command.
The non-blocking part is not a good idea (or at least, you should spin around you pipe in order to retrieve its content.)
Once you're done with your command, you should close the output pipe so the shell get the end-of-file on its input.
Other remarks: you should wait for the child termination (using wait(2)), you should leave after your execl in the child process (use with err(3) for the error message) to handle exec errors. And, seriously, calling strlen on string literal ? I know that gcc is replacing it at compile time, but …
Here is a modified version of your code:
#include <err.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
#define BUF_SIZE 1024
int main(int argc, char **argv)
{
int rpipefd[2], wpipefd[2], pid;
pipe(rpipefd);
pipe(wpipefd);
char buffer[BUF_SIZE] = {0};
pid = fork();
if(pid == 0)
{
close(rpipefd[0]);
dup2(rpipefd[1],STDOUT_FILENO);
dup2(rpipefd[1],STDERR_FILENO);
close(wpipefd[1]);
dup2(wpipefd[0],STDIN_FILENO);
close(rpipefd[1]);
close(wpipefd[0]);
execl("/bin/sh","/bin/sh",NULL);
err(1, "execl()");
}
close(wpipefd[0]);
close(rpipefd[1]);
write(wpipefd[1], "echo helloWorld\n", 16);
close(wpipefd[1]); // we're done, say it to the shell
int r;
while ( (r = read(rpipefd[0],buffer,BUF_SIZE)) )
{
if (r == -1)
{
if (errno == EAGAIN || errno == EINTR) continue;
err(1, "read()");
}
write(STDOUT_FILENO, buffer, r);
}
wait(NULL);
return 0;
}
Maybe this is not a compact title, I am very sorry about that:). I try redirecting stdin/stdout of a child process to its parent process with pipes. The child process execute a system command from the father process input and return the exec result to the father process with a pipe. Here I implemented "cat -n" and "tr /a-z/ /A-Z/", the former works fine, but later haven't return any results. What has caused this? Thank you.
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <signal.h>
#include <assert.h>
#include <sys/sem.h>
#define ERR_EXIT(m) \
do { \
perror(m); \
exit(EXIT_FAILURE); \
} while( 0)
int main(int argc, char *argv[])
{
int chi_pipe[2], par_pipe[2];
if (pipe(chi_pipe) == -1 || pipe(par_pipe) == -1)
ERR_EXIT("pipe error");
/* Set O_NONBLOCK flag for the read end (pfd[0]) of the pipe. */
if (fcntl(chi_pipe[0], F_SETFL, O_NONBLOCK) == -1) {
fprintf(stderr, "Call to fcntl failed.\n"); exit(1);
}
/* Set O_NONBLOCK flag for the read end (pfd[0]) of the pipe. */
if (fcntl(chi_pipe[1], F_SETFL, O_NONBLOCK) == -1) {
fprintf(stderr, "Call to fcntl failed.\n"); exit(1);
}
pid_t pid;
pid = fork();
if (pid == -1)
ERR_EXIT("fork error");
if (pid == 0)
{
close(chi_pipe[0]); // I don't read in channel 1
close(par_pipe[1]); // I don't write in channel 2
dup2(chi_pipe[1], STDOUT_FILENO);
close(STDIN_FILENO);
dup2(par_pipe[0], STDIN_FILENO);
execlp("cat", "cat" , "-n", NULL);
//execlp("tr", "tr" , "/a-z/", "/A-Z/", NULL);
sleep(10);
close(chi_pipe[1]);
close(par_pipe[0]);
_exit(0);
}
close(par_pipe[0]);
close(chi_pipe[1]);
while(1) {
char input[1024];
memset(input, 0 , 1024);
fgets(input, 1024 ,stdin);
write(par_pipe[1], input, strlen(input));
char buf[3*1024];
int count = 0;
while (count <= 0)
count=read(chi_pipe[0], buf, 1024*3);
if (count >= 1)
{
printf("buf=%s", buf);
printf("\n");
}
}
close(par_pipe[1]);
close(chi_pipe[0]);
return 0;
}
A couple of points:
You are suffering from the need to perform non-blocking I/O. You are reading a line from a file, then writing it to a pipe. But there is no guarantee tr will conveniently write that line back translated. It might wait for the next line to come in. There is no line discipline in place. What you need to do is read from your file, write to tr (if the pipe is not full) and read from tr (if bytes are ready) at the same time. Or, more accurately, according to availability of data on the fd (to read) or the availability of space in the pipe (to write). Otherwise you will run into deadlock problems. tr isn't writing because it would rather read more first, and it hasn't got EOF. You aren't reading from tr because it hasn't written yet, so you aren't reading any more from the file either. To do this, you want to use select() (or poll()).
The only way execlp will return is if the exec fails; in that case you don't want to exit(0) as it's necessarily an error.
I am trying to figure out piping in C by playing around with it. I want to write a program that takes the output from the shell command 'cat', saves it as a string, then prints that string. The command should look like this:
cat foo.txt | ./my_prog
I am having issues sending the output from the cat command to my_prog. Here is what I have tried so far.
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(int argc, char *argv[])
{
int pipe_1[2];
pid_t pid = -1;
char catString[200];
catString [199] = '\0';
// dup stdout to pipe_1
if( dup2(STDOUT_FILENO, pipe_1[1]) == -1 ){
perror("Could not create pipe 1");
exit(-1);
}
// fork a new process
pid = fork();
switch(pid){
case -1:
perror("Fork 1 failed");
exit(-1);
case 0: // child
// close stdin and write stdout to the string
close(pipe_1[0]);
write(pipe_1[1], catString, 200);
break;
default: // parent
// wait for child process to finish, close stdout, then print the string
wait(NULL);
close(pipe_1[1]);
printf("Parent recieved %s\n", catString);
break;
}
return 0;
}
This doesn't print anything and gives me the output:
Parent recieved
On a side note, am I using the wait() function correctly? I wanted to make sure the child is done writing to catString before the parent process executed.
The shell will send the output of cat foo.txt to stdin of your program. You don't have to do anything with "pipes" inside your program, just accept the input in the way the shell delivers it.