The program first creates a pipe, and then creates a child process of the current process
through fork().
Then each process closes the file descriptors that are not needed for the read
and write pipes.
The child process executes the ls -a command under the current path, and
writes the command execution output to the pipe by copying the pipe write descriptor fd[1]
to standard output; the parent process reads the pipe data and displays it through fd[0].
Every process in Linux is provided with three standard file descriptor including standard
input, output and error files.
By default:
Standard Input is the keyboard, abstracted as a file to make it easier to write shell
scripts.
Standard Output is the shell window or the terminal from which the script runs,
abstracted as a file to again make writing scripts & program easier
Standard error is the same as standard output: the shell window or terminal from
which the script runs.
A file descriptor is simply a number that refers to an open file. By default, file descriptor
0 (zero) refers to the standard input & often abbreviated as stdin.
File descriptor 1 refers
to standard output (stdout) and file descriptor 2 refers to standard error (stderr).
You can use dup(2) function to duplicate a file descriptor with the pipe write descriptor
fd[1] by using function dup2 in order to relocate the standard output
The issue is how can I execute the command and after that I read the stdout in the child
process.
I am executing it using exec function, the line after the exec function will not
execute because the child process memory image is now ls -l.
#include <sys/types.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#define BUFFER_SIZE 25
#define READ_END 0
#define WRITE_END 1
int main(void)
{
char writemsg[BUFFER_SIZE] = "Greetings!";
char readmsg[BUFFER_SIZE];
int fd[2];
pid_t pid;
/* create the pipe */
if (pipe(fd) == -1) {
fprintf(stderr,"Pipe failed");
return 1;
}
/* fork a child process */
pid = fork();
if (pid < 0) { /* error occurred */
fprintf(stderr, "Fork Failed");
return 1;
}
if (pid > 0) { /* parent process */
/* close the unused end of the pipe */
close(fd[READ_END]);
/* write to the pipe */
write(fd[WRITE_END], writemsg, strlen(writemsg)+1);
/* close the write end of the pipe */
close(fd[WRITE_END]);
}
else { /* child process */
/* close the unused end of the pipe */
close(fd[WRITE_END]);
/* read from the pipe */
read(fd[READ_END], readmsg, BUFFER_SIZE);
printf("read %s\n",readmsg);
/* close the write end of the pipe */
close(fd[READ_END]);
}
return 0;
}
this code is working fine. this is the simple code of pipe communication. kindly help me to solve the command execution portion and writing it in child from console
Related
A pipe connects the stdout of one process to the stdin of another: https://superuser.com/a/277327
Here is a simple program to take input from stdin and print it:
int main( ) {
char str[100];
gets( str );
puts( str );
return 0;
}
I can use a unix pipe to pass the input from another process:
echo "hi" | ./a.out
My question is, what is the difference between the simple code above and using the pipe() system call? Does the system call essentially do the same job without writing to the terminal? More on Pipes: https://tldp.org/LDP/lpg/node11.html
The pipe() system call allows you to get file descriptors (one for reading and one for writing) for a channel (a pipe) that allows to stream bytes through multiple processes. This is an example where a parent process creates a pipe and its child writes to it so the parent can read from it:
int main() {
int fd[2];
pipe(fd);
int pid = fork();
if (pid == 0) { // Child:
close(fd[0]); // Close reading descriptor as it's not needed
write(fd[1], "Hello", 5);
} else { // Parent:
char buf[5];
close(fd[1]); // Close writing descriptor as it's not needed
read(fd[0], buf, 5); // Read the data sent by the child through the pipe
write(1, buf, 5); // print the data that's been read to stdout
}
}
When a shell encounters the pipe (|) operator, it does use the pipe() system call, but also does additional things, in order to redirect the left operand's stdout and the right operand's stdin to the pipe. Here's a simplified example of what the shell would do for the command echo "hi" | ./a.out (keep in mind that when duplicating a file descriptor it gets duplicated to the first index available in the open files structure of the process):
int main() {
int fd[2];
pipe(fd);
int pid_echo = fork();
if (pid_echo == 0) {
// Close reading descriptor as it's not needed
close(fd[0]);
// Close standard output
close(1);
// Replace standard output with the pipe by duplicating its writing descriptor
dup(fd[1]);
// Execute echo;
// now when echo prints to stdout it will actually print to the pipe
// because now file descriptor 1 belongs to the pipe
execlp("echo", "echo", "hi", (char*)NULL);
exit(-1);
}
int pid_aout = fork();
if (pid_aout == 0) {
// Close standard input
close(0);
// Replace standard input with the pipe by duplicating its reading descriptor
dup(fd[0]);
// Execute a.out;
// Now when a.out reads from stdin it will actually read from the pipe
// because now file descriptor 0 belongs to the pipe
execl("./a.out", "./a.out", (char*)NULL);
exit(-1);
}
}
A pipe is an inter-process communication mechanism that leverages I/O redirection. However, pipes are not involved in all I/O redirection.
Since child processes may inherit file descriptors from their parent process, a parent process may change what files the child's standard streams point to, unbeknownst to the child process. This is I/O redirection.
I've been trying to implement shell-like functionality with pipes in an application and I'm following this example. I will reproduce the code here for future reference in case the original is removed:
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
/**
* Executes the command "cat scores | grep Villanova | cut -b 1-10".
* This quick-and-dirty version does no error checking.
*
* #author Jim Glenn
* #version 0.1 10/4/2004
*/
int main(int argc, char **argv)
{
int status;
int i;
// arguments for commands; your parser would be responsible for
// setting up arrays like these
char *cat_args[] = {"cat", "scores", NULL};
char *grep_args[] = {"grep", "Villanova", NULL};
char *cut_args[] = {"cut", "-b", "1-10", NULL};
// make 2 pipes (cat to grep and grep to cut); each has 2 fds
int pipes[4];
pipe(pipes); // sets up 1st pipe
pipe(pipes + 2); // sets up 2nd pipe
// we now have 4 fds:
// pipes[0] = read end of cat->grep pipe (read by grep)
// pipes[1] = write end of cat->grep pipe (written by cat)
// pipes[2] = read end of grep->cut pipe (read by cut)
// pipes[3] = write end of grep->cut pipe (written by grep)
// Note that the code in each if is basically identical, so you
// could set up a loop to handle it. The differences are in the
// indicies into pipes used for the dup2 system call
// and that the 1st and last only deal with the end of one pipe.
// fork the first child (to execute cat)
if (fork() == 0)
{
// replace cat's stdout with write part of 1st pipe
dup2(pipes[1], 1);
// close all pipes (very important!); end we're using was safely copied
close(pipes[0]);
close(pipes[1]);
close(pipes[2]);
close(pipes[3]);
execvp(*cat_args, cat_args);
}
else
{
// fork second child (to execute grep)
if (fork() == 0)
{
// replace grep's stdin with read end of 1st pipe
dup2(pipes[0], 0);
// replace grep's stdout with write end of 2nd pipe
dup2(pipes[3], 1);
// close all ends of pipes
close(pipes[0]);
close(pipes[1]);
close(pipes[2]);
close(pipes[3]);
execvp(*grep_args, grep_args);
}
else
{
// fork third child (to execute cut)
if (fork() == 0)
{
// replace cut's stdin with input read of 2nd pipe
dup2(pipes[2], 0);
// close all ends of pipes
close(pipes[0]);
close(pipes[1]);
close(pipes[2]);
close(pipes[3]);
execvp(*cut_args, cut_args);
}
}
}
// only the parent gets here and waits for 3 children to finish
close(pipes[0]);
close(pipes[1]);
close(pipes[2]);
close(pipes[3]);
for (i = 0; i < 3; i++)
wait(&status);
}
I have trouble understanding why the pipes are being closed just before calling execvp and reading or writing any data. I believe it has something to do with passing EOF flags to processes so that they can stop reading writing however I don't see how that helps before any actual data is pushed to the pipe. I'd appreciate a clear explanation. Thanks.
I have trouble understanding why the pipes are being closed just before calling execvp and reading or writing any data.
The pipes are not being closed. Rather, some file descriptors associated with the pipe ends are being closed. Each child process is duping pipe-end file descriptors onto one or both of its standard streams, then closing all pipe-end file descriptors that it is not actually going to use, which is all of the ones stored in the pipes array. Each pipe itself remains open and usable as long as each end is open in at least one process, and each child process holds at least one end of one pipe open. Those are closed when the child processes terminate (or at least under the control of the child processes, post execvp()).
One reason to perform such closures is for tidiness and resource management. There is a limit on how many file descriptors a process may have open at once, so it is wise to avoiding leaving unneeded file descriptors open.
But also, functionally, a process reading from one of the pipes will not detect end of file until all open file descriptors associated with the write end of the pipe, in any process, are closed. That's what EOF on a pipe means, and it makes sense because as long as the write end is open anywhere, it is possible that more data will be written to it.
I have to modify this code. The child process should redirect standard output to text file.
I think that I should do sth with dup2 and exec but I don't know what.
I read this ref and also this.
But it didn't helped me, probably I'm doing sth wrong.
#include <stdio.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
int main ()
{
int fds[2];
pid_t pid;
/* Create a pipe. File descriptors for the two ends of the pipe are placed in fds. */
/* TODO add error handling for system calls like pipe, fork, etc. */
pipe (fds);
/* Fork a child process. */
pid = fork ();
if (pid == (pid_t) 0) {
/* This is the child process. Close our copy of the write end of the file descriptor. */
close (fds[1]);
/* Connect the read end of the pipe to standard input. */
dup2 (fds[0], STDIN_FILENO);
/* Replace the child process with the "sort” program. */
execlp ("sort", "sort", NULL);
} else {
/* This is the parent process. */
FILE* stream;
/* Close our copy of the read end of the file descriptor. */
close (fds[0]);
/* Convert the write file descriptor to a FILE object, and write to it. */
stream = fdopen (fds[1], "w");
fprintf (stream, "This is a test.\n");
fprintf (stream, "Hello, world.\n");
fprintf (stream, "My dog has fleas.\n");
fprintf (stream, "This program is great.\n");
fprintf (stream, "One fish, two fish.\n");
fflush (stream);
close (fds[1]);
/* Wait for the child process to finish. */
waitpid (pid, NULL, 0);
}
return 0;
}
what you have done with dup2 is connecting the parent's stdout to child's stdin, leaving the child's stdout without a redirection. so the childr would print sorted strings to stdout.
what you should do next is to open a textfile , and do a dup2 with it's stdout. such as something before execlp
int outfd=open("/tmp/output",O_WRONLY|O_TRUNC|O_CREAT,0600);
dup2(outfd,STDOUT_FILENO);
execlp ("sort", "sort", NULL);
you will also need to #include <fcntl.h> to have file flags.
My goal is to write a program that performs a bash pipe using anonymous pipes in C.
For now I'm testing my function with this command: ls -l | wc [EDIT], but the goal is to concatenate multiple pipes or just execute a program without them.
I'm using a main program that calls a function defined in routines.h header that execute fork and exec:
Main.c
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "routines.h"
#define READ 0 // for file descriptor index
#define WRITE 1 // for file descriptor index
typedef enum eMode{ // typedef of exec modes
LEFT_PIPE=0, // cmd1|... start pipe
CENTER_PIPE=1, // cmd1|cmd2|... center pipes
RIGHT_PIPE=2, // ...|cmd2 end of the pipe
NORMAL_COMMAND=3// normal cmd
} Mode;
int main()
{
char * commandTest1[]={"ls","-l",NULL};
char * commandTest2[]={"wc",NULL};
int pipeFileDesc[2];
pipe(pipeFileDesc);
executeCommand(commandTest1,pipeFileDesc[READ],pipeFileDesc[WRITE],LEFT_PIPE);
executeCommand(commandTest2,pipeFileDesc[READ],pipeFileDesc[WRITE],RIGHT_PIPE);
return 0;
}
executeCommand in Routines.c
int executeCommand(char** args,int inputDataFD,int outputDataFD,int mode){ //inputDataFD is READ PIPE BUFFER
//outputDATA is WRITE PIPE BUFFER
int pid,retCode,status;
pid=fork(); // fork MONOLITIC: monolitic (father) remains unchanged the son will be transformed in exec of args
if(pid==-1){
perror("Fork Error, for command: ");
return 1; // TODO: in main read strerror(errno)
}else{
if(pid==0){ //son process
printf("Child process: pid %i, father pid %i , pid fork %i \n",getpid(),getppid(),pid);
////////////////////// SETTING OF STDOUT/IN REDIRECTS //////////////////////
if(mode==LEFT_PIPE){ // nothing to read for the process-> left side of the pipe -> PIPE has NO contet and another pipe is found
close(inputDataFD); //close the READ id/channel of the pipe Buffer: !!!! for a process the read side is input !!!!
dup2(outputDataFD,1); // redirects stdout on pipe write: fd 1 for every process is STDOUT, dup2 closes the file linked previously by stdout (closes stdout file)
// and assign fd=1 to the file linked by outputDataFD (pipeFileDesc[WRITE]), beacuse of that outputDataFD (pipeFileDesc[WRITE])
// is a duplicate and it can be unlinked with close()
close(outputDataFD);
}
if(mode==CENTER_PIPE){ //repiping: PIPE has contet and another pipe is found
dup2(inputDataFD,0); // redirects stdin on pipe READ: fd0 for every process is STDIN, dup2 closes the file linked previously by stdin (closes stdin file)
// and assign the file linked by inputDataFD (pipeFileDesc[READ]) to fd id=0 (where processes get data for input (exstdin)
// beacuse of that inputDataFD (pipeFileDesc[READ]) is a duplicate and it can be unlinked with close()
close(inputDataFD);
dup2(outputDataFD,1); // redirects stdout on pipe write: fd 1 for every process is STDOUT, dup2 closes the file linked previously by stdout (closes stdout file)
// and assign fd=1 to the file linked by outputDataFD (pipeFileDesc[WRITE]), beacuse of that outputDataFD (pipeFileDesc[WRITE])
// is a duplicate and it can be unlinked with close()
close(outputDataFD);
}
if(mode==RIGHT_PIPE){ //nothing to write for the process-> right side of pipe ->PIPE has contet and NO other pipe is found
close(outputDataFD); // close the WRITE id/channel of the pipe Buffer: !!! for a process the write side is output !!!!
dup2(inputDataFD,0); // redirects stdin on pipe READ: fd0 for every process is STDIN, dup2 closes the file linked previously by stdin (closes stdin file)
// and assign the file linked by inputDataFD (pipeFileDesc[READ]) to fd id=0 (where processes get data for input (exstdin)
// beacuse of that inputDataFD (pipeFileDesc[READ]) is a duplicate and it can be unlinked with close()
close(inputDataFD);
}
if(mode==NORMAL_COMMAND){ // non pipes-> no STDOUT/IN redirections -> PIPE has NO contet and NO other pipe is found
}
retCode=execvp(args[0],args); // exec command arg[0] with process name arg[0] and arg[1->i] parameters
// stdout of process is redirected onto outputDataFD that is pipeBuffer WRITE side
if(mode==RIGHT_PIPE) printf("error TWO %s",strerror(errno));
if(retCode == -1){
perror("Exec error, unable to execute:");
return 1; //smthing went wrong same TODO in main read output strerror(errno)
}
}else{ //father process
printf("Parent process: pid %i, pid fork %i \n",getpid(), pid );
waitpid(pid,&status,0); // The wait system-call puts the process to sleep and waits for a child-process to end.
// It then fills in the argument with the exit code of the child-process (if the argument is not NULL).
perror("\n");
printf("\nout of wait\n");
if(WIFEXITED(status)==1){//this macro translate the value int of status in TRUE if child ends normaly and FALSE otherwise
return WEXITSTATUS(status); //returns to the main the status of the call
}else{
printf("wait error\n");
}
}
}//end else pid==0
}//end executeCommand
So my problem is that when executing this file it runs all the way till the execvp() of the second call of executeCommand()and than the process (the childre one) freezes and appears to be in sleeping mode.
Removing the wait statement helps but the output is wrong....could it be a waiting problem?
I'm implementing piping on a simulated file system in C++ (with mostly C). It needs to run commands in the host shell but perform the piping itself on the simulated file system.
I could achieve this with the pipe(), fork(), and system() system calls, but I'd prefer to use popen() (which handles creating a pipe, forking a process, and passing a command to the shell). This may not be possible because (I think) I need to be able to write from the parent process of the pipe, read on the child process end, write the output back from the child, and finally read that output from the parent. The man page for popen() on my system says a bidirectional pipe is possible, but my code needs to run on a system with an older version supporting only unidirectional pipes.
With the separate calls above, I can open/close pipes to achieve this. Is that possible with popen()?
For a trivial example, to run ls -l | grep .txt | grep cmds I need to:
Open a pipe and process to run ls -l on the host; read its output back
Pipe the output of ls -l back to my simulator
Open a pipe and process to run grep .txt on the host on the piped output of ls -l
Pipe the output of this back to the simulator (stuck here)
Open a pipe and process to run grep cmds on the host on the piped output of grep .txt
Pipe the output of this back to the simulator and print it
man popen
From Mac OS X:
The popen() function 'opens' a
process by creating a bidirectional
pipe, forking, and invoking the shell.
Any streams opened by previous popen()
calls in the parent process are closed
in the new child process.
Historically, popen() was implemented
with a unidirectional pipe; hence,
many implementations of popen() only
allow the mode argument to specify
reading or writing, not both. Because
popen() is now implemented using a
bidirectional pipe, the mode argument
may request a bidirectional data flow.
The mode argument is a pointer to a
null-terminated string which must be
'r' for reading, 'w' for writing, or
'r+' for reading and writing.
I'd suggest writing your own function to do the piping/forking/system-ing for you. You could have the function spawn a process and return read/write file descriptors, as in...
typedef void pfunc_t (int rfd, int wfd);
pid_t pcreate(int fds[2], pfunc_t pfunc) {
/* Spawn a process from pfunc, returning it's pid. The fds array passed will
* be filled with two descriptors: fds[0] will read from the child process,
* and fds[1] will write to it.
* Similarly, the child process will receive a reading/writing fd set (in
* that same order) as arguments.
*/
pid_t pid;
int pipes[4];
/* Warning: I'm not handling possible errors in pipe/fork */
pipe(&pipes[0]); /* Parent read/child write pipe */
pipe(&pipes[2]); /* Child read/parent write pipe */
if ((pid = fork()) > 0) {
/* Parent process */
fds[0] = pipes[0];
fds[1] = pipes[3];
close(pipes[1]);
close(pipes[2]);
return pid;
} else {
close(pipes[0]);
close(pipes[3]);
pfunc(pipes[2], pipes[1]);
exit(0);
}
return -1; /* ? */
}
You can add whatever functionality you need in there.
You seem to have answered your own question. If your code needs to work on an older system that doesn't support popen opening bidirectional pipes, then you won't be able to use popen (at least not the one that's supplied).
The real question would be about the exact capabilities of the older systems in question. In particular, does their pipe support creating bidirectional pipes? If they have a pipe that can create a bidirectional pipe, but popen that doesn't, then I'd write the main stream of the code to use popen with a bidirectional pipe, and supply an implementation of popen that can use a bidirectional pipe that gets compiled in an used where needed.
If you need to support systems old enough that pipe only supports unidirectional pipes, then you're pretty much stuck with using pipe, fork, dup2, etc., on your own. I'd probably still wrap this up in a function that works almost like a modern version of popen, but instead of returning one file handle, fills in a small structure with two file handles, one for the child's stdin, the other for the child's stdout.
POSIX stipulates that the popen() call is not designed to provide bi-directional communication:
The mode argument to popen() is a string that specifies I/O mode:
If mode is r, when the child process is started, its file descriptor STDOUT_FILENO shall be the writable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the readable end of the pipe.
If mode is w, when the child process is started its file descriptor STDIN_FILENO shall be the readable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the writable end of the pipe.
If mode is any other value, the result is unspecified.
Any portable code will make no assumptions beyond that. The BSD popen() is similar to what your question describes.
Additionally, pipes are different from sockets and each pipe file descriptor is uni-directional. You would have to create two pipes, one configured for each direction.
In one of netresolve backends I'm talking to a script and therefore I need to write to its stdin and read from its stdout. The following function executes a command with stdin and stdout redirected to a pipe. You can use it and adapt it to your liking.
static bool
start_subprocess(char *const command[], int *pid, int *infd, int *outfd)
{
int p1[2], p2[2];
if (!pid || !infd || !outfd)
return false;
if (pipe(p1) == -1)
goto err_pipe1;
if (pipe(p2) == -1)
goto err_pipe2;
if ((*pid = fork()) == -1)
goto err_fork;
if (*pid) {
/* Parent process. */
*infd = p1[1];
*outfd = p2[0];
close(p1[0]);
close(p2[1]);
return true;
} else {
/* Child process. */
dup2(p1[0], 0);
dup2(p2[1], 1);
close(p1[0]);
close(p1[1]);
close(p2[0]);
close(p2[1]);
execvp(*command, command);
/* Error occured. */
fprintf(stderr, "error running %s: %s", *command, strerror(errno));
abort();
}
err_fork:
close(p2[1]);
close(p2[0]);
err_pipe2:
close(p1[1]);
close(p1[0]);
err_pipe1:
return false;
}
https://github.com/crossdistro/netresolve/blob/master/backends/exec.c#L46
(I used the same code in popen simultaneous read and write)
Here's the code (C++, but can be easily converted to C):
#include <unistd.h>
#include <cstdlib>
#include <cstdio>
#include <cstring>
#include <utility>
// Like popen(), but returns two FILE*: child's stdin and stdout, respectively.
std::pair<FILE *, FILE *> popen2(const char *__command)
{
// pipes[0]: parent writes, child reads (child's stdin)
// pipes[1]: child writes, parent reads (child's stdout)
int pipes[2][2];
pipe(pipes[0]);
pipe(pipes[1]);
if (fork() > 0)
{
// parent
close(pipes[0][0]);
close(pipes[1][1]);
return {fdopen(pipes[0][1], "w"), fdopen(pipes[1][0], "r")};
}
else
{
// child
close(pipes[0][1]);
close(pipes[1][0]);
dup2(pipes[0][0], STDIN_FILENO);
dup2(pipes[1][1], STDOUT_FILENO);
execl("/bin/sh", "/bin/sh", "-c", __command, NULL);
exit(1);
}
}
Usage:
int main()
{
auto [p_stdin, p_stdout] = popen2("cat -n");
if (p_stdin == NULL || p_stdout == NULL)
{
printf("popen2() failed\n");
return 1;
}
const char msg[] = "Hello there!";
char buf[32];
printf("I say \"%s\"\n", msg);
fwrite(msg, 1, sizeof(msg), p_stdin);
fclose(p_stdin);
fread(buf, 1, sizeof(buf), p_stdout);
fclose(p_stdout);
printf("child says \"%s\"\n", buf);
return 0;
}
Possible Output:
I say "Hello there!"
child says " 1 Hello there!"
No need to create two pipes and waste a filedescriptor in each process. Just use a socket instead. https://stackoverflow.com/a/25177958/894520