My goal is to write a program that performs a bash pipe using anonymous pipes in C.
For now I'm testing my function with this command: ls -l | wc [EDIT], but the goal is to concatenate multiple pipes or just execute a program without them.
I'm using a main program that calls a function defined in routines.h header that execute fork and exec:
Main.c
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "routines.h"
#define READ 0 // for file descriptor index
#define WRITE 1 // for file descriptor index
typedef enum eMode{ // typedef of exec modes
LEFT_PIPE=0, // cmd1|... start pipe
CENTER_PIPE=1, // cmd1|cmd2|... center pipes
RIGHT_PIPE=2, // ...|cmd2 end of the pipe
NORMAL_COMMAND=3// normal cmd
} Mode;
int main()
{
char * commandTest1[]={"ls","-l",NULL};
char * commandTest2[]={"wc",NULL};
int pipeFileDesc[2];
pipe(pipeFileDesc);
executeCommand(commandTest1,pipeFileDesc[READ],pipeFileDesc[WRITE],LEFT_PIPE);
executeCommand(commandTest2,pipeFileDesc[READ],pipeFileDesc[WRITE],RIGHT_PIPE);
return 0;
}
executeCommand in Routines.c
int executeCommand(char** args,int inputDataFD,int outputDataFD,int mode){ //inputDataFD is READ PIPE BUFFER
//outputDATA is WRITE PIPE BUFFER
int pid,retCode,status;
pid=fork(); // fork MONOLITIC: monolitic (father) remains unchanged the son will be transformed in exec of args
if(pid==-1){
perror("Fork Error, for command: ");
return 1; // TODO: in main read strerror(errno)
}else{
if(pid==0){ //son process
printf("Child process: pid %i, father pid %i , pid fork %i \n",getpid(),getppid(),pid);
////////////////////// SETTING OF STDOUT/IN REDIRECTS //////////////////////
if(mode==LEFT_PIPE){ // nothing to read for the process-> left side of the pipe -> PIPE has NO contet and another pipe is found
close(inputDataFD); //close the READ id/channel of the pipe Buffer: !!!! for a process the read side is input !!!!
dup2(outputDataFD,1); // redirects stdout on pipe write: fd 1 for every process is STDOUT, dup2 closes the file linked previously by stdout (closes stdout file)
// and assign fd=1 to the file linked by outputDataFD (pipeFileDesc[WRITE]), beacuse of that outputDataFD (pipeFileDesc[WRITE])
// is a duplicate and it can be unlinked with close()
close(outputDataFD);
}
if(mode==CENTER_PIPE){ //repiping: PIPE has contet and another pipe is found
dup2(inputDataFD,0); // redirects stdin on pipe READ: fd0 for every process is STDIN, dup2 closes the file linked previously by stdin (closes stdin file)
// and assign the file linked by inputDataFD (pipeFileDesc[READ]) to fd id=0 (where processes get data for input (exstdin)
// beacuse of that inputDataFD (pipeFileDesc[READ]) is a duplicate and it can be unlinked with close()
close(inputDataFD);
dup2(outputDataFD,1); // redirects stdout on pipe write: fd 1 for every process is STDOUT, dup2 closes the file linked previously by stdout (closes stdout file)
// and assign fd=1 to the file linked by outputDataFD (pipeFileDesc[WRITE]), beacuse of that outputDataFD (pipeFileDesc[WRITE])
// is a duplicate and it can be unlinked with close()
close(outputDataFD);
}
if(mode==RIGHT_PIPE){ //nothing to write for the process-> right side of pipe ->PIPE has contet and NO other pipe is found
close(outputDataFD); // close the WRITE id/channel of the pipe Buffer: !!! for a process the write side is output !!!!
dup2(inputDataFD,0); // redirects stdin on pipe READ: fd0 for every process is STDIN, dup2 closes the file linked previously by stdin (closes stdin file)
// and assign the file linked by inputDataFD (pipeFileDesc[READ]) to fd id=0 (where processes get data for input (exstdin)
// beacuse of that inputDataFD (pipeFileDesc[READ]) is a duplicate and it can be unlinked with close()
close(inputDataFD);
}
if(mode==NORMAL_COMMAND){ // non pipes-> no STDOUT/IN redirections -> PIPE has NO contet and NO other pipe is found
}
retCode=execvp(args[0],args); // exec command arg[0] with process name arg[0] and arg[1->i] parameters
// stdout of process is redirected onto outputDataFD that is pipeBuffer WRITE side
if(mode==RIGHT_PIPE) printf("error TWO %s",strerror(errno));
if(retCode == -1){
perror("Exec error, unable to execute:");
return 1; //smthing went wrong same TODO in main read output strerror(errno)
}
}else{ //father process
printf("Parent process: pid %i, pid fork %i \n",getpid(), pid );
waitpid(pid,&status,0); // The wait system-call puts the process to sleep and waits for a child-process to end.
// It then fills in the argument with the exit code of the child-process (if the argument is not NULL).
perror("\n");
printf("\nout of wait\n");
if(WIFEXITED(status)==1){//this macro translate the value int of status in TRUE if child ends normaly and FALSE otherwise
return WEXITSTATUS(status); //returns to the main the status of the call
}else{
printf("wait error\n");
}
}
}//end else pid==0
}//end executeCommand
So my problem is that when executing this file it runs all the way till the execvp() of the second call of executeCommand()and than the process (the childre one) freezes and appears to be in sleeping mode.
Removing the wait statement helps but the output is wrong....could it be a waiting problem?
Related
The program first creates a pipe, and then creates a child process of the current process
through fork().
Then each process closes the file descriptors that are not needed for the read
and write pipes.
The child process executes the ls -a command under the current path, and
writes the command execution output to the pipe by copying the pipe write descriptor fd[1]
to standard output; the parent process reads the pipe data and displays it through fd[0].
Every process in Linux is provided with three standard file descriptor including standard
input, output and error files.
By default:
Standard Input is the keyboard, abstracted as a file to make it easier to write shell
scripts.
Standard Output is the shell window or the terminal from which the script runs,
abstracted as a file to again make writing scripts & program easier
Standard error is the same as standard output: the shell window or terminal from
which the script runs.
A file descriptor is simply a number that refers to an open file. By default, file descriptor
0 (zero) refers to the standard input & often abbreviated as stdin.
File descriptor 1 refers
to standard output (stdout) and file descriptor 2 refers to standard error (stderr).
You can use dup(2) function to duplicate a file descriptor with the pipe write descriptor
fd[1] by using function dup2 in order to relocate the standard output
The issue is how can I execute the command and after that I read the stdout in the child
process.
I am executing it using exec function, the line after the exec function will not
execute because the child process memory image is now ls -l.
#include <sys/types.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#define BUFFER_SIZE 25
#define READ_END 0
#define WRITE_END 1
int main(void)
{
char writemsg[BUFFER_SIZE] = "Greetings!";
char readmsg[BUFFER_SIZE];
int fd[2];
pid_t pid;
/* create the pipe */
if (pipe(fd) == -1) {
fprintf(stderr,"Pipe failed");
return 1;
}
/* fork a child process */
pid = fork();
if (pid < 0) { /* error occurred */
fprintf(stderr, "Fork Failed");
return 1;
}
if (pid > 0) { /* parent process */
/* close the unused end of the pipe */
close(fd[READ_END]);
/* write to the pipe */
write(fd[WRITE_END], writemsg, strlen(writemsg)+1);
/* close the write end of the pipe */
close(fd[WRITE_END]);
}
else { /* child process */
/* close the unused end of the pipe */
close(fd[WRITE_END]);
/* read from the pipe */
read(fd[READ_END], readmsg, BUFFER_SIZE);
printf("read %s\n",readmsg);
/* close the write end of the pipe */
close(fd[READ_END]);
}
return 0;
}
this code is working fine. this is the simple code of pipe communication. kindly help me to solve the command execution portion and writing it in child from console
A pipe connects the stdout of one process to the stdin of another: https://superuser.com/a/277327
Here is a simple program to take input from stdin and print it:
int main( ) {
char str[100];
gets( str );
puts( str );
return 0;
}
I can use a unix pipe to pass the input from another process:
echo "hi" | ./a.out
My question is, what is the difference between the simple code above and using the pipe() system call? Does the system call essentially do the same job without writing to the terminal? More on Pipes: https://tldp.org/LDP/lpg/node11.html
The pipe() system call allows you to get file descriptors (one for reading and one for writing) for a channel (a pipe) that allows to stream bytes through multiple processes. This is an example where a parent process creates a pipe and its child writes to it so the parent can read from it:
int main() {
int fd[2];
pipe(fd);
int pid = fork();
if (pid == 0) { // Child:
close(fd[0]); // Close reading descriptor as it's not needed
write(fd[1], "Hello", 5);
} else { // Parent:
char buf[5];
close(fd[1]); // Close writing descriptor as it's not needed
read(fd[0], buf, 5); // Read the data sent by the child through the pipe
write(1, buf, 5); // print the data that's been read to stdout
}
}
When a shell encounters the pipe (|) operator, it does use the pipe() system call, but also does additional things, in order to redirect the left operand's stdout and the right operand's stdin to the pipe. Here's a simplified example of what the shell would do for the command echo "hi" | ./a.out (keep in mind that when duplicating a file descriptor it gets duplicated to the first index available in the open files structure of the process):
int main() {
int fd[2];
pipe(fd);
int pid_echo = fork();
if (pid_echo == 0) {
// Close reading descriptor as it's not needed
close(fd[0]);
// Close standard output
close(1);
// Replace standard output with the pipe by duplicating its writing descriptor
dup(fd[1]);
// Execute echo;
// now when echo prints to stdout it will actually print to the pipe
// because now file descriptor 1 belongs to the pipe
execlp("echo", "echo", "hi", (char*)NULL);
exit(-1);
}
int pid_aout = fork();
if (pid_aout == 0) {
// Close standard input
close(0);
// Replace standard input with the pipe by duplicating its reading descriptor
dup(fd[0]);
// Execute a.out;
// Now when a.out reads from stdin it will actually read from the pipe
// because now file descriptor 0 belongs to the pipe
execl("./a.out", "./a.out", (char*)NULL);
exit(-1);
}
}
A pipe is an inter-process communication mechanism that leverages I/O redirection. However, pipes are not involved in all I/O redirection.
Since child processes may inherit file descriptors from their parent process, a parent process may change what files the child's standard streams point to, unbeknownst to the child process. This is I/O redirection.
so in this code snippet a process forked a child process. the child process calculated a random number r, and called linux command 'head -r "file' with an exec function which destroys the process itself, but to send the result back to the parent process the child process first duplicated the writing end of a pipe p, shared with the parent process and then closed both ends of the pipe p and closed the stdout file descriptor too...after execlp the parent process could read the result of the command 'head -r "fil2"' from the pipe p.
How is this possible?
if (pid == 0)
{
/* code of child */
srand(time(NULL));
nr=atoi(argv[(i*2)+2]);
r=mia_random(nr); //calc random value
close(1); //closing standard output???
dup(p[1]); //duplicating write end of inherited pipe from parent
close(p[0]);//closing read end of inherited pipe
close(p[1]);//closing write end of inherited pipe
//creating a variable to hold an argument for `head`
sprintf(option, "-%d", r);
//calling head on a file given as argument in main
execlp("head", "head", option, argv[(i*2)+1], (char *)0);
/* must not be here anymore*/
/* using perror to check for errors since stdout is closed? or connected to the pipe?*/
perror("Problem esecuting head by child process");
exit(-1);
}
Why wasn't the result of head written to stderr instead? How come it was written to the dup(p[1])???
The system is guaranteed to open each new file at the lowest possible file descriptor.
Effectively, that means that if fd 0 and 1 are open and p[1] != 1, then
close(1);
dup(p[1]);
in a single-threaded process is equivalent to
dup2(p[1],1);
or in other words, if the dup call in this context succeeds, it will return (filedescriptor) 1.
I need to create two child processes. One child needs to run the command "ls -al" and redirect its output to the input of the next child process, which in turn will run the command "sort -r -n -k 5" on its input data. Finally, the parent process needs to read that (data already sorted) and display it in the terminal. The final result in the terminal (when executing the program) should be the same as if I entered the following command directly in the shell: "ls -al | sort -r -n -k 5". For this I need to use the following methods: pipe(), fork(), execlp().
My program compiles, but I don't get the desired output to the terminal. I don't know what is wrong. Here is the code:
#include <sys/types.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int main()
{
int fd[2];
pid_t ls_pid, sort_pid;
char buff[1000];
/* create the pipe */
if (pipe(fd) == -1) {
fprintf(stderr, "Pipe failed");
return 1;
}
/* create child 2 first */
sort_pid = fork();
if (sort_pid < 0) { // error creating Child 2 process
fprintf(stderr, "\nChild 2 Fork failed");
return 1;
}
else if(sort_pid > 0) { // parent process
wait(NULL); // wait for children termination
/* create child 1 */
ls_pid = fork();
if (ls_pid < 0) { // error creating Child 1 process
fprintf(stderr, "\nChild 1 Fork failed");
return 1;
}
else if (ls_pid == 0) { // child 1 process
close(1); // close stdout
dup2(fd[1], 1); // make stdout same as fd[1]
close(fd[0]); // we don't need this end of pipe
execlp("bin/ls", "ls", "-al", NULL);// executes ls command
}
wait(NULL);
read(fd[0], buff, 1000); // parent reads data
printf(buff); // parent prints data to terminal
}
else if (sort_pid == 0) { // child 2 process
close(0); // close stdin
dup2(fd[0], 0); // make stdin same as fd[0]
close(fd[1]); // we don't need this end of pipe
execlp("bin/sort", "sort", "-r", "-n", "-k", "5", NULL); // executes sort operation
}
return 0;
}
Your parent process waits for the sort process to finish before creating the ls process.
The sort process needs to read its input before it can finish. And its input is coming from the ls that won't be started until after the wait. Deadlock.
You need to create both processes, then wait for both of them.
Also, your file descriptor manipulations aren't quite right. In this pair of calls:
close(0);
dup2(fd[0], 0);
the close is redundant, since dup2 will automatically close the existing fd 0 if there is one. You should do a close(fd[0]) after ther dup2, so you only have one file descriptor tied to that end of the pipe. And if you want to be really robust, you should test wither fd[0]==0 already, and in that case skip the dup2 and close.
Apply all of that to the other dup2 also.
Then there's the issue of the parent process holding the pipe open. I'd say you should close both ends of the pipe in the parent after you've passed them on to the children, but you have that weird read from fd[0] after the last wait... I'm not sure why that's there. If the ls|sort pipeline has run correctly, the pipe will be empty afterward, so there will be nothing to read. In any case, you definitely need to close fd[1] in the parent, otherwise the sort process won't finish because the pipe won't indicate EOF until all writers are closed.
After the weird read is a printf that will probably crash, since the read buffer won't be '\0'-terminated.
And the point of using execlp is that it does the $PATH lookup for you so you don't have to specify /bin/. My first test run failed because my sort is in /usr/bin/. Why hardcode paths when you don't have to?
This is the code i found for my own shell. It works fine, but the thing i can't understand is pipe section of the code.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
char* cmndtkn[256];
char buffer[256];
char* path=NULL;
char pwd[128];
int main(){
//setting path variable
char *env;
env=getenv("PATH");
putenv(env);
system("clear");
printf("\t MY OWN SHELL !!!!!!!!!!\n ");
printf("_______________________________________\n\n");
while(1){
fflush(stdin);
getcwd(pwd,128);
printf("[MOSH~%s]$",pwd);
fgets(buffer,sizeof(buffer),stdin);
buffer[sizeof(buffer)-1] = '\0';
//tokenize the input command line
char* tkn = strtok(buffer," \t\n");
int i=0;
int indictr=0;
// loop for every part of the command
while(tkn!=NULL)
{
if(strcoll(tkn,"exit")==0 ){
exit(0);
}
else if(strcoll(buffer,"cd")==0){
path = buffer;
chdir(path+=3);
}
else if(strcoll(tkn,"|")==0){
indictr=i;
}
cmndtkn[i++] = tkn;
tkn = strtok(NULL," \t\n");
}cmndtkn[i]='\0';
// execute when command has pipe. when | command is found indictr is greater than 0.
if(indictr>0){
char* leftcmnd[indictr+1];
char* rightcmnd[i-indictr];
int a,b;
for(b=0;b<indictr;b++)
leftcmnd[b]=cmndtkn[b];
leftcmnd[indictr]=NULL;
for(a=0;a<i-indictr-1;a++)
rightcmnd[a]=cmndtkn[a+indictr+1];
rightcmnd[i-indictr]=NULL;
if(!fork())
{
fflush(stdout);
int pfds[2];
pipe(pfds);
if(!fork()){
close(1);
dup(pfds[1]);
close(pfds[0]);
execvp(leftcmnd[0],leftcmnd);
}
else{
close(0);
dup(pfds[0]);
close(pfds[1]);
execvp(rightcmnd[0],rightcmnd);
}
}else
wait(NULL);
//command not include pipe
}else{
if(!fork()){
fflush(stdout);
execvp(cmndtkn[0],cmndtkn);
}else
wait(NULL);
}
}
}
What is the purpose of the calls to close() with parameters of 0 and 1 mean and what does the call to dup() do?
On Unix, the dup() call uses the lowest numbered unused file descriptor. So, the close(1) before the call to dup() is to coerce dup() to use file descriptor 1. Similarly for close(0).
So, the aliasing is to get the process to use the write end of the pipe for stdout (file descriptor 1 is used for console output), and the read end of the pipe for stdin (file descriptor 0 is used for console input).
The code may have been more clearly expressed with dup2() instead.
dup2(fd[1], 1); /* alias fd[1] to 1 */
From your question about how ls | sort works, your question is not limited to why the dup() system call is being made. Your question is actually how pipes in Unix work, and how a shell command pipeline works.
A pipe in Unix is a pair of file descriptors that are related in that writing data on tje writable descriptor allows that data to be read from the readable descriptor. The pipe() call returns this pair in an array, where the first array element is readable, and second array element is writable.
In Unix, a fork() followed by some kind of exec() is the only way to produce a new process (there are other library calls, such as system() or popen() that create processes, but they call fork() and do an exec() under the hood). A fork() produces a child process. The child process sees the return value of 0 from the call, while the parent sees a non-zero return value that is either the PID of the child process, or a -1 indicating that an error has occurred.
The child process is a duplicate of the parent. This means that when a child modifies a variable, it is modifying a copy of the variable that resides in its own process. The parent does not see the modification occur, as the parent has the original copy). However, a duplicated pair of file descriptors that form a pipe can be used to allow a child process its parent to communicate with each other.
So, ls | sort means that there are two processes being spawned, and the output written by ls is being read as input by sort. Two processes means two calls to fork() to create two child processes. One child process will exec() the ls command, the other child process will exec() the sort command. A pipe is used between them to allow the processes to talk to each other. The ls process writes to the writable end of the pipe, the sort process reads from the readable end of the pipe.
The ls process is coerced into writing into the writable end of the pipe with the dup() call after issuing close(1). The sort process is coerced into reading the readable end of the pipe with the dup() call after close(0).
In addition, the close() calls that close the pipe file descriptors are used to make sure that the ls process is the only process to have an open reference to the writable fd, the the sort process is the only process to have an open reference to the readable fd. That step is important because after ls exits, it will close the writable end of the fd, and the sort process will expect to see an EOF as a result. However, this will not occur if some other process still has the writable fd open.
http://en.wikipedia.org/wiki/Standard_streams#Standard_input_.28stdin.29
stdin is file descriptor 0.
stdout is file descriptor 1.
In the !fork section, the process closes stdout then calls dup on pfds[1] which according to:
http://linux.die.net/man/2/dup
Creates a duplicate of the specified file descriptor at the lowest available position, which will be 1, since it was just closed (and stdin hasn't been closed yet). This means everything sent to stdout will really go to pfds[1].
So, basically, it's setting up the two new processes to talk to each other. the !fork section is for the new child which will send data to stdout (file descriptor 1), the parent (the else block) closes stdin, so it really reads from pfds[0] when it tries to read from stdout.
Each process has to close the file descriptor in pfds it's not using, as there are two open handles to the file now that the process has forked. Each process now execs to left/right-cmnd, but the new stdin and stdout mappings remain for the new processes.
Forking twice is explained here: Why fork() twice