I'm trying to build a minishell and I've mostly everything but pipes working. I've read a lot of answers in this site and many others, but I'm not able to find my particular problem.
The sequence should end when the right-most process ends, this is: sleep 3 | sleep 3 | sleep 0 should immediately end, and not wait for any of the sleep 3 processes.
*argvv is a pointer to the first process (*argvv would be sleep 3, *(argvv + 1) is the next sleep 3, etc. I don't know what's going on, but something is wrong with file descriptors and everything breaks after a test.
For instance, when the input is sh> ls | wc | wc, the output is
sh> ls | wc | wc
sh> 1 3 25
, then it waits for some input. It's not until I press enter again it finishes. After that, sleep 3 does not work anymore.
Any help would be welcome, thanks a lot!
Edit:
Ok, I edited the code and everything seems to work fine, except sleep 3 | sleep 0 lasts 3 seconds, instead of immediately ending. I don't know how to fix this.
void execute(char ***argvv)
{
pid_t pid;
int pd[2];
int in = 0;
while (*argvv != NULL) {
pipe(pd);
if ((pid = fork()) < 0)
{
perror();
exit(1);
}
else if (pid == 0) {
dup2(in, 0);
if (*(argvv + 1) != NULL)
dup2(pf[1], 1);
close(pd[0]);
close(pd[1]);
execvp((*argvv)[0], *argvv);
perror();
exit(1);
}
else
{
close(pd[1]);
in = pd[0];
if (*(argvv + 1) == NULL)
close(pd[0]);
argvv++;
}
wait(&pid);
}
}
I think part of the problem may be in the use of wait(). It's pid parameter is an OUTPUT, reporting which fork()ed process ended. Every forked processed ends, and needs to be wait()ed for. When you run a second pipeline, all but one piece of the preceding pipeline is still ready to be wait()ed for.
Generally, you have several choices:
wait() in a loop until the process you want to end ends.
wait() in a loop until all the subprocesses end.
Use waitpid() to look for a particular process ending.
Arrange that only the last process in the chain is a child of your process, making the rest children of that. (This is done by some shell, though I don't remember which.) The problem with this is that it leaves extra children for the last process in the chain, which may confuse it.
Track all the child processes, and which have ended. When you background a pipeline, this allows you to display which parts of the pipeline are still running. (tcsh does this. bash does half.)
fork() twice for the earlier pipeline members, execvp() in the grandchild, then exit() in the child, and have the parent wait() for the child to finish before going on. This disconnects the grandchild from the parent, so it never needs to be waited for.
Do things involving ignoring SIGCHLD, but this will probably cause you a lot of grief, as it is all or nothing.
Somehow it's easier to call fork and then unshare because many arguments are copied via fork that would otherwise be manually wrapped to clone. My question is, what is the difference between (1) calling clone which forks a new process in separate namespaces and (2) fork+unshare which forks a new process and then leaves parent's namespaces. Assume all namespace flags passed to clone and unshare are the same.
auto flag = CLONE_NEWUSER | CLONE_NEWUTS | CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWNET | SIGCHLD;
So, with fork, its very easy for child to reuse data inherited from parents.
int mydata; // this could be even more complicated, class, struct, etc.
auto pid = fork();
if(pid == 0) {
// reuse inherited data in a copy-on-write manner
unshare(flag);
}
For clone, we sometimes have to wrap data into another struct and pass it as void* to the clone system call.
int mydata;
clone(func, stack+stack_size, flag, &wrapper_of_data);
It seems to me in the above example the only difference is the performance hit, where fork can be a bit more expensive. However, the nature of fork saves me many efforts in the way that both can create a process in new namespaces.
Somehow it's easier to call fork and then unshare because many arguments are copied via fork that would otherwise be manually wrapped to clone.
This is only true for the libc wrapper function.
The underlying system call is more like fork(), and would probably work better in this case:
long flags = CLONE_NEWUSER | ... | SIGCHLD;
long pid = syscall(__NR_clone, flags, 0, 0, 0, 0);
if(pid == 0) {
/* child with unshared namespaces */
}
Since about the time clone syscall was introduced, fork() is just a specific case of clone with flags=SIGCHLD and 0s for all other arguments.
There is probably no reason to prefer fork/unshare pair over just passing the same flags to clone.
This question already has answers here:
Output of fork() calls
(3 answers)
Closed 8 years ago.
from the man page of fork i read fork create duplicate of parent process. But not able to understand why below program printf execute 8 times. I read Working of fork() in linux link also.
#include <stdio.h>
int main()
{
fork();
fork();
fork();
printf("process\n");
}
In general for n forks in this manner will execute the next statements (in this case printf) 2^n times.
Here is how:
|
+-fork()----------------------------------+
| |
+-fork()-------------+ +-fork()-------------+
| | | |
+-fork()---+ +-fork()---+ +-fork()---+ +-fork()---+
| | | | | | | |
print() print() print() print() print() print() print() print()
Fork works like a binary tree. So its always 2^x number of processes on every x number of fork calls.
Lets understand with your example.
First fork() call:
When the first fork() is called. The parent process creates a new process. So we have 2
threads.
Second fork() call:
At this point we have two processes(main process and a new created process).These two threads will call second fork individually and create 2 new processes each. so we have 4 threads.
You might have got the idea by now. Every time a fork() is encountered all the processes create their respective child processes(double themselves).
I try write a command interpreter in C. I must create dwo and three redirects (e.g. ls | grep ^d | wc -l and ls -ps | grep / | pr -3 | more)
I have code to operate one redirects
if(k==1)
{
int pfds[2];
pipe(pfds);
child_pid = fork();
if(child_pid==0)
{
close(1);
dup(pfds[1]);
close(pfds[0]);
execlp(arg1[0], *arg1, NULL);
}
else
{
close(0);
dup(pfds[0]);
close(pfds[1]);
execlp(arg2[0], *arg2, NULL);
}
}
My question is how make two and three redirects using pipe and fork?
I try do this using only one pipe but this in not work.
you will have to create as many pipe variables as there are "diversion".
then create a list of commands.
if you want parent process leave,you would fork a process for each command. otherwise one less.
for the very first command, dup or 'dup2` only for STDOUT.
and for the very last command dup or 'dup2` only for STDIN.
for the rest, dup or 'dup2` for STDIN and STDOUT both.
you can do this with a for loop from 2nd to second-last.
For Example:
shell $> cat file.txt | grep 'pattern1|pattern2' | grep 'pattern1' | wc -l
I am assuming you are not using parent process for exec
So, you would create a list/array/vector of commands. list would have all 4 commands.
create 4 pipes for each commands in parent process itself.
run a loop for 4 iteration, each for a command.
fork one process.
if parent process, continue.
else(child)
dup/dup2 only STDOUT for first command(cat).
dup/dup2 only STDIN for last command(wc -l).
dup/dup2 both STDIN and STDOUT otherwise.
then run exec.
CHEERS :)
What are the differences between fork and exec?
The use of fork and exec exemplifies the spirit of UNIX in that it provides a very simple way to start new processes.
The fork call basically makes a duplicate of the current process, identical in almost every way. Not everything is copied over (for example, resource limits in some implementations) but the idea is to create as close a copy as possible.
The new process (child) gets a different process ID (PID) and has the PID of the old process (parent) as its parent PID (PPID). Because the two processes are now running exactly the same code, they can tell which is which by the return code of fork - the child gets 0, the parent gets the PID of the child. This is all, of course, assuming the fork call works - if not, no child is created and the parent gets an error code.
The exec call is a way to basically replace the entire current process with a new program. It loads the program into the current process space and runs it from the entry point.
So, fork and exec are often used in sequence to get a new program running as a child of a current process. Shells typically do this whenever you try to run a program like find - the shell forks, then the child loads the find program into memory, setting up all command line arguments, standard I/O and so forth.
But they're not required to be used together. It's perfectly acceptable for a program to fork itself without execing if, for example, the program contains both parent and child code (you need to be careful what you do, each implementation may have restrictions). This was used quite a lot (and still is) for daemons which simply listen on a TCP port and fork a copy of themselves to process a specific request while the parent goes back to listening.
Similarly, programs that know they're finished and just want to run another program don't need to fork, exec and then wait for the child. They can just load the child directly into their process space.
Some UNIX implementations have an optimized fork which uses what they call copy-on-write. This is a trick to delay the copying of the process space in fork until the program attempts to change something in that space. This is useful for those programs using only fork and not exec in that they don't have to copy an entire process space.
If the exec is called following fork (and this is what happens mostly), that causes a write to the process space and it is then copied for the child process.
Note that there is a whole family of exec calls (execl, execle, execve and so on) but exec in context here means any of them.
The following diagram illustrates the typical fork/exec operation where the bash shell is used to list a directory with the ls command:
+--------+
| pid=7 |
| ppid=4 |
| bash |
+--------+
|
| calls fork
V
+--------+ +--------+
| pid=7 | forks | pid=22 |
| ppid=4 | ----------> | ppid=7 |
| bash | | bash |
+--------+ +--------+
| |
| waits for pid 22 | calls exec to run ls
| V
| +--------+
| | pid=22 |
| | ppid=7 |
| | ls |
V +--------+
+--------+ |
| pid=7 | | exits
| ppid=4 | <---------------+
| bash |
+--------+
|
| continues
V
fork() splits the current process into two processes. Or in other words, your nice linear easy to think of program suddenly becomes two separate programs running one piece of code:
int pid = fork();
if (pid == 0)
{
printf("I'm the child");
}
else
{
printf("I'm the parent, my child is %i", pid);
// here we can kill the child, but that's not very parently of us
}
This can kind of blow your mind. Now you have one piece of code with pretty much identical state being executed by two processes. The child process inherits all the code and memory of the process that just created it, including starting from where the fork() call just left off. The only difference is the fork() return code to tell you if you are the parent or the child. If you are the parent, the return value is the id of the child.
exec is a bit easier to grasp, you just tell exec to execute a process using the target executable and you don't have two processes running the same code or inheriting the same state. Like #Steve Hawkins says, exec can be used after you forkto execute in the current process the target executable.
I think some concepts from "Advanced Unix Programming" by Marc Rochkind were helpful in understanding the different roles of fork()/exec(), especially for someone used to the Windows CreateProcess() model:
A program is a collection of instructions and data that is kept in a regular file on disk. (from 1.1.2 Programs, Processes, and Threads)
.
In order to run a program, the kernel is first asked to create a new process, which is an environment in which a program executes. (also from 1.1.2 Programs, Processes, and Threads)
.
It’s impossible to understand the exec or fork system calls without fully understanding the distinction between a process and a program. If these terms are new to you, you may want to go back and review Section 1.1.2. If you’re ready to proceed now, we’ll summarize the distinction in one sentence: A process is an execution environment that consists of instruction, user-data, and system-data segments, as well as lots of other resources acquired at runtime, whereas a program is a file containing instructions and data that are used to initialize the instruction and user-data segments of a process. (from 5.3 exec System Calls)
Once you understand the distinction between a program and a process, the behavior of fork() and exec() function can be summarized as:
fork() creates a duplicate of the current process
exec() replaces the program in the current process with another program
(this is essentially a simplified 'for dummies' version of paxdiablo's much more detailed answer)
Fork creates a copy of a calling process.
generally follows the structure
int cpid = fork( );
if (cpid = = 0)
{
//child code
exit(0);
}
//parent code
wait(cpid);
// end
(for child process text(code),data,stack is same as calling process)
child process executes code in if block.
EXEC replaces the current process with new process's code,data,stack.
generally follows the structure
int cpid = fork( );
if (cpid = = 0)
{
//child code
exec(foo);
exit(0);
}
//parent code
wait(cpid);
// end
(after exec call unix kernel clears the child process text,data,stack and fills with foo process related text/data)
thus child process is with different code (foo's code {not same as parent})
They are use together to create a new child process. First, calling fork creates a copy of the current process (the child process). Then, exec is called from within the child process to "replace" the copy of the parent process with the new process.
The process goes something like this:
child = fork(); //Fork returns a PID for the parent process, or 0 for the child, or -1 for Fail
if (child < 0) {
std::cout << "Failed to fork GUI process...Exiting" << std::endl;
exit (-1);
} else if (child == 0) { // This is the Child Process
// Call one of the "exec" functions to create the child process
execvp (argv[0], const_cast<char**>(argv));
} else { // This is the Parent Process
//Continue executing parent process
}
The main difference between fork() and exec() is that,
The fork() system call creates a clone of the currently running program. The original program continues execution with the next line of code after the fork() function call. The clone also starts execution at the next line of code.
Look at the following code that i got from http://timmurphy.org/2014/04/26/using-fork-in-cc-a-minimum-working-example/
#include <stdio.h>
#include <unistd.h>
int main(int argc, char **argv)
{
printf("--beginning of program\n");
int counter = 0;
pid_t pid = fork();
if (pid == 0)
{
// child process
int i = 0;
for (; i < 5; ++i)
{
printf("child process: counter=%d\n", ++counter);
}
}
else if (pid > 0)
{
// parent process
int j = 0;
for (; j < 5; ++j)
{
printf("parent process: counter=%d\n", ++counter);
}
}
else
{
// fork failed
printf("fork() failed!\n");
return 1;
}
printf("--end of program--\n");
return 0;
}
This program declares a counter variable, set to zero, before fork()ing. After the fork call, we have two processes running in parallel, both incrementing their own version of counter. Each process will run to completion and exit. Because the processes run in parallel, we have no way of knowing which will finish first. Running this program will print something similar to what is shown below, though results may vary from one run to the next.
--beginning of program
parent process: counter=1
parent process: counter=2
parent process: counter=3
child process: counter=1
parent process: counter=4
child process: counter=2
parent process: counter=5
child process: counter=3
--end of program--
child process: counter=4
child process: counter=5
--end of program--
The exec() family of system calls replaces the currently executing code of a process with another piece of code. The process retains its PID but it becomes a new program. For example, consider the following code:
#include <stdio.h>
#include <unistd.h>
main() {
char program[80],*args[3];
int i;
printf("Ready to exec()...\n");
strcpy(program,"date");
args[0]="date";
args[1]="-u";
args[2]=NULL;
i=execvp(program,args);
printf("i=%d ... did it work?\n",i);
}
This program calls the execvp() function to replace its code with the date program. If the code is stored in a file named exec1.c, then executing it produces the following output:
Ready to exec()...
Tue Jul 15 20:17:53 UTC 2008
The program outputs the line ―Ready to exec() . . . ‖ and after calling the execvp() function, replaces its code with the date program. Note that the line ― . . . did it work‖ is not displayed, because at that point the code has been replaced. Instead, we see the output of executing ―date -u.‖
fork() creates a copy of the current process, with execution in the new child starting from just after the fork() call. After the fork(), they're identical, except for the return value of the fork() function. (RTFM for more details.) The two processes can then diverge still further, with one unable to interfere with the other, except possibly through any shared file handles.
exec() replaces the current process with a new one. It has nothing to do with fork(), except that an exec() often follows fork() when what's wanted is to launch a different child process, rather than replace the current one.
fork():
It creates a copy of running process. The running process is called parent process & newly created process is called child process. The way to differentiate the two is by looking at the returned value:
fork() returns the process identifier (pid) of the child process in the parent
fork() returns 0 in the child.
exec():
It initiates a new process within a process. It loads a new program into the current process, replacing the existing one.
fork() + exec():
When launching a new program is to firstly fork(), creating a new process, and then exec() (i.e. load into memory and execute) the program binary it is supposed to run.
int main( void )
{
int pid = fork();
if ( pid == 0 )
{
execvp( "find", argv );
}
//Put the parent to sleep for 2 sec,let the child finished executing
wait( 2 );
return 0;
}
The prime example to understand the fork() and exec() concept is the shell,the command interpreter program that users typically executes after logging into the system.The shell interprets the first word of command line as a command name
For many commands,the shell forks and the child process execs the command associated with the name treating the remaining words on the command line as parameters to the command.
The shell allows three types of commands. First, a command can be an
executable file that contains object code produced by compilation of source code (a C program for example). Second, a command can be an executable file that
contains a sequence of shell command lines. Finally, a command can be an internal shell command.(instead of an executable file ex->cd,ls etc.)