I am trying to program a shell in C , and I found that each command is executed in a new process, my question is why do we make a new process to execute the command? can't we just execute the command in the current process?
It's because of how the UNIX system was designed, where the exec family of calls replace the current process. Therefore you need to create a new process for the exec call if you want the shell to continue afterward.
When you execute a command, one of the following happens:
You're executing a builtin command
You're executing an executable program
An executable program needs many things to work: different memory sections (stack, heap, code, ...), it is executed with a specific set of privileges, and many more things are happening.
If you run this new executable program in your current process, you're going to replace the current program (your shell) with the new one. It works perfectly fine but when the new executable program is done, you cannot go back to your shell since it's not in memory anymore. This is why we create a new process and run the executable program in this new process. The shell waits for this new process to be done, then it collects its exit status and prompts you again for a new command to execute.
can't we just execute the command in the current process?
Sure we can, but that would then replace the shell program with the program of the command called. But that's probably not something you want in this particular application. There are in fact, many situations in which replacing the process program via execve is a the most straightforward way to implement something. But in the case of a shell, that's likely not what you want.
You should not think processes to be something to be avoided or "feared". As a matter of fact, segregating different things into different processes is the foundation of reliability and security features. Processes are (mostly) isolated from each other, so if a process gets terminated for whatever reason (bug, crash, etc.) this in the first degree affects only that particular process.
Here's something to try out:
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int segfault_crash()
{
fprintf(stderr, "I will SIGSEGV...\n");
fputs(NULL, stderr);
return 0;
}
int main(int argc, char *argv)
{
int status = -1;
pid_t const forked_pid = fork();
if( -1 == forked_pid ){
perror("fork: ");
return 1;
}
if( 0 == forked_pid ){
return segfault_crash();
}
waitpid(forked_pid, &status, 0);
if( WIFSIGNALED(status) ){
fprintf(stderr, "Child process %lld terminated by signal %d\n",
(long long)forked_pid,
(int)WTERMSIG(status) );
} else {
fprintf(stderr, "Child process %lld terminated normally\n",
(long long)forked_pid);
}
return 0;
}
This little program forks itself, then calls a function that deliberately performs undefined behavior, that on commonplace systems triggers some kind of memory protection fault (Access Violation on Windows, Segmentation Fault on *nix systems). But because this crash has been isolated into dedicated process, the parent process (and also siblings) are not crashing together with it.
Furthermore processes may drop their privileges, limit themselves to only a subset of system calls, and be moved into namespaces/containers, each of which prevents a bug in the process to damage the rest of the system. This is how modern browsers (for example) implement sandboxing, to improve security.
Related
I want to build my own debugger, from scratch, so I am trying to pick up some of the concepts behind it. First, I am starting easy, using the ptrace library. But even at this point I am having some issues, let me run through this code:
int main(int argc, char** argv)
{
pid_t child_pid;
if (argc < 2) {
fprintf(stderr, "Expected a program name as argument\n");
return -1;
}
child_pid = fork();
if (child_pid == 0)
run_target(argv[1]);
else if (child_pid > 0)
run_debugger(child_pid);
else {
perror("fork");
return -1;
}
return 0;
}
this is nothing really special, I am creating a child process using fork()
the next function is what really I cannot understand
void run_target(const char* programname)
{
procmsg("target started. will run '%s'\n", programname);
/* Allow tracing of this process */
if (ptrace(PTRACE_TRACEME, 0, 0, 0) < 0) {
perror("ptrace");
return;
}
/* Replace this process's image with the given program */
execl(programname, programname, 0);
}
The last call is the issue. this call represents the concept of overlaying process image.I am not fully getting what is happening. This is what the author says:
I've highlighted the part that interests us in this example. Note that the very next thing run_target does after ptrace is invoke the program given to it as an argument
with execl. This, as the highlighted part explains, causes the OS kernel to stop the process just before it begins executing the program in execl and send a signal to the parent.
To run the debugger basically the parent process must trace the child process, which acknowledges that it wants to be traced using PTRACEME. But I can’t figure out what that execl is doing. I can understand the purpose and the output but can’t figure out HOW. I consulted the man pages but could not wrap my head around this.
I would appreciate if someone could give me a clear explanation of what’s going on with this execl function.
I think you agree on this: the concept is that the debugger must debug program TARGET, that can only be debugged if it calls PTRACE_TRACEME.
Naturally, TARGET does not call ptrace with PTRACE_TRACEME argument in its source code.
So, the debugger must do it for it.
Initially, the debugger forks. At this time we have two processes:
Father: it calls run_debugger()
Child: it calls run_target()
Child is a process that the debugger has control on it, therefore it can call ptrace with argument PTRACE_TRACEME (in run_target()). But this process is not TARGET.
Thus, next step is associating to child the "image" of TARGET (namely the program we want to debug). A process image is an executable file required while executing the program, and it's composed of the 4 classical segments:
Code (text segment)
Data
Stack
Heap
execl and friends belong to exec family, namely functions which replace the current process image with a new process image. execl differs from its friend execv for the way the arguments are passed to the best of my knowledge, but the concept is the same.
So, what you need to know is that:
exec replaces the currently running program by another program (i..e., TARGET)
inside an EXISTING process.
The latter has called ptrace with argument PTRACE_TRACEME so the new program will not ignore the future ptrace calls made by the debugger.
If your question is the implementation details of exec systemcall, I have not a perfect knowledge for it, but I can give some suggestions:
Reading "The exec-like Functions" in the book "Understanding Linux Kernel"
Having a look to the source code (this question) directionates you to the source code of execve which is totally fine for you, same concept of execl.
If you want to create your own exec, this question can be useful as well.
#include <unistd.h>
#include <stdio.h>
int main(){
fork();
return 0;
}
In my understanding, fork() will copy the parent's process, and run it as a child process; if that was the case, would the program above break? Because how I am understanding this program is: the program above will call fork() indefinitely, and eventually cause a Stack Overflow.
According to the POSIX specification:
Both processes shall continue to execute from the fork() function.
So, both processes will continue after the call to fork(), and both will immediately terminate.
The fork call does not make either the child or the parent process go back to the beginning of main and start over. It returns like a normal function, but it does it twice, once in the child and once in the parent, with different return values so you can tell which is which.
So, in your program, fork succeeds and then both processes go on to the return 0 and exit. Nothing bad will happen.
A variation will cause problems, though:
#include <unistd.h>
int
main(void)
{
for (;;)
fork();
/* not reached */
}
This is called a "fork bomb". Because it calls fork inside an infinite loop, never checking whether it's the parent or the child, the original process becomes two processes, and then four, and then eight, and ... until you run out of RAM, or at least process IDs. And it doesn't check for failure either, so it doesn't stop after the fork calls start failing. All of these processes will continue chewing up CPU forever, and none of the other programs running on the computer will be able to make forward progress.
Back in the days of mammoths and SunOS 4 it was even worse than that, a fork bomb would be liable to tickle a kernel bug and outright crash the minicomputer, and then the BOFH would come looking for you and he or she would not be happy. I would expect a modern kernel not to crash, and you might even be able to kill off the entire exponential process tree with control-C, but I'm not going to try it just to find out.
Incidentally, return_type whatever() is bad style in C, because for historical reasons it means whatever takes any number of arguments. Always write return_type whatever(void) instead.
When I write a small script with fork, the syscall returns twice processes (once per process):
#include <stdio.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int pid = fork();
if (pid == 0) {
// child
} else if (pid > 0) {
// parent
}
}
If I instrument that with systemtap, I only find one return value:
// fork() in libc calls clone on Linux
probe syscall.clone.return {
printf("Return from clone\n")
}
(SystemTap installes probes on _do_fork instead of clone, but that shouldn't change anything.)
This confuses me. A couple of related questions:
Why does the syscall only return once?
If I understand the _do_fork code correctly, the process is cloned in the middle of the function. (copy_process and wake_up_new_task). Shouldn't the subsequent code run in both processes?
Does the kernel code after a syscall run in the same thread / process as the user code before the syscall?
creation of the child can fail, thus errors have to be detected and handled
the child has a different return value and this also has to be handled
it may be the parent has clean ups / additional actions to do
Thus the code would have to differentiate between executing as a parent and a child. But there are no checks of the sort, which is already a strong hint that the child does not execute this code in the first place. Thus one should look for a dedicated place new children return to.
Since the code is quite big and hairy, one can try to cheat and just look for 'fork' in arch-specific code, which quickly reveals ret_from_fork.
It is set a starting point by -> do_fork -> copy_process -> copy_thread_tls http://lxr.free-electrons.com/source/arch/x86/kernel/process_64.c#L158
Thus
Why does the syscall only return once?
It does not return once. There are 2 returning threads, except the other one uses a different code path. Since the probe is installed only on the first one, you don't see the other one. Also see below.
If I understand the _do_fork code correctly, the process is cloned in the middle of the function. (copy_process and wake_up_new_task). Shouldn't the subsequent code run in both processes?
I noted earlier this is false. The real question is what would be the benefit of making the child return in the same place as the parent. I don't see any and it would troublesome (extra special casing, as noted above). To re-state: making the child return elsehwere lets callers not have to handle the returning child. They only need to check for errors.
Does the kernel code after a syscall run in the same thread / process as the user code before the syscall?
What is 'kernel code after a syscall'? If you are thread X and enter the kernel, you are still the thread X.
I have a program in C which utilizes the fork() system call:
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
void doit(void)
{
pid_t pid;
fork();
fork();
printf("Unix System Programming\n");
return;
}
int main(void)
{
doit();
printf("WTF\n");
exit(0);
}
Now, this program gives me 8 lines of output. I think that is because of two forks 2^2 = 4 Times * 2 Print Statement = 8 Times. If I am wrong, please correct me and explain why.
Now, the question is why am I getting different outputs on each run? Let's say I execute this code: the first time I get output
Unix System Programming
WTF
Unix System Programming
Unix System Programming
Unix System Programming
WTF
WTF
WTF
and the 2nd time I get:
Unix System Programming
WTF
Unix System Programming
Unix System Programming
WTF
Unix System Programming
WTF
WTF
And third time again different. Why does this happen? I am clueless; kindly explain in detail.
When you fork a new process, the parent and child both run concurrently. The order that they execute their respective printf() statements is unpredictable -- sometimes the parent will print first, sometimes the child will.
You might understand better if you included the PID in the output, so you could see which process is printing each line. SO change it to:
printf("%d: Unix System Programming\n", getpid());
and
printf("%d: WTF\n", getpid());
What you should see is that each process prints Unix System Programming before WTF, but the order of processes will be mixed up.
The outputs that you are seeing are from different processes. Once fork has succeeded, you get a different child process (as well as the parent process). Since they are from different processes, you can't guarantee when one process gets its turn and executes. So the outputs of different processes get intermixed and the order might be different at each run.
If you want to make sure that parent and child processes run in some specific order then you have to synchronize them, which is typically a lot more work than just forking. (It will probably require a wait operation or the use of pipes — depending on the nature of synchronization that you want.)
How do you run an external program and pass it command line parameters using C? If you have to use operating system API, include a solution for Windows, Mac, and Linux.
It really depends on what you're trying to do, exactly, as it's:
OS dependent
Not quite clear what you're trying to do.
Nevertheless, I'll try to provide some information for you to decide.
On UNIX, fork() creates a clone of your process from the place where you called fork. Meaning, if I have the following process:
#include <unistd.h>
#include <stdio.h>
int main()
{
printf( "hi 2 u\n" );
int mypid = fork();
if( 0 == mypid )
printf( "lol child\n" );
else
printf( "lol parent\n" );
return( 0 );
}
The output will look as follows:
hi 2 u
lol child
lol parent
When you fork() the pid returned in the child is 0, and the pid returned in the parent is the child's pid. Notice that "hi2u" is only printed once... by the parent.
execve() and its family of functions are almost always used with fork(). execve() and the like overwrite the current stackframe with the name of the application you pass to it. execve() is almost always used with fork() where you fork a child process and if you're the parent you do whatever you need to keep doing and if you're the child you exec a new process. execve() is also almost always used with waitpid() -- waitpid takes a pid of a child process and, quite literally, waits until the child terminates and returns the child's exit status to you.
Using this information, you should be able to write a very basic shell; one that takes process names on the command line and runs processes you tell it to. Of course, shells do more than that, like piping input and output, but you should be able to accomplish the basics using fork(), execve() and waitpid().
NOTE: This is *nix specific! This will NOT work on Windows.
Hope this helped.
If you want to perform more complicated operations, like reading the output of the external program, you may be better served by the popen system call. For example, to programmatically access a directory listing (this is a somewhat silly example, but useful as an example), you could write something like this:
#include <stdio.h>
int main()
{
int entry = 1;
char line[200];
FILE* output = popen("/usr/bin/ls -1 /usr/man", "r");
while ( fgets(line, 199, output) )
{
printf("%5d: %s", entry++, line);
}
}
to give output like this
1: cat1
2: cat1b
3: cat1c
4: cat1f
5: cat1m
6: cat1s
...
#include <stdlib.h>
int main()
{
system("echo HAI");
return 0;
}
I want to give a big warning to not use system and 100% never use system when you write a library. It was designed 30 years ago when multithreading was unknown to the toy operating system called Unix. And it is still not useable even when almost all programs are multithreaded today.
Use popen or do a fork+execvp, all else is will give you hard to find problems with signal handling, crashs in environment handling code etc. It's pure evil and a shame that the selected and most rated answer is promoting the use of "system". It's more healthy to promote the use of Cocain on the workplace.
On UNIX, I think you basically need to fork it if you want the spawned process to run detached from your the spawing one : For instance if you don't want your spawned process to be terminate when you quit your spawning process.
Here is a page that explains all the subtle differences between Fork, System, Exec.
If you work on Win,Mac and linux, I can recommend you the Qt Framework and its QProcess object, but I don't know if that's an option for you. The great advantages is that you will be able to compile the same code on windows linux and mac :
QString program = "./yourspawnedprogram";
QProcess * spawnedProcess = new QProcess(parent);
spawnedProcess->start(program);
// or spawnedProcess->startDetached(program);
And for extra, you can even kill the child process from the mother process,
and keep in communication with it through a stream.
One solution is the system function defined in stdlib.h
int system(const char *string);
system api example
If you need to check/read/parse the output of your external command, I would suggest to use popen() instead of system().
Speaking of platform-dependent recipes, on Windows use CreateProcess, on Posix (Linux, Mac) use fork + execvp. But system() should cover your basic needs and is part of standard library.