the man page says that "The exec() family of functions replaces the current process image with a new process image." but I am not quite understand the meaning of "replaces the current process image with a new process image". For example, if exec succeed, perror would not be reached
execl("/bin/ls", /* Remaining items sent to ls*/ "/bin/ls", ".", (char *) NULL);
perror("exec failed");
Correct. If the exec works, the perror will not be called, simply because the call to perror no longer exists.
I find it's sometimes easier when educating newcomers to these concepts, to think of the UNIX execution model as being comprised of processes, programs and program instances.
Programs are executable files such as /bin/ls or /sbin/fdisk (note that this doesn't include things like bash or Python scripts since, in that case, the actual executable would be the bash or python interpreter, not the script).
Program instances are programs that have been loaded into memory and are basically running. While there is only one program like /bin/ls, there may be multiple instances of it running at any given time if, for example, both you and I run it concurrently.
That "loaded into memory" phrase is where processes come into the picture. Processes are just "containers" in which instances of programs can run.
So, when you fork a process, you end up with two distinct processes but they're still each running distinct instances of the same program. The fork call is often referred to as one which one process calls but two processes return from.
Likewise, exec will not have an effect on the process itself but it will discard the current program instance in that process and start a new instance of the requested program.
This discard in a successful exec call is what dictates that the code following it (perror in this case) will not be called.
It means your current process becomes the new process instead of what it was. You stop doing what you're doing and start doing,really being, something else instead, never to rebecome what that process once was.
Instead of starting a whole new process, however, your current pid and environment become the new process instead. That let's you setup things the way the new process will need it before doing the exec
You are correct. perror will not be called unless the execl fails. The exec functions are the means for starting new processes in a POSIX compliant OS (typically combined with a fork call). Maybe an example will help. Suppose your program, call it programX, is running. It then calls one of the exec functions like the one you have above. programX will no longer exist as a running process. Instead, ls will be running. It will have the same exact PID as programX, but pretty much be a whole new process otherwise.
Related
Is there a way to start a child process without fork(), using execvp() exclusively?
The pedantic answer to your question is no. The only system call that creates a new process is fork. The system call underlying execvp (called execve) loads a new program into an existing process, which is a different thing.
Some species of Unix have additional system calls besides fork (e.g. vfork, rfork, clone) that create a new process, but they are only small variations on fork itself, and none of them are part of the POSIX standard that specifies the functionality you can count on on anything that calls itself a Unix.
The slightly more helpful answer is that you might be looking for posix_spawn, which is a library routine wrapping fork and exec into a single operation, but I find it more troublesome to use that correctly than to write my own fork+exec subroutine. YMMV.
posix_spawn is the only posix compliant way to create a child process without calling fork directly. I say 'directly' because historically posix_spawn would itself just call fork, or vfork. However, that is no longer the case in GNU/linux. posix_spawn itself may be more efficient than fork, in addition to perhaps being a stronger fit conceptually when code is attempting to run a different executable.
If you aren't worried about portability, you can abandon posix and couple yourself directly to the kernel you are targeting. On linux the system call to create a child process is clone. At the time of this answer the manual page provides documentation for three variants, including the relatively new clone3.
I believe you can take the example from the manual page and add an execvp call to childFunc. I have not tried it yet, though!
Unlike Windows systems, where creating a new process and executing a new process image happen in a single step, Linux and other UNIX-like systems do them as two distinct steps.
The fork function makes an exact duplicate of the calling process and actually returns twice, once to the parent process and once to the child process. The execvp function (and other functions in the exec family) executes a new process image in the same process, overwriting the existing process image.
You can call execvp without calling fork first. If so, that just means the currently running program goes away and is replaced with the given program. However, fork is the way to create a new process.
As user zwol has already explained, execve() does not fork a new process. Rather, it replaces the address space and CPU state of current process,
loads the new address space from the executable filename and starts it from
main() with argument list argv and environment variable list envp.
It keeps pid and open files.
int execve(const char *filename,char *const argv [],char *const envp[]);
filename: name of executable file to run
argv: Command line arguments
envp: environment variable settings (e.g., $PATH, $HOME, etc.)
posix_spawn. But it ignores failures of execvp() -- potentially because implementing this was regarded as too complicated.
the man page says that "The exec() family of functions replaces the current process image with a new process image." but I am not quite understand the meaning of "replaces the current process image with a new process image". For example, if exec succeed, perror would not be reached
execl("/bin/ls", /* Remaining items sent to ls*/ "/bin/ls", ".", (char *) NULL);
perror("exec failed");
Correct. If the exec works, the perror will not be called, simply because the call to perror no longer exists.
I find it's sometimes easier when educating newcomers to these concepts, to think of the UNIX execution model as being comprised of processes, programs and program instances.
Programs are executable files such as /bin/ls or /sbin/fdisk (note that this doesn't include things like bash or Python scripts since, in that case, the actual executable would be the bash or python interpreter, not the script).
Program instances are programs that have been loaded into memory and are basically running. While there is only one program like /bin/ls, there may be multiple instances of it running at any given time if, for example, both you and I run it concurrently.
That "loaded into memory" phrase is where processes come into the picture. Processes are just "containers" in which instances of programs can run.
So, when you fork a process, you end up with two distinct processes but they're still each running distinct instances of the same program. The fork call is often referred to as one which one process calls but two processes return from.
Likewise, exec will not have an effect on the process itself but it will discard the current program instance in that process and start a new instance of the requested program.
This discard in a successful exec call is what dictates that the code following it (perror in this case) will not be called.
It means your current process becomes the new process instead of what it was. You stop doing what you're doing and start doing,really being, something else instead, never to rebecome what that process once was.
Instead of starting a whole new process, however, your current pid and environment become the new process instead. That let's you setup things the way the new process will need it before doing the exec
You are correct. perror will not be called unless the execl fails. The exec functions are the means for starting new processes in a POSIX compliant OS (typically combined with a fork call). Maybe an example will help. Suppose your program, call it programX, is running. It then calls one of the exec functions like the one you have above. programX will no longer exist as a running process. Instead, ls will be running. It will have the same exact PID as programX, but pretty much be a whole new process otherwise.
the man page says that "The exec() family of functions replaces the current process image with a new process image." but I am not quite understand the meaning of "replaces the current process image with a new process image". For example, if exec succeed, perror would not be reached
execl("/bin/ls", /* Remaining items sent to ls*/ "/bin/ls", ".", (char *) NULL);
perror("exec failed");
Correct. If the exec works, the perror will not be called, simply because the call to perror no longer exists.
I find it's sometimes easier when educating newcomers to these concepts, to think of the UNIX execution model as being comprised of processes, programs and program instances.
Programs are executable files such as /bin/ls or /sbin/fdisk (note that this doesn't include things like bash or Python scripts since, in that case, the actual executable would be the bash or python interpreter, not the script).
Program instances are programs that have been loaded into memory and are basically running. While there is only one program like /bin/ls, there may be multiple instances of it running at any given time if, for example, both you and I run it concurrently.
That "loaded into memory" phrase is where processes come into the picture. Processes are just "containers" in which instances of programs can run.
So, when you fork a process, you end up with two distinct processes but they're still each running distinct instances of the same program. The fork call is often referred to as one which one process calls but two processes return from.
Likewise, exec will not have an effect on the process itself but it will discard the current program instance in that process and start a new instance of the requested program.
This discard in a successful exec call is what dictates that the code following it (perror in this case) will not be called.
It means your current process becomes the new process instead of what it was. You stop doing what you're doing and start doing,really being, something else instead, never to rebecome what that process once was.
Instead of starting a whole new process, however, your current pid and environment become the new process instead. That let's you setup things the way the new process will need it before doing the exec
You are correct. perror will not be called unless the execl fails. The exec functions are the means for starting new processes in a POSIX compliant OS (typically combined with a fork call). Maybe an example will help. Suppose your program, call it programX, is running. It then calls one of the exec functions like the one you have above. programX will no longer exist as a running process. Instead, ls will be running. It will have the same exact PID as programX, but pretty much be a whole new process otherwise.
In *nix systems, processes are created by using fork() system call. Consider for example, init process creates another process.. First it forks itself and creates the a process which has the context like init. Only on calling exec(), this child process turns out to be a new process. So why is the intermediate step ( of creating a child with same context as parent ) needed? Isn't that a waste of time and resource, because we are creating a context ( consumes time and wastes memory ) and then over writing it?
Why is this not implemented as allocating a vacant memory area and then calling exec()? This would save time and resources right?
The intermediate step enables you to set up shared resources in the child process without the external program being aware of it. The canonical example is constructing a pipe:
// read output of "ls"
// (error checking omitted for brevity)
int pipe_fd[2];
pipe(&pipe_fd);
if (fork() == 0) { // child:
close(pipe_fd[0]); // we don't want to read from the pipe
dup2(pipe_fd[1], 1); // redirect stdout to the write end of the pipe
execlp("ls", "ls", (char *) NULL);
_exit(127); // in case exec fails
}
// parent:
close(pipe_fd[1]);
fp = fdopen(pipe_fd[0], "r");
while (!feof(fp)) {
char line[256];
fgets(line, sizeof line, fp);
...
}
Note how the redirection of standard output to the pipe is done in the child, between fork and exec. Of course, for this simple case, there could be a spawning API that would simply do this automatically, given the proper parameters. But the fork() design enables arbitrary manipulation of per-process resources in the child — one can close unwanted file descriptors, modify per-process limits, drop privileges, manipulate signal masks, and so on. Without fork(), the API for spawning processes would end up either extremely fat or not very useful. And indeed, the process spawning calls of competing operating systems typically fall somewhere in between.
As for the waste of memory, it is avoided with the copy on write technique. fork() doesn't allocate new memory for the child process, but points the child to the parent's memory, with the instructions to make a copy of a page only if the page is ever written to. This makes fork() not only memory-efficient, but also fast, because it only needs to copy a "table of contents".
This is an old complaint. Many people have asked Why fork() first? and typically they suggest an operation that will both create a new process from scratch and run a program in it. This operation is called something like spawn().
And they always say, Won't that be faster?
And in fact, every system other than the Unix family does go the "spawn" way. Only Unix is based on fork() and exec().
But it's funny, Unix has always been much faster than other full-featured systems. It has always handled way more users and load.
And Unix has been made even faster over the years. Fork() no longer really duplicates the address space, it just shares it using a technique called copy-on-write. (A very old fork optimization called vfork() is also still around.)
Drink the Kool-Aid.
I don't know exactly how the init process works on a kernel in terms of forking but to answer you question of why you need to call fork then exec is simply because once you exec there is no turning back.
If you check out the documentation here, it essentially requires a new process to be spawned (the fork call) in order for the parent process to resume control and either wait for it to finish or sit as a daemon probably would.
Only on calling exec(), this child process turns out to be a new
process.
Not really. After a fork, you already have new process, even not that much different from its parent. There are some cases where no exec need to follow a fork.
So why is the intermediate step ( of creating a child with same
context as parent ) needed?
One reason would be because it is an efficient way to create the whole shebang. Cloning is usually less complex than creating from scratch.
Isn't that a waste of time and resource, because we are creating a
context ( consumes time and wastes memory ) and then over writing it?
It is not a waste of time and resource as most of this resource is virtual, due to the copy on write mechanism used. Moreover, it is incorrect to state the created context is overwritten. Nothing is rewritten given the fact nothing was actually written in the first place. That's the whole point of COW. "Only" the process address space (code, heap and stack) are substituted, not overwritten. A lot of the process context is partially or totally preserved, including environment, file descriptors, priority, ignored signals, current and root directory, limits, various masks, processor bindings, privileges and several other things foreign to the process address space.
What are the reasons that an exec (execl,execlp, etc.) can fail? If you make a call to exec and it returns, are there any best practices other than just panicking and calling exit?
The problem with handling exec failure is that usually exec is performed in a child process, and you want to do the error handling in the parent process. But you can't just exit(errno) because (1) you don't know if error codes fit in an exit code, and (2), you can't distinguish between failure to exec and failure exit codes from the new program you exec.
The best solution I know is using pipes to communicate the success or failure of exec:
Before forking, open a pipe in the parent process.
After forking, the parent closes the writing end of the pipe and reads from the reading end.
The child closes the reading end and sets the close-on-exec flag for the writing end.
The child calls exec.
If exec fails, the child writes the error code back to the parent using the pipe, then exits.
The parent reads eof (a zero-length read) if the child successfully performed exec, since close-on-exec made successful exec close the writing end of the pipe. Or, if exec failed, the parent reads the error code and can proceed accordingly. Either way, the parent blocks until the child calls exec.
The parent closes the reading end of the pipe.
From the exec(3) man page:
The execl(), execle(), execlp(), execvp(), and execvP() functions may fail and set errno for any of the errors specified for the library functions execve(2) and malloc(3).
The execv() function may fail and set errno for any of the errors specified for the library function execve(2).
And then from the execve(2) man page:
ERRORS
Execve() will fail and return to the calling process if:
[E2BIG] - The number of bytes in the new process's argument list is larger than the system-imposed limit. This limit is specified by the sysctl(3) MIB variable KERN_ARGMAX.
[EACCES] - Search permission is denied for a component of the path prefix.
[EACCES] - The new process file is not an ordinary file.
[EACCES] - The new process file mode denies execute permission.
[EACCES] - The new process file is on a filesystem mounted with execution disabled (MNT_NOEXEC in <sys/mount.h>).
[EFAULT] - The new process file is not as long as indicated by the size values in its header.
[EFAULT] - Path, argv, or envp point to an illegal address.
[EIO] - An I/O error occurred while reading from the file system.
[ELOOP] - Too many symbolic links were encountered in translating the pathname. This is taken to be indicative of a looping symbolic link.
[ENAMETOOLONG] - A component of a pathname exceeded {NAME_MAX} characters, or an entire path name exceeded {PATH_MAX} characters.
[ENOENT] - The new process file does not exist.
[ENOEXEC] - The new process file has the appropriate access permission, but has an unrecognized format (e.g., an invalid magic number in its header).
[ENOMEM] - The new process requires more virtual memory than is allowed by the imposed maximum (getrlimit(2)).
[ENOTDIR] - A component of the path prefix is not a directory.
[ETXTBSY] - The new process file is a pure procedure (shared text) file that is currently open for writing or reading by some process.
malloc() is a lot less complicated, and uses only ENOMEM. From the malloc(3) man page:
If successful, calloc(), malloc(), realloc(), reallocf(), and valloc() functions return a pointer to allocated memory. If there is an error, they return a NULL pointer and set errno to ENOMEM.
What you do after the exec() call returns depends on the context - what the program is supposed to do, what the error is, and what you might be able to do to work around the problem.
One source of trouble could be that you specified a simple program name instead of a pathname; maybe you could retry with execvp(), or convert the command into an invocation of sh -c 'what you originally specified'. Whether any of these is reasonable depends on the application. If there are major security issues involved, probably you don't try again.
If you specified a pathname and there is a problem with that (ENOTDIR, ENOENT, EPERM), then you may not have any sensible fallback, but you can report the error meaningfully.
In the old days (10+ years ago), some systems did not support the '#!' shebang notation, and if you were not sure whether you were executing an executable or a shell script, you tried it as an executable and then retried it as a shell script. That might or might not work if you were running a Perl script, but in those days, you wrote your Perl scripts to detect that they were being run by a shell and to re-exec themselves with Perl. Fortunately, those days are mostly over.
To the extent possible, it is important to ensure that the process reports the problem so that it can be traced - writing its message to a log file or just to stderr (or maybe even syslog()), so that those who have to work out what went wrong have more information to help them other than the hapless end user's report "I tried X and it didn't work". It is crucial that if nothing works, then the exit status is not 0 as that indicates success. Even that might be ignored - but you did what you could.
Other than just panicking, you could take a decision based on errno's value.
Exec should always succeed
(except for shells, e.g. if the user entered a bogus command).
If exec does fail, it indicates:
a "fault" with the program (missing or bad component, wrong pathname, bad memory, ...), or
a serious system error (out of memory, too many processes, disk fault, ...)
For any serious error, the normal approach is to write the error message on stderr, then exit with a failure code. Almost all of the standard tools do this. For exec:
execl("bork", "bork", NULL);
perror("failed: exec");
exit(127);
The shell does that, too (more or less).
Normally if a child process fails, the parent has failed too and should exit. It does not matter whether the child failed in exec, or while running the program. If exec failed, it does not matter why exec failed. If the child process failed for any reason, the calling process is in trouble and needs to stop.
Don't waste lots of time trying to anticipate all possible error conditions. Don't write code that tries to handle each error code in the best possible way. You'll just bloat the code, and introduce many new bugs. If your program is broken, or it's being abused, it should simply fail. If you force it to continue, worse trouble will come of that.
For example, if the system is out of memory and thrashing swap, we don't want to cycle over and over trying to run a process; it would just make the situation worse. If we get a filesystem error, we don't want to continue running on that filesystem; it might make the corruption worse. If the program was installed wrongly, or has a bug, or has memory corruption, we want to stop as soon as possible, before that broken program does some real damage (such as sending a corrupted report to a client, trashing a database, ...).
One possible alternative: a failing process might call for help, pause itself (SIGSTOP), then retry the operation if told to continue. This could help when the system is out of memory, or disks are full, or perhaps even if there is a fault in the program. Few operations are so expensive and important that this would be worthwhile.
If you're making an interactive GUI program, try to do it as a thin wrapper over reusable command-line tools (which exit if something goes wrong). Every function in your program should be accessible through the GUI, through the command-line, and as a function call. Write your functions. Write a few tools to make commmand-line and GUI wrappers for any function. Use sub-processes too.
If you are making a truly critical system, such as a controller for a nuclear power station, or a program to predict tsunamis, then what are you doing reading my dumb advice? Critical systems should not depend entirely on computers or software. There needs to be a 'manual override', with someone to drive it. Especially, do not attempt to build a critical system on MS Windows; that is like building sandcastles underwater.