I'm writing a code that mimics a shell behavior, specifically & and |.
My function receives user commands, and checks if there's an & at the end, then the child process should run in the background and the parent should not wait for it to finish and continue executing commands.
Also it's supposed to check if there's a | in the input array and run two child processes while piping their stdin and stdout.
I have implemented the behavior for &, but whenever I compile and run my code, I only get the printf sentence from the parent's process.
I would like to hear ideas how to fix this, in addition I would appreciate any suggestions regarding the implementation of | (pipes) and how to prevent zombies.
int process_arglist(int count, char** arglist) {
int pid = fork();
printf("%d", pid);
switch (pid) {
case -1:
fprintf(stderr, "ERROR: fork failed\n");
return 1;
break;
case 0: // Son's proccess
printf("I got to son");
//check last arglist argument
if (strcmp(arglist[count - 1], "&") == 0) {
setpgid(0, 0);
arglist[count - 1] = NULL;
if (execvp(*arglist, arglist) < 0) { //execute the command
fprintf(stderr, "ERROR: execvp failed\n");
exit(1);
}
} else { //There's no & at the end, look for pipes
int i = 0;
while (i < count) {
if (strcmp(arglist[i], "|") == 0) {
int pid2 = fork();
if (pid2 < 0) {
//fork failed, handle error
}
if (pid2 == 0) { // Son's proccess
} else { //Parent's code
}
}
}
}
break;
//in case no & and no |, call execvp
default: //Parent's code
printf("I go to parent");
return 1;
break;
}
return 0;
}
The output is always "I go to parent"
I assume your code is for Linux or some other POSIX system. Read some good book on Linux programming (perhaps the old Advanced Linux Programming, freely downloadable, or something newer).
stdio(3) is buffered, and stdout and printf is often (but not always) line-buffered. Buffering happens for efficiency reasons (calling write(2) very often, e.g. once per output byte, is very slow; you should prefer doing write-s on chunks of several kilobytes).
BTW you'll better handle failure of system calls (see intro(2) and syscalls(2)) by using errno(3) thru perror(3) (or strerror(3) on errno). You (and the user of your shell) needs to be informed of the failure reason (and your current code don't show it).
I recommend to often end your printf format control strings with \n (this works when stdout is line-buffered) or to call fflush(3) at appropriate places.
As a rule of thumb, I suggest doing fflush(NULL); before every call to fork(2).
The behavior you observe is consistent with the hypothesis that some printf-ed data is staying in buffers (e.g. of stdout).
You could use strace(1) on your program (or on other ones, e.g. some existing shell process) to understand what system calls are done.
You should compile with all warnings and debug info (e.g. gcc -Wall -Wextra -g with GCC), improve your code to get no warnings, and use the debugger gdb (with care, it can be used on forking processes).
I'm writing a code that mimics a shell behavior
You probably are coding some shell. Then study for inspiration the source code of existing free software shells (most -probably all- Linux shells are free software).
I would appreciate any suggestions regarding the implementation of | (pipes) and how to prevent zombies.
Explaining all that requires a lot of space (several chapters of a book, or perhaps, an entire book) and don't fit here or on any other forum. So read a good Linux or POSIX programming book. Regarding pipes, read pipe(7) (it should be created with pipe(2) before the fork). Regarding avoiding zombie processes, you need to carefully call waitpid(2) or some similar call.
Related
I've really searched for this and all I can find is that you can execvp() shell commands.
I'm wondering if I can fork processes and then have them run a function that's internal to the program? (As in, a function I've written myself within the code)
Of course you can have the child execute one function and the parent execute a different (or even the same) function in the same executable.
pid_t pid = fork();
if (pid < 0)
err_syserr("failed to fork: ");
else if (pid == 0)
be_childish();
else
be_parental();
You can add arguments to be_childish() and be_parental() as needed. Before the code executes fork(), you can create pipes or sockets to communicate between them — or semaphores, and shared memory, or whatever IPC you want.
When you call fork, a new process is created inheriting the current context of the parent process. The child and parent processes can execute independently calling any function within your program. However, if they need to communicate/synchronize with each other, they need to use one of the IPC mechanisms such as shared memories, pipes, semaphores etc.
I suspect the simplest answer here is an example:
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
static int child_foo(void)
{
/* This child process just counts to three. */
int i;
for (i = 1; i <= 3; i++) {
printf("Foo: %d\n", i);
fflush(stdout);
sleep(1);
}
printf("Foo is done.\n");
return EXIT_SUCCESS;
}
static int child_bar(unsigned long n)
{
/* This child process checks if n is prime or not. */
const unsigned long imax = (n + 1) / 2;
unsigned long i;
for (i = 2; i <= imax; i++)
if (!(n % i)) {
printf("Bar: Because %lu is divisible by %lu, %lu is not a prime.\n", n, i, n);
return EXIT_FAILURE;
}
printf("Bar: %lu is prime.\n", n);
return EXIT_SUCCESS;
}
int main(void)
{
pid_t p, foo, bar;
int status;
printf("Forking child processes.\n");
fflush(stdout);
foo = fork();
if (foo == -1) {
fprintf(stderr, "Cannot fork: %s.\n", strerror(errno));
return EXIT_FAILURE;
} else
if (!foo)
return child_foo();
bar = fork();
if (bar == -1) {
fprintf(stderr, "Cannot fork: %s.\n", strerror(errno));
/* Wait until all child processes (here, foo only) have completed. */
do {
p = wait(NULL);
} while (p != -1 || errno == EINTR);
return EXIT_FAILURE;
} else
if (!bar)
return child_bar(227869319);
/* Wait until all child processes have completed. */
do {
p = wait(&status);
if (p == foo || p == bar) {
/* Report exit status. */
if (p == foo)
printf("child_foo()");
else
printf("child_bar()");
if (WIFEXITED(status)) {
if (WEXITSTATUS(status) == EXIT_SUCCESS)
printf(" exited successfully (EXIT_SUCCESS).\n");
else
if (WEXITSTATUS(status) == EXIT_FAILURE)
printf(" exited with failure (EXIT_FAILURE).\n");
else
printf(" exited with status %d.\n", WEXITSTATUS(status));
} else
if (WIFSIGNALED(status))
printf(" died from signal %d (%s).\n", WTERMSIG(status), strsignal(WTERMSIG(status)));
else
printf(" was lost.\n");
fflush(stdout);
}
} while (p != -1 || errno == EINTR);
printf("All done.\n");
return EXIT_SUCCESS;
}
Above, main() forks two child processes. One of them runs child_foo() and exits, the other runs child_bar(227869319) and exits. The parent process reaps all child processes, and returns the cause of exit and exit status if available, then exits itself as well.
The fflush(stdout) before the fork() is there to remind that internal caches (including caching done by the C library) should be flushed before forking, or else the child process will inherit the cache contents. In the case of stdout, this would mean duplicated outputs.
In practice, it is often better for maintainability (human programmer understanding) to "return" from the child process functions using exit(EXIT_SUCCESS) or exit(EXIT_FAILURE), than return. Then, in main(), instead of say return child_foo(), you'd have say
child_foo(); /* Never returns. */
exit(EXIT_FAILURE);
where the exit(EXIT_FAILURE) is just a bug catcher, in case a modification to child_foo() causes it to return rather than exit.
In some cases the return pattern is useful, if there is cleanup work that must always be done before the child process exits. In that case, you usually put that cleanup work in a separate function, and then replace return child_foo() with
int exitstatus;
exitstatus = child_foo();
cleanup_function();
exit(exitstatus);
Note that in main(), return exitstatus; and exit(exitstatus); are exactly equivalent (C89 2.1.2.2, C99 5.1.2.2.3p1, C11 5.1.2.2.3p1). The only difference is to us humans; it may be easier for us humans to correctly interpret the intent behind exit(exitstatus) in the middle of main(), compared to return exitstatus;. Myself, I "parse" them the same, but I seem to use return more.
The strsignal() function is defined in POSIX-1.2008, and is only called if you manage to kill one of the child processes with an external signal without killing the parent as well. If your C library does not support it, just remove it (also remove the %s printf specifier, too).
As usual, the second most important thing to do, is to ensure future developers understand the intent correctly. I included pretty minimal comments, but both the function names and the initial comment in the child process functions should make the intent clear. I'm sure you can do even better. (I do consider commenting to be the hardest part in programming for me. It is so easy to write useless comments that describe what the code does, but omit the intent. When you look at even your own code months later, you forget your thoughts then, and must rely on the comments (and indirectly infer from the code) to get a proper understanding of the intent. Good comments are hard to write!)
In my opinion, the most important thing is to ensure the program is robust, and does not silently corrupt data. This means sufficient error checking (in my case, paranoid -- I really don't want to silently corrupt data) and minimal assumptions on what "ought to" succeed. Sometimes, it leads to "nests" of code, like the status checks in the wait() loop in main(). I deliberately left out the comments there, because I believe you should go through it, keeping a browser or terminal window open to man 2 wait, and add the necessary comments so you understand exactly what that nest of code does. It will definitely help you understand a lot about how processes can terminate or be terminated, and how a parent process can detect that.
You fork a process (a running, active, instance of a program) and you execve an executable (a passive file, usually in ELF format) - not some functions. Read again fork(2) and execve(2) and credentials(7). See also Operating Systems: Three Easy Pieces to understand better the role of an OS.
that you can execvp() shell commands.
Wrong. execvp calls execve and runs executables (not shell commands; e.g. you cannot execvp the cd shell builtin). It does not use the shell (but execvp search your PATH variable as your shell does).
Notice that every process has its own virtual address space. And fork creates a new process with its own new virtual address space (which happens to be a copy of the virtual address space of the parent. That copy happens lazily, read about copy-on-write techniques). execve is replacing the virtual address process by a new one (described in the executable file).
Since that new virtual address space (after a successful fork, in the child process) is a copy of the parent one, it also contains the code of every internal function you dream of from your program. So after a fork you can call these internal functions.
Use pmap(1) and proc(5) to understand the virtual address space of processes. At first, run cat /proc/$$/maps then pmap $$ in a terminal, and try to understand its output (it describes the virtual address space of your shell process).
When a process is running, you could extend its virtual address space using mmap(2). This is used by malloc, and also by dlopen(3) (which enables you to load plugins into your process).
PS. I am guessing you are using Linux.
I am currently writing my own shell implementation in C. I understood the principle behind piping and redirecting the fds. However, some specific behavior with pipes has attracted my attention:
cat | ls (or any command that does not read from stdin as final element of the pipe).
In that case, what happens in the shell is that ls executes and cat asks for a single line before exiting (resulting from a SIGPIPE I guess). I have tried to follow this tutorial to better understand the principle behind multiple pipes: http://web.cse.ohio-state.edu/~mamrak.1/CIS762/pipes_lab_notes.html
Below is some code I have written to try to replicate the behavior I am looking for:
char *cmd1[] = {"/bin/cat", NULL};
char *cmd2[] = {"/bin/ls", NULL};
int pdes[2];
pid_t child;
if (!(child = fork()))
{
pipe(pdes);
if (!fork())
{
close(pdes[0]);
dup2(pdes[1], STDOUT_FILENO);
/* cat command gets executed here */
execvp(cmd1[0], cmd1);
}
else
{
close(pdes[1]);
dup2(pdes[0], STDIN_FILENO);
/* ls command gets executed here */
execvp(cmd2[0], cmd2);
}
}
wait(NULL);
I am aware of the security flaws of that implementation but this is just for testing. The problem with that code as I understand it is that whenever ls gets executed, it just exits and then cat runs in the background somehow (and in my case fail because it tries to read during the prompt of zsh as my program exits). I cannot find a solution to make it work like it should be. Because if I wait for the commands one by one, such commands as cat /dev/random | head -c 10 would run forever...
If anyone has a solution for this issue or at least some guidance it would be greatly appreciated.
After consideration of comments from #thatotherguy here is the solution I found as implemented in my code. Please bear in mind that pipe and fork calls should be checked for errors but this version is meant to be as simple as possible. Extra exit calls are also necessary for some of my built-in commands.
void exec_pipe(t_ast *tree, t_sh *sh)
{
int pdes[2];
int status;
pid_t child_right;
pid_t child_left;
pipe(pdes);
if (!(child_left = fork()))
{
close(pdes[READ_END]);
dup2(pdes[WRITE_END], STDOUT_FILENO);
/* Execute command to the left of the tree */
exit(execute_cmd(tree->left, sh));
}
if (!(child_right = fork()))
{
close(pdes[WRITE_END]);
dup2(pdes[READ_END], STDIN_FILENO);
/* Recursive call or execution of last command */
if (tree->right->type == PIPE_NODE)
exec_pipe(tree->right, sh);
else
exit(execute_cmd(tree->right, sh));
}
/* Should not forget to close both ends of the pipe */
close(pdes[WRITE_END]);
close(pdes[READ_END]);
wait(NULL);
waitpid(child_right, &status, 0);
exit(get_status(status));
}
I was confused with the original link I posted and the different ways to handle chained pipes. From the link to the POSIX documented posted below my original question (http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_02) it appears that:
If the pipeline is not in the background (see Asynchronous Lists), the shell shall wait for the last command specified in the pipeline to complete, and may also wait for all commands to complete.
Both behavior are therefore accepted: waiting for last command, or waiting for all of them. I chose to implement the second behavior to stick to what bash/zsh would do.
I have to migrate a C-program from OpenVMS to Linux, and have now difficulties with a program generating subprocesses. A subprocess is generated (fork works fine), but execve fails (which is correct, as the wrong program name is given).
But to reset the number of active subprocesses, I afterwards call a wait() which does not return. When I look at the process via ps, I see that there are no more subprocesses, but wait() does not return ECHILD as I had thought.
while (jobs_to_be_done)
{
if (running_process_cnt < max_process_cnt)
{
if ((pid = vfork()) == 0)
{
params[0] = param1 ;
params[1] = NULL ;
if ((cstatus = execv(command, params)) == -1)
{
perror("Child - Exec failed") ; // this happens
exit(EXIT_FAILURE) ;
}
}
else if (pid < 0)
{
printf("\nMain - Child process failed") ;
}
else
{
running_process_cnt++ ;
}
}
else // no more free process slot, wait
{
if ((pid = wait(&cstatus)) == -1) // does not return from this statement
{
if (errno != ECHILD)
{
perror("Main: Wait failed") ;
}
anz_sub = 0 ;
}
else
{
...
}
}
}
Is the anything that has to be done to tell the wait-command that there are no more subprocesses?
With OpenVMS the program works fine.
Thanks a lot in advance for your help
I don't recommend using vfork these days on Linux, since fork(2) is efficient enough, thanks to lazy copy-on-write techniques in the Linux kernel.
You should check the result of fork. Unless it is failing, a process has been created, and wait (or waitpid(2), perhaps with WNOHANG if you don't want to really wait, but just find out about already ended child processes ...) should not fail (even if the exec function in the child has failed, the fork did succeed).
You might also carefully use the SIGCHLD signal, see signal(7). A defensive way of using signals is to set some volatile sigatomic_t flag in signal handlers, and test and clear these flags inside your loop. Recall that only async signal safe functions (and there are quite few of them) can be called -even indirectly- inside a signal handler. Read also about POSIX signals.
Take time to read Advanced Linux Programming to get a wider picture in your mind. Don't try to mimic OpenVMS on POSIX, but think in a POSIX or Linux way!
You probably may want to always waitpid in your loop, perhaps (sometimes or always) with WNOHANG. So waitpid should not be only called in the else part of your if (running_process_cnt < max_process_cnt) but probably in every iteration of your loop.
You might want to compile with all warnings & debug info (gcc -Wall -Wextra -g) then use the gdb debugger. You could also strace(1) your program (probably with -f)
You might want to learn about memory overcommitment. I dislike this feature and usually disable it (e.g. by running echo 0 > /proc/sys/vm/overcommit_memory as root). See also proc(5) -which is very useful to know about...
From man vfork:
The child must not return from the current function or call exit(3), but may call _exit(2)
You must not call exit() when the call to execv (after vfork) fails - you must use _exit() instead. It is quite possible that this alone is causing the problem you see with wait not returning.
I suggest you use fork instead of vfork. It's much easier and safer to use.
If that alone doesn't solve the problem, you need to do some debugging or reduce the code down until you find the cause. For example the following should run without hanging:
#include <sys/wait.h>
int main(int argc, char ** argv)
{
pid_t pid;
int cstatus;
pid = wait(&cstatus);
return 0;
}
If you can verify that this program doesn't hang, then it must be some aspect of your program that is causing a hang. I suggest putting in print statements just before and after the call to wait.
This is the relevant snippet of code:
if(!strcmp(args[0],"run")){
pid_t pid = fork();
if(pid == 0){
execvp(args[1], args);
fprintf(stdout, "Process could not be found in this directory\n");
kill((int)getpid(), SIGKILL);
}
else{
if(pid < 0)
exit(1);
fprintf(stdout, "PID of Process = %d\n", pid);
int success = waitpid(pid, &childExitStatus, WUNTRACED | WCONTINUED);
if(success == -1)
exit(EXIT_FAILURE);
}
}
Now when I run a process like Libre Office math, it will only open one instance of it. However, when I try to open xterm using this code, it will continue to exec over and over, creating many instances of xterm before I do an interrupt and exit. I don't see any loops that would cause this to happen. Any insight into why this would be?
The execvp() call is incorrect because it passes an additional argument "run" at the start. When executing xterm in this manner, the effect is similar to xterm xterm and has xterm use xterm as the shell. The new xterm inherits a SHELL environment variable that causes it to start another xterm, until limits are exhausted.
In addition to jilles' answer to your immediate problem, you should know that calling fprintf if execvp fails is unwise as it can lead to duplication of previously-buffered output. It also means you cannot safely use vfork, which can still be a valuable optimization (particularly if the parent process uses a lot of memory).
There is no good way to report an exec failure, unfortunately. I tend to just do
if (pid == 0) {
execvp(...);
/* if we got here, something bad happened */
_exit(127);
}
This question already has an answer here:
fork(), problems with multiple children
(1 answer)
Closed 8 years ago.
The code :
for ( ii = 0; ii < 24; ++ii) {
switch (fork()) {
case -1 : {
printf("\n\nproblem with fork() !!! \n\n");
exit(0);
};
case 0 : {
WriteOnShared_Mem(ii);
}break;
default : {
ChildPidTab[ii] = p;
usleep(50000);
ReadShared_MemMp(nbSect, 24,ChildPidTab);
};
}
}
My problem is that i get too many child (nbenfant = 24), i got much more than 24 :/
This is my 3rd post today here but still not solved :(
Thanks
Read carefully the fork(2) man page. Read that page several times, it is hard to understand. Read also the wikipage on fork (system call) and on processes (computing).
Please understand -and that takes time- that fork is returning simultaneously twice on success: once in the parent and once in the child
The fork syscall can fail (and then returns -1) for a number of reasons. On failure of fork please use perror or some other way to show the errno. And you should always keep the result of fork. So code
for (ii = 0; ii < 24; ++ii) {
fflush(NULL);
pid_t p = fork();
switch (p) {
case -1 : // fork failed
printf("\n\nproblem with fork() in pid %d error %s!!! \n\n",
(int) getpid(), strerror(errno));
exit(EXIT_FAILURE);
break;
case 0: // child process
WriteOnShared_Mem(ii);
ii = MAX_INT; // to stop the for loop
break;
default: // parent process
ChildPidTab[ii] = p;
/// etc.... some synchronization is needed
break;
}
In particular, fork can fail because
EAGAIN fork() cannot allocate sufficient memory to copy the
parent's page tables and allocate a task structure for
the child.
EAGAIN It was not possible to create a new process because the
caller's RLIMIT_NPROC resource limit was encountered. To
exceed this limit, the process must have either the
CAP_SYS_ADMIN or the CAP_SYS_RESOURCE capability.
ENOMEM fork() failed to allocate the necessary kernel structures
because memory is tight.
If you want to be able to fork more processes, try to:
increase the RLIMIT_NPROC resource limit with setrlimit(2) (which might be called by system facilities, so look also into /etc/pam.d/login etc
lower the resources required by the fork-ing program. In particular, lower the heap memory requirements
increase some system resources, like perhaps swap. You could swapon some temporary file for testing.
As Joachim Pileborg replied you should avoid forking too much (the forked process continues the loop so is also forking again).
Don't forget that stdio routines are buffered. Use fflush(3) appropriately.
I suggest reading the Advanced Linux Programming book (available online) which has a full chapter explaining process handling on Linux.
BTW, check with ps or top or pstree how many processes you have (and with the free command how much memory is used, but read http://linuxatemyram.com/ before complaining). It could happen that your particular system is not able to fork more than 24 times your particular program (because of lack of resources)
Study also the source code of simple shells (like sash) and use strace -f (e.g. on some shell, or on your program) to understand more what syscalls are done. Also learn how to use the gdb debugger.
It's because each child continues with the loop and so in turn fork their own children. When the children are done, you should either return from the main function or call exit.
The child process coutinue fork new child process, you just need stop it .
Like this:
switch (fork()) {
case -1 : {
printf("\n\nproblem with fork() !!! \n\n");
exit(0);
};
case 0 : {
i = 24 ; //--- here I set i = 24 , so child process will stop fork new child process.
WriteOnShared_Mem(ii);
}break;
default : {
ChildPidTab[ii] = p;
usleep(50000);
ReadShared_MemMp(nbSect, 24,ChildPidTab);
};
}
}