I'm writing a program in C on Linux which includes a module that
allows a shell command to be executed on a remote machine. The
easiest way to actually execute the command would of course be to
simply use the system() function, or use popen and then grab the
output. However, I chose to use a more low-level approach due to other design requirements which are not relevant to the current
problem.
So basically, I set up a pipe and fork, and then call execl. This all
works perfectly, except for one annoying exception. It doesn't work
properly if the shell command to be executed is a daemon. In that
case, it just hangs. I can't figure out why. My understanding is
that when a daemon starts, it typically forks and then the parent exits. Since my application has an open pipe to the parent, the call
to read() should fail when the parent exits. But instead
the application just hangs.
Here is some bare bones code that reproduces the problem:
int main(int argc, char** argv)
{
// Create a pipe and fork
//
int fd[2];
int p = pipe(fd);
pid_t pid = fork();
if (pid > 0)
{
// Read from the pipe and output the result
//
close(fd[1]);
char buf[1024] = { 0 };
read(fd[0], buf, sizeof(buf));
printf("%s\n", buf);
// Wait for child to terminate
int status;
wait(&status);
}
else if (pid == 0)
{
// Redirect stdout and stderr to the pipe and execute the shell
// command
//
dup2(fd[1], STDOUT_FILENO);
dup2(fd[1], STDERR_FILENO);
close(fd[0]);
execl("/bin/sh", "sh", "-c", argv[1], 0);
}
}
The code works fine if you use it with a normal shell command. But if
you try to run a daemon, it just hangs instead of returning to the
prompt as it should.
The most probable solution is adding close(fd[1]); above the execl().
The reason why your program hangs is that the read() function waits for the daemon to write something to its stdout/stderr. If the daemon (including the child process of your program, and also the child process' forked children who keep their stdout/stderr) doesn't write anything and there is at least one process holding the writable end of the pipe open, read() will never return. But which is that process, which is holding the writable end of the pipe open? It is most probably the child of your program's child, the long-running daemon process. Although it may have called close(0); and close(1); when daemonizing itself, most probably it hasn't called close(fd[1]);, so the writable end of the pipe is still open.
Your problem is proably here:-
// Wait for child to terminate
int status;
wait(&status);
As the child process is a deamon it wont terminate anytime soon.
Also your "read()" is likely to hang. You are going to have to decide how long you wait before abandoning any attempt to display output.
As the child process is a deamon it wont terminate anytime soon.
Are you sure? Of course I would agree that a daemon won't terminate anytime soon - but when a daemon starts up it forks so the child can disassociate itself with the terminal, and then the parent exits. Since the wait() system call is waiting on the parent daemon process, it should exit.
Regardless, the same problem occurs without the call to wait().
Also, why doesn't the read() get an EOF? The read() is reading from an open pipe that is connected with the parent daemon process. So when the parent daemon process exits, the read() should return immediately with an EOF.
I think you should receive the SIGPIPE signal when waiting on the read to complete, since the other end of the pipe is closed. Did you make anything unusual with the signal ? I suggest you run your code with the strace command.
Related
I have been tasked with creating my own shell in c. I am to use fork(), pipe(), exec(), and wait() to achieve this. I have a good start, but the more I research about pipes, the more confused I get. Every example of piping to a child processes looks like this:
I completely understand this. I have implemented it before. My problem is with how simple the example is. In creating a shell, I need two children to communicate with each other through a pipe in order to run a command like "cat file | grep hello". I can imagine a few ways of doing this. This was my first idea:
This doesn't seem to work. I could just be that my code is flawed, but I suspect my
understanding of pipes and file descriptors is insufficient. I figured that since pipe was called in main() and fd[] is a file variable, this strategy should work. The Linux manual states "At the time of fork() both memory spaces have the same content." Surely my child processes can access the pipe through the same file descriptors.
Is there a flaw in my understanding? I could try to make the processes run concurrently like so:
But I'm not sure why this would behave differently.
Question: If a process writes to a pipe, but there is no immediate second process to read that data, does the data get lost?
Most examples online show that each process needs to close the end of the pipe that it is not using. However, occasionally I see an example that closed both ends of the pipe in both processes:
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
As best I can tell, dup2 duplicates the file descriptor, making 2 open file descriptors to the same file. If I don't close BOTH, then execvp() continues to expect input and never exits. This means that when I am done with the reading, I should close(stdin).
Question: With 2 children communicating over a pipe, does the main process need anything with the pipe, such as close(fd[0])?
If a process writes to a pipe, but there is no immediate second process to read that data, does the data get lost?
No.
By default, writing to a pipe is a blocking action. That is, writing to a pipe will block execution of the calling process until there is enough room in the pipe to write the requested data.
The responsibility is on the reading side to drain the pipe to make room, or close their side of the pipe to signal they no longer wish to receive data.
With 2 children communicating over a pipe, does the main process need anything with the pipe, such as close(fd[0])?
Each process involved will have its own copy of the file descriptors.
As such, the parent process should close both ends of the pipe (after both forks), since it has no reason to hold onto those file descriptors. Failing to do so could result in the parent process running out of file descriptors (ulimit -n).
Your understanding of dup2 appears to be correct.
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
Both ends of the pipe are closed because after dup2 the file descriptor usually associated with stdin now refers to the same file description that the file descriptor for the read end of the pipe does.
stdin is of course closed closed when the replacement process image (exec*) exits.
Your second example of forking two processes, where they run concurrently, is the correct understanding.
In your typical shell, piped commands run concurrently. Otherwise, as stated earlier, the writer may fill the pipe and block before completing its task.
Generally, the parent waits for both processes to finish.
Here's a toy example. Run as ./program FILE STRING to emulate cat FILE | grep STRING.
#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>
int main(int argc, char **argv) {
int fds[2];
pipe(fds);
int left = fork();
if (0 == left) {
close(fds[0]);
dup2(fds[1], fileno(stdout));
close(fds[1]);
execlp("cat", "cat", argv[1], NULL);
return 1;
}
int right = fork();
if (0 == right) {
close(fds[1]);
dup2(fds[0], fileno(stdin));
close(fds[0]);
execlp("grep", "grep", argv[2], NULL);
return 1;
}
close(fds[0]);
close(fds[1]);
waitpid(left, NULL, 0);
waitpid(right, NULL, 0);
}
I am writing a shell in C and I am trying to add signal handling. In the shell, fork() is called and the child process executes a shell command. The child process is put into its own process group. This way, if Ctrl-C is pressed when a child process is in the foreground, it closes all of the processes that share the same process group id. The shell executes the commands as expected.
The problem is the signals. When, for example, I execute "sleep 5", and then I press Ctrl-C for SIGINT, the "shell>" prompt comes up as expected but the process is still running in the background. If I quickly run "ps" after I press Ctrl-C, the sleep call is still there. Then after the 5 seconds are up and I run "ps" again, it's gone. The same thing happens when I press Ctrl-Z (SIGTSTP). With SIGTSTP, the process goes to the background, as expected, but it doesn't pause execution. It keeps running until it's finished.
Why are these processes being sent to the background like this and continuing to run?
Here is the gist of my code...
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
int status;
void sig_handler_parent()
{
printf("\n");
}
void sig_handler_sigchild(int signum)
{
waitpid(-1, &status, WNOHANG);
}
int main()
{
signal(SIGCHLD, sig_handler_sigchild);
signal(SIGINT, sig_handler_parent);
signal(SIGQUIT, sig_handler_parent);
signal(SIGTERM, sig_handler_parent);
signal(SIGCONT, sig_handler_parent);
signal(SIGTSTP, sig_handler_parent);
while (1)
{
printf("shell> ");
// GET COMMAND INPUT HERE
pid = fork();
if (pid == 0)
{
setpgid(getpid(), getpid());
execvp(cmd[0], cmd);
printf("%s: unknown command\n", cmd[0]);
exit(1);
}
else
waitpid(0, &status, WUNTRACED);
}
return 0;
}
p.s. I have already tried setting all of the signal handlers to SIG_DFL before the exec command.
The code you provide does not compile, and an attempt to fix it shows
that you omitted a lot. I am only guessing.
In order to bring you forward, I'll point out a number of facts that
you might have misunderstood. Together with a couple of documentation
links, I hope this is helpful.
Error Handling
First: please make a habit of handling errors, especially when you
know there's something that you don't understand. For example, the
parent (your shell) waits until the child terminates,
waitpid(0, &status, WUNTRACED);
You say,
When, for example, I execute "sleep 5", and then I press Ctrl-C for
SIGINT, the "shell>" prompt comes up as expected but the process is
still running in the background.
What actually happens is that once you press Ctrl-C, the parent (not the
child; see below for why) receives SIGINT (the kernel's terminal
subsystem handles keyboard input, sees that someone holds "Ctrl" and
"C" at the same time, and concludes that all processes with that
controlling terminal must be sent SIGINT).
Change the parent branch to,
int error = waitpid(0, &status, WUNTRACED);
if (error != 0)
perror("waitpid");
With this, you'd see perror() print something like:
waitpid: interrupted system call
You want SIGINT to go to the child, so something must be wrong.
Signal Handlers, fork(), and exec()
Next, what happens to your signal handlers across fork() and
exec()?
The signal overview man
page states,
A child created via fork(2) inherits a copy of its parent's signal
dispositions. During an execve(2), the dispositions of handled
signals are reset to the default; the dispositions of ignored
signals are left unchanged.
So, ideally, what this means is that:
The parent (shell) sees SIGINT, as observed above, and prints
"interrupted system call".
The child's signal handlers are reset back to their defaults. For SIGINT,
this means to terminate.
You do not fiddle with the controlling terminal, so the child
inherits the controlling terminal of the parent. This means that
SIGINT is delivered to both parent and child. Given that the
child's SIGINT behavior is to terminate, I'd bet that no process is
left running.
Except when you use setpgid() to create a new process group.
Process Groups, Sessions, and Controlling Terminal
Someone once called me a UNIX greybeard. While this is true form a
visual point of view, I must reject that compliment because I rarely
hang around in one of the darkest corners of UNIX - the terminal
subsystem. Shell writers have to understand that too though.
In this context, it's the "NOTES" section of the setpgid() man
page. I suggest
you read that, especially where it says,
At any time, one (and only one) of the process groups in the session
can be the foreground process group for the terminal; (...)
The shell (bash maybe) from which you start your shell program has
done so for the foreground invocation of your program, and marked that
as "foreground process group". Effectively this means, "Please, dear
terminal, whenever someone presses Ctrl-C, send a SIGINT to all
processes in that group. I (your parent) just sit and wait (waitpid()) until all is over, and will take control again then.".
You create a process group for the child, but don't tell the terminal
about it. What you want is to
Detach the parent from the terminal.
Set the child process group as the foregroud process group of the terminal.
Wait for the child (you already do).
Regain terminal foreground.
Further down in the "NOTES" section of said man page, they give links
to how that is done. Follow those, read thoroughly, try out things,
and make sure you handle errors. In most cases, such errors are signs
of misunderstanding. And, in most cases, such errors are fixed by
re-reading the documentation.
Are you sure that your child process is actually receiving the signals from your tty? I believe you need to make a call to tcsetpgrp to actually tell the controlling terminal to send signals to the process group of your child process.
For example, after you call fork, and before exec, try this from within your child.
tcsetpgrp(STDIN_FILENO, getpid())
Here is the man page for tcsetpgrp(3)
I am writing a Linux application. What happens if I call fork() and then run an application that takes console input? Consider the code below:
int process_id = fork();
if (process_id != 0) {
/* this is the parent process */
error = execv("../my_other_app", "parameter1", NULL);
if (error < 0) {
printf("error!");
}
} else {
/* this is the child process. Wait for my_other_app to set up */
sleep(3);
/* now continue */
}
printf("########## press ENTER to stop ##########\n");
getchar();
exit(0);
The thing is, my_other_app also has a press ENTER to stop message. So when I do the getchar() call, which application is reading it? The main application or the my_other_app that I launched with execv?
EDIT: It appears through testing that my_other_app takes priority over the console. Does this happen every time? Is there a way to make sure the console is instead owned by the main process?
Both processes have their stdin connected to the terminal (or whatever the original process's stdin was connected to). This doesn't change when you call execv. If both processes try to read from stdin at the same time, it's unpredictable which one will get the input.
If you want to disconnect the child process from the terminal, you should call setsid() before calling execv to put it in its own session and remove its controlling terminal.
fork() calls dup() on every single file descriptor. In effect you get a copy of all the files in the child. Some "processes" (via hooks) may detect the fork() and close some file descriptors, but that's very unlikely. Some files may be opened with a specific flag saying that it should be closed on execv(). stdin is not one of them.
You have two solutions, just close stdin in the child process, but that can cause problems, or replace it with /dev/null.
freopen("/dev/null", "r", stdin);
You can do the same for stdout and stderr.
Adding a 'wait forever' at the end of the program (since you cannot do getchar() anymore):
for(;;) sleep(0x7FFFFFFF); // for(;;) is probably superfluous
That's probably the simplest way, there are many others such as using select() on a file you know will never change...
I think it APPEARS as though my_other_app has priority since the other child process has sleep(3).
I am trying to implement a simple two stage pipe in a shell.
When I don't do the second fork and just do the rest of the implementation of the pipe in the parent, it works fine but I exit the shell. That's why I want to do the second fork so I don't exit the shell. But for some reason nothing happens with the above code. Can you help me figure out what may be going wrong? I have a feeling it doesn't wait for both my processes to finish before exiting but I could be wrong.
Solution: close fd[0] and fd[1] in the parent.
In the twin fork model, which you want, your parent process (the shell) is keeping its copy of fd[1] open. With this open, the child pid2 will never see EOF on its standard input fd.
Comments:
both children should close their pipe fds after dup2'ing
the code after execvp, both above and in your pastie suggests that you think that execvp will return control under ordinary circumstances. It does not. For this code, at most you probably want to follow the execvp with a perror and exit.
Let's suppose we have a code doing something like this:
int pipes[2];
pipe(pipes);
pid_t p = fork();
if(0 == p)
{
dup2(pipes[1], STDOUT_FILENO);
execv("/path/to/my/program", NULL);
...
}
else
{
//... parent process stuff
}
As you can see, it's creating a pipe, forking and using the pipe to read the child's output (I can't use popen here, because I also need the PID of the child process for other purposes).
Question is, what should happen if in the above code, execv fails? Should I call exit() or abort()? As far as I know, those functions close the open file descriptors. Since fork-ed process inherits the parent's file descriptors, does it mean that the file descriptors used by the parent process will become unusable?
UPD
I want to emphasize that the question is not about the executable loaded by exec() failing, but exec itself, e.g. in case the file referred by the first argument is not found or is not executable.
You should use exit(int) since the (low byte) of the argument can be read by the parent process using waitpid(). This lets you handle the error appropriately in the parent process. Depending on what your program does you may want to use _exit instead of exit. The difference is that _exit will not run functions registered with atexit nor will it flush stdio streams.
There are about a dozen reasons execv() can fail and you might want to handle each differently.
The child failing is not going to affect the parent's file descriptors. They are, in effect, reference counted.
You should call _exit(). It does everything exit() does, but it avoids invoking any registered atexit() functions. Calling _exit() means that the parent will be able to get your failed child's exit status, and take any necessary steps.