Can you give me some ideas on the following code? The code runs but it doesn't exit. The other string, s="ls -1" works well. Running the shell snippet via sh(1) works just fine also.
#include <unistd.h>
#include <string.h>
int
main(int argc, char *argv[])
{
int fd[2];
char *s = "ls -1 \"/usr/bin\" | while IFS= read -r fp\ndo\ncat <<- EOF\n\t$fp\nEOF\ndone;";
//char *s = "ls -1";
pipe(fd);
switch(fork()) {
case 0:
close(fd[1]);
dup2(fd[0], 0);
execl("/bin/sh", "sh", NULL);
close(fd[0]);
break;
default:
close(fd[0]);
write(fd[1], s, strlen(s) + 1);
close(fd[1]);
break;
}
return 0;
}
When I was converting my comments into an answer, I tested the proposed change, and it didn't fix the problem. One fix that ought to work is to add ; exit at the end of the string, even though that's tantamount to cheating. However, testing that also shows it doesn't finish; it is as if the ls isn't terminating.
I went to another terminal to see whether the processes for ls or pipe97 (my name for your code) were still around; they weren't.
Try typing ps to your 'hung' process.
You should get a normal output and your prompt.
Because the parent doesn't wait for the child to exit, the prompt is lost somewhere in the output from ls (which is quite long and produced quite slowly). Redirect the output to /dev/null and you'll see your prompt.
The best fix is probably to add a wait() loop in the parent process, so it doesn't exit until after the child does.
#include <sys/wait.h>
…
int corpse;
int status;
while ((corpse = wait(&status)) > 0)
;
While debugging, you can print corpse and status so you can see what's going on.
I observe that this comment (without its preamble) remains valid:
…the close(fd[0]); needs to be before the execl() — remember, if it is successful, execl() never returns (it only returns on failure). You should arguably have an error report and exit(1); or similar after the execl(); at the moment, you report success even on failure.
By itself, the Rule of Thumb comment is valid, but it is not actually applicable to this code — unclosed pipe descriptors aren't causing the trouble, even though one pipe descriptor that should have been closed was not closed.
#WilliamPursell Thanks, you're right. I wasn't focusing on that on this example.
#user3629249 I'm aware I'm not doing proper error checking, thank you!
#mosvy It was OpenBSD, indeed it ran on a Mac.
This version works:
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>
int
main(int argc, char *argv[])
{
int fd[2];
char *s = "ls -1 '/usr/bin' | while IFS= read -r fp\ndo\ncat <<- EOF\n\t$fp\nEOF\ndone;";
//char *s = "ls -1";
pipe(fd);
switch(fork()) {
case 0:
close(fd[1]);
dup2(fd[0], 0);
execl("/bin/sh", "sh", NULL);
close(fd[0]);
break;
default:
close(fd[0]);
write(fd[1], s, strlen(s));
close(fd[1]);
wait(NULL);
break;
}
return 0;
}
It was the wait(2) that did it. I remember I did wait(2) at some point, however I think I screwed the close(1) order so it was blocking.
Anyway, problem fixed, thanks!
I need to create a program that takes input from the user and executes it just like it does in the terminal. I am using the execvp() function for this purpose. The requirement of the program is to keep taking input from the user unless the quit call is encountered. The problem here is that the current program is replaced after the execvp() call. So, using a goto is not an option either. I found this Fork–exec article but it doesn't tell how to create an indefinite number of processes. Here is my code:
#include <unistd.h>
#include <stdio.h>
#include <string.h>
void main() {
char *args[4];
char inputCommand[100];
fgets (inputCommand, 100, stdin);
printf ("Splitting string \"%s\" into tokens:\n",inputCommand);
/* Perfrom string tokenization here */
execvp(args[0], args);
}
fork() can be called an indefinite number of times; as long as the return value indicates that you're the parent process, you can continue to operate as usual and call it again.
Thus, you can have a loop within which you fork, call an execvp() if you're the child process, and continue to the next iteration if you're the parent.
Execvp replaces the current process image with the command you run. So it cancels your C program. To produce the desired effect, you should fork before execvp. It would look something like this:
int status = 0;
pid_t pid = fork();
if(pid > 0) {
waitpid(pid, &status, 0);
// is parent
} else if(pid == 0) {
execvp(*args, args);
// is child
} else {
// fork didn't work
}
I am trying to make my own shell in C. It uses one pipe and the input (for now) is static. I execute commands using execvp.
Everything is fine except when I run the command ls |grep ".c" I get no results. Can anyone show me where is the problem and find a solution.
The shell so far:
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
int p[2];
int pid;
int r;
main()
{
char *ls[] = {"ls", NULL};
char *grep[] = {"grep", "\".c\"", NULL};
pipe(p);
pid = fork();
if (pid != 0) {
// Parent: Output is to child via pipe[1]
// Change stdout to pipe[1]
dup2(p[1], 1);
close(p[0]);
r = execvp("ls", ls);
} else {
// Child: Input is from pipe[0] and output is via stdout.
dup2(p[0], 0);
close(p[1]);
r = execvp("grep", grep);
close(p[0]);
}
return r;
}
Remove the quotes in the argument to grep. i.e., use
char *grep[] = {"grep", ".c", NULL};
If you are calling execvp, the usual shell expansion of arguments (i.e., globbing, removal of quotes, etc) does not happen, so effectively what you are doing is the same as
ls | grep '".c"'
In a normal shell.
Also be aware that nothing that comes after the call to execvp will execute, execvp replaces the current process, it will never return.
You have multiple problems:
One problem is that you have far too few calls to close(). When you use dup2() to replicate a file descriptor from a pipe to standard input or standard output, you should close both file descriptors returned by pipe().
A second problem is that the shell removes double quotes around arguments but you've added them around your. You are looking for files whose name contains ".c" (where the double quotes are part of the file name being searched for). Use:
char *grep[] = { "grep", "\\.c$", NULL };
This looks for a dot and a c at the end of the line.
You should report failures after execvp(). If any of the exec*() functions returns, it failed. It can happen when the user mistypes a command name, for example. It is crucial that you report the error and that the child process then exits. If you don't do that, you can end up in a normal iterative shell (rather than this one-shot, non-iterative, non-interactive shell) with multiple shell processes all trying to read from the terminal at the same time, which leads to chaos and confusion.
I'm having problems understanding the right use of the pipe in UNIX Systems.
I have a main process which create a child process. The child process must run a different program from the father, he has to make some operation and then the child must communicate to the father the results.
However in the child process I have to print on the terminal the partial results of these operations.
I'm trying with a test program to do so, but I'm a bit stuck right now. This is the main test program
TEST.C
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
int main(){
int mypipe[2];
pid_t pid;
char readbuffer[6];
pipe(mypipe);
if((pid = fork()) == 0){
close(mypipe[0]);
dup2(mypipe[1], STDOUT_FILENO);
execlp("./proc", "./proc", NULL);
} else {
wait(-1);
close(mypipe[1]);
read(mypipe[0], readbuffer, sizeof(readbuffer));
printf("%s", readbuffer);
}
}
And the c file of the ./proc program is this:
PROC.C
#include <stdio.h>
int main(int argc, char* argv[]){
printf("check\n");
return 0;
}
With this solution, the proc program can't print anything on the terminal. How do I make the proc program to print on the terminal AND on the pipe so the main program can read from there???
Thank you!
Even when you redirect stdout to the parent program, you can still use stderr. Just call fprintf(stderr, ...) instead of printf(...) when you want to print to the console.
If you want your "proc" program to print in terminal for log/debug infos, u can use fprintf and stderr:
fprintf(stderr, "What you need to print\n");
You can also see where your program is writing, use strace
So I'm making a UNIX minishell, and am trying to add pipelines, so I can do things like this:
ps aux | grep dh | grep -v grep | cut -c1-5
However I'm having trouble wrapping my head around the piping part. I replace all the "|" characters with 0, and then run each line as a normal line. However, I am trying to divert the output and input. The input of a command needs to be the output of the previous command, and the output of a command needs to be the input of the next command.
I'm doing this using pipes, however I can't figure out where to call pipe() and where to close them. From the main processing function, processline(), I have this code:
if((pix = findUnquotChar(line_itr, '|')))
{
line_itr[pix++] = 0;
if(pipe (fd) < 0) perror("pipe");
processline(line_itr, inFD, fd[1], pl_flags);
line_itr = &(line_itr[pix]);
while((pix = findUnquotChar(line_itr, '|')) && pix < line_len)
{
line_itr[pix++] = 0;
//? if(pipe (fd) < 0) perror("pipe");
processline(line_itr, fd[0], fd[1] pl_flags);
line_itr = &(line_itr[pix]);
//? close(fd[0]);
//? close(fd[1]);
}
return;
}
So, I'm recursively(the code above is in processline) sending the commands in between the "|" to be processed by processline. You can see where I commented out the code above, I'm not sure how to make it work. The 2nd and 3rd parameter of processline are the inputFD and outputFD respectively, so I need to process a command, write the output to a pipe, and then call processline again on the next command, however this time the output of the previous command is the input. This just doesn't seem like it can work though, because each time I close fd[0] I'm losing the previous output. Do I need two separate pipes, that I can flip flop back and forth with?
I'm just having trouble seeing how this is possible with a single pipe, if you guys need any additional info just ask. Here's the entire processline function in case you want to take a look:
http://pastebin.com/YiEdaYdj
EDIT: If anybody has an example of a shell that implements pipelines I would love a link to the source, I haven't been able to find one on google so far.
EDIT2: Here's an example of my predicament:
echo a | echo b | echo c
So first I would call the shell like this:
processline("echo a", 0, fd[1], flags);
....
processline("echo b", fd[0], NOT_SURE_GOES_HERE[1], flags);
....
processline("echo c", NOT_SURE_GOES_HERE[0], NOT_SURE_EITHER[1], flags);
Each of these occurs once per iteration, and as you can see I can't figure out what to pass for the input-file-descriptors and the output-file-descriptors for the 2nd and 3rd(and so on) iteration.
Here's some moderately generic but simple code to execute pipelines, a program I'm calling pipeline. It's an SSCCE in a single file as presented, though I'd have the files stderr.h and stderr.c as separate files in a library to be linked with all my programs. (Actually, I have a more complex set of functions in my 'real' stderr.c and stderr.h, but this is a good starting point.)
The code operates in two ways. If you supply no arguments, then it runs a built-in pipeline:
who | awk '{print $1}' | sort | uniq -c | sort -n
This counts the number of times each person is logged in on the system, presenting the list in order of increasing number of sessions. Alternatively, you can invoke with a sequence of arguments that are the command line you want invoked, use a quoted pipe '|' (or "|") to separate commands:
Valid:
pipeline
pipeline ls '|' wc
pipeline who '|' awk '{print $1}' '|' sort '|' uniq -c '|' sort -n
pipeline ls
Invalid:
pipeline '|' wc -l
pipeline ls '|' '|' wc -l
pipeline ls '|' wc -l '|'
The last three invocations enforce 'pipes as separators'. The code does not error check every system call; it does error check fork(), execvp() and pipe(), but skips checking on dup2() and close(). It doesn't include diagnostic printing for the commands that are generated; a -x option to pipeline would be a sensible addition, causing it to print out a trace of what it does. It also does not exit with the exit status of the last command in the pipeline.
Note that the code starts with a child being forked. The child will become the last process in the pipeline, but first creates a pipe and forks another process to run the earlier processes in the pipeline. The mutually recursive functions are unlikely to be the only way of sorting things out, but they do leave minimal code repetition (earlier drafts of the code had the content of exec_nth_command() largely repeated in exec_pipeline() and exec_pipe_command()).
The process structure here is such that the original process only knows about the last process in the pipeline. It is possible to redesign things in such a way that the original process is the parent of every process in the pipeline, so the original process can report separately on the status of each command in the pipeline. I've not yet modified the code to allow for that structure; it will be a little more complex, though not hideously so.
/* One way to create a pipeline of N processes */
/* stderr.h */
#ifndef STDERR_H_INCLUDED
#define STDERR_H_INCLUDED
static void err_setarg0(const char *argv0);
static void err_sysexit(char const *fmt, ...);
static void err_syswarn(char const *fmt, ...);
#endif /* STDERR_H_INCLUDED */
/* pipeline.c */
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <unistd.h>
/*#include "stderr.h"*/
typedef int Pipe[2];
/* exec_nth_command() and exec_pipe_command() are mutually recursive */
static void exec_pipe_command(int ncmds, char ***cmds, Pipe output);
/* With the standard output plumbing sorted, execute Nth command */
static void exec_nth_command(int ncmds, char ***cmds)
{
assert(ncmds >= 1);
if (ncmds > 1)
{
pid_t pid;
Pipe input;
if (pipe(input) != 0)
err_sysexit("Failed to create pipe");
if ((pid = fork()) < 0)
err_sysexit("Failed to fork");
if (pid == 0)
{
/* Child */
exec_pipe_command(ncmds-1, cmds, input);
}
/* Fix standard input to read end of pipe */
dup2(input[0], 0);
close(input[0]);
close(input[1]);
}
execvp(cmds[ncmds-1][0], cmds[ncmds-1]);
err_sysexit("Failed to exec %s", cmds[ncmds-1][0]);
/*NOTREACHED*/
}
/* Given pipe, plumb it to standard output, then execute Nth command */
static void exec_pipe_command(int ncmds, char ***cmds, Pipe output)
{
assert(ncmds >= 1);
/* Fix stdout to write end of pipe */
dup2(output[1], 1);
close(output[0]);
close(output[1]);
exec_nth_command(ncmds, cmds);
}
/* Execute the N commands in the pipeline */
static void exec_pipeline(int ncmds, char ***cmds)
{
assert(ncmds >= 1);
pid_t pid;
if ((pid = fork()) < 0)
err_syswarn("Failed to fork");
if (pid != 0)
return;
exec_nth_command(ncmds, cmds);
}
/* Collect dead children until there are none left */
static void corpse_collector(void)
{
pid_t parent = getpid();
pid_t corpse;
int status;
while ((corpse = waitpid(0, &status, 0)) != -1)
{
fprintf(stderr, "%d: child %d status 0x%.4X\n",
(int)parent, (int)corpse, status);
}
}
/* who | awk '{print $1}' | sort | uniq -c | sort -n */
static char *cmd0[] = { "who", 0 };
static char *cmd1[] = { "awk", "{print $1}", 0 };
static char *cmd2[] = { "sort", 0 };
static char *cmd3[] = { "uniq", "-c", 0 };
static char *cmd4[] = { "sort", "-n", 0 };
static char **cmds[] = { cmd0, cmd1, cmd2, cmd3, cmd4 };
static int ncmds = sizeof(cmds) / sizeof(cmds[0]);
static void exec_arguments(int argc, char **argv)
{
/* Split the command line into sequences of arguments */
/* Break at pipe symbols as arguments on their own */
char **cmdv[argc/2]; // Way too many
char *args[argc+1];
int cmdn = 0;
int argn = 0;
cmdv[cmdn++] = &args[argn];
for (int i = 1; i < argc; i++)
{
char *arg = argv[i];
if (strcmp(arg, "|") == 0)
{
if (i == 1)
err_sysexit("Syntax error: pipe before any command");
if (args[argn-1] == 0)
err_sysexit("Syntax error: two pipes with no command between");
arg = 0;
}
args[argn++] = arg;
if (arg == 0)
cmdv[cmdn++] = &args[argn];
}
if (args[argn-1] == 0)
err_sysexit("Syntax error: pipe with no command following");
args[argn] = 0;
exec_pipeline(cmdn, cmdv);
}
int main(int argc, char **argv)
{
err_setarg0(argv[0]);
if (argc == 1)
{
/* Run the built in pipe-line */
exec_pipeline(ncmds, cmds);
}
else
{
/* Run command line specified by user */
exec_arguments(argc, argv);
}
corpse_collector();
return(0);
}
/* stderr.c */
/*#include "stderr.h"*/
#include <stdio.h>
#include <stdarg.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
static const char *arg0 = "<undefined>";
static void err_setarg0(const char *argv0)
{
arg0 = argv0;
}
static void err_vsyswarn(char const *fmt, va_list args)
{
int errnum = errno;
fprintf(stderr, "%s:%d: ", arg0, (int)getpid());
vfprintf(stderr, fmt, args);
if (errnum != 0)
fprintf(stderr, " (%d: %s)", errnum, strerror(errnum));
putc('\n', stderr);
}
static void err_syswarn(char const *fmt, ...)
{
va_list args;
va_start(args, fmt);
err_vsyswarn(fmt, args);
va_end(args);
}
static void err_sysexit(char const *fmt, ...)
{
va_list args;
va_start(args, fmt);
err_vsyswarn(fmt, args);
va_end(args);
exit(1);
}
Signals and SIGCHLD
The POSIX Signal Concepts section discusses SIGCHLD:
Under SIG_DFL:
If the default action is to ignore the signal, delivery of the signal shall have no effect on the process.
Under SIG_IGN:
If the action for the SIGCHLD signal is set to SIG_IGN, child processes of the calling processes shall not be transformed into zombie processes when they terminate. If the calling process subsequently waits for its children, and the process has no unwaited-for children that were transformed into zombie processes, it shall block until all of its children terminate, and wait(), waitid(), and waitpid() shall fail and set errno to [ECHILD].
The description of <signal.h> has a table of default dispositions for signals, and for SIGCHLD, the default is I (SIG_IGN).
I added another function to the code above:
#include <signal.h>
typedef void (*SigHandler)(int signum);
static void sigchld_status(void)
{
const char *handling = "Handler";
SigHandler sigchld = signal(SIGCHLD, SIG_IGN);
signal(SIGCHLD, sigchld);
if (sigchld == SIG_IGN)
handling = "Ignored";
else if (sigchld == SIG_DFL)
handling = "Default";
printf("SIGCHLD set to %s\n", handling);
}
I called it immediately after the call to err_setarg0(), and it reports 'Default' on both Mac OS X 10.7.5 and Linux (RHEL 5, x86/64). I validated its operation by running:
(trap '' CHLD; pipeline)
On both platforms, that reported 'Ignored', and the pipeline command no longer reported the exit status of the child; it didn't get it.
So, if the program is ignoring SIGCHLD, it does not generate any zombies, but does wait until 'all' of its children terminate. That is, until all of its direct children terminate; a process cannot wait on its grandchildren or more distant progeny, nor on its siblings, nor on its ancestors.
On the other hand, if the setting for SIGCHLD is the default, the signal is ignored, and zombies are created.
That's the most convenient behaviour for this program as written. The corpse_collector() function has a loop that collects the status information from any children. There's only one child at a time with this code; the rest of the pipeline is run as a child (of the child, of the child, ...) of the last process in the pipeline.
However I'm having trouble with zombies/corpses. My teacher had me implement it the same way you did, as cmd1 isn't the parent of cmd2 in the case of: "cmd1 | cmd2 | cmd3". Unless I tell my shell to wait on each process (cmd1, cmd2, and cmd3), rather than just waiting on the last process (cmd3), the entire pipeline shuts down before the output can reach the end. I'm having trouble figuring out a good way to wait on them; my teacher said to use WNOHANG.
I'm not sure I understand the problem. With the code I provided, cmd3 is the parent of cmd2, and cmd2 is the parent of cmd1 in a 3-command pipeline (and the shell is the parent of cmd3), so the shell can only wait on cmd3. I did state originally:
The process structure here is such that the original process only knows about the last process in the pipeline. It is possible to redesign things in such a way that the original process is the parent of every process in the pipeline, so the original process can report separately on the status of each command in the pipeline. I've not yet modified the code to allow for that structure; it will be a little more complex, though not hideously so.
If you've got your shell able to wait on all three commands in the pipeline, you must be using the alternative organization.
The waitpid() description includes:
The pid argument specifies a set of child processes for which status is requested. The waitpid() function shall only return the status of a child process from this set:
If pid is equal to (pid_t)-1, status is requested for any child process. In this respect, waitpid() is then equivalent to wait().
If pid is greater than 0, it specifies the process ID of a single child process for which status is requested.
If pid is 0, status is requested for any child process whose process group ID is equal to that of the calling process.
If pid is less than (pid_t)-1, status is requested for any child process whose process group ID is equal to the absolute value of pid.
The options argument is constructed from the bitwise-inclusive OR of zero or more of the following flags, defined in the header:
...
WNOHANG
The waitpid() function shall not suspend execution of the calling thread if status is not immediately available for one of the child processes specified by pid.
...
This means that if you're using process groups and the shell knows which process group the pipeline is running in (for example, because the pipeline is put into its own process group by the first process), then the parent can wait for the appropriate children to terminate.
...rambling... I think there's some useful information here; there probably should be more that I'm writing, but my mind's gone blank.