I'm trying to write a shell in c and it mostly works except for grep. Whenever I give a grep command in the shell, it just doesn't output anything. Here is the part of the code I use to create a new child process and then run execvp() in it.
The file descriptors(fd_in and fd_out) in dup2 are passed as arguments to the function which has this code. And most interestingly, when I give 'grep' or 'grep --help' it displays as usual. Am I missing anything? Or something special has to be done with grep?
This is what happens with my shell: The last command outputs when run from bash.
--> grep
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
--> wc /etc/hosts
11 33 314 /etc/hosts
--> grep -i "perror" shell.c
-->
Here is the code :
void
create_process(char *cmd_argv[], int fd_in, int fd_out, char *buffer_copy) {
/*Flag bit for Background processes*/
int FLAG = 0;
pid_t cpid;
int status;
int i = 0,j = 0;
/*Find the no. of arguments*/
while(cmd_argv[j] != NULL)
j++;
/*Set the flag bit*/
if(strcmp("&", cmd_argv[j-1]) == 0) {
FLAG = 1;
cmd_argv[j-1] = NULL;
}
//Create a child process
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
//In the child...
if (cpid == 0) {
/*Checking if the file descriptors are already assigned*/
/*For stdin*/
if (fd_in != STDIN_FILENO) {
dup2(fd_in, STDIN_FILENO);
close(fd_in);
}
/*For stdout*/
if (fd_out != STDOUT_FILENO) {
dup2(fd_out, STDOUT_FILENO);
close(fd_out);
}
/*Run the cmd specified*/
status = execvp(cmd_argv[0], cmd_argv);
/*In case of errors*/
if(status < 0) {
perror("execvp ");
exit(1);
}
}
//In the parent...
else {
if(FLAG == 1) {
/*Find where the new bg process can be inserted*/
while(1) {
if (bgprocess[i].pid == 0) {
bgprocess[i].pid = cpid;
strcpy(bgprocess[i].cmd, buffer_copy);
break;
}
i++;
}
printf("[%d] : %s\n", cpid, cmd_argv[0] );
}
/*If not bg, wait for the process to exit*/
else
waitpid(cpid, NULL, 0);
}
}
The problem is using the quotes in your shell. Bash does a lot of things in background.
grep -i perror shell.c should give you output on your shell, whatever is anticipated when run from bash.
Quoth man dup2:
After a successful return from one of these system calls, the old and
new file descriptors may be used interchangeably. They refer to the
same open file description (see open(2))…
Therefore after after your call to dup2 (which you should be checking for error returns) you close oldfd and newfd because they are the exact same descriptor.
What I can't figure out is why grep --help works at all but you aren't showing enough code to answer that question.
Added following comment: You still aren't giving enough code. If grep isn't working than what is? Does wc /etc/hosts work? There is nothing special about grep at all, indeed it is a completely unspecial Unix filter.
Related
I'm trying to implement piping in a simple shell program that I'm writing in C.
But for some reason, I'm not getting output when I try to run ls | wc -l.
I'm really not sure why this is happening since I'm basically putting the child process's output to pipe[1] which does the command before pipe indicator and I'm putting parent's input to pipe[0] which does the command after pipe indicator and it should be printing to the terminal since the output of parent's never been changed, my approach right now is if piping is flagged the call fork in child and do the piping.
code below
int pipe1[2];
int pipepid;
int piping; /*flag for piping*/
int pipeposition;/*index of pipe indicator*/
//* code... */
if(pipe(pipe1)!= 0){
perror("pipe");
exit(1);
};
/* split commands to before pipe indicator and after */
for(int p = 0;p<pipeposition;p++){
argsbefore[p]=args[p];
}
/* after */
int e=0;
for(int h = pipeposition+1; h<cnt;h++){
argsafter[e]=args[h];
e++;
}
/* code ... */
if(piping){
pipepid = fork();
if(pid == 0){
/* do child */
if(dup2(pipe1[1],1)==-1){
perror("dup2 child");
exit(1);
}
close(pipe1[1]);
if (execvp(argsbefore[0], argsbefore) < 0) {
printf("exec failed\n");
exit(1);
}
exit(0);
}/* else if error */
else if(pid == -1){
printf("ERROR: fork failed\n");
exit(1);
}/* parent */
else{
if(dup2(pipe1[0],0)==-1){
perror("dup2 parent");
exit(1);
}
close(pipe1[0]);
if (execvp(argsafter[0], argsafter) < 0) {
printf("exec failed\n");
exit(1);
}
}
}
you seem to be doing that on a unix-like system. If you're lucky, your system might have a tool that reports every system call your program perform (strace -f my_program my_ar gu_ments would do that on Linux).
That would give you a list of what process did what and when, and whether there have been error code for some operations. That usually helps a lot with these multi-process setups.
It turns out I didn't close all the pipes so the second command wasn't able to finish, after putting close for both ends in the main parent process it was fixed
I am writing a function to my shell where I need to redirect output to file. For example user write: ls -l >> file and ls -l should be written to file. Almost all things are correct but after first calling my function program stop and I can't write anything else. Below I present my function and I would appreciate any clues to resolve the problem:
void execute2(char *command, char **argv, char **argv2)
{
pid_t pid;
int status;
if ((pid = fork()) < 0)
{
printf("*** ERROR ***\n");
exit(1);
}
else if (pid == 0)
{
close(1);
parse(command, argv, argv2);
int output = open(*argv2, O_APPEND | O_WRONLY);
dup2(output,1);
if (strcmp(argv[0], "exit") == 0)
exit(0);
if (execvp(*argv, argv) < 0)
{
printf("*** ERROR ***\n");
exit(1);
}
close(output);
}
else
{
while (wait(&status) != pid);
}
}
command is command getting from user, argv is part of instruction and parameters and argv2 is the output file.
#Jonathan Leffler I checked this and I think it doesn 't resolve problem. I suggest that when I call first time execute2 it works as well as I want but I never finish that proces. Or I don 't undestand something
im currently implementing a shell in C.
My problem arises when i try to run a command like this:
SHELL$: sort < txtFile | grep key
im running sort < txtFile in a process (child), and in the parent i.e else if(pid > 0) im running the other command to the right of the pipe.
The program runs fine, but it exits the infinite loop that i set up in main to keep receiving input from the user.
How could i solve this problem?
this is the code i have so far to deal with the pipe, i didnt include the code that i have to deal with the redirects:
c2p is the pipe i setup for this.
if(pid == 0)
{
if( PIPE_FLAG )
{
close(c2p[0]);
if(dup2(c2p[1], STDOUT_FILENO) == -1){
perror("dup2() failed");
exit(2);
}
}
/* Execute command */
execvp(cmd_args[0], cmd_args);
perror("exec failed 1. "); /* return only when exec fails */
exit(-1);
}
else if(pid > 0)
{
if(PIPE_FLAG)
{
close(c2p[1]);
if(dup2(c2p[0], STDIN_FILENO) == -1){
perror("dup2() failed");
exit(-1);
}
execvp(nxt_args[0], nxt_args);
perror("exec failed 2. ");
exit(-1);
}
}
else
{
/* error occurred */
perror("fork failed");
exit(1);
}
I'm running sort < txtFile in the child process, and in the parent I'm running the command to the right of the pipe.
What happens to your shell process, then? The parent process is the shell. By running the right-side command in the parent process you're having it take over the shell's process. Remember that exec() replaces the current process.
You'll need to fork() twice, and execute the two sides of the pipe in the child processes. The parent must remain the shell, which will then wait() for the children to exit before presenting the next command prompt.
/* How shell works */
#include<stdio.h>
#include<unistd.h>
main (int argc, char **argv)
{
if (argc < 2)
{
fprintf (stderr, "\nUsage: ./a.out cmd [options]...\n");
}
if (!fork ())
{
argv++;
execvp (argv[0], argv);
}
}
I created a pipe between two child processes,
first, I run ls, which writes to the proper fd,
then, I run grep r, which reads from the proper fd,
I can see in the terminal that the grep command works fine (the output)
The problem is that grep doesn't quit, it stays there, even though ls isn't running anymore
for other programs the pipe works fine..
for (i = 0; i < commands_num ; i++) { //exec all the commands instants
if (pcommands[i]._flag_pipe_out == 1) { //creates pipe if necessary
if (pipe(pipe_fd) == -1) {
perror("Error: \"pipe()\" failed");
}
pcommands[i]._fd_out = pipe_fd[1];
pcommands[i+1]._fd_in = pipe_fd[0];
}
pid = fork(); //the child exec the commands
if (pid == -1) {
perror("Error: \"fork()\" failed");
break;
} else if (!pid) { //child process
if (pcommands[i]._flag_pipe_in == 1) { //if there was a pipe to this command
if (dup2(pcommands[i]._fd_in, STDIN) == -1) {
perror("Error: \"dup2()\" failed");
exit(0);
}
close(pcommands[i]._fd_in);
}
if (pcommands[i]._flag_pipe_out == 1) { //if there was a pipe from this command
if (dup2(pcommands[i]._fd_out, STDOUT) == -1) {
perror("Error: \"dup2()\" failed");
exit(0);
}
close(pcommands[i]._fd_out);
}
execvp(pcommands[i]._commands[0] , pcommands[i]._commands); //run the command
perror("Error: \"execvp()\" failed");
exit(0);
} else if (pid > 0) { //father process
waitpid(pid, NULL, WUNTRACED);
}
}
//closing all the open fd's
for (i = 0; i < commands_num ; i++) {
if (pcommands[i]._fd_in != STDIN) { //if there was an other stdin that is not 0
close(pcommands[i]._fd_in);
}
if (pcommands[i]._fd_out != STDOUT) { //if there was an other stdout that is not 1
close(pcommands[i]._fd_out);
}
}
So, I have a "command" instant pcommands[i]
It has:
a flag of pipein,pipeout
fdin,fdout,
and a char** (for the real command, like "ls -l")
lets say everything is good,
that means that:
pcommands[0]:
pipein=0
pipeout=1
char** = {"ls","-l",NULL}
pcommands[1]:
pipein=1
pipeout=0
char** = {"grep","r",NULL}
now, the loop will go twice (because I have two commands instants)
at the first time, it will see the pcommands[0] has pipeout==1
create pipe
do fork
pcommands[0] has pipeout==1
child: dup2 to the stdout
execvp
second time:
doesn't create pipe
do fork
child:
the pcomands[1] has pipein==1
then: dup2 to the input
exevp
..
this command works, my output is:
errors.log exer2.pdf multipal_try
(all the things with 'r')
but then it get stuck, and doesn't get out of grep..
in an other terminal i can see grep is still working
I hope I close all the fd's I need to close...
I don't understand why doesn't it work, it seems like I do it right (well, it works for other commands..)
can someone please help? thanks
You aren't closing enough pipe file descriptors.
Rule of Thumb:
If you use dup() or dup2() to duplicate a pipe file descriptor to standard input or standard output, you should close both of the original pipe file descriptors.
You also need to be sure that if the parent shell creates the pipe, it closes both of its copies of the pipe file descriptors.
Also note that the processes in a pipeline should be allowed to run concurrently. In particular, pipes have a limited capacity, and a process blocks when there's no room left in the pipe. The limit can be quite small (POSIX mandates it must be at least 4 KiB, but that's all). If your programs deal with megabytes of data, they must be allowed to run concurrently in the pipeline. Therefore, the waitpid() should occur outside the loop that launches the children. You also need to close the pipes in the parent process before waiting; otherwise, the child reading the pipe will never see EOF (because the parent could, in theory, write to the pipe, even though it won't).
You have structure members whose names start with an underscore. That's dangerous. Names starting with an underscore are reserved for the implementation. The C standard says:
ISO/IEC 9899:2011 §7.1.3 Reserved Identifiers
— All identifiers that begin with an underscore and either an uppercase letter or another
underscore are always reserved for any use.
— All identifiers that begin with an underscore are always reserved for use as identifiers
with file scope in both the ordinary and tag name spaces.
That means that if you run into problems, then the trouble is yours, not the system's. Obviously, your code works, but you should be aware of the problems you could run into and it is wisest to avoid them.
Sample Code
This is a fixed SSCCE based on the code above:
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
typedef struct Command Command;
struct Command
{
int _fd_out;
int _fd_in;
int _flag_pipe_in;
int _flag_pipe_out;
char **_commands;
};
typedef int Pipe[2];
enum { STDIN = STDIN_FILENO, STDOUT = STDOUT_FILENO, STDERR = STDERR_FILENO };
int main(void)
{
char *ls_cmd[] = { "ls", 0 };
char *grep_cmd[] = { "grep", "r", 0 };
Command commands[] =
{
{
._fd_in = 0, ._flag_pipe_in = 0,
._fd_out = 1, ._flag_pipe_out = 1,
._commands = ls_cmd,
},
{
._fd_in = 0, ._flag_pipe_in = 1,
._fd_out = 1, ._flag_pipe_out = 0,
._commands = grep_cmd,
}
};
int commands_num = sizeof(commands) / sizeof(commands[0]);
/* Allow valgrind to check memory */
Command *pcommands = malloc(commands_num * sizeof(Command));
for (int i = 0; i < commands_num; i++)
pcommands[i] = commands[i];
for (int i = 0; i < commands_num; i++) { //exec all the commands instants
if (pcommands[i]._flag_pipe_out == 1) { //creates pipe if necessary
Pipe pipe_fd;
if (pipe(pipe_fd) == -1) {
perror("Error: \"pipe()\" failed");
}
pcommands[i]._fd_out = pipe_fd[1];
pcommands[i+1]._fd_in = pipe_fd[0];
}
pid_t pid = fork(); //the child exec the commands
if (pid == -1) {
perror("Error: \"fork()\" failed");
break;
} else if (!pid) { //child process
if (pcommands[i]._flag_pipe_in == 1) { //if there was a pipe to this command
assert(i > 0);
assert(pcommands[i-1]._flag_pipe_out == 1);
assert(pcommands[i-1]._fd_out > STDERR);
if (dup2(pcommands[i]._fd_in, STDIN) == -1) {
perror("Error: \"dup2()\" failed");
exit(0);
}
close(pcommands[i]._fd_in);
close(pcommands[i-1]._fd_out);
}
if (pcommands[i]._flag_pipe_out == 1) { //if there was a pipe from this command
assert(i < commands_num - 1);
assert(pcommands[i+1]._flag_pipe_in == 1);
assert(pcommands[i+1]._fd_in > STDERR);
if (dup2(pcommands[i]._fd_out, STDOUT) == -1) {
perror("Error: \"dup2()\" failed");
exit(0);
}
close(pcommands[i]._fd_out);
close(pcommands[i+1]._fd_in);
}
execvp(pcommands[i]._commands[0] , pcommands[i]._commands); //run the command
perror("Error: \"execvp()\" failed");
exit(1);
}
else
printf("Child PID %d running\n", (int)pid);
}
//closing all the open pipe fd's
for (int i = 0; i < commands_num; i++) {
if (pcommands[i]._fd_in != STDIN) { //if there was another stdin that is not 0
close(pcommands[i]._fd_in);
}
if (pcommands[i]._fd_out != STDOUT) { //if there was another stdout that is not 1
close(pcommands[i]._fd_out);
}
}
int status;
pid_t corpse;
while ((corpse = waitpid(-1, &status, 0)) > 0)
printf("Child PID %d died with status 0x%.4X\n", (int)corpse, status);
free(pcommands);
return(0);
}
Just for my knowledge, how would you do it, so it won't get "indisputably messy"?
I'd probably keep the pipe information so that I the child didn't need to worry about the conditionals contained in the asserts (accessing the child information for the child before or after it in the pipeline). If each child only needs to access information in its own data structure, it is cleaner. I'd reorganize the 'struct Command' so it contained two pipes, plus indicators for which pipe contains information that needs closing. In many ways, not radically different from what you've got; just tidier in that child i only needs to look at pcommands[i].
You can see a partial answer in a different context at C Minishell adding pipelines.
I need to implement my shell that handles multiple pipe commands. For example I need to be able to handle this: ls | grep -i cs340 | sort | uniq | cut -c 5. I am assuming the problem is that I am not passing output of the previous command to the input of the next command.
When I execute my code, it gives me no output. I am using this pseudo code:
for cmd in cmds
if there is a next cmd
pipe(new_fds)
fork
if child
if there is a previous cmd
dup2(old_fds[0], 0)
close(old_fds[0])
close(old_fds[1])
if there is a next cmd
close(new_fds[0])
dup2(new_fds[1], 1)
close(new_fds[1])
exec cmd || die
else
if there is a previous cmd
close(old_fds[0])
close(old_fds[1])
if there is a next cmd
old_fds = new_fds
if there are multiple cmds
close(old_fds[0])
close(old_fds[1])
Here is the source code of the function that handles multiple pipes.
void execute_multiple_commands(struct command ** commands_to_exec,
int num_commands_p)
{
pid_t status;
int i, err;
int new_fd[2], old_fd[2];
pid_t pid, cpid;
// creating child process
if ( (cpid = fork()) == -1)
{
fprintf(stderr, "Could not create child process, exiting...");
exit(1);
}
if (cpid == 0) // in the child process we run multiple pipe handling
{
for (i = 0; i < num_commands_p; i++) // for each cmd in cmds
{
if (i+1 < num_commands_p) // if there is next cmd
pipe(new_fd);
if ( (pid = fork()) == -1)
{
fprintf(stderr, "Could not create child process, exiting...");
exit(1);
}
if (pid == 0) // if child
{
if (i != 0) // if there is a previous command
{
dup2(old_fd[0], 0); // setting up old_pipe to input into the child
close(old_fd[0]);
close(old_fd[1]);
}
if (i+1 < num_commands_p) // if there is a next cmd
{
close(new_fd[0]); // setting up new_pipe to get output from child
dup2(new_fd[1], 1);
close(new_fd[1]);
err = execvp(commands_to_exec[i]->args[0], commands_to_exec[i]->args);
status = err;
exit(err);
}
}
else
{
waitpid(pid, &status, 0);
if (status == -1)
exit(1);
if (i != 0) // if there a previous command
{
close(old_fd[0]);
close(old_fd[1]);
}
if (i+1 < num_commands_p) // if there a next cmd
{
old_fd[0] = new_fd[0];
old_fd[1] = new_fd[1];
}
exit(0);
} // end if
} // end for
if (i) // if there a multiple commands
{
close(old_fd[0]);
close(old_fd[1]);
}
}
else // in the parent process we are waiting for child to handle multiple pipes
waitpid(cpid, &status, 0);
}
Function execvp() takes array of structures. Ive checked all my parsing part, and it works fine. It is the execute_multiple_commands() function that I am having trouble with.
Here is the code for struct:
// name: command
// desc: holds one command (meaning that it can be
// more than one token in that command)
// "ls -la" will be an example of one command
// holds num of tokens in command array
struct command
{
char ** args;
int num_args;
};
I suggest a new strategy, R2:
function do(commands)
if commands is of size 1
exec commands[0] || die
split commands into c1 (first command) c2 (the rest of them)
open
if fork
close input end of pipe
dup output of pipe to stdin
do (c2) || die
close output end of pipe
dup input of pipe to stdout
exec c1 || die
Using a recursive function, especially if you're maintaining a list, will help you simplify your logic. You don't really have to worry about stack depth here, as your whole address space will be overwritten anyway.
In other news, from the man page:
After a successful return from one of these system calls, the old and
new file descriptors may be used interchangeably. They refer to the
same open file description (see open(2)) and thus share file offset
and file status flags; for example, if the file offset is modified by
using lseek(2) on one of the descriptors, the offset is also changed
for the other.
Which meanse when you say you're closing both ends of the pipe? You really are closing it - it AND the standard in/out that your program is intending on using.
--> MUCH LATER EDIT <--
As Jonathan Leffler pointed out, the above information is in correct. I have confirmed it with the following program:
#include <unistd.h>
int main(){
dup2(0, 7);
write(7, "Hey, 1\n", 7);
close(0);
write(7, "Hey, 2\n", 7);
close(7);
write(7, "Hey, 3\n", 7);
}
Which results in the following output:
$ gcc dup2Test.c && ./a.out
Hey, 1
Hey, 2
Thanks, Jonathan!