I'm fine executing commands like "ls" and stuff like that but I want to do something like "ls | sort" but the execvp system call doesn't support "|". How can I do this using only system calls? when I try something like
char *arg[] = {"ls","|","sort",NULL};
execvp(arg[0],arg);
it doesn't work, how can I do this?
Edit:
char* execString (char string[]){
int link[2];
pipe(link);
if (fork() == 0){
int i = 0;
char *p = strtok(string," ");
char *x[spacecount(string)+2];
while(p){
x[i++] = p;
p = strtok(NULL," ");
}
x[i] = NULL;
dup2(link[1],1);
close(link[0]);
close(link[0]);
execvp(x[0],x);
_exit(0);
} else {
wait(NULL);
close(link[1]);
char buf[512];
int i = 0;
while (read(link[0],&buf[i++],1) == 1);
close(link[0]);
buf[i-2] = '\0';
return strdup(buf);
}
}
This is the function i'm executing to exec a string that contains a command, its return value is a pointer to a string that contains the output from that command, how can I use that output as the input to a new command using execvp or another function from the exec family?
Edit2: So I made a new function that receives two strings as argument and execs the first one then the second one using as input the output from the first exec, I thought it was working fine it worked with ls | head -1 and other variations of ls but when I do something like ls | sort -R it doesn't work, i've tried several things and I can't understand why this is happening, here is the code:
char* execStrings (char previousstring[], char string[]){
int link[2];
pipe(link);
if (fork() == 0){
int i = 0;
char *previouscommand[spacecount(previousstring)+2];
char *temp = strtok(previousstring," ");
while(temp){
previouscommand[i++] = temp;
temp = strtok(NULL," ");
}
previouscommand[i] = NULL;
dup2(link[1],1); /* stdout result redrecting to write end of pipe */
close(link[1]);
close(link[0]);
execvp(previouscommand[0],previouscommand);
} else {
wait(NULL);
int res[2];
pipe(res);
if(fork() == 0){
int i = 0;
char *temp = strtok(string," ");
char *command[spacecount(string)+2];
while(temp){
command[i++] = temp;
temp = strtok(NULL," ");
}
command[i] = NULL;
dup2(link[0],0);
close(link[0]);
close(link[1]);
dup2(res[1],1);
close(res[1]);
close(res[0]);
execvp(command[0],command)
} else {
wait(NULL);
close(res[1]);
char buf[512];
int i = 0;
while (read(res[0],&buf[i++],1) == 1);
close(res[0]);
buf[i-2] = '\0';
return strdup(buf);
}
}
}
you want to do something like ls | sort but the way you are doing like
char *arg[] = {"ls","|","sort",NULL};
execvp(arg[0],arg); /*it won't work */
won't work because here you are calling execvp on ls and sort which are two separate process not single process. Also
ls | sort => output of process-1 make as input to process-2 & execute it
| |
process-1 process-2
To achieve the above create two process by calling fork() and use exec() family function to replace ls and sort in child & parent process.
here is the sample code
int main(void) {
int p[2];
pipe(p);
char *arg[] = {"ls","sort",NULL};
if(fork()==0) {
close(0);/* close the stdin stream so that this
process shoulbn't read from stdin */
dup(p[0]);/* read from read end of pipe */
close(p[1]);
execlp(arg[1],arg[1],(char*)NULL);
}
else{
close(1);/* close the stdout stream, so that o/p shouldn't print on monitor */
dup(p[1]); /* stdout result redrecting to write end of pipe */
close(p[0]);
execlp(arg[0],arg[0],(char*)NULL);
}
return 0;
}
| is a shell feature. You need to do the same thing the shell does, i.e. use pipe, fork, dup2, execvp to create a pipe, spawn a new process, and connect the pipe to the processes' stdin and stdout, respectively.
Take the code from your execString function but replace the else block. In the parent process you should dup2(link[0], 0) (connect the read end of your pipe to stdin), then execvp the other program (e.g. sort).
Remember to close the pipe ends you don't need! I.e. in the producer process, close(link[0]); in the consumer process, close(link[1]). If you forget this part, things may get "stuck" (commands seemingly hanging forever).
Of course, if you want your original program to keep going, you need to wrap that code inside another fork (and waitpid for it).
As an aside, your execString is broken if the command outputs a lot of data. At some point the pipe will get full and the command will pause, waiting for another process to drain the pipe (by reading from it). Your program will be stuck in wait, waiting for the command to terminate. The result is deadlock.
To fix this issue, only call wait after you're done reading from the pipe.
Related
I want to understand how pipe works but i don't catch why waitpid is blocked in the case you execute my program with "ls -l /usr/bin" "grep ls".... If you take out the option -l it's works !
my_str_tab just put each word of the string into a charstar array.
void command_levels(int *pipe_fd, char **av, int idx, int pipe_save, int pipe_one)
{
if (idx == 1)
dup2(pipe_fd[1], 1);
else if (idx > 1 && av[idx + 1] != NULL) {
dup2(pipe_save, 0);
dup2(pipe_fd[1], 1);
}
if (idx > 1 && av[idx + 1] == NULL) {
dup2(pipe_save, 0);
dup2(pipe_one, 1);
}
}
void multiple_pipe_handle(char **av, char **env, int idx, int pipe_one)
{
int pipe_fd[2] = {0, 0};
char **command = NULL;
static int pipe_save = 0;
if (av[idx] == NULL)
return;
command = my_str_tab(av[idx], " ");
pipe(pipe_fd);
command_levels(pipe_fd, av, idx, pipe_save, pipe_one);
if (fork() == 0) {
close(pipe_fd[0]);
close(pipe_fd[1]);
execve(command[0], command, env);
} else {
wait(NULL);
close(pipe_fd[1]);
pipe_save = pipe_fd[0];
multiple_pipe_handle(av, env, idx + 1, pipe_one);
close(pipe_fd[0]);
}
}
int main(int ac, char **av, char **env)
{
int pipe_one = dup(1);
multiple_pipe_handle(av, env, 1, pipe_one);
}
I expect the output of all word contains 'ls' but i'm in infinite loop..
This is a common mistake when implementing pipelines.
When you pass -l to ls, it produces more output than when you don't pass that option. That makes it completely fill up the pipe's internal buffer. The kernel "blocks" it from continuing to execute until something reads from the other end of the pipe. But nothing is reading from the other end of the pipe yet, because your parent program is waiting for ls to finish execution before it starts the grep process. But ls will not finish execution until it can write more data to the pipe, so the overall program is deadlocked.
To fix this bug you must start all of the processes in the pipeline before you wait for any of them. You can't do that with a single recursive call to multiple_pipe_handle. You need two loops, one calling fork and one calling waitpid, and an array of subprocess PIDs. If you intend to read from the final pipeline process's output in your parent process, you must read all of the data produced (until read signals EOF by returning zero bytes of data) before you start calling waitpid.
I am trying to write a shell script in
C language. In short details, if user enters a command I send it to the system() call, if user enters more than one command like "ls;whoami" I parse it and create child processes to execute all of them. Now it works but my methods such as gets() and getting input by the user does not seem well and when I put multi commands, prompt text becomes unseen. Do you have any suggestion or if you see any mistakes or wrong usage because I am not the C guy then I would be grateful.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define RESET "\033[0m"
#define BOLDGREEN "\033[1m\033[32m" /* Bold Green */
#define BOLDMAGENTA "\033[1m\033[35m" /* Bold Magenta */
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
void welcome(){
char* welcomestr = "\n\TEST\n\n";
printf("%s%s%s",BOLDMAGENTA,welcomestr,RESET);
}
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
}
int main(int argc, char* argv[]){
inputget=input;
welcome();
prompt();
gets(inputget);
if(argc == 1) {
while(strcmp(inputget, "quit")!=0){
p = strsep(&inputget, ";\n");
while(p != NULL){
pid_t parent = getpid();
pid = fork();
if (pid==-1){
perror("failed to fork");
}
else if (pid==0){
system(p);
exit(0);
}else{
p = strsep(&inputget, ";\n");
}
}
wait(NULL);
prompt();
scanf("%s",input);
inputget=input;
}
exit(0);
}else{
//get argc 2 and and read-run commands from text.file
}
}
Let's begin with the worst part: using gets. Don't use this function anymore,
it's dangerous and deprecated. You should use fgets instead.
fgets(input, sizeof input, stdin);
There is no reason why any of these variables
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
have to be global variables, declare them in main.
Stream buffers like stdout are buffered, content is physically written on
the device once the buffer is full or you call fflush to flush the buffer.
One exception is stdout when is connected to a terminal, in that case printf
will flush immediately when a newline is printed. That's why you almost always
see that the format of printf statements end with \n, like
printf("Your age is %d\n", age);
When you don't want to print a newline, because you are printing something like
a prompt, then you should flush stdout yourself.
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
fflush(stdout); // <-- you need this here
}
The last thing is where you are executing wait. The problem is the
synchronization between the children and the parent process. Once a child is
created, it begins to run immediately. If the child is also printing to stdout
and you don't synchronize with the parent, then there's no guarantee which
output will be printed first.
In your case, if the user enters cmd1;cmd2;cmd3, you are forking 3 times
but you are only doing one wait after you've forked all children. That means
that the three children will run concurrently and the order of their output is
undefined. After all children are forked, you finally do wait(NULL), but this
only waits for one child, then you execute prompt() but remember, the other
children might be still running and hence the output of the prompt might come
before the output of the other children that are running. That is perhaps what
you've been observing.
If you want to emulate the shell, then cmd2 can only start after cmd1 is
finished and cmd3 only after cmd2 is finished. Only when the three commands
are finished you can execute prompt(). That means that you have to wait for
every child to end before the next child can be forked. That's why you have to
move the wait in the parent block before the next fork is called.
// "quit\n" because fgets does not remove the newline
while(strcmp(inputget, "quit\n") != 0) {
p = strsep(&inputget, ";\n");
while(p != NULL) {
if(p[0] == 0)
{
// handles "empty fields", when
// two delimiters come one after
// the other
p = strsep(&inputget, ";\n");
continue;
}
pid = fork();
if (pid==-1) {
perror("failed to fork");
}
else if (pid==0) {
system(p);
exit(0);
} else {
wait(NULL); // <-- here the parent waits
// until the child is finished
p = strsep(&inputget, ";\n");
}
}
prompt();
fgets(input, sizeof input, stdin);
inputget = input;
}
Also note that I'm not using scanf here. scanf("%s"... reads until the first
non-white character, so a command like cat /etc/fstab will only read cat and
your shell will only execute cat and it would block until you close stdin
(by pressing Ctrl+D). The next time, scanf won't wait
for user input and will read /etc/fstab instead and try to execute
/etc/fstab, which will fail, as /etc/fstab is not a script or binary.
That's why it's better to use fgets.
I'd also use a longer buffer for the user input, depending on the command
length, 49 bytes is too short. I'd use 1024 or more. Or you can use getline to
fetch a whole line without the worry about buffer sizes.
I am trying to implement a shell in C. I can execute simple commands just fine with a simple execvp() but one of the requirements is to manage commands like this: "ls -l | head | tail -4" with a 'for' loop and only one 'pipe()' statement redirecting stdin and stdout. Now after days I'm a bit lost.
N = Number of simple commands (3 in the example: ls, head, tail)
commands = a list of structs with the commands, like this:
commands[0].argv[0]: ls
commands[0].argv[1]: -l
commands[1].argv[0]: head
commands[2].argv[0]: tail
commands[2].argv[1]: -4
So, I made the for loop, and started to redirect stdin and stdout in order to connect all the commands with pipes, but...I'm just clueless why it doesn't work.
for (i=0; i < n; i++){
pipe(pipe);
if(fork()==0){ // CHILD
close(pipe[0]);
close(1);
dup(pipe[1]);
close(pipe[1]);
execvp(commands[i].argv[0], &commands[i].argv[0]);
perror("ERROR: ");
exit(-1);
}else{ // FATHER
close(pipe[1]);
close(0);
dup(pipe[0]);
close(pipe[0]);
}
}
What I want to create is a 'line' of childed processes:
[ls -l] ----pipe----> [head] ----pipe----> [tail -4]
All this processes have a root (the process runing my shell) so, the first father is also a child of the shell process, I'm a bit exhausted already, can anyone help me here please?
I'm not even sure if the childs should be the ones executing the commands.
Thanks guys !!
Nothing complex here, just have in mind that the last command should output to the original process' file descriptor 1 and the first should read from original process file descriptor 0. You just spawn the processes in order, carrying along the input side of the previous pipe call.
So, here's are the types:
#include <unistd.h>
struct command
{
const char **argv;
};
Make a helper function with a simple well defined semantics:
int
spawn_proc (int in, int out, struct command *cmd)
{
pid_t pid;
if ((pid = fork ()) == 0)
{
if (in != 0)
{
dup2 (in, 0);
close (in);
}
if (out != 1)
{
dup2 (out, 1);
close (out);
}
return execvp (cmd->argv [0], (char * const *)cmd->argv);
}
return pid;
}
And here's the main fork routine:
int
fork_pipes (int n, struct command *cmd)
{
int i;
pid_t pid;
int in, fd [2];
/* The first process should get its input from the original file descriptor 0. */
in = 0;
/* Note the loop bound, we spawn here all, but the last stage of the pipeline. */
for (i = 0; i < n - 1; ++i)
{
pipe (fd);
/* f [1] is the write end of the pipe, we carry `in` from the prev iteration. */
spawn_proc (in, fd [1], cmd + i);
/* No need for the write end of the pipe, the child will write here. */
close (fd [1]);
/* Keep the read end of the pipe, the next child will read from there. */
in = fd [0];
}
/* Last stage of the pipeline - set stdin be the read end of the previous pipe
and output to the original file descriptor 1. */
if (in != 0)
dup2 (in, 0);
/* Execute the last stage with the current process. */
return execvp (cmd [i].argv [0], (char * const *)cmd [i].argv);
}
And a small test:
int
main ()
{
const char *ls[] = { "ls", "-l", 0 };
const char *awk[] = { "awk", "{print $1}", 0 };
const char *sort[] = { "sort", 0 };
const char *uniq[] = { "uniq", 0 };
struct command cmd [] = { {ls}, {awk}, {sort}, {uniq} };
return fork_pipes (4, cmd);
}
Appears to work. :)
First, you are prematurely closing the pipes. Close only the end that you don't need in the current process, and remember to close stdin/stdout in the child.
Secondly, you need to remember the fd from the previous command. So, for two processes, this looks like:
int pipe[2];
pipe(pipe);
if ( fork() == 0 ) {
/* Redirect output of process into pipe */
close(stdout);
close(pipe[0]);
dup2( pipe[1], stdout );
execvp(commands[0].argv[0], &commands[0].argv[0]);
}
if ( fork() == 0 ) {
/* Redirect input of process out of pipe */
close(stdin);
close(pipe[1]);
dup2( pipe[0], stdin );
execvp(commands[1].argv[0], &commands[1].argv[0]);
}
/* Main process */
close( pipe[0] );
close( pipe[1] );
waitpid();
Now your job is to add error handling to this and generate n-1 pipes for n processes to start. The code in the first fork() block needs to be run for the appropriate pipe for processes 1..n-1, and the code in the second fork() block for the processes 2..n.
I want to preface this with the fact that I have no formal education in the use of pipes, so this is my first venture. Not to mention that I couldn't find any similar questions to my situation.
Note: This IS part of a larger project for a school assignment, so I am NOT asking for anyone to do this for me. I would just like some direction/helpful code segments. (I have tried to make this as generic as possible to avoid "cheater" remarks.)
I am trying to run a for-loop over int k elements in which a parent process spawns off k children with fork() and execl(), and then use a pipe() to send the output back to the parent. Here is some generic code that I am trying to use and the error/problem in which I encounter:
Note: helloworld= an executable compiled with GCC that produces printf("hello world\n");
int k = 10; //The number of children to make
int fd[2]; //Used for each pipe
int readFromMe[k]; //Holds the file IDs of each pipe fd[0]
int writeToMe[k]; //Holds the file IDs of each pipe fd[1]
int processID[k]; //Holds the list of child process IDs
//Create children
int i;
for(i = 0; i < k; i++)
{
if(pipe(fd) == -1)
{
printf("Error - Pipe error.\n");
exit(EXIT_FAILURE);
}
//Store the pipe ID
readFromMe[i] = fd[0];
writeToMe[i] = fd[1];
if((processID[i] = fork()) == -1)
{
fprintf(stderr, "fork failure");
exit(EXIT_FAILURE);
}
//If it is a child, change the STDOUT to the pipe-write file descriptor, and exec
if(processID[i] == 0)
{
dup2 (writeToMe[i], STDOUT_FILENO);
close(readFromMe[i]);
execl("./helloworld", (char *)0);
}
//If it is the parent, just close the unnecessary pipe-write descriptor and continue itterating
else
{
close(writeToMe[i]);
}
}
//Buffer for output
char output[100000];
//Read from each pipe and print out the result
for(i = 0; i < k; i++)
{
int r = read(readFromMe[i], &output, (sizeof(char) * 100000));
if(r > 0)
{
printf("result = %s\n", output);
}
close(readFromMe[i]);
}
I get no output from my program at all, so I am trying to figure out why this issue is occurring.
Probably unrelated, but you call execl wrong. The extra arguments after the program is what will the the argv array to the other programs main function. And as you know it always have one entry, the program name. So you need to call it like this:
execl("./helloworld", "helloworld", NULL);
More related to your problem, you should also check for errors, it might actually fail.
Try printing the value of 'r' in your printout function. I suspect the read is returning an error (perhaps EPIPE) that you're not seeing. Also, you example code is trying to printf 'c', not output like it looks like you meant.
I am trying to implement a shell in C. I can execute simple commands just fine with a simple execvp() but one of the requirements is to manage commands like this: "ls -l | head | tail -4" with a 'for' loop and only one 'pipe()' statement redirecting stdin and stdout. Now after days I'm a bit lost.
N = Number of simple commands (3 in the example: ls, head, tail)
commands = a list of structs with the commands, like this:
commands[0].argv[0]: ls
commands[0].argv[1]: -l
commands[1].argv[0]: head
commands[2].argv[0]: tail
commands[2].argv[1]: -4
So, I made the for loop, and started to redirect stdin and stdout in order to connect all the commands with pipes, but...I'm just clueless why it doesn't work.
for (i=0; i < n; i++){
pipe(pipe);
if(fork()==0){ // CHILD
close(pipe[0]);
close(1);
dup(pipe[1]);
close(pipe[1]);
execvp(commands[i].argv[0], &commands[i].argv[0]);
perror("ERROR: ");
exit(-1);
}else{ // FATHER
close(pipe[1]);
close(0);
dup(pipe[0]);
close(pipe[0]);
}
}
What I want to create is a 'line' of childed processes:
[ls -l] ----pipe----> [head] ----pipe----> [tail -4]
All this processes have a root (the process runing my shell) so, the first father is also a child of the shell process, I'm a bit exhausted already, can anyone help me here please?
I'm not even sure if the childs should be the ones executing the commands.
Thanks guys !!
Nothing complex here, just have in mind that the last command should output to the original process' file descriptor 1 and the first should read from original process file descriptor 0. You just spawn the processes in order, carrying along the input side of the previous pipe call.
So, here's are the types:
#include <unistd.h>
struct command
{
const char **argv;
};
Make a helper function with a simple well defined semantics:
int
spawn_proc (int in, int out, struct command *cmd)
{
pid_t pid;
if ((pid = fork ()) == 0)
{
if (in != 0)
{
dup2 (in, 0);
close (in);
}
if (out != 1)
{
dup2 (out, 1);
close (out);
}
return execvp (cmd->argv [0], (char * const *)cmd->argv);
}
return pid;
}
And here's the main fork routine:
int
fork_pipes (int n, struct command *cmd)
{
int i;
pid_t pid;
int in, fd [2];
/* The first process should get its input from the original file descriptor 0. */
in = 0;
/* Note the loop bound, we spawn here all, but the last stage of the pipeline. */
for (i = 0; i < n - 1; ++i)
{
pipe (fd);
/* f [1] is the write end of the pipe, we carry `in` from the prev iteration. */
spawn_proc (in, fd [1], cmd + i);
/* No need for the write end of the pipe, the child will write here. */
close (fd [1]);
/* Keep the read end of the pipe, the next child will read from there. */
in = fd [0];
}
/* Last stage of the pipeline - set stdin be the read end of the previous pipe
and output to the original file descriptor 1. */
if (in != 0)
dup2 (in, 0);
/* Execute the last stage with the current process. */
return execvp (cmd [i].argv [0], (char * const *)cmd [i].argv);
}
And a small test:
int
main ()
{
const char *ls[] = { "ls", "-l", 0 };
const char *awk[] = { "awk", "{print $1}", 0 };
const char *sort[] = { "sort", 0 };
const char *uniq[] = { "uniq", 0 };
struct command cmd [] = { {ls}, {awk}, {sort}, {uniq} };
return fork_pipes (4, cmd);
}
Appears to work. :)
First, you are prematurely closing the pipes. Close only the end that you don't need in the current process, and remember to close stdin/stdout in the child.
Secondly, you need to remember the fd from the previous command. So, for two processes, this looks like:
int pipe[2];
pipe(pipe);
if ( fork() == 0 ) {
/* Redirect output of process into pipe */
close(stdout);
close(pipe[0]);
dup2( pipe[1], stdout );
execvp(commands[0].argv[0], &commands[0].argv[0]);
}
if ( fork() == 0 ) {
/* Redirect input of process out of pipe */
close(stdin);
close(pipe[1]);
dup2( pipe[0], stdin );
execvp(commands[1].argv[0], &commands[1].argv[0]);
}
/* Main process */
close( pipe[0] );
close( pipe[1] );
waitpid();
Now your job is to add error handling to this and generate n-1 pipes for n processes to start. The code in the first fork() block needs to be run for the appropriate pipe for processes 1..n-1, and the code in the second fork() block for the processes 2..n.