Pipes and processes - c

Premise:
Write a program to query the user for two input strings. Each input string should be a unix command, with arguments allowed. For example, input 1 could be ls -l and input 2 could be more. The program will then create a pipe, and two child processes. The first child process will run the command specified in the first input. It will output to the pipe instead of standard output. The second child process will run the command specified in the second input. It will take its input from the pipe rather than standard input. The parent process will wait on its two children to complete, then the whole thing will repeat. Execution will stop when the '#' symbol is entered as the first command. Here is the code I have:
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int main(){
/* Program Termination Symbol */
const char terminate = '#';
/* String delimiter */
const char delimiter = ' ';
/* Pipe file ID's */
int fileID[2];
/* Parent ID's */
int pid1, pid2;
/* String token */
char * token, * token2;
/* User input */
char * user_input, line[100];
user_input = (char *) malloc(100);
/* Unix Commands */
char * command1[10], *command2[10];
for (int i=0; i<10; i++)
{
command1[i] = (char *)malloc(100*sizeof(char));
command2[i] = (char *)malloc(100*sizeof(char));
}
/* Begin main program logic */
printf("Please enter the first command: \n");
user_input = gets(line);
while (user_input[0] != terminate)
{
token = (char *) malloc(100*sizeof(char));
for (int i=0; i<10; i++)
{
if (i == 0)
{
token = strtok(user_input, &delimiter);
} else {
token = strtok(NULL, &delimiter);
}
if (token != NULL)
{
strcpy(command1[i], token);
} else {
command1[i] = 0;
}
}
printf("Please enter the second command: \n");
user_input = gets(line);
token2 = (char *) malloc(100*sizeof(char));
for (int i=0; i<10; i++)
{
if (i == 0)
{
token2 = strtok(user_input, &delimiter);
} else {
token2 = strtok(NULL, &delimiter);
}
if (token2 != NULL)
{
strcpy(command2[i], token2);
} else {
command2[i] = 0;
}
}
/* Pipe and execute user commands */
/* Create pipe */
pipe(fileID);
/* Create child processes */
pid1 = fork();
if (pid1 != 0)
{
pid2 = fork();
}
/* First child process */
if (pid1 == 0)
{
dup2(fileID[1], 1);
execvp(command1[0], command1);
}
/* Second child process */
if (pid2 == 0)
{
dup2(fileID[0], 0);
execvp(command2[0], command2);
}
/* Wait for children to terminate */
wait(&pid1);
wait(&pid2);
/* Repeat */
printf("Please enter the first command: \n");
user_input = gets(line);
}
return 0;
}
The problem I'm running into is with my waits. If I have both, which would make sense to me (one wait per child) then the program freezes after executing the first pipe. If I remove the second wait, then the program will begin its loop again, but will not accept keyboard input other than enter, and will produce a segfault. So, with both waits, input and output is...
Please enter the first command:
ls
Please enter the second command:
more
Pipe
Pipe.c
Pipe.c~
...and then it locks up. If I remove the second wait, input/output is...
Please enter the first command:
ls
Please enter the second command:
more
Pipe
Pipe.c
Pipe.c~
Please enter the first command:
(I hit enter, nothing else will work)
Segmentation fault
Anyone have any suggestions? It's clearly related to waiting on the two processes, but I'm at a loss as to how to handle it.
This program is now 100% functional - thank you so much for your help, everyone! Stack overflow has been one of the best resources on the internet. Thank you all so much for taking the time to look over my code and give me your suggestions.

I agree with everything torak said, but to address your problem, you need to close your pipes. I think you are "hanging" because the pipe is still open.
So in the parent, right before the "waits", I would close the pipes.
close(fileID[0]);
close(fileID[1]);
wait(&pid_status);
wait(&pid_status);
Then, right before each execvp, I would close the ends of the pipe the child will not be using:
close(fileID[0]);
dup2(fileID[1], 1);
execvp(command1[0], command1);
close(fileID[1]);
dup2(fileID[0], 0);
execvp(command2[0], command2);
That should resolve your hanging. In addition to the suggestions made by torak, I would also recommend fgets instead of gets to prevent a buffer overflow.

A couple of things. Not sure that they are the cause of your problems, but still things to consider before submitting your homework.
I don't think you are using wait correctly. According to http://linux.die.net/man/2/wait it doesn't take pid pointer as an argument.
Each time around the loop you call malloc for token and token2, but I don't see a corresponding release of the memory.
You've writen a single monolithic function. Good coding practice would suggest breaking it out into a collection of subroutines
Finally, and its possibly related to point 3, the following two lines of code appear twice in your code. Again it's not an error, but unnecessary duplication, and inelegant.
printf("Please enter the first command: \n");
user_input = gets(line);

First of all, you're calling wait from the child processes as well [edit: no, you're not, since each child calls execvp].
Also, wait doesn't take a pointer to the child's pid, but to a variable where the process's status will be written to (which means you're throwing away your child's pid).
Finally, try using waitpid with the "WNOHANG" option. It won't hang, you can put both on a loop while you do other stuff, and you can check to see if the child processes have exited by inspecting the status variables. man waitpid.

Related

C executing a command with another commands output with execvp

I'm fine executing commands like "ls" and stuff like that but I want to do something like "ls | sort" but the execvp system call doesn't support "|". How can I do this using only system calls? when I try something like
char *arg[] = {"ls","|","sort",NULL};
execvp(arg[0],arg);
it doesn't work, how can I do this?
Edit:
char* execString (char string[]){
int link[2];
pipe(link);
if (fork() == 0){
int i = 0;
char *p = strtok(string," ");
char *x[spacecount(string)+2];
while(p){
x[i++] = p;
p = strtok(NULL," ");
}
x[i] = NULL;
dup2(link[1],1);
close(link[0]);
close(link[0]);
execvp(x[0],x);
_exit(0);
} else {
wait(NULL);
close(link[1]);
char buf[512];
int i = 0;
while (read(link[0],&buf[i++],1) == 1);
close(link[0]);
buf[i-2] = '\0';
return strdup(buf);
}
}
This is the function i'm executing to exec a string that contains a command, its return value is a pointer to a string that contains the output from that command, how can I use that output as the input to a new command using execvp or another function from the exec family?
Edit2: So I made a new function that receives two strings as argument and execs the first one then the second one using as input the output from the first exec, I thought it was working fine it worked with ls | head -1 and other variations of ls but when I do something like ls | sort -R it doesn't work, i've tried several things and I can't understand why this is happening, here is the code:
char* execStrings (char previousstring[], char string[]){
int link[2];
pipe(link);
if (fork() == 0){
int i = 0;
char *previouscommand[spacecount(previousstring)+2];
char *temp = strtok(previousstring," ");
while(temp){
previouscommand[i++] = temp;
temp = strtok(NULL," ");
}
previouscommand[i] = NULL;
dup2(link[1],1); /* stdout result redrecting to write end of pipe */
close(link[1]);
close(link[0]);
execvp(previouscommand[0],previouscommand);
} else {
wait(NULL);
int res[2];
pipe(res);
if(fork() == 0){
int i = 0;
char *temp = strtok(string," ");
char *command[spacecount(string)+2];
while(temp){
command[i++] = temp;
temp = strtok(NULL," ");
}
command[i] = NULL;
dup2(link[0],0);
close(link[0]);
close(link[1]);
dup2(res[1],1);
close(res[1]);
close(res[0]);
execvp(command[0],command)
} else {
wait(NULL);
close(res[1]);
char buf[512];
int i = 0;
while (read(res[0],&buf[i++],1) == 1);
close(res[0]);
buf[i-2] = '\0';
return strdup(buf);
}
}
}
you want to do something like ls | sort but the way you are doing like
char *arg[] = {"ls","|","sort",NULL};
execvp(arg[0],arg); /*it won't work */
won't work because here you are calling execvp on ls and sort which are two separate process not single process. Also
ls | sort => output of process-1 make as input to process-2 & execute it
| |
process-1 process-2
To achieve the above create two process by calling fork() and use exec() family function to replace ls and sort in child & parent process.
here is the sample code
int main(void) {
int p[2];
pipe(p);
char *arg[] = {"ls","sort",NULL};
if(fork()==0) {
close(0);/* close the stdin stream so that this
process shoulbn't read from stdin */
dup(p[0]);/* read from read end of pipe */
close(p[1]);
execlp(arg[1],arg[1],(char*)NULL);
}
else{
close(1);/* close the stdout stream, so that o/p shouldn't print on monitor */
dup(p[1]); /* stdout result redrecting to write end of pipe */
close(p[0]);
execlp(arg[0],arg[0],(char*)NULL);
}
return 0;
}
| is a shell feature. You need to do the same thing the shell does, i.e. use pipe, fork, dup2, execvp to create a pipe, spawn a new process, and connect the pipe to the processes' stdin and stdout, respectively.
Take the code from your execString function but replace the else block. In the parent process you should dup2(link[0], 0) (connect the read end of your pipe to stdin), then execvp the other program (e.g. sort).
Remember to close the pipe ends you don't need! I.e. in the producer process, close(link[0]); in the consumer process, close(link[1]). If you forget this part, things may get "stuck" (commands seemingly hanging forever).
Of course, if you want your original program to keep going, you need to wrap that code inside another fork (and waitpid for it).
As an aside, your execString is broken if the command outputs a lot of data. At some point the pipe will get full and the command will pause, waiting for another process to drain the pipe (by reading from it). Your program will be stuck in wait, waiting for the command to terminate. The result is deadlock.
To fix this issue, only call wait after you're done reading from the pipe.

C Fork Usage With User Input

I am trying to write a shell script in
C language. In short details, if user enters a command I send it to the system() call, if user enters more than one command like "ls;whoami" I parse it and create child processes to execute all of them. Now it works but my methods such as gets() and getting input by the user does not seem well and when I put multi commands, prompt text becomes unseen. Do you have any suggestion or if you see any mistakes or wrong usage because I am not the C guy then I would be grateful.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define RESET "\033[0m"
#define BOLDGREEN "\033[1m\033[32m" /* Bold Green */
#define BOLDMAGENTA "\033[1m\033[35m" /* Bold Magenta */
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
void welcome(){
char* welcomestr = "\n\TEST\n\n";
printf("%s%s%s",BOLDMAGENTA,welcomestr,RESET);
}
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
}
int main(int argc, char* argv[]){
inputget=input;
welcome();
prompt();
gets(inputget);
if(argc == 1) {
while(strcmp(inputget, "quit")!=0){
p = strsep(&inputget, ";\n");
while(p != NULL){
pid_t parent = getpid();
pid = fork();
if (pid==-1){
perror("failed to fork");
}
else if (pid==0){
system(p);
exit(0);
}else{
p = strsep(&inputget, ";\n");
}
}
wait(NULL);
prompt();
scanf("%s",input);
inputget=input;
}
exit(0);
}else{
//get argc 2 and and read-run commands from text.file
}
}
Let's begin with the worst part: using gets. Don't use this function anymore,
it's dangerous and deprecated. You should use fgets instead.
fgets(input, sizeof input, stdin);
There is no reason why any of these variables
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
have to be global variables, declare them in main.
Stream buffers like stdout are buffered, content is physically written on
the device once the buffer is full or you call fflush to flush the buffer.
One exception is stdout when is connected to a terminal, in that case printf
will flush immediately when a newline is printed. That's why you almost always
see that the format of printf statements end with \n, like
printf("Your age is %d\n", age);
When you don't want to print a newline, because you are printing something like
a prompt, then you should flush stdout yourself.
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
fflush(stdout); // <-- you need this here
}
The last thing is where you are executing wait. The problem is the
synchronization between the children and the parent process. Once a child is
created, it begins to run immediately. If the child is also printing to stdout
and you don't synchronize with the parent, then there's no guarantee which
output will be printed first.
In your case, if the user enters cmd1;cmd2;cmd3, you are forking 3 times
but you are only doing one wait after you've forked all children. That means
that the three children will run concurrently and the order of their output is
undefined. After all children are forked, you finally do wait(NULL), but this
only waits for one child, then you execute prompt() but remember, the other
children might be still running and hence the output of the prompt might come
before the output of the other children that are running. That is perhaps what
you've been observing.
If you want to emulate the shell, then cmd2 can only start after cmd1 is
finished and cmd3 only after cmd2 is finished. Only when the three commands
are finished you can execute prompt(). That means that you have to wait for
every child to end before the next child can be forked. That's why you have to
move the wait in the parent block before the next fork is called.
// "quit\n" because fgets does not remove the newline
while(strcmp(inputget, "quit\n") != 0) {
p = strsep(&inputget, ";\n");
while(p != NULL) {
if(p[0] == 0)
{
// handles "empty fields", when
// two delimiters come one after
// the other
p = strsep(&inputget, ";\n");
continue;
}
pid = fork();
if (pid==-1) {
perror("failed to fork");
}
else if (pid==0) {
system(p);
exit(0);
} else {
wait(NULL); // <-- here the parent waits
// until the child is finished
p = strsep(&inputget, ";\n");
}
}
prompt();
fgets(input, sizeof input, stdin);
inputget = input;
}
Also note that I'm not using scanf here. scanf("%s"... reads until the first
non-white character, so a command like cat /etc/fstab will only read cat and
your shell will only execute cat and it would block until you close stdin
(by pressing Ctrl+D). The next time, scanf won't wait
for user input and will read /etc/fstab instead and try to execute
/etc/fstab, which will fail, as /etc/fstab is not a script or binary.
That's why it's better to use fgets.
I'd also use a longer buffer for the user input, depending on the command
length, 49 bytes is too short. I'd use 1024 or more. Or you can use getline to
fetch a whole line without the worry about buffer sizes.

Trouble creating a C program using fork() and execvp() functions

Here is the following code I am current having issues with:
#include <stdio.h>
#include <unistd.h>
#include<sys/types.h>
#include<sys/wait.h>
#define MAX_LINE 80
int main(void)
{
char *args[MAX_LINE/2+1];
int background= 0;//integer that acts a boolean for if & appears
int should_run = 1;
int status;
while(should_run)//Just to keep the interface going until the user chooses to stop
{
printf("osh>");//prompt
fflush(stdout);
int i = 0;
while(getchar() != '\n')//Use scanf until a new line is found
{
scanf("%s", &args[i]);
if(strcmp(&args[i], "exit") == 0)//temporary exit protocal
{
printf("Exiting now...");
return 0;
}
if(strcmp(&args[i], "&") == 0)//If we find a & in our input then we changed background to 1 or true
background = 1;
printf("Args[%i] = %s\n", i, &args[i]);//Tester
i++;
}
printf("Background = %i\n",background);//Test
pid_t pid= fork();// Create new child process
printf("process %i created\n", pid);//Test
if(pid < 0)//fork() failed
{
printf("Fork Failed.\n");
return 1;
}
else if(pid == 0)//Child process id
{
printf("Child process started %s command\n", &args[0]);
if(execvp(args[0], args) < 0)//change the current child process to execute the input given by the user
//with args[0] being the command and args being the parameters(ls -l is an example).
{
printf("Command failed\n");
}
return 0;
}
else//Parent Process
{
if(background == 1)//If the user typed in a & then the parent will wait for a change in state from the child, if there is no &
//then we will just finish the parent process
{
printf("Parent process waiting on child\n");
wait(NULL);
}
}
}
return 0;
I have one major issue and one minor issue right now. The major issue is that I have a printf method before execvp starts that says "Child Process started" and I get this line to print, but then nothing else happens. No interrupts are thrown, the program just seems to be frozen on my execvp command.
My minor issue is that when my program starts a prompt "osh>" before asking for input. Now if, for example, I would type in "osh>ls -l" then I get args[0] = s, args1 = -l. Now if I put "osh> ls -l" in that exact format I get args[0] = ls, args1 = -l. Is that a part of scanf() that I am not using properly here to make sure I get ever character after "osh>" and between blank spaces as strings?
EDIT:
here is my output for user input "ls -l"
The problem you're having with the missing character is because getchar() is consuming the first character of your input before scanf gets to take a stab at it. You probably want to do something like:
while (scanf("%s", &buffer) > 0)
{
strcpy(args[i], buffer);
/* then do stuff with args[i] */
}

fork() system call and while loop ( part 2)

Does anyone know why the printf("Type q to quit") line prints twice in the terminal when I run this code:
#include <stdio.h>
#include <unistd.h>
int main (int argc, char *argv[])
{
char run[2];
run[0]='a';
int pid=0;
while (run[0]!= 'q')
{
printf("Type q to quit \n");
fgets (run, 2, stdin);
pid=fork();
//wait();
if(pid==0) { break;}
}
}
I would like the child to break from the loop and the parent to continue looping (to create new children). If I call wait() the execution ends after the first iteration regardless of whether I enter 'q' or not. Otherwise it works as expected but prints the "Type q to quit" line twice every time. Why does this happen?
You have three bugs. First, the carriage return that the user types, after whatever letter, counts as input. Assuming the user types only one character per line, your fgets calls will alternate between returning the character you care about, and returning '\n'. You need to read and discard characters until you reach the end of each line of input, on each iteration. This is what is causing the double printouts.
Second, you need to test for 'q' in between reading from stdin and calling fork; right now, you fork once more after reading the 'q'. Right now this is invisible, but once the child process starts doing something useful it won't be.
Third, this program will go into an infinite loop if the user types ^D at it, because you're not checking for EOF.
Putting that together, corrected code looks like this. I've also fixed some style nits, and arranged so that the parent process exits immediately rather than dropping out of the for loop; this means that when control reaches the point marked with a comment, you know you're in a child process, rather than in the parent on its way out.
#include <stdio.h>
#include <unistd.h>
int
main(void)
{
pid_t pid;
int c;
for (;;)
{
puts("Type q to quit");
c = getchar();
if (c == 'q')
return 0;
if (c == '\n')
continue;
while (c != EOF && c != '\n')
c = getchar();
if (c == EOF)
return 0;
pid = fork();
if (pid == 0)
break;
else if (pid == -1)
{
perror("fork");
return 1;
}
}
/* control reaches this point only in child processes */
return 0;
}
When you type x<enter> on your terminal, you'll get (assuming ordinary encodings all over) two bytes sent to your process: one for the x, one for the newline.
Your fgets call reads at most one byte (so, the x), forks of a child that dies "instantly", prints the messages and calls fgets. fgets picks up where it left: it reads the newline char without blocking, your code forks again for no reason, and loops back.
At that point there's nothing left in the input stream, so fgets waits for I/O.
See for example: I am not able to flush stdin for ways to "clear out" the input stream that you could use here.
You need to wait for these forked process down the line:
int main (int argc, char *argv[])
{
char run[2];
run[0]='a';
int pid=0;
int pids[256]; // should be enough.
int j,i = 0;
while (run[0]!= 'q')
{
printf("Type q to quit \n");
fgets (run, 2, stdin);
pid=fork();
//wait();
if(pid==0) { break;}
else { pids[i] = pid; i++; }
}
for (j = 0 ; j < i && pid != 0 ; j++)
wait(pids[j]);
if(pid == 0){
// do something
}
}
You could instead have the child call another function (such as a program wrapper or even itself in non-forkable version) instead of breaking. The waiting would remain the same i.e. wait only once all forks are, well, forked.

Multiprocess, Forks and Pipes in C

I posted a question previously about using fork() and pipes in C
. I changed the design a little bit so that it reads a regular txt file and sorts the words in the file. So far, this is what I came up with:
for (i = 0; i < numberOfProcesses; ++i) {
// Create the pipe
if (pipe(fd[i]) < 0) {
perror("pipe error");
exit(1);
}
// fork the child
pids[i] = fork();
if (pids[i] < 0) {
perror("fork error");
} else if (pids[i] > 0) {
// Close reading end in parent
close(fd[i][0]);
} else {
// Close writing end in the child
close(fd[i][1]);
int k = 0;
char word[30];
// Read the word from the pipe
read(fd[i][0], word, sizeof(word));
printf("[%s]", word); <---- **This is for debugging purpose**
// TODO: Sort the lists
}
}
// Open the file, and feed the words to the processes
file_to_read = fopen(fileName, "rd");
char read_word[30];
child = 0;
while( !feof(file_to_read) ){
// Read each word and send it to the child
fscanf(file_to_read," %s",read_word);
write(fd[child][1], read_word, strlen(read_word));
++child;
if(child >= numberOfProcesses){
child = 0;
}
}
where numberOfProcesses is a command-line argument. So what it does is that it reads each word in the file and send it to a process. This however, does not work. When I print the word in the child process, it doesn't give me the correct output. Am I writing/reading the words correctly to/from the pipes?
Are the words being printed in the wrong order or interleaved? The thing is that when you write a word to a pipe, you are expecting the process handling that pipe to be scheduled immediately and to print the word. Then you expect the main process to run again and the next word to be written to the next pipe etc.
But that is not guaranteed to happen. Your main loop might write all of the words to all of the pipes before any of the other processes is scheduled. Those processes might not be scheduled in the order you expect. And the printf calls might interfere with each other so that their output becomes interleaved.
If you really want to do what you set out to, then Posix threads would be better. If you just wanted to learn something about using multiple processes, then I guess you have :-)
In the parent, you write strlen() bytes, which might be less than 30 bytes. In the child, however, you always try to read 30 bytes. You also must NUL-terminate the word, otherwise you might see garbage or a runaway string in your printf() statement.
In the child, you must either parse and split the input at word boundary or use stdio as #JonathanLeffler proposed. When you use stdio, you get all this buffering and word reading for free.
int n;
char word[31];
/* Read the word from the pipe */
n = read(fd[i][0], word, sizeof(word) - 1);
if (n == -1) {
perror("read");
/* do error handling */
} else {
word[n] = 0;
printf("[%s]", word);
}

Resources