I have Implemented a simple program which simulates $ls -l | wc -c command execution using simple pipes and execvp calls.
Now After redirecting stdin and stdout ,when executes the program , shell prompt disappears and it waits for the enter key to be pressed.
Any way of solving this issue . Plz criticize my code also..
Thanks
/* Create pipe */
ret_val=pipe(pipe_fd);
/*Error check */
if(ERROR==ret_val)
{
perror("Pipe creation error \n");
_exit(FAILURE);
}
/*Fork First Child */
pid_one = fork() ;
if( 0 == pid_one ) /*child process block */
{
/* First sub process */
/*printf("First sub process is %d \n",getpid());*/
/* redirect stdout to pipe's write end for sub process one*/
dup2(pipe_fd[1],1);
/*close pipe read end */
close(pipe_fd[0]);
execvp(cmd_one_tokens[0],cmd_one_tokens);
/* if execvp returns then if must have failed */
printf("Unknown Command \n ");
//exit(10);
}
else /*main process block */
{
/*printf(" Main process is %d \n",getpid());*/
/*Wait for first sub process to finish */
//wait(&status);
/*printf("Exit status of first child is %d \n ", WEXITSTATUS(status) );*/
/*Fork second subprocess */
pid_two = fork();
if( 0 == pid_two ) /*second child process block */
{
/* redirect stdin to pipe's read end for sub process two */
dup2(pipe_fd[0],0);
// close(0); /* close normal stdin */
// dup(pipe_fd[0]); /* make stdib same as pfds[0] */
/*close pipe write end */
close(pipe_fd[1]);
/* Second sub process */
/*printf("Second sub process is %d \n",getpid()); */
execvp(cmd_two_tokens[0] , cmd_two_tokens);
/* if execvp returns then if must have failed */
printf("Unknown Command \n ");
}
else /*main process block */
{
/* printf(" Main process is %d \n",getpid()); */
status=-1; /*reset status */
/*Waiting for the second sub process to finish in No hang fashion */
waitpid ( -1 , &status ,WNOHANG);
/*printf("Exit status of second child is %d \n ", WEXITSTATUS(status) ); */
}
}
You have to close the pipe file descriptors in the main process, after the second is forked. Until you have closed them, the child process (wc) will wait for input on the pipe which the main process still has open. You must be very careful to close all the unneeded ends of the pipe.
Your code does not do what you describe you want to do:
You create a pipe, fork a new process, redirect it's stdout to the pipe and make it execute some program (so far so good), then in the parent process you wait for your child to finish and only then fork second process, redirect it's stdin to the pipe other end and make it execute another program.
This is not what "ls | wc" does - in the shell they are running concurrently. Remove the first wait().
pid_one = fork() ;
if( 0 == pid_one ) /*child process block */
You're not checking fork(2) for an error return, which is a very real possibility. (The user could be bumping up against their RLIMIT_NPROC limit, kernel.threads-max, run out of memory for holding task structures, etc.)
More idiomatic use of fork(2) looks like this:
if(-1 == (pid = fork()) {
perror("fork");
exit(1); /* or return -1 or similar */
} else if (0 == pid) {
/* executing as child */
} else {
/* parent, pid is child */
}
execvp(cmd_two_tokens[0] , cmd_two_tokens);
/* if execvp returns then if must have failed */
printf("Unknown Command \n ");
Note that there are many reasons why execvp(3) can fail; simply printing "Unknown Command" may leave your users very confused in the future. It'd be better to call perror("execvp"); and give your users a chance to discover the real reason why their execvp(3) call failed.
waitpid ( -1 , &status ,WNOHANG);
Use of WNOHANG here might be dangerous; if the system is running "just right", your parent might get to this code before the child has even had a chance to begin executing. Because you've asked for it to return immediately if no child has exited, the child will probably be turned into a zombie when it finally does exit -- your code doesn't take the opportunity to wait for the child again.
I'm not sure what the best solution is: if you use SA_NOCLDWAIT to sigaction(2) to avoid creating zombies completely, you won't have an opportunity to ever collect the child's exit status. Installing a SIGCHLD signal handler might interfere with the rest of the process; your clients might have reason to set it themselves. Using a blocking waitpid(2) might stall processing elsewhere. And using a non-blocking waitpid(2) means you still have to collect the child's status sometime, possibly through polling. (But you can't use -1 for the pid in that case, as you might accidentally reap another child process.)
Related
Like many others, I'm trying to simulate a shell. I've gotten to use the execvp properly on a string coming from the user. The string is parsed and an array of strings is generated (each word has its array, split on the space character), including a NULL at the very end.
When I find that the last word entered by the user is &, I set a flag up to notify my shell that the command is to be executed in the background while letting the user input another command right away. The "background-executed" command sees its & replaced by a NULL character within the array of strings passed to execvp.
As it is, I've been trying to use a pthread to run the process in the background, but it's acting somewhat weird: the command passed to execvp through the thread's function requires me to press two times ENTER after sending the command.
Here is my simplified main function that is to simulate a shell:
int main (void) {
fprintf (stdout, "%% ");
bool running = true;
while(running) {
/* Ask for an instruction and parses it. */
char** args = query_and_split_input();
/* Executing the commands. */
if (args == NULL) { // error while reading input
running = false;
} else {
printf("shell processing new command\n");
int count = count_words(args);
split_line* line = form_split_line(args, count);
Expression* ast = parse_line(line, 0, line->size - 1);
if(line->thread_flag) {
pthread_t cmd_thr;
/* Setting up the content of the thread. */
thread_data_t thr_data;
thr_data.ast = *ast;
thr_data.line = *line;
/* Executing the thread. */
int thr_err;
if ((thr_err = pthread_create(&cmd_thr, NULL, thr_func, &thr_data))) {
fprintf(stderr, "error: pthread_create, rc: %d\n", thr_err);
return EXIT_FAILURE;
}
printf("thread has been created.\n");
} else {
run_shell(args);
}
free(line);
printf("done running shell on one command\n");
}
}
/* We're all done here. See you! */
printf("Bye!\n");
exit (0);
}
Here is my thread's function:
void *thr_func(void *arg) {
thread_data_t *data = (thread_data_t *)arg;
data->line.content[data->line.size-1] = NULL; // to replace the trailing '&' from the command
run_shell(data->line.content);
printf("thread should have ran the command\n");
pthread_exit(NULL);
}
And the actual line that runs a command:
void run_shell(char** args) {
/* Forking. */
int status;
pid_t pid; /* Right here, the created THREAD somehow awaits a second 'ENTER' before going on and executing the next instruction that forks the process. This is the subject of my first question. */
pid = fork();
if (pid < 0) {
fprintf(stderr, "fork failed");
} else if (pid == 0) { // child
printf("Child executing the command.\n");
/* Executing the commands. */
execvp(args[0], args);
/* Child process failed. */
printf("execvp didn't finish properly: running exit on child process\n");
exit(-1);
} else { // back in parent
waitpid(-1, &status, 0); // wait for child to finish
if (WIFEXITED(status)) { printf("OK: Child exited with exit status %d.\n", WEXITSTATUS(status)); }
else { printf("ERROR: Child has not terminated correctly. Status is: %d\n", status); }
free(args);
printf("Terminating parent of the child.\n");
}
}
So basically, as an example, what run_shell(args) receives is either ["echo","bob","is","great",NULL] (in the case of a sequential execution) or ["echo","bob","is","great",NULL,NULL] (in the case of a command to be executed in the background).
I've left the printf traces since it might help you understand the execution flow.
If I input echo bob is great, the output (printf traces) is:
shell processing new command
Child executing the command.
bob is great
OK: Child exited with exit status 0.
Terminating parent of the child.
done running shell on one command
However, if I input echo bob is great &, the output is:
shell processing new command
thread has been created.
done running shell on one command
And then I actually need to press ENTER again to obtain the following output:
Child executing the command.
bob is great
OK: Child exited with exit status 0.
Terminating parent of the child.
thread should have ran the command
(On that last execution, I also get traces of my function that queries and parses the input of the user, but that seemed irrelevant so I abstracted this whole part.)
So my questions are:
How comes the created thread awaits a second ENTER before running the execvp ? (thr_func stops executing run_shell and awaits the second ENTER right before the pid = fork(); instruction)
Do I have the right approach to solve the problem at hand? (Trying to execute a shell command in the background.)
You cannot use a thread to simulate a process. Well, strictly you can, but there's no use on doing that. The problem is that all the threads belonging to a process share the same virtual address space. There's no reason to create a thread, as you finally need to fork() to create a new process (you'll need this for reasons explained below), so why to create two threads of execution if one of them will be stopped all the time just waiting for the subprocess to finish. There's no use on this schema.
The need of a fork() system call comes historically to make a simple call to create a new process (with different virtual memory map) to allow for a new program to be able to be executed. You need to create a new, complete process before calling exec(2) system call, because the process address space will be overwritten by the text and data segments of the new program. If you do this in a thread, you'll be overwriting the whole process address space (this is the shell) and killing all the threads you can have running on behalf of that process. The schema to follow is (pseudocode):
/* create pipes for redirection here, before fork()ing, so they are available
* in the parent process and the child process */
int fds[2];
if (pipe(fds) < 0) { /* error */
... /* do error treatment */
}
pid_t child_pid = fork();
switch(child_pid) {
case -1: /* fork failed for some reason, no subprocess created */
...
break;
case 0: /* this code is executed in the childd process, do redirections
* here on pipes acquired ***before*** the fork() call */
if (dup2(0 /* or 1, or 2... */, fds[0 /* or 1, or 2... */]) < 0) { /* error */
... /* do error management, considering you are in a different process now */
}
execvpe(argc, argv, envp);
... /* do error management, as execvpe failed (exec* is non-returning if ok) */
break; /* or exit(2) or whatever */
default: /* we are the parent, use the return value to track the child */
save_child_pid(child_pid);
... /* close the unused file descriptors */
close(fds[1 /* or 0, or 2, ... */]);
... /* more bookkeeping */
/* next depends on if you have to wait for the child or not */
wait*(...); /* wait has several flavours */
} /* switch */
Exec and fork system calls are separated by two reasons:
you need to be able to do housekeeping between both calls to execute the actual redirections in the child before exec().
there was a time when unix was not multitasking or protected, and the exec call just replaced all the memory in the system with the new program to execute (including kernel code, to cope with the fact that an unprotected system could be corrupted by the executing program) This was common in old operating systems and I've seen it on systems like CP/M or TRS-DOS. The implementation in unix conserved almost all the semantics of exec() call and added with fork() the unavailable functionality only. This was good, as it allowed both, parent and child processes to do the necessary bookkeeping when the time for pipes came.
Only if you need a different thread to communicate with each child is when you probably can use a different thread to do the task. But think that a thread shares all the virtual space with the parent (case we can talk about a parent/child relationship between threads) and if you do an exec call you'll get that virtual space overwritten for the whole process (all threads there)
I'm having trouble with my program waiting for a child process (gzip) to finish and taking a very long time in doing so.
Before it starts waiting it closes the input stream to gzip so this should trigger it to terminate pretty quickly. I've checked the system and gzip isn't consuming any CPU or waiting on IO (to write to disk).
The very odd thing is the timing on when it stops waiting...
The program us using pthreads internally. It's processing 4 pthreads side by side. Each thread processes many units of work and for each unit of work one it kicks off a new gzip process (using fork() and execve()) to write the result. Threads hang when gzip doesn't terminate, but it suddenly does terminate when other threads close their instance.
For clarity, I'm setting up a pipeline that goes: my program(pthread) --> gzip --> file.gz
I guess this could be explained in part by CPU load. But when processes are kicked off minutes apart and the whole system ends up using only 1 core of 4 because of this locking issue, that seems unlikely.
The code to kick off gzip is below. The execPipeProcess is called such that the child writes direct to file, but reads from my program. That is:
execPipeProcess(&process, "gzip", -1, gzFileFd)
Any suggestions?
typedef struct {
int processID;
const char * command;
int stdin;
int stdout;
} ChildProcess;
void closeAndWait(ChildProcess * process) {
if (process->stdin >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdin)) {
exitError(-1,errno, "Failed to close stdin for %s", process->command);
}
}
if (process->stdout >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdout)) {
exitError(-1,errno, "Failed to close stdout for %s", process->command);
}
}
int status;
stdLog("waiting on post process %d", process->processID);
if (waitpid(process->processID, &status, 0) == -1) {
exitError(-1, errno, "Could not wait for %s", process->command);
}
stdLog("post process finished");
if (!WIFEXITED(status)) exitError(-1, 0, "Command did not exit properly %s", process->command);
if (WEXITSTATUS(status)) exitError(-1, 0, "Command %s returned %d not 0", process->command, WEXITSTATUS(status));
process->processID = 0;
}
void execPipeProcess(ChildProcess * process, const char* szCommand, int in, int out) {
// Expand any args
wordexp_t words;
if (wordexp (szCommand, &words, 0)) exitError(-1, 0, "Could not expand command %s\n", szCommand);
// Runs the command
char nChar;
int nResult;
if (in < 0) {
int aStdinPipe[2];
if (pipe(aStdinPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdin = aStdinPipe[PIPE_WRITE];
in = aStdinPipe[PIPE_READ];
}
else {
process->stdin = -1;
}
if (out < 0) {
int aStdoutPipe[2];
if (pipe(aStdoutPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdout = aStdoutPipe[PIPE_READ];
out = aStdoutPipe[PIPE_WRITE];
}
else {
process->stdout = -1;
}
process->processID = fork();
if (0 == process->processID) {
// child continues here
// these are for use by parent only
if (process->stdin >= 0) close(process->stdin);
if (process->stdout >= 0) close(process->stdout);
// redirect stdin
if (STDIN_FILENO != in) {
if (dup2(in, STDIN_FILENO) == -1) {
exitError(-1, errno, "redirecting stdin failed");
}
close(in);
}
// redirect stdout
if (STDOUT_FILENO != out) {
if (dup2(out, STDOUT_FILENO) == -1) {
exitError(-1, errno, "redirecting stdout failed");
}
close(out);
}
// we're done with these; they've been duplicated to STDIN and STDOUT
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execvp(words.we_wordv[0], words.we_wordv);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exitError(-1, errno, "could not run %s", szCommand);
} else if (process->processID > 0) {
wordfree(&words);
// parent continues here
// close unused file descriptors, these are for child only
close(in);
close(out);
process->command = szCommand;
} else {
exitError(-1,errno, "Failed to fork");
}
}
Child process inherits open file descriptors.
Every subsequent gzip child process inherits not only pipe file descriptors intended for communication with that particular instance but also file descriptors for pipes connected to previous child process instances.
It means that stdin pipe is still open when the main process performs close since there are some other file descriptors for the same pipe in a few child processes. Once those ones terminate the pipe is finally closed.
A quick fix is to prevent child processes from inheriting pipe file descriptors intended for the master process by setting close-on-exec flag.
Since there are multiple threads involved spawning child processes should be serialized to prevent child process from inheriting pipe fds intended for another child process.
You have not given us enough information to be sure, as the answer depends on how you use the functions presented. However, your closeAndWait() function looks a bit suspicious. It may be reasonable to suppose that that the child process in question will exit when it reaches the end of its stdin, but what is supposed to happen to data it has written or even may still write to its stdout? It is possible that your child processes hang because their standard output is blocked, and it is slow for them to recognize it.
I think this reflects a design problem. If you are capturing the child processes' output, as you seem at least to support doing, then after you close the parent's end of a child's input stream you'll want the parent to continue reading the child's output to its end, and performing whatever processing it intends to do on it. Otherwise you may lose some of it (which for a child performing gzip would mean corrupted data). You cannot do that if you make closing both streams part of the process of terminating the child.
Instead, you should to close the parent's end of the child's stdin first, continue processing its output until you reach its end, and only then try to collect the child. You can make closing the parent's end of the child's output stream part of the process of collecting that child if you like. Alternatively, if you really do want to discard any remaining output from the child, then you should drain its output stream between closing the input and closing the output.
I need a little help with my plumbing.
I'm trying to create a program that begins with a single process, spawns child processes based on a user-defined number, and then flow back into another (single) child process. The data flow looks something like this:
/-->-█->-\
█-->--█->--█
\-->-█->-/
I've got the first part of the process creation finished. The fork works well - I run it through a loop limited to the number specified by the user. It's the piping that's throwing me off.
For simplicity, I'm focusing on the first part (from parent to multiple children). So I create the pipe before forking the process - that much is given. Then I close the write-end of the child process, close stdin, dup so the child's 0 redirects to stdin, then close the child's 0. That should take care of the plumbing on the child-side, right?
So then back in the parent process, I need to plumb for the pipes going to the children. For whatever reason, this is the harder part for me. Would someone mind walking me through it a little?
Here's what I've got for this part of the code:
for (i = 0; i < numberOfChildren; ++i) {
(void) pipe(workFDs[i]); /* Creates a pipe before the fork */
if ((workPIDs[i] = fork()) < 0) {
perror("Error: failure when forking workPID #: \n");
exit(-1);
}
if (workPIDs[i] == 0) {
/* ************************* WORKER PROCESS *********************** */
close(workFDs[i][1]); /* Closes the write-end for worker proc */
close(0); /* Closes stdin */
dup(workFDs[i][0]); /* Redirects workFDs 0 to stdin */
close(workFDs[i][0]);
//Fgets to get from the pipe
//Exec sort stuff here
}
} else {
/* *********************** PARENT PROCESS *********************** */
assert(inputPID == getpid()); /* Just to be sure */
close(workFDs[i][0]);
//Fputs to put into the pipe
}
}
I want to send exit status 0 from the execve process to the process which started it.
Because on Success execve never returns, So I am not able to do anything after that. But I want if execve ran successful.
You can verify that exec succeeded using FD_CLOEXEC:
create a pipe.
fork.
in the child, close the read end of the pipe and set the FD_CLOEXEC flag on the write end of the pipe. Then proceed with exec. If exec succeeds, the pipe will be automatically closed due to FD_CLOEXEC. If exec fails, write a byte to the pipe and exit.
in the parent, close the write end of the pipe and read from the read end. If you read 0 bytes (EOF), it means that exec succeeded. If you read a byte in the pipe, it means that exec failed.
The data written over the pipe in case of failure can be used to transmit error information, such as value of errno after exec.
The code, with error checking omitted for brevity, would look like this:
int pipe_fds[2];
pipe(pipe_fds);
if (!fork()) {
close(pipe_fds[0]);
fcntl(pipe_fds[1], F_SETFD, F_CLOEXEC);
execve(...);
write(pipe_fds[1], "", 1);
_exit(1);
}
else {
int n;
char out;
close(pipe_fds[1]);
n = read(pipe_fds[0], &out, 1);
if (n == 0)
printf("exec successful\n");
else
printf("exec failed\n");
close(pipe_fds[0]);
}
Note that this technique can be unsafe if other threads may exec their own processes in parallel. The issue is that there is a delay between when the pipe is created and when the close-on-exec flag gets set. If an unrelated thread forks and execs during that critical window, the child will inherit pipe_fds[1] and won't close it, causing the parent to hang. This can be fixed using the Linux-specific pipe2 call which allows atomically creating the pipe with the close-on-exec flag set.
to wait you process launched by fork() finish, you can use wait():
/* parent */
int status;
while (wait(&status) != uproc.pid) {
printf("waiting for child to exit");
}
and based on this question
The exit status of the child is provided by the wait function, in the status variable.
You get the exit status by using the WEXITSTATUS macro, but only if the program exited normally (i.e. called exit or returned from its main function):
if (WIFEXITED(status))
printf("Child exit status: %d\n", WEXITSTATUS(status));
else
printf("Child exited abnormally\n");
I am trying to create a simple client/server program that allows the client to connect to the server using a TCP socket and then allows the user to issue system calls form the client side to the server side and return the reply to the user. For example:
Client issues: ls
Server will find ls in /usr/bin or w/e and then execute it using execve()
I will also have something liks lls, or lmkdir, ect..which will issue the system calls on the client side.
The problem is my execve() is not appearing to run correctly because 'ls' or any other command is not actually being called. I have done this same kind of program before with only a local side (no server or anything) and execve() worked fine. Here is some code:
pid = fork();
if(pid){ // Child
printf("child wait");
pid = wait(&status);
printf("Child dead\n");
}else{ // Parent
if(execPath){
execve(execPath, arglist, env);
printf("Command Complete\n");
}
}
For some reason the printfs in the child section of the PID statement are not executing at all. I do not think the system is actually ever forking a process. Is there something special I would have to do to make this work since it is a client/server type of program or should it work exactly the same?
Thanks
exactly, execve does not fork. It replaces current image with the one specified as its argument and starts from its start (i.e. main()). It never returns to your origial program.
You probably want to use system() in your use case.
There are several problems in the code:
fork() returns pid for the parent and zero for the child. So parent runs the true branch of the if. And child runs the else branch. Swap those comments.
The stdout is line buffered. Add new line (\n) to printf which is before the wait. Or else you don't see the printout before waiting is done and 2nd printf is under call.
Be sure that child will exit also in error cases, or else the child will run the code of parent, and parent is still waiting exit of the child.
execve does not return if it success. It will return, if it fails.
So, fixed code could be something like that:
pid = fork();
if(pid){ // Parent
printf("child wait\n");
pid = waitpid(pid, &status, 0);
printf("Child dead\n");
}else{ // Child
if(execPath){
execve(execPath, arglist, env);
printf("execve failed!\n");
}
_exit(1);
}
Or you could use system(3).
Since the child process has not spawned any children of its own, the wait() call is unlikely to return without some other external event (like a signal interrupting the call). You should have the parent wait on the child process instead.
Note that fork() may fail, and you should account for that. Also note that if execve succeeds, it won't return. So, the print statement after it should indicate failure if it is to print anything at all.
Using system() probably would not save you the fork, since you are likely to want the output of the command to be directed to the socket associated with the connected client. But, your code is missing the steps that would allow the output to flow to the client.
switch ((pid = fork())) {
case -1: /* todo: handle error */
break;
case 0: /* child */
dup2(socket, 0); /* todo: check return value */
dup2(socket, 1); /* todo: check return value */
dup2(socket, 2); /* todo: check return value */
close(socket); /* todo: check return value */
execve(...);
/* todo: handle failure */
exit(EXIT_FAILURE);
default: /* parent */
if (pid != waitpid(pid, 0, 0)) {
/* todo: handle error */
}
}