I want to execute a command using system() command or execl and want to capture the output directly in a buffer in C. Is ther any possibility to capture the output in a buffer using dup() system call or using pipe(). I dont want to use any file in between using mkstemp or any other temporary file. please help me in this.Thanks in advance.
I tried it with fork() creating two process and piping the output and it is working.However I dont want to use fork system call since i am going to run the module infinitely using seperate thread and it is invoking lot of fork() and system is running out of resources sometimes after.
To be clear about what i am doing is capturing an output of a shell script in a buffer processing the ouput and displaying it in a window which i have designed using ncurses.Thankyou.
Here is some code for capturing the output of program; it uses exec() instead of system(), but that is straightforward to accomodate by invoking the shell directly:
How can I implement 'tee' programmatically in C?
void tee(const char* fname) {
int pipe_fd[2];
check(pipe(pipe_fd));
const pid_t pid = fork();
check(pid);
if(!pid) { // our log child
close(pipe_fd[1]); // Close unused write end
FILE* logFile = fname? fopen(fname,"a"): NULL;
if(fname && !logFile)
fprintf(stderr,"cannot open log file \"%s\": %d (%s)\n",fname,errno,strerror(errno));
char ch;
while(read(pipe_fd[0],&ch,1) > 0) {
//### any timestamp logic or whatever here
putchar(ch);
if(logFile)
fputc(ch,logFile);
if('\n'==ch) {
fflush(stdout);
if(logFile)
fflush(logFile);
}
}
putchar('\n');
close(pipe_fd[0]);
if(logFile)
fclose(logFile);
exit(EXIT_SUCCESS);
} else {
close(pipe_fd[0]); // Close unused read end
// redirect stdout and stderr
dup2(pipe_fd[1],STDOUT_FILENO);
dup2(pipe_fd[1],STDERR_FILENO);
close(pipe_fd[1]);
}
}
A simple way is to use popen ( http://www.opengroup.org/onlinepubs/007908799/xsh/popen.html), which returns a FILE*.
You can try popen(), but your fundamental problem is running too many processes. You have to make sure your commands finish, otherwise you will end up with exactly the problems you're having. popen() internally calls fork() anyway (or the effect is as if it did).
So, in the end, you have to make sure that the program you want to run from your threads exits "soon enough".
You want to use a sequence like this:
Call pipe once per stream you want to create (eg. stdin, stdout, stderr)
Call fork
in the child
close the parent end of the handles
close any other handles you have open
set up stdin, stdout, stderr to be the appropriate child side of the pipe
exec your desired command
If that fails, die.
in the parent
close the child side of the handles
Read and write to the pipes as appropriate
When done, call waitpid() (or similar) to clean up the child process.
Beware of blocking and buffering. You don't want your parent process to block on a write while the child is blocked on a read; make sure you use non-blocking I/O or threads to deal with those issues.
If you are have implemented a C program and you want to execute a script, you want to use a fork(). Unless you are willing to consider embedding the script interpreter in your program, you have to use fork() (system() uses fork() internally).
If you are running out of resources, most likely, you are not reaping your children. Until the parent process get the exit code, the OS needs keeps the child around as a 'zombie' process. You need to issue a wait() call to get the OS to free up the final resources associated with the child.
Related
How can I close my child file descriptors when killing the parent process?
I've created a program that does the following:
Fork 2 child processes.
Process 1 is a reader. It reads from STDIN_FILENO and writes to STDOUT_FILENO with scanf/printf. But I use dup2 to redirect to a named pipe (let's call it npipe).
Process 2 is a writer. I redirect STDIN_FILENO to npipe so that I can use scanf from that pipe. I redirect STDOUT_FILENO to another file called "output_file" so that I can use printf to write.
Reading / Writing is done in a while(1) loop;
while(1){
scanf("%s",word);
printf("%s",word);
}
If I use CTRL+C (send SIGTERM signal to the parent), any data that was written in "output_file" is lost. Nothing is inside there. I think it's because I kill the processes while the files are opened?
I used a word that stops both child processes when read; in this case, I do have everything in my "output_file".
How can I close the files when using SIGTERM? / Can I somehow force my process to write into "output_file" so that no data is lost when suddenly closing?
I tried closing and opening the file after each writing. Still losing all the written data? Is this because of the redirect?
void read_from_pipe(void)
{
char command[100];
int new_fd = open(pipe_name,O_RDONLY,0644);
int output_file = open("new_file",O_CREAT|O_WRONLY|O_APPEND,0644);
dup2(new_fd,0);
dup2(output_file,1);
while(1)
{
scanf("%s",command);
if(strcmp(command,"stop")==0)
{
close(output_file);
close(new_fd);
exit(0);
}
else
printf("%s ",command);
close(output_file);
output_file = open("new_file",O_CREAT|O_WRONLY|O_APPEND,0644);
}
}
Managed to attach correct signal handlers, yet data is still not written even though I close my files! Why is that?
You problem is (likely) not to do with closing the file descriptors, it's with flushing the FILE pointers. Note that FILE pointers and file descriptors are two different things.
You call printf, which does NOT write to STDOUT_FILENO (a file descriptor), it writes to stdout (a FILE pointer), which generally just buffers the data until it has a reason to write it to the underlying file descriptor (STDOUT_FILENO).
So you need to ensure your data that you write to stdout is actually written to STDOUT_FILENO. The easiest way to do that is to stick an fflush(stdout); after the printf call to manually flush it, giving you fine-grained control over when stuff is written. Alternately, you can call setbuf(stdout, 0); to make stdout unbuffered, so every write will be flushed immediately.
Shortened Question:
I have a parent process that creates a child process as seen below:
int me2them[2], them2me[2];
pipe(me2them);pipe(them2me);
if (!fork()){
close(0); dup2(me2them[0],0); close(me2them[0]);
close(1); dup2(them2me[1],1); close(them2me[1]);
char * cmds[] = {"wish", "myProg.tcl",NULL};
execvp(cmds[0], cmds);
fprintf(stderr, "Unable to exec 1\n");
exit(-1);
}
close(0); dup2(them2me[0],0); close(them2me[0]);
close(1); dup2(me2them[1],1); close(me2them[1]);
But, I need the child process to be able to recieve input from the user. With this method, the stdin for the child is changed from the keyboard to the stdout of the parent. How can I maintain communication with both the keyboard and the parent?
Also, the parent is the client of a server, and thus multiple parents can be running on the same or different machines, making a shared file between parent and child difficult because the child of any parent would be able to access any other parent's file.
NOTE: I'd prefer to keep the parent's stdout being mapped to the child's input because I did not write the c code and I want to re-route its printf statements to the child.
Original Version:
I am using tcl to make a GUI for a c code. The tcl is a child process of the c code and I use I/O redirection to make the stdout of the c to be the stdin of the tcl and the stdout of the tcl to be the stdin of the c. However, there is a part where the c requests the user's name and it sends the request via stdout to the stdin of the tcl code, no problems, then the tcl requests the name. The tcl name request presents two problems:
1) tcl is in effect sending the request to the c code, causing the c code to mistake the request as being the actual name (solved by sending the request to stderr instead of stdout)
2) When tcl attempts to get the user input for the name, it will be checking stdin, which is mapped to receive from the c code not the keyboard, and will not be able to read the response from the user.
Is there a way to specify to get the response from the keyboard? Or should I map the stdout of the c code to a different fd for the tcl? And if so, how do I specify to get from the keyboard/new fd.
Here is how I make the tcl a child process of the c code:
int me2them[2], them2me[2];
pipe(me2them);pipe(them2me);
if (!fork()){
close(0); dup2(me2them[0],0); close(me2them[0]);
close(1); dup2(them2me[1],1); close(them2me[1]);
char * cmds[] = {"wish", "myProg.tcl",NULL};
execvp(cmds[0], cmds);
fprintf(stderr, "Unable to exec 1\n");
exit(-1);
}
close(0); dup2(them2me[0],0); close(them2me[0]);
close(1); dup2(me2them[1],1); close(me2them[1]);
It sounds as if the child would have a conventional command-line interface, e.g.,line-buffered. I suggest these design changes:
modify the two-way pipe to the child to something other than its standard input and output (you can read/write on other streams)
it might be simplest to make that change within the child
you can use dup2, etc., within the child to modify the pipe. That leaves the question of how to get a usable keyboard interface for the child.
you can solve that problem by opening /dev/tty directly, and (again with dup2 and friends) making the file opened on /dev/tty into the child's standard input and output.
As an example, the dialog program has a feature for reading data via a pipe (at the shell level, that is its standard input), and in initialization, changing that into a different stream and opening /dev/tty for a "real" standard input. Your problem is a little more complicated (with both input and output pipes), but reading the dialog source may be helpful. For reference, that is the init_dialog function in util.c (source here).
I am writing my own shell for a homework assignment, and am running into issues.
My shell program gets an input cat scores | grep 100 from the console and prints the output as expected but the grep command doesn't terminate and I can see it running infinitely from ps command.
EDIT - There was an error while closing fds. Now grep command is not executing and console output is -
grep: (standard input): Bad file descriptor
I am reading the number of commands from the console and creating necessary pipes and storing them in a two dimensional int array fd[][] before forking the first process.
fd[0][0] will contain read end of 1st pipe and fd[0][1] will contain write end of 1st pipe. fd[1][0] will contain read end of 2nd pipe and fd[1][1] will contain write end of 2nd pipe and so on.
Each new process duplicates its stdin with the read end of its pipe with the previous process and duplicates its stdout with the write end of its pipe with the next process.
Below is my function:
void run_cmds(char **args, int count,int pos)
{
int pid,status;
pid = fork();
if ( pid == 0 )
{
if(pos != 0) dup2(fd[pos-1][0],0); // not changing stdin for 1st process
if(pos != count) dup2(fd[pos][1],1); //not changing stdout for last process
close_fds(pos);
execvp(*args,args);
}
else
{
waitpid(pid,&status,0);
count--;
pos++;
//getting next command and storing it in args
if(count > 0)
run_cmds(args,count,pos);
}
}
}
args will contain the arguments for the command.
count is the number of commands I need to create.
pos is the position of the command in the input
I am not able to figure out the problem. I used this same approach for hard coded values before this and it was working.
What am I missing with my understanding/implementation of dup2/fork and why is the command waiting infinitely?
Any inputs would be greatly helpful. Struck with this for the past couple of days!
EDIT : close_fds() function is as below -
For any process , I am closing both the pipes linking the process.
void close_fds(int pos)
{
if ( pos != 0 )
{
close(fd[pos-1][0]);
close(fd[pos-1][1]);
}
if ( pos != count)
{
close(fd[pos][0]);
close(fd[pos][1]);
}
}
First diagnosis
You say:
Each new process duplicates its stdin with the read end of its pipe with the previous process and duplicates its stdout with the write end of its pipe with the next process.
You don't mention the magic word close().
You need to ensure that you close both the read and the write end of each pipe when you use dup() or dup2() to connect it to standard input. That means with 2 pipes you have 4 calls to close().
If you don't close the pipes correctly, the process that is reading won't get EOF (because there's a process, possibly itself, that could write to the pipe). It is crucial to have enough (not too few, not too many) calls to close().
I am calling close_fds() after dup2 calls. The function will go through the fd[][2] array and do a close() call for each fd in the array.
OK. That is important. It means my primary diagnosis probably wasn't spot on.
Second diagnoses
Several other items:
You should have code after the execvp() that reports an error and exits if the execvp() returns (which means it fails).
You should not immediately call waitpid(). All the processes in a pipeline should be allowed to run concurrently. You need to launch all the processes, then wait for the last one to exit, cleaning up any others as they die (but not necessarily worrying about everything in the pipeline exiting before continuing).
If you do force the first command to execute in its entirety before launching the second, and if the first command generates more output than will fit into the pipe, you will have a deadlock — the first process can't exit because it is blocked writing, and the second process can't be started because the first hasn't exited. Interrupts and reboots and the end of the universe will all solve the problem somewhat crudely.
You decrement count as well as incrementing pos before you recurse. That might be bad. I think you should just increment pos.
Third diagnosis
After update showing close_fds() function.
I'm back to "there are problems with closing pipes" (though the waiting and error reporting problems are still problems). If you have 6 processes in a pipeline and all 5 connecting pipes are created before any processes are run, each process has to close all 10 pipe file descriptors.
Also, don't forget that if the pipes are created in the parent shell, rather than in a subshell that executes one of the commands in the pipeline, then the parent must close all the pipe descriptors before it waits for the commands to complete.
Please manufacture an MCVE (How to create a Minimal, Complete, and Verifiable Example?) or
SSCCE (Short, Self-Contained, Correct Example) — two names and links for the same basic idea.
You should create a program that manufactures the data structures that you're passing to the code that invokes run_cmds(). That is, you should create whatever data structures your parsing code creates, and show the code that creates the pipe or pipes for the 'cat score | grep 100' command.
I am no longer clear how the recursion works — or whether it is invoked in your example. I think it is unused, in fact in your example, which is probably as well since you would end up with the same command being executed multiple times, AFAICS.
Most probable reasons why grep doesn't terminate:
You don't call waitpid with the proper PID (even though there is such a call in your code, it may not get executed for some reason), so grep becomes a zombie process. Maybe your parent shell process is waiting for another process first (infinitely, because the other one never terminates), and it doesn't call waitpid with the PID of grep. You can find Z in the output of ps if grep is a zombie.
grep doesn't receive an EOF on its stdin (fd 0), some process is keeping the write end of its pipe open. Have you closed all file descriptors in the fd array in the parent shell process? If not closed everywhere, grep will never receive an EOF, and it will never terminate, because it will be blocked (forever) waiting for more data on its stdin.
Right now I'm writing a C program that must execute a child process. I'm not doing multiple child processes simultaneously or anything, so this is fairly straightforward. I am definitely executing the built-in shell programs (i.e. things like cat and echo) successfully, but I also need to be able to tell when one of these programs fails to execute successfully. I'm trying this with the following simplified code:
int returnStatus; // The return status of the child process.
pid_t pid = fork();
if (pid == -1) // error with forking.
{
// Not really important for this question.
}
else if (pid == 0) // We're in the child process.
{
execvp(programName, programNameAndCommandsArray); // vars declared above fork().
// If this code executes the execution has failed.
exit(127); // This exit code was taken from a exec tutorial -- why 127?
}
else // We're in the parent process.
{
wait(&returnStatus); // Wait for the child process to exit.
if (returnStatus == -1) // The child process execution failed.
{
// Log an error of execution.
}
}
So for example, if I try to execute rm fileThatDoesntExist.txt, I would like to consider that a failure since the file didn't exist. How can I accomplish this? Also, while that execvp() call successfully executes built-in shell programs, it doesn't execute programs in the current directory of the executable (i.e. the program that this code is running inside of); Is there something else that I have to do in order to get it to run programs in the current directory?
Thanks!
This is a classic problem with a very elegant solution. Before forking, create a pipe in the parent. After fork, the parent should close the writing end of the pipe, and block attempting to read from the reading end. The child should close the reading end and set the close-on-exec flag, using fcntl, for the writing end.
Now, if the child calls execvp successfully, the writing end of the pipe will be closed with no data, and read in the parent will return 0. If execvp fails in the child, write the error code to the pipe, and read in the parent will return nonzero, having read the error code for the parent to handle.
wait(2) gives you more than just the exit status of the child process. In order to get the real exit status, you need to use the WIFEXITED() macro to test if the child exited normally (as opposed to abnormally via a signal etc.), and then use the WEXITSTATUS() macro to get the real exit status:
wait(&status);
if(WIFEXITED(status))
{
if(WEXITSTATUS(status) == 0)
{
// Program succeeded
}
else
{
// Program failed but exited normally
}
}
else
{
// Program exited abnormally
}
In order for execvp(3) to run a program in the current directory, you either need to add the current directory to your $PATH environment (generally not a good idea), or pass it the full path, e.g. use ./myprogram instead of just myprogram.
In terms of failure detection, if an exec() function replaces the current process with a new one, then the current process is gone; it doesn't matter if the executed program decides that what it has been asked to do is a success or failure. However, the parent process from before the fork can discover the child's exit code which would likely have the command success/failure information.
In terms of finding executables, execvp() duplicates the action of the shell, searching the current path. If it is not finding executables in the working directory, it is likely that directory is not in the search path (or the files are not actually executable). You can try specifying them by a full path name.
If you simply want to run a command and wait for the result, you might want to use the system() function which handles this for you, instead of building it yourself with fork/exec.
I'm using fork(). However, before executing fork(), I open a file (say a.txt) using freopen for writing. Now the child process redirects the output of execlp to a.txt. After terminating the child process, the parent process closes a.txt. Now how can the parent process read a.txt and show some information in stdout?
If the parent process opened the file with freopen(3), then the rewind(3) library call can be used to re-wind the stream's pointer to the start of the file, for use with fread(3) or fgets(3) or whatever API you'd like to use.
freopen does not belong in this code at all. Instead, you should do something like:
FILE *tmp = tmpfile();
if (!(pid=fork())) {
dup2(fileno(tmp), 1);
close(fileno(tmp));
execlp(...);
_exit(1);
}
wait(&status);
/* read from tmp */
However it would actually be a lot better to use a pipe if possible.