I'm writing a function, which, given an argument, will either redirect the stdout to a file or read the stdin from a file. To do this I close the file descriptor associated with the stdout or stdin, so that when I open the file it opens under the descriptor that I just closed. This works, but the problem is that once this is done, I need to restore the stdout and stdin to what they should really be.
What I can do for stdout is open("/dev/tty",O_WRONLY); But I'm not sure why this works, and more importantly I don't know of an equivalent statement for stdin.
So I have, for stdout
close(1);
if (creat(filePath, O_RDWR) == -1)
{
exit(1);
}
and for stdin
close(0);
if (open(filePath, O_RDONLY) == -1)
{
exit(1);
}
You should use dup() and dup2() to clone a file descriptor.
int stdin_copy = dup(0);
int stdout_copy = dup(1);
close(0);
close(1);
int file1 = open(...);
int file2 = open(...);
< do your work. file1 and file2 must be 0 and 1, because open always returns lowest unused fd >
close(file1);
close(file2);
dup2(stdin_copy, 0);
dup2(stdout_copy, 1);
close(stdin_copy);
close(stdout_copy);
However, there's a minor detail you might want to be careful with (from man dup):
The two descriptors do not share file descriptor flags (the
close-on-execflag). The close-on-exec flag (FD_CLOEXEC; see fcntl(2))
for the duplicate descriptor is off.
If this is a problem, you might have to restore the close-on-exec flag, possibly using dup3() instead of dup2() to avoid race conditions.
Also, be aware that if your program is multi-threaded, other threads may accidentally write/read to your remapped stdin/stdout.
I think you can "save" the descriptors before redirecting:
int save_in, save_out;
save_in = dup(STDIN_FILENO);
save_out = dup(STDOUT_FILENO);
Later on you can use dup2 to restore them:
/* Time passes, STDIN_FILENO isn't what it used to be. */
dup2(save_in, STDIN_FILENO);
I am not doing any error checking in that example - you should.
You could create a child process, and set up the redirection inside the child only. Then wait for the child to terminate, and continue working in the parent process. That way you don't have to worry about reversing your redirection at all.
Just look for examples of code using fork() and wait ().
Related
I'm writing a C program where I fork(), exec(), and wait(). I'd like to take the output of the program I exec'ed to write it to file or buffer.
For example, if I exec ls I want to write file1 file2 etc to buffer/file. I don't think there is a way to read stdout, so does that mean I have to use a pipe? Is there a general procedure here that I haven't been able to find?
For sending the output to another file (I'm leaving out error checking to focus on the important details):
if (fork() == 0)
{
// child
int fd = open(file, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
dup2(fd, 1); // make stdout go to file
dup2(fd, 2); // make stderr go to file - you may choose to not do this
// or perhaps send stderr to another file
close(fd); // fd no longer needed - the dup'ed handles are sufficient
exec(...);
}
For sending the output to a pipe so you can then read the output into a buffer:
int pipefd[2];
pipe(pipefd);
if (fork() == 0)
{
close(pipefd[0]); // close reading end in the child
dup2(pipefd[1], 1); // send stdout to the pipe
dup2(pipefd[1], 2); // send stderr to the pipe
close(pipefd[1]); // this descriptor is no longer needed
exec(...);
}
else
{
// parent
char buffer[1024];
close(pipefd[1]); // close the write end of the pipe in the parent
while (read(pipefd[0], buffer, sizeof(buffer)) != 0)
{
}
}
You need to decide exactly what you want to do - and preferably explain it a bit more clearly.
Option 1: File
If you know which file you want the output of the executed command to go to, then:
Ensure that the parent and child agree on the name (parent decides name before forking).
Parent forks - you have two processes.
Child reorganizes things so that file descriptor 1 (standard output) goes to the file.
Usually, you can leave standard error alone; you might redirect standard input from /dev/null.
Child then execs relevant command; said command runs and any standard output goes to the file (this is the basic shell I/O redirection).
Executed process then terminates.
Meanwhile, the parent process can adopt one of two main strategies:
Open the file for reading, and keep reading until it reaches an EOF. It then needs to double check whether the child died (so there won't be any more data to read), or hang around waiting for more input from the child.
Wait for the child to die and then open the file for reading.
The advantage of the first is that the parent can do some of its work while the child is also running; the advantage of the second is that you don't have to diddle with the I/O system (repeatedly reading past EOF).
Option 2: Pipe
If you want the parent to read the output from the child, arrange for the child to pipe its output back to the parent.
Use popen() to do this the easy way. It will run the process and send the output to your parent process. Note that the parent must be active while the child is generating the output since pipes have a small buffer size (often 4-5 KB) and if the child generates more data than that while the parent is not reading, the child will block until the parent reads. If the parent is waiting for the child to die, you have a deadlock.
Use pipe() etc to do this the hard way. Parent calls pipe(), then forks. The child sorts out the plumbing so that the write end of the pipe is its standard output, and ensures that all other file descriptors relating to the pipe are closed. This might well use the dup2() system call. It then executes the required process, which sends its standard output down the pipe.
Meanwhile, the parent also closes the unwanted ends of the pipe, and then starts reading. When it gets EOF on the pipe, it knows the child has finished and closed the pipe; it can close its end of the pipe too.
Since you look like you're going to be using this in a linux/cygwin environment, you want to use popen. It's like opening a file, only you'll get the executing programs stdout, so you can use your normal fscanf, fread etc.
After forking, use dup2(2) to duplicate the file's FD into stdout's FD, then exec.
You could also use the linux sh command and pass it a command that includes the redirection:
string cmd = "/bin/ls > " + filepath;
execl("/bin/sh", "sh", "-c", cmd.c_str(), 0);
For those such as myself who like a complete example with includes, here's this fantastic answer with a runnable example (still without error handling, left as an exercise):
#include <fcntl.h>
#include <sys/wait.h>
#include <unistd.h>
int main() {
if (fork() == 0) { // child
int fd = open("test.txt", O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
dup2(fd, 1); // make stdout go to file
dup2(fd, 2); // make stderr go to file - you may choose to not do this
// or perhaps send stderr to another file
close(fd); // fd no longer needed - the dup'ed handles are sufficient
execlp("ls", "ls", NULL);
}
else {
while (wait(NULL) > 0) {} // wait for each child process
}
return 0;
}
I am creating a C program and with it I am setting up a pipe between separately forked process for interprocess communication.
The first process has written the data I need into the pipe.
However, with the second process reading from the pipe, I am trying to exec the process to become the UNIX sort command. I want to somehow call sort on the data in the pipe.
How can I call sort on a pipe? On the commandline, I can sort by supplying the filename to sort as a commandline argument e.g. "sort -r MyFileToSort". I know that pipes are essentially considered files, but they are only described by their file descriptor, and as far as I know, sort won't know what to do with a fd.
Thanks for any help/feedback
int p[2];
if (pipe(p) != 0) ...report error and do not continue...
pid_t pid = fork();
if (pid < 0) ...report error, close pipe descriptors, and do not continue...
if (pid == 0)
{
/* Child - becomes sort */
dup2(p[0], 0);
close(p[0]);
close(p[1]);
int fd = open("output-file", O_CREAT | O_EXCL | O_WRONLY, 0644);
if (fd < 0) ...report error and exit...
dup2(fd, 1);
close(fd);
execlp("sort", "sort", (char *)0);
...report error and exit...
}
else
{
/* Parent - writes data to sort */
close(fd[0]);
...write data to fd[1]...
close(fd[1]);
int status;
int corpse;
while ((corpse = wait(&status)) > 0 && corpse != pid)
...consider reporting which child died...
...consider reporting sort status...
...continue with the rest of the program...
}
You can decide whether to report errors related to dup2() failing, or close() failing. There isn't much you can do in either case except report the problem and exit. Unless someone has subjected your program to cruel and unusual punishment by not supplying it with standard input, standard output and standard error (or something elsewhere in the program has closed any of the standard channels), then the pipe and file descriptors can't be the standard I/O descriptors, so the closes are safe. If you're not sure how sick your users are, you might protect the closes:
if (p[0] > FILENO_STDERR)
close(p[0]);
That is normally unnecessarily paranoid (but it can be fun trying programs with missing standard I/O).
You don't need to pass sort any arguments to specify input source or output sink at all in this case. Instead, before execing it, you should make attach your pipeline's file descriptors to its stdin (FD 0, if receiving data from a pipe) or stdout (FD 1, if writing data to a pipe), as appropriate.
See the dup2() call, which lets you set the destination to which you're copying a FD, for this purpose. As #JonathanLeffler points out, you'll want to be sure to close the original FDs (after duplicating them to the numbers you want) before your exec call.
Since you've clarified, in comments, that your goal is to write to a file, you would attach FD 1 to that destination file before calling exec, with FD 0 attached to the output side of the pipeline containing input.
I have code where I want to encapsulate a particular piece of code into a forked child so that if the code causes issues, it will not affect the parent and the parent will be able to report.
I am not execing after the fork call. Just plain fork and then the code runs that would have normally run in the parent.
The code in the child outputs stuff to stdout and stderr. Due to issues with interleaving output and stdio buffering, I was now looking into TLPI for a better solution.
The idea I came up with was roughly:
pipe() for each stream
fork()
in child:
close() read end of pipes
dup2() the write ends onto the file descriptors for stdout and stderr respectively
setbuf(fd, NULL) to turn off stdio stream buffering inside the child.
in parent:
close() write ends of pipes
use select/pselect/epoll etc (has to run across Linux, BSDs, AIX, Solaris etc) to watch the read ends of the pipes for new data and when it arrives write() it straight to the respective file descriptors in the parent process.
Now, I presume there is one step missing between the dup2 and the setbuf inside the child. What is it?
setbuf takes a FILE* whereas of course dup2 acts on int.
freopen came to mind, but it requires a path. If I merely want to assign a new fileno to the streams, how can I do that?
The extern variables stdin, stdout and stderr are FILE * pointers. You can pass these to setbuf / setvbuf. However, note that the child has a separate address space so whilst it will inherit the state of the buffers at the time of the fork(), both can continue to use the buffers safely, and if you fflush STDOUT and STDERR, the output buffers will be empty anyway.
This is the way I would do it in the child (untested, and please do add some error handling):
void
child ( ... )
{
const char *devnull = "/dev/null";
/* turn off buffering */
setvbuf (stdin, NULL, _IONBF);
setvbuf (stdout, NULL, _IONBF);
setvbuf (stderr, NULL, _IONBF);
for (i = getdtablesize () - 1; i >= 0; i--)
{
if ((i != write_end_of_pipe1) && (i != write_end_of_pipe2))
close (i);
}
i = open (devnull, O_RDWR);
if (i == -1)
{
fprintf (stderr, "Unable to open /dev/null\n");
return;
}
i = open (devnull, O_RDONLY);
if (i != 0)
{
dup2 (i, 0);
close (i);
}
dup2 (write_end_of_pipe1, 1);
close (write_end_of_pipe1);
dup2 (write_end_of_pipe2, 2);
close (write_end_of_pipe2);
return;
}
This is the code i found for my own shell. It works fine, but the thing i can't understand is pipe section of the code.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
char* cmndtkn[256];
char buffer[256];
char* path=NULL;
char pwd[128];
int main(){
//setting path variable
char *env;
env=getenv("PATH");
putenv(env);
system("clear");
printf("\t MY OWN SHELL !!!!!!!!!!\n ");
printf("_______________________________________\n\n");
while(1){
fflush(stdin);
getcwd(pwd,128);
printf("[MOSH~%s]$",pwd);
fgets(buffer,sizeof(buffer),stdin);
buffer[sizeof(buffer)-1] = '\0';
//tokenize the input command line
char* tkn = strtok(buffer," \t\n");
int i=0;
int indictr=0;
// loop for every part of the command
while(tkn!=NULL)
{
if(strcoll(tkn,"exit")==0 ){
exit(0);
}
else if(strcoll(buffer,"cd")==0){
path = buffer;
chdir(path+=3);
}
else if(strcoll(tkn,"|")==0){
indictr=i;
}
cmndtkn[i++] = tkn;
tkn = strtok(NULL," \t\n");
}cmndtkn[i]='\0';
// execute when command has pipe. when | command is found indictr is greater than 0.
if(indictr>0){
char* leftcmnd[indictr+1];
char* rightcmnd[i-indictr];
int a,b;
for(b=0;b<indictr;b++)
leftcmnd[b]=cmndtkn[b];
leftcmnd[indictr]=NULL;
for(a=0;a<i-indictr-1;a++)
rightcmnd[a]=cmndtkn[a+indictr+1];
rightcmnd[i-indictr]=NULL;
if(!fork())
{
fflush(stdout);
int pfds[2];
pipe(pfds);
if(!fork()){
close(1);
dup(pfds[1]);
close(pfds[0]);
execvp(leftcmnd[0],leftcmnd);
}
else{
close(0);
dup(pfds[0]);
close(pfds[1]);
execvp(rightcmnd[0],rightcmnd);
}
}else
wait(NULL);
//command not include pipe
}else{
if(!fork()){
fflush(stdout);
execvp(cmndtkn[0],cmndtkn);
}else
wait(NULL);
}
}
}
What is the purpose of the calls to close() with parameters of 0 and 1 mean and what does the call to dup() do?
On Unix, the dup() call uses the lowest numbered unused file descriptor. So, the close(1) before the call to dup() is to coerce dup() to use file descriptor 1. Similarly for close(0).
So, the aliasing is to get the process to use the write end of the pipe for stdout (file descriptor 1 is used for console output), and the read end of the pipe for stdin (file descriptor 0 is used for console input).
The code may have been more clearly expressed with dup2() instead.
dup2(fd[1], 1); /* alias fd[1] to 1 */
From your question about how ls | sort works, your question is not limited to why the dup() system call is being made. Your question is actually how pipes in Unix work, and how a shell command pipeline works.
A pipe in Unix is a pair of file descriptors that are related in that writing data on tje writable descriptor allows that data to be read from the readable descriptor. The pipe() call returns this pair in an array, where the first array element is readable, and second array element is writable.
In Unix, a fork() followed by some kind of exec() is the only way to produce a new process (there are other library calls, such as system() or popen() that create processes, but they call fork() and do an exec() under the hood). A fork() produces a child process. The child process sees the return value of 0 from the call, while the parent sees a non-zero return value that is either the PID of the child process, or a -1 indicating that an error has occurred.
The child process is a duplicate of the parent. This means that when a child modifies a variable, it is modifying a copy of the variable that resides in its own process. The parent does not see the modification occur, as the parent has the original copy). However, a duplicated pair of file descriptors that form a pipe can be used to allow a child process its parent to communicate with each other.
So, ls | sort means that there are two processes being spawned, and the output written by ls is being read as input by sort. Two processes means two calls to fork() to create two child processes. One child process will exec() the ls command, the other child process will exec() the sort command. A pipe is used between them to allow the processes to talk to each other. The ls process writes to the writable end of the pipe, the sort process reads from the readable end of the pipe.
The ls process is coerced into writing into the writable end of the pipe with the dup() call after issuing close(1). The sort process is coerced into reading the readable end of the pipe with the dup() call after close(0).
In addition, the close() calls that close the pipe file descriptors are used to make sure that the ls process is the only process to have an open reference to the writable fd, the the sort process is the only process to have an open reference to the readable fd. That step is important because after ls exits, it will close the writable end of the fd, and the sort process will expect to see an EOF as a result. However, this will not occur if some other process still has the writable fd open.
http://en.wikipedia.org/wiki/Standard_streams#Standard_input_.28stdin.29
stdin is file descriptor 0.
stdout is file descriptor 1.
In the !fork section, the process closes stdout then calls dup on pfds[1] which according to:
http://linux.die.net/man/2/dup
Creates a duplicate of the specified file descriptor at the lowest available position, which will be 1, since it was just closed (and stdin hasn't been closed yet). This means everything sent to stdout will really go to pfds[1].
So, basically, it's setting up the two new processes to talk to each other. the !fork section is for the new child which will send data to stdout (file descriptor 1), the parent (the else block) closes stdin, so it really reads from pfds[0] when it tries to read from stdout.
Each process has to close the file descriptor in pfds it's not using, as there are two open handles to the file now that the process has forked. Each process now execs to left/right-cmnd, but the new stdin and stdout mappings remain for the new processes.
Forking twice is explained here: Why fork() twice
I'm writing a C program where I fork(), exec(), and wait(). I'd like to take the output of the program I exec'ed to write it to file or buffer.
For example, if I exec ls I want to write file1 file2 etc to buffer/file. I don't think there is a way to read stdout, so does that mean I have to use a pipe? Is there a general procedure here that I haven't been able to find?
For sending the output to another file (I'm leaving out error checking to focus on the important details):
if (fork() == 0)
{
// child
int fd = open(file, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
dup2(fd, 1); // make stdout go to file
dup2(fd, 2); // make stderr go to file - you may choose to not do this
// or perhaps send stderr to another file
close(fd); // fd no longer needed - the dup'ed handles are sufficient
exec(...);
}
For sending the output to a pipe so you can then read the output into a buffer:
int pipefd[2];
pipe(pipefd);
if (fork() == 0)
{
close(pipefd[0]); // close reading end in the child
dup2(pipefd[1], 1); // send stdout to the pipe
dup2(pipefd[1], 2); // send stderr to the pipe
close(pipefd[1]); // this descriptor is no longer needed
exec(...);
}
else
{
// parent
char buffer[1024];
close(pipefd[1]); // close the write end of the pipe in the parent
while (read(pipefd[0], buffer, sizeof(buffer)) != 0)
{
}
}
You need to decide exactly what you want to do - and preferably explain it a bit more clearly.
Option 1: File
If you know which file you want the output of the executed command to go to, then:
Ensure that the parent and child agree on the name (parent decides name before forking).
Parent forks - you have two processes.
Child reorganizes things so that file descriptor 1 (standard output) goes to the file.
Usually, you can leave standard error alone; you might redirect standard input from /dev/null.
Child then execs relevant command; said command runs and any standard output goes to the file (this is the basic shell I/O redirection).
Executed process then terminates.
Meanwhile, the parent process can adopt one of two main strategies:
Open the file for reading, and keep reading until it reaches an EOF. It then needs to double check whether the child died (so there won't be any more data to read), or hang around waiting for more input from the child.
Wait for the child to die and then open the file for reading.
The advantage of the first is that the parent can do some of its work while the child is also running; the advantage of the second is that you don't have to diddle with the I/O system (repeatedly reading past EOF).
Option 2: Pipe
If you want the parent to read the output from the child, arrange for the child to pipe its output back to the parent.
Use popen() to do this the easy way. It will run the process and send the output to your parent process. Note that the parent must be active while the child is generating the output since pipes have a small buffer size (often 4-5 KB) and if the child generates more data than that while the parent is not reading, the child will block until the parent reads. If the parent is waiting for the child to die, you have a deadlock.
Use pipe() etc to do this the hard way. Parent calls pipe(), then forks. The child sorts out the plumbing so that the write end of the pipe is its standard output, and ensures that all other file descriptors relating to the pipe are closed. This might well use the dup2() system call. It then executes the required process, which sends its standard output down the pipe.
Meanwhile, the parent also closes the unwanted ends of the pipe, and then starts reading. When it gets EOF on the pipe, it knows the child has finished and closed the pipe; it can close its end of the pipe too.
Since you look like you're going to be using this in a linux/cygwin environment, you want to use popen. It's like opening a file, only you'll get the executing programs stdout, so you can use your normal fscanf, fread etc.
After forking, use dup2(2) to duplicate the file's FD into stdout's FD, then exec.
You could also use the linux sh command and pass it a command that includes the redirection:
string cmd = "/bin/ls > " + filepath;
execl("/bin/sh", "sh", "-c", cmd.c_str(), 0);
For those such as myself who like a complete example with includes, here's this fantastic answer with a runnable example (still without error handling, left as an exercise):
#include <fcntl.h>
#include <sys/wait.h>
#include <unistd.h>
int main() {
if (fork() == 0) { // child
int fd = open("test.txt", O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
dup2(fd, 1); // make stdout go to file
dup2(fd, 2); // make stderr go to file - you may choose to not do this
// or perhaps send stderr to another file
close(fd); // fd no longer needed - the dup'ed handles are sufficient
execlp("ls", "ls", NULL);
}
else {
while (wait(NULL) > 0) {} // wait for each child process
}
return 0;
}