I have a problem with this little code for educational purposes. I can not understand how it works.
#include <stdio.h>
#include <fcntl.h>
#define FNAME "info.txt"
#define STDIN 0
int main(){
int fd;
fd = open(FNAME, O_RDONLY);
close(STDIN); //entry 0 on FDT is now free
dup(fd); //fd duplicate is now stored at entry 0
execlp("more","more",0);
}
By starting this program it prints the contents of the file "info.txt" on terminal. I can not understand why! Where is the link between "more" and STDIN (keyboard or file)?
Why if i use more with no args and without redirection on file it just shows a help screen but whit redirection it uses the file as input?
dup always gives you the lowest available file descriptor number.
By default all process will have 0, 1 and 2 for stdin, stdout and stderr. You are opening a file from that you will get a file descriptor value 3. After that you have closed stdin. Now calling dup after that will give you a lowest available value as a duplicate file descriptor for 3, so you will be getting stdin as duplicate file descriptor for 3.
int main()
{
int fd, fd2;
fd = open(FNAME, O_RDONLY); //This will be 3
fd2 = dup(fd); //This will be 4 because 4 is the lowest available value
close(STDIN); //entry 0 on FDT is now free
dup(fd); //fd duplicate is now stored at entry 0
execlp("more","more",0);
}
And here why its displaying the content of the file is, more command can be used in two ways.
more filename
command | more
In your exec, you are not giving any filename as command line argument for more command. So its executing in pipe mode, by reading it from stdin.
Related
I want to pipe the output of a child process to the parent's stdout. I know there are other ways of doing this, but why can't a pipe's read-end be duplicated to stdout? Why doesn't the program print what is written to the pipes write end?
Here i have a minimal example (without any subprocesses) of what I'm trying to do. Im expecting to see test in the output when running, but the program outputs nothing.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(void) {
int fds[2];
if(pipe(fds) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
if(write(fds[1], "test", 5) == -1) {
perror("write");
exit(EXIT_FAILURE);
}
if(dup2(fds[0], STDOUT_FILENO) == -1) {
perror("dup2");
exit(EXIT_FAILURE);
}
return 0;
}
A pipe is two “files” that share a buffer and some locking or control semantics. When you write into the pipe, the data is put into the buffer. When you read from a pipe, the data is taken from a buffer.
There is nothing in the pipe that moves data to some output device.
If you use dup2 to duplicate the read side of the pipe into the standard output file descriptor (number 1), then all you have is the read side of the pipe on file descriptor 1. That means you can issue read operations to file descriptor 1, and the system will give your program data from the pipe.
There is nothing “special” about file descriptor 1 in this regard. Putting any file on file descriptor 1 does not cause that file to be automatically sent anywhere. The way standard output works normally is that you open a terminal or some chosen output file or other device on file descriptor 1, and then you send things to that device or file by writing to file descriptor 1. The operating system does not automatically write things to file descriptor 1; you have to issue write operations.
I am writing a porgram with takes two arguments - the name of commands. The program should redirect the output of the first to a file 'tmp' than execute it, than redirect the stdin of the second command to 'tmp' and execute the second command.
#include<unistd.h>
#include<fcntl.h>
#include<wait.h>
#include<stdio.h>
int main(int argc, char** argv){
int fd = open("tmp", O_RDWR |O_TRUNC | O_CREAT, S_IRUSR | S_IWUSR);
int cpid = fork();
if(cpid == 0){
dup2(fd, 1);
execlp(argv[1], "", NULL);
}
wait(NULL);
//If we uncoment this line the program gives correct output
//fd = open("tmp", O_RDWR, S_IRUSR | S_IWUSR);
dup2(fd, 0);
execlp(argv[2], "", NULL);
}
However when i run the program like ./main ls wc instead of
5 5 50
i get output 0 0 0 which means the wc command reads 0 bytes from stdin.
But if I instead create the file descriptor anew on the same file 'tmp' the program gives the correct output. How can this behaviour be explained?
This question is basically a duplicate of Can anyone explain a simple description regarding 'file descriptor' after fork()? but since this is a bit subtle I'll explain this specific case.
The process opens a file. This creates a file description. File descriptions are an intermediate concept between files and file descriptors. They are not directly exposed in the Unix API, but they have an important property in addition to the file that they point to, which we'll see in a minute.
The child writes to this file description. The parent waits.
The parent reads from the file description.
At the end of step 2, the file position on this file description is the end of the file. So at step 3, the parent starts reading at the end of the file.
If you add a call to rewind(fd) after wait(NULL), the child will read from the beginning of the file.
If you open the same file with a new open call, this creates a new file description. open puts the position on the new file description at the beginning of the file unless you set append mode.
The same file descriptions can be accessed through any number of file descriptors, potentially in different processes. The file position is a property of the file description, so anything that moves it (reading, writing, seeking) through one file descriptor also moves it for the other file descriptors, even in different processes.
int main (void) {
int rc=fork();
if(rc==0){
close(1); //close stdout BEFORE opening my file
open("./c.txt", O_CREAT|O_WRONLY|O_TRUNC, S_IRWXU);
//execve "wc"
char *cmd[3];
cmd[0] = strdup("wc"); //file to execuable
cmd[1]=strdup("c.c"); //first arg to command 'wc' -> c.c
cmd[2]=NULL;
execvp(cmd[0], cmd);
}
If I close() stdout, then the output of execve ("wc"), will be in file c.txt but ONLY if I close stdout BEFORE open()ing. If I call it AFTER
open("./c.txt", O_CREAT|O_WRONLY|O_TRUNC, S_IRWXU);
close(1);
then -> wc: write error: Bad file descriptor.
I have read, that for open() (probably in my case for wc output) is OS reaching file descriptor from 0, and so it first find 1 as stdout to printf() to screen. So I need to close() it, in order to use file descriptor from open("./c.txt") for wc. But If that is correct (I do not know where I have understood it correctly), then it would not matter whether I close stdout before or after open() call, does it? Once it is closed, OS has no other FD to use as output. I maybe does not understand it clearly.
question: why must be fd1 closed first in order to make redirection to c.txt?
A few concepts to establish first.
stdout is one of the streams automatically opened as part of program startup. The program startup code uses file descriptor 1 for stdout).
execve creates a new process with the same open file descriptors as the parent/calling process (there are exceptions and naunces which can be read from the execve man page.
open will look for the lowest available file descriptor to use.
Ok, so now to your code.
Case 1 - close, open, execve
In this case the following sequence of events happens:
Program starts with stdout=>fd 1.
close(1) makes fd 1 available.
open("c.txt") returns 1 which effectively redirects stdout to the file.
execve creates a new process which has 1 open and redirected to the file.
wc writes to fd 1 which now ends up in the file.
Case 2 - open,close,execve
In this case the following sequence of events happens:
Program starts with stdout=>fd 1.
open("c.txt")is called but fd 1 is not available so it returns 2.
close(1) means there is now effectively no stdout.
execve creates a new process which has no open stream on fd 1 (ie no stdout).
wc tries to write to fd 1 and gets a bad file descriptor error since fd 1 is not open.
Ok guys, there are a billion demos relating to dup, dup2, fcntl, pipe and all kinds of stuff that are wonderful when multiple processes exist. However, I have yet to see one very basic thing that I think will help explain the behavior of pipe and its relationship to standard out and in.
My goal is to simply (in the same process) reroute standard output through a pipe back to standard output directly. I have already accomplished this
with intermediate stages which redirect the pipe output to a file or write into a buffer... and then put standard output back to where it started. At that point, of course I can write the buffer back to stdout, but I don't want to do this.
Since I moved standard output to another location in the file table, I'd like to direct the output of the pipe to feed directly into the new standard output position and have it print like it normally would.
I feel like there is some kind of layer surrounding the file table that I am not understanding.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main() {
int pipeEnds_arr1[2];
char str1[] = "STRING TO FEED INTO PIPE \n"; // make a string array
pipe(pipeEnds_arr1);
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
/* now my goal is to shift the input of the pipe into the position of
* standard output, so that the print command feeds the pipe, then I
* would like to redirect the other end of the pipe to standard out.
*/
int someInt = dup(1); // duplicates stdout to next available file table position
printf ("Some Int FD: %d\n", someInt); // print out the fd for someInt just for knowing where it is
/* This is the problem area. The out end of the pipe never
* makes it back to std out, and I see no way to do so.
* Stdout should be in the file table position 5, but when
* I dup2 the output end of the pipe into this position ,
* I believe I am actually overwriting std out completely.
* But I don't want to overwrite it, i want to feed the output
* of the pipe into std out. I think I am fundamentally
* misunderstanding this issue.
*/
dup2(pipeEnds_arr1[1], 1); //put input end of pipe into std out position
//dup2(pipeEnds_arr1[0], 5); // this will not work
//and other tests I have conducted do not work
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
fflush(stdout);
close(pipeEnds_arr1[0]);
close(pipeEnds_arr1[1]);
return 0;
}
EDIT*********
OK, what I know is that somehow std out takes information from commands like printf and then routs it into a buffer that is then flushed to the shell.
What I believe is that there must be a way to rout the "read" or output end of the pipe to that same buffer that then gets to the shell. I have figured out how to rout the pipe output into a string, and then I can do as I please. In the example code I post below, I will first rout the pipe out to a string and then open a file and write the string to the open file descriptor of that file...
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main() {
/* Each pipe end array has to have 2 positions in it. The array
* position represents the two pipe ends with the 0 index
* position representing the output of the pipe (the place you want
* read your data from), and 1 index position representing the
* input file descriptor of the pipe (the place you want to write
* your data).
*/
int pipeEnds_arr1[2];
char str1[] = "Hello, we are feeding this into the pipe that we are through stdout into a pipe and then reading from the pipe and then feeding that output into a file \n"; // make a string array
/* Here we want to actually do the pipe command. We feed it the array
* with the 2 positions in it which will now hold file descriptors
* attached to the current process which allow for input and output
* through the new pipe. At this point, we don't know what the
* exact file decriptors are, but we can look at them by printing
*/
pipe(pipeEnds_arr1);
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
/* now my goal is to shift the input of the pipe into the position of
* standard output, so that the print command feeds the pipe, then we
* will try to read from the pipe and redirect the output to the std
* or in this test case out to a file.
*/
int someInt = dup(1); // we moved what was stdout into someInt;
/* put the write end of the pipe in the old stdout position by
* using dup2 so we will print directly into the pipe
*/
dup2(pipeEnds_arr1[1], 1);
/* this is where id like to re-rout the pipe back to stdout but
* im obviously not understanding this correctly
*/
//dup2(someInt, 3);
/* since std out has now been replaced by the pipe write end, this
* printf will print into the pipe
*/
printf("%s", str1);
/* now we read from the pipe into a new string we make */
int n;
char str2[strlen(str1)];
n = read(pipeEnds_arr1[0], str2, sizeof(str2)-1);
str2[n] = 0;
/* open a file and then write into it from the output of the pipe
* that we saved into the str2
*/
int fd = open("tmp.out", O_WRONLY | O_CREAT | O_TRUNC, 0644);
write(fd, str2, strlen(str2));
/* not sure about these last commands and their relevance */
fflush(stdout);
close(pipeEnds_arr1[0]);
close(pipeEnds_arr1[1]);
close(fd);
return 0;
}
Pipes aren't between file descriptors. They are between processes. So it doesn't make any sense to "reroute standard out through a pipe".
What you can do is modify a process's file descriptor table so that its stdout (fd 1) is the write side of a pipe. And you can modify another process's file descriptor table so that some file descriptor, perhaps even stdin (fd 0) is the read side of the same pipe. That allows you to pass data through the pipe between the two processes. (You can set up a pipe between two fds in the same process, if you want to; it's occasionally useful but watch out for deadlocking.)
stdout is not some sort of magical entity. It's just entry 1 in the fd table, and it might refer to any "file", in the Unix sense of the word, which includes regular files, devices (including the console and the pseudoterminal your shell is communicating with), sockets, pipes, FIFOs, and whatever else the operating system feels worthy of allowing streaming access to.
Normally, when the shell starts a running a command-line utility, it first clones fds 0, 1 and 2 (stdin, stdout and stderr) from its own fd 0, 1, and 2, which are normally all the same device: the console, or more commonly these days, the pseudoterminal provided by the graphical console application you are using. But you can change those assignments with, for example, shell redirection operators, shell pipe operators, and some shell-provided special files.
Finally, pipes do have small buffers in the kernel, but the key is rhe word "small" -- the buffer might hold as little as 4096 bytes. If gets full, attempts to write to the pipe will hang until space becomes available, which only happens when data is read from the other sude. That's why it is so easy to deadlock if the same process is using both sides of the pipe: if the process is hanging waiting for the pileto be emptied, it wikk neverbe able to read the pipe.
I know that dup, dup2, dup3 "create a copy of the file descriptor oldfd"(from man pages). However I can't digest it.
As I know file descriptors are just numbers to keep track of file locations and their direction(input/output). Wouldn't it be easier to just
fd=fd2;
Whenever we want to duplicate a file descriptor?
And something else..
dup() uses the lowest-numbered unused descriptor for the new descriptor.
Does that mean that it can also take as value stdin, stdout or stderr if we assume that we have close()-ed one of those?
Just wanted to respond to myself on the second question after experimenting a bit.
The answer is YES. A file descriptor that you make can take a value 0, 1, 2 if stdin, stdout or stderr are closed.
Example:
close(1); //closing stdout
newfd=dup(1); //newfd takes value of least available fd number
Where this happens to file descriptors:
0 stdin .--------------. 0 stdin .--------------. 0 stdin
1 stdout =| close(1) :=> 2 stderr =| newfd=dup(1) :=> 1 newfd
2 stderr '--------------' '--------------' 2 stderr
A file descriptor is a bit more than a number. It also carries various semi-hidden state with it (whether it's open or not, to which file description it refers, and also some flags). dup duplicates this information, so you can e.g. close the two descriptors independently. fd=fd2 does not.
Let's say you're writing a shell program and you want to redirect stdin and stdout in a program you want to run. It could look something like this:
fdin = open(infile, O_RDONLY);
fdout = open(outfile, O_WRONLY);
// Check for errors, send messages to stdout.
...
int pid = fork(0);
if(pid == 0) {
close(0);
dup(fdin);
close(fdin);
close(1);
dup(fdout);
close(fdout);
execvp(program, argv);
}
// Parent process cleans up, maybe waits for child.
...
dup2() is a little more convenient way to do it the close() dup() can be replaced by:
dup2(fdin, 0);
dup2(fdout, 1);
The reason why you want to do this is that you want to report errors to stdout (or stderr) so you can't just close them and open a new file in the child process. Secondly, it would be a waste to do the fork if either open() call returned an error.
The single most important thing about dup() is it returns the smallest integer available for a new file descriptor. That's the basis of redirection:
int fd_redirect_to = open("file", O_CREAT);
close(1); /* stdout */
int fd_to_redirect = dup(fd_redirect_to); /* magically returns 1: stdout */
close(fd_redirect_to); /* we don't need this */
After this anything written to file descriptor 1 (stdout), magically goes into "file".
Example:
close(1); //closing stdout
newfd=dup(1); //newfd takes value of least available fd number
Where this happens to file descriptors:
0 stdin .--------------. 0 stdin .--------------. 0 stdin
1 stdout =| close(1) :=> 2 stderr =| newfd=dup(1) :=> 1 newfd
2 stderr '--------------' '--------------' 2 stderr
A question arose again: How can I dup() a file descriptor that I already closed?
I doubt that you conducted the above experiment with the shown result, because that would not be standard-conforming - cf. dup:
The dup() function shall fail if:
[EBADF]
The fildes argument is not a valid open file descriptor.
So, after the shown code sequence, newfd must be not 1, but rather -1, and errno EBADF.
see this page, stdout can be aliased as dup(1)...
Just a tip about "duplicating standard output".
On some Unix Systems (but not GNU/Linux)
fd = open("/dev/fd/1", O_WRONLY);
it is equivalent to:
fd = dup(1);
dup() and dup2() system call
•The dup() system call duplicates an open file descriptor and returns the new file
descriptor.
•The new file descriptor has the following properties in common with
the original
file descriptor:
1. refers to the same open file or pipe.
2. has the same file pointer -- that is, both file descriptors share one file pointer.
3. has the same access mode, whether read, write, or read and write.
• dup() is guaranteed to return a file descriptor with the lowest integer value available.It is because of this feature of returning the lowest unused file descriptor available that processes accomplish I/O redirection.
int dup(file_descriptor)
int dup2(file_descriptor1, file_descriptor2)