I am trying to use a pipe to rederect stdout into a pipe and read it later. I will use this later with fork(), where the child process starts a different program that I need to comunicate with. This is my Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
int main(){
printf("Starting Test\n");
int myPipe[2], nbytes;
char readbuffer[80];
pipe(myPipe);
int backup = dup(1); //store stdout
if (dup2(1,myPipe[1])< 1){printf("error");} //copy stdout in the input end of my pipe
printf("in pipe\n"); //print something in stdout -> in my pipe
nbytes = read(myPipe[0],readbuffer,sizeof(readbuffer)); //read output of my pipe
dup2(myPipe[1],backup); // restore stdout
printf("recived: %s",readbuffer); //prit out what I recived
return 0;
}
I expected it to print out:
Starting Test
recived: in pipe
But the output that I get is:
Starting Test
in pipe
recived: #����U
So I assume that stdout was not copied properly, as I get the "in pipe" before the "recived: ..." But the dup2() call throws no errors.
I read some tutorials, mostly this one https://tldp.org/LDP/lpg/node11.html but I can't find my error... Thank you for your help!
The code has a couple of problems:
In dup2(1,myPipe[1]) the parameters are back to front. That makes mypipe[1] be the same as 1. But instead you need it to be the other way around: dup2(myPipe[1],1)
dup2(myPipe[1],backup) is also wrong. That makes backup be the same as mypipe[1]. What you want instead is to make 1 the same as backup: dup2(backup, 1).
Smaller problem but printf does not output a NUL character. So the read will not result in a valid NUL terminated string. Solve that by initialising: char readbuffer[80] = "";
Related
I'm trying to understand what is behind this behaviour in my parent process.
Basically, I create a child process and connect its stdout to my pipe. The parent process continuously reads from the pipe and does some stuff.
I noticed that when inserting the while loop in the parent the stdout seems to be lost, nothing appears on the terminal etc I thought that the output of stdout would somehow go to the pipe (maybe an issue with dup2) but that doesn't seem to be the issue. If I don't continuously fflush(stdout) in the parent process, whatever I'm trying to get to the terminal just won't show. Without a while loop in the parent it works fine, but I'm really not sure why it's happening or if the rest of my implementation is problematic somehow.
Nothing past the read system call seems to be going to the stdout in the parent process. Assuming the output of inotifywait in the pipe is small enough ( 30 > bytes ), what exactly is wrong with this program?
What I expect to happen is the stdout of inotifywait to go to the pipe, then for the parent to read the message, run strtok and print the file name (which only appears in stdout when I fflush)
Running the program with inotify installed and creating any file in the current directory of the program should be enough. Removing the while loop does print the created file's name (as expected).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
char b[100];
int pipefd;
if (mkfifo("fifo", 0666) == -1) {
if (errno != EEXIST) {
perror("mkfifo");
exit(EXIT_FAILURE);
}
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
exit(1);
}
if ((pipefd = open("fifo", O_RDWR)) < 0) {
perror("open pipe");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(pipefd, 1);
const char* dir = ".";
const char* args[] = {"inotifywait", dir, "-m", "-e",
"create", "-e", "moved_to", NULL};
execvp("inotifywait", (char**)args);
perror("inotifywait");
} else {
while (1) {
fflush(stdout); // the output only appears in stdout with this here
if (read(pipefd, b, 30) < 0) {
perror("problem # read");
exit(1);
}
char filename[30];
printf("anything");
sscanf(b, "./ CREATE %s", filename);
printf("%s", filename);
}
}
}
The streams used by the C standard library are designed in such a way that they are normally buffered (except for the standard error stream stderr).
The standard output stream is normally line buffered, unless the output device is not an interactive device, in which case it is normally fully buffered. Therefore, in your case, it is probably line buffered.
This means that the buffer will only be flushed
when it is full,
when an \n character is encountered,
when the stream is closed (e.g. during normal program termination),
when reading input from an unbuffered or line-buffered stream (in certain situations), or
when you explicitly call fflush.
This explains why you are not seeing the output, because none of the above are happening in your infinite loop (when you don't call fflush). Although you are reading input, you are not doing this from a C standard library FILE * stream. Instead, you are bypassing the C runtime library (e.g. glibc) by using the read system call directly (i.e. you are using a file descriptor instead of a stream).
The simplest solution to your problem would probably be to replace the line
printf("%s", filename);
with:
printf("%s\n", filename);
If stdout is line-buffered (which should be the case if it is connected to a terminal), then the input should automatically be flushed after every line and an explicit call to fflush should no longer be necessary.
I'm learning about pipe programming in Linux, and am having trouble understanding pipe / FIFO management.
I wrote a small program which opens a FIFO I created (I did mkfifo newfifo in my terminal before executing the program). I then repeatedly read and dump my character buffer. I'm filling the FIFO using echo "message" > newfifo from another terminal's cmd line.
The problem is that when I write to the FIFO, I can read that data in the buffer, but then the read doesn't block anymore. My understanding was that after I read the data from the FIFO, the FIFO should be empty and the read should block. Am I thinking about this wrong, or am I incorrectly managing the FIFO?
Code is below:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#define NEWPIPE "./newfifo"
void main()
{
int great_success = 0;
int fd;
char buffer[20];
fd = open(NEWPIPE, O_RDONLY);
while (1) {
great_success = read(fd, buffer, 20);
if (great_success < 0) {
printf("pipe failed\n");
} else {
printf("buffer : %s\n", buffer);
printf("great_success = %d\n", great_success);
great_success = 0;
}
}
}
Your understanding of how fifos works is incorrect. They are much like pipes: if the write end is closed (the echo command has terminated), the read end will read end-of-file (EOF), i.e. return 0.
Note that when you open the fifo, it isn't read that is blocking. The blocking system call is the open() system call, as explained in http://linux.die.net/man/4/fifo
Because the process(echo "message" > newfifo) is a short program, it terminated quickly. Once the process terminated, there is no write end for the pipe, so the read end in another process gets an EOF.
I am currently trying to understand combincation of dup2 and C pipes, but not even the simplest program seems to work. Already when reading example codes I am pretty confused on when they close ends of the pipe and where the output should be printed.
Sometimes the write end is closed, even though one line later output should be generated which should go into the pipe. In other examples, the unused end is closed (which makes more sense to me).
Then, I do not understand when dup2 should be executed. I guess it should become before the output I want to redirect, but I have the feeling I also saw that differently today.
So in the end I came up with this little test with printf and fflush in each line, where nothing gets redirected through the pipe. Why's that? What am I doing wrong?
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void) {
int out_pipe[2];
char *output[101];
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello");
fflush(stdout);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello");
fflush(stdout);
close(out_pipe[1]);
printf("Hello");
fflush(stdout);
read(out_pipe[0], output, 100);
close(out_pipe[0]);
printf("PIPE: %s", output);
fflush(stdout);
return 0;
}
End your printf() messages with newlines; the fflush() is still a good idea as you're about to change where standard output goes, though it's not usually necessary if the standard output of the program is going to a terminal. If the standard output was going to a file and the fflush() was not in place, then you'd get three copies of "Hello\n" written to the pipe.
When you change standard output to the pipe, your message is indeed written to the pipe.
When you close the write file descriptor, you don't run into any issues. You then write a second Hello to the pipe. You need this fflush() to ensure that the standard I/O package has actually written its buffered data to the pipe.
You then read from the pipe into the output buffer. You should check how many bytes you read since the string is not going to be null terminated. You should get 10 bytes read (when you don't have any newlines in the messages).
You then write to the pipe again with the PIPE: prefix.
To fix, write messages to standard error.
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int out_pipe[2];
char output[101];
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello\n");
fflush(stdout);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello\n");
fflush(stdout);
close(out_pipe[1]);
printf("Hello\n");
fflush(stdout);
int n = read(out_pipe[0], output, sizeof(output));
close(out_pipe[0]);
fprintf(stderr, "PIPE: %.*s\n", n, output);
return 0;
}
Note that I changed the definition of output from an array of char * to a simple array of char. With the changes, I got the output:
$ ./pipe3
Hello
PIPE: Hello
Hello
$
That's because I included newlines in the messages written to the pipe, as well as in the format string that ends up on standard error.
Is there a possibility to "reenable" stdout?
Yes; simply preserve a copy of the original file descriptor for standard output before using dup2(), and then reinstate the copy once you've done with the pipe.
I've removed the two leading fflush() calls, and the sample output demonstrates the difference between terminal and file output:
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int out_pipe[2];
char output[101];
int old_stdout;
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello\n");
old_stdout = dup(STDOUT_FILENO);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello\n");
close(out_pipe[1]);
printf("Hello\n");
fflush(stdout);
int n = read(out_pipe[0], output, sizeof(output));
close(out_pipe[0]);
dup2(old_stdout, STDOUT_FILENO);
printf("PIPE: %d <<%.*s>>\n", n, n, output);
return 0;
}
Sample outputs:
$ ./pipe3Hello
PIPE: 12 <<Hello
Hello
>>
$./pipe3 > output
'pipe3' is up to date.
$ cat output
PIPE: 18 <<Hello
Hello
Hello
>>
$
If you remove the remaining fflush(), the program hangs. There is nothing in the pipe (because standard I/O hasn't flushed its buffer because it isn't full and the output isn't a terminal any more), but the pipe is open for writing, so the kernel considers that input could appear on it — if only the program that has the pipe open for writing wasn't waiting on the read end of the pipe for the input to appear. The program has deadlocked on itself.
I am trying to pass data from my perl script to my c program using a pipe (uni-directional).
I need to find a way to to do this without messing with the child programs STDIN or STDOUT, so I try creating a new handle and passing the fd.
I create 2 IO::Handles and create a pipe. I write to one end of the pipe and attempt to pass the File descriptor of the other end of the pipe to my child program that is being execed. I pass the file descriptor by setting an ENV variable. Why does this not work? (It does not print out 'hello world'). As far as I know, file descriptors and pipes are inherited by the child when exec'd.
Perl script:
#!/opt/local/bin/perl
use IO::Pipe;
use IO::Handle;
my $reader = IO::Handle->new();
my $writer = IO::Handle->new();
$reader->autoflush(1);
$writer->autoflush(1);
my $pipe = IO::Pipe->new($reader, $writer);
print $writer "hello world";
my $fh = $reader->fileno;
$ENV{'MY_FD'} = $fh;
exec('./child') or print "error opening app\n";
# No more code after this since exec replaces the current process
C Program, app.c (Compiled with gcc app.c -o child):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d\n", fd);
printf("message: %s\n", buf);
}
Output:
fd: 3
message:
The message is never passed through the pipe to the C program. Any suggestions?
Your pipe file descriptors are set FD_CLOEXEC, and so are closed upon exec().
Perl's $^F controls this behavior. Try something like this, before you call IO::Pipe->new:
$^F = 10; # Assumes we don't already have a zillion FDs open
Alternatively, you can with Fcntl clear the FD_CLOEXEC flag yourself after creating the pipe.
I found the solution. Some people said that it was not possible with exec, that it would not see pipes or file descriptors, but that was not correct.
Turns out that perl closes/invalidates all fd > 2 automatically unless you say otherwise.
Adding the following flags to the FD fixes this problem (where READ is the handle here, NOT STDIN):
my $flags = fcntl(READ, F_GETFD, 0);
fcntl(READ, F_SETFD, $flags & ~FD_CLOEXEC);
Your program is failing because exec calls another program and never returns. It isn't designed for communication with another process at all.
You probably wrote the above code based on the IO::Pipe documentation, which says "ARGS are passed to exec". That isn't what it means, though. IO::Pipe is for communication between two processes within your Perl script, which are created by fork. They mean the execution of the new process, rather than a call to exec in your own code.
Edit: for one-directional communication, all you need is open with a pipe:
open my $prog, '|-', './child' or die "can't run program: $!";
print {$prog} "Hello, world!";
Rodrigo, I can tell you that your file descriptor is no longer valid when you exec into the c app.
Please be aware that I just say it is INVALID, but it still exists in the environment variables. The FD=3 will continue existing until the whole process ends.
You can check the fd by fcntl. The code is listing below
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d, if fd still valid: %d\n", fd, fcntl(fd, F_GETFD));
printf("strlen %d\n", (int)strlen(buf));
printf("message: %s\n", buf);
}
You can see that MY_FD=3 will always in ENV as the process doesn't destroy itself, so you can get fd as 3. But, this file descriptor has been invalid. so the result of fcntl(fd, F_GETFD) will be -1, and the length you read from fd will be 0.
That's why you will never see the "hello world" sentence.
One more thing, #dan1111 is right, but you don't need to open a new pipe, as you have already done so.
All you need to is just set MY_FD=0, like
$ENV{'MY_FD'} = 0;
The STDIN/OUT is another independent process that always exists, so the pipe will not broken down when your perl app exec into c app. That's why you can read from what you input in app.
If your requirement is writing from another file hanle, please try to make that file handle an independent process and always exist, just like STDIN.
I have the following c code. I want to display my file with less by calling execv()
however the following seems never work. The program terminates and noting pop out.
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
int main(void){
int pid;
if(pid=fork()>0){
//read in from stdin and pass to pipe
}else if(pid==0){
//read from pipe
//write to out.txt
//everything up to here works fine
char* para[]={"less","/Desktop/out.txt"};
execv("/bin/less",para);
}
return 0;
}
(The original code contained execv("bin/less", para);.) Unless the current directory is the root directory, /, or unless there is a program less in the subdirectory ./bin/less, then one of your problems is that you have a probable typo in the name of the executable. That assumes the program is /bin/less and not /usr/bin/less. You might even use execvp() to do a PATH-based search for the program.
There's an additional problem: you need to include a null pointer to mark the end of the argument list.
Finally, you can print an error message after the execv() returns. The mere fact that it returns tells you it failed.
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
int main(void)
{
int pid;
if ((pid = fork()) != 0)
{
// read in from stdin and pass to pipe
// Need to test for fork() error here too
}
else
{
// read from pipe
// write to out.txt
// everything up to here works fine
char *para[] = { "/bin/less", "Desktop/out.txt", 0 };
execv(para[0], para);
fprintf(stderr, "Failed to execute %s\n", para[0]);
exit(1);
}
return 0;
}
Or:
char *para[] = { "less", "Desktop/out.txt", 0 };
execvp(para[0], para);
fprintf(stderr, "Failed to execute %s\n", para[0]);
The remarks in the code about pipes are puzzling since there is no sign of pipes other than in the comments. As it stands, less will read the file it is told to read. Note that less will not paginate its output if the output is not going to a terminal. Since we can see no I/O redirection, we have to assume, then, that less will ignore anything the program tries to write to it, and will not send any data back to the program.
char* para[]={"less","/Desktop/out.txt"};
execv("/bin/less",para);
How does execv know when to stop reading parameters?
I think if you'd put code in there to handle execv() returning an error you'd have found this. You're also not testing for errors from fork().