No output in the parent process without fflush(stdout) - c

I'm trying to understand what is behind this behaviour in my parent process.
Basically, I create a child process and connect its stdout to my pipe. The parent process continuously reads from the pipe and does some stuff.
I noticed that when inserting the while loop in the parent the stdout seems to be lost, nothing appears on the terminal etc I thought that the output of stdout would somehow go to the pipe (maybe an issue with dup2) but that doesn't seem to be the issue. If I don't continuously fflush(stdout) in the parent process, whatever I'm trying to get to the terminal just won't show. Without a while loop in the parent it works fine, but I'm really not sure why it's happening or if the rest of my implementation is problematic somehow.
Nothing past the read system call seems to be going to the stdout in the parent process. Assuming the output of inotifywait in the pipe is small enough ( 30 > bytes ), what exactly is wrong with this program?
What I expect to happen is the stdout of inotifywait to go to the pipe, then for the parent to read the message, run strtok and print the file name (which only appears in stdout when I fflush)
Running the program with inotify installed and creating any file in the current directory of the program should be enough. Removing the while loop does print the created file's name (as expected).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
char b[100];
int pipefd;
if (mkfifo("fifo", 0666) == -1) {
if (errno != EEXIST) {
perror("mkfifo");
exit(EXIT_FAILURE);
}
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
exit(1);
}
if ((pipefd = open("fifo", O_RDWR)) < 0) {
perror("open pipe");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(pipefd, 1);
const char* dir = ".";
const char* args[] = {"inotifywait", dir, "-m", "-e",
"create", "-e", "moved_to", NULL};
execvp("inotifywait", (char**)args);
perror("inotifywait");
} else {
while (1) {
fflush(stdout); // the output only appears in stdout with this here
if (read(pipefd, b, 30) < 0) {
perror("problem # read");
exit(1);
}
char filename[30];
printf("anything");
sscanf(b, "./ CREATE %s", filename);
printf("%s", filename);
}
}
}

The streams used by the C standard library are designed in such a way that they are normally buffered (except for the standard error stream stderr).
The standard output stream is normally line buffered, unless the output device is not an interactive device, in which case it is normally fully buffered. Therefore, in your case, it is probably line buffered.
This means that the buffer will only be flushed
when it is full,
when an \n character is encountered,
when the stream is closed (e.g. during normal program termination),
when reading input from an unbuffered or line-buffered stream (in certain situations), or
when you explicitly call fflush.
This explains why you are not seeing the output, because none of the above are happening in your infinite loop (when you don't call fflush). Although you are reading input, you are not doing this from a C standard library FILE * stream. Instead, you are bypassing the C runtime library (e.g. glibc) by using the read system call directly (i.e. you are using a file descriptor instead of a stream).
The simplest solution to your problem would probably be to replace the line
printf("%s", filename);
with:
printf("%s\n", filename);
If stdout is line-buffered (which should be the case if it is connected to a terminal), then the input should automatically be flushed after every line and an explicit call to fflush should no longer be necessary.

Related

C writing into files ignores new line?

Given the following code, I have: printf("hello\n"); so I am expecting to see hello\n in myfile. but when I run my program I see haha which means \n was ignored, why is that?
Worth Noting: when I replace printf("hello\n"); with printf("hellos"); I don't see the s letter being printed as well. So I think maybe something is writing on top of it but the question is who and why?
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/file.h>
#include <string.h>
int main() {
// creates a new file having full read/write permissions
int fd = open("myfile", O_RDWR | O_CREAT, 0666);
write(fd, "haha\n", 5);
close(fd); // line 6
fd = open("myfile", O_RDWR); // line 7
close(0);
close(1);
dup(fd);
dup(fd);
if (fork() == 0) {
char s[100];
dup(fd);
scanf("%s", s);
printf("hello\n");
write(2, s, strlen(s)); // line 18
return 0; // line 19
}
wait(NULL);
printf("Father finished\n");
close(fd);
return 0;
}
contents of myfile after running:
haha
helloFather finished (new line after this)
First of all, the behavior is undefined. You are starting to use stdout that refers to the same file descriptor as stdin without closing or flushing stdin before doing it. Let's try to take the important stuff from POSIX 2.5.1 Interaction of File Descriptors and Standard I/O Streams:
[...] if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined.
[...]
For a handle to become the active handle, the application shall ensure that the actions below are performed between the last use of the handle (the current active handle) and the first use of the second handle (the future active handle). [...]
[...]
For the first handle, the first applicable condition below applies. [...]
[...]
If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.
Your code does:
scanf("%s", s); // associates stdin with fd
// Ups - no flush(stdin) nor fclose(stdin)
printf("hello\n"); // associates stdout with fd - undefined behavior
The result you are seeing comes from that scanf calls ungetc which increments file position but also "remembers" to un-increment file position once the stream is flushed. Because it is flushed when child terminates, the file position is decremented, and parent overwrites last character of the child.

what will happen calling printf after close stdout?

I tried below code, and screen showed nothing.
close(STDOUT_FILENO);
printf("Child output something\n");
is it just can not find the stdout,then abort the data?
I want to find wether printf write some data, since I can not print the return value so I output it to some file.
close(STDOUT_FILENO);
int res = printf("output something\n");
open("./log.output", O_CREAT|O_WRONLY|O_TRUNC, S_IRWXU);
printf("%d", res); // return 17
So printf work, but I don't know where it write to.
The reason you're seeing this result has to do with buffering. In general, a file which is attached to a terminal is line buffered and all other files are block buffered. stderr is unbuffered.
When you close stdout, it's no longer attached to a terminal, so it's block buffered, not line buffered. You've attempted to write fewer bytes than the buffer size (which is usually some multiple of 512), so printf happily copied it to the buffer and did nothing else. If you wrote a suitable amount of data using printf, you'd find that it did indeed fail at that point.
You can verify a similar behavior by calling fflush(stdout):
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int main(void)
{
close(STDOUT_FILENO);
int res = printf("output something\n");
fprintf(stderr, "%d\n", res);
res = fflush(stdout);
fprintf(stderr, "%d %s\n", res, strerror(errno));
}
The last line will output -1 Bad file descriptor, which shows that the attempt to write out to stdout failed with EBADF, as expected. If you need to verify that data has been written, you must call fflush or fsync as appropriate.
Note that in general, you don't want to close any of the three default file descriptors, because any time you open a new file descriptor, it will use the lowest unused number and take the place of one of the standard streams. If a separate part of your program attempts to write to one of those streams without checking, it can write into an unexpected file, corrupting it. The safe thing to do is redirect those streams to /dev/null instead.
Your open call for log.output does exactly the thing I just mentioned in that it opens file descriptor 1 (stdout) again.

Should a read from FIFO block after all the data was just read from that FIFO?

I'm learning about pipe programming in Linux, and am having trouble understanding pipe / FIFO management.
I wrote a small program which opens a FIFO I created (I did mkfifo newfifo in my terminal before executing the program). I then repeatedly read and dump my character buffer. I'm filling the FIFO using echo "message" > newfifo from another terminal's cmd line.
The problem is that when I write to the FIFO, I can read that data in the buffer, but then the read doesn't block anymore. My understanding was that after I read the data from the FIFO, the FIFO should be empty and the read should block. Am I thinking about this wrong, or am I incorrectly managing the FIFO?
Code is below:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#define NEWPIPE "./newfifo"
void main()
{
int great_success = 0;
int fd;
char buffer[20];
fd = open(NEWPIPE, O_RDONLY);
while (1) {
great_success = read(fd, buffer, 20);
if (great_success < 0) {
printf("pipe failed\n");
} else {
printf("buffer : %s\n", buffer);
printf("great_success = %d\n", great_success);
great_success = 0;
}
}
}
Your understanding of how fifos works is incorrect. They are much like pipes: if the write end is closed (the echo command has terminated), the read end will read end-of-file (EOF), i.e. return 0.
Note that when you open the fifo, it isn't read that is blocking. The blocking system call is the open() system call, as explained in http://linux.die.net/man/4/fifo
Because the process(echo "message" > newfifo) is a short program, it terminated quickly. Once the process terminated, there is no write end for the pipe, so the read end in another process gets an EOF.

How does stuff have to be ordered when using dup2 and pipe?

I am currently trying to understand combincation of dup2 and C pipes, but not even the simplest program seems to work. Already when reading example codes I am pretty confused on when they close ends of the pipe and where the output should be printed.
Sometimes the write end is closed, even though one line later output should be generated which should go into the pipe. In other examples, the unused end is closed (which makes more sense to me).
Then, I do not understand when dup2 should be executed. I guess it should become before the output I want to redirect, but I have the feeling I also saw that differently today.
So in the end I came up with this little test with printf and fflush in each line, where nothing gets redirected through the pipe. Why's that? What am I doing wrong?
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void) {
int out_pipe[2];
char *output[101];
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello");
fflush(stdout);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello");
fflush(stdout);
close(out_pipe[1]);
printf("Hello");
fflush(stdout);
read(out_pipe[0], output, 100);
close(out_pipe[0]);
printf("PIPE: %s", output);
fflush(stdout);
return 0;
}
End your printf() messages with newlines; the fflush() is still a good idea as you're about to change where standard output goes, though it's not usually necessary if the standard output of the program is going to a terminal. If the standard output was going to a file and the fflush() was not in place, then you'd get three copies of "Hello\n" written to the pipe.
When you change standard output to the pipe, your message is indeed written to the pipe.
When you close the write file descriptor, you don't run into any issues. You then write a second Hello to the pipe. You need this fflush() to ensure that the standard I/O package has actually written its buffered data to the pipe.
You then read from the pipe into the output buffer. You should check how many bytes you read since the string is not going to be null terminated. You should get 10 bytes read (when you don't have any newlines in the messages).
You then write to the pipe again with the PIPE: prefix.
To fix, write messages to standard error.
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int out_pipe[2];
char output[101];
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello\n");
fflush(stdout);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello\n");
fflush(stdout);
close(out_pipe[1]);
printf("Hello\n");
fflush(stdout);
int n = read(out_pipe[0], output, sizeof(output));
close(out_pipe[0]);
fprintf(stderr, "PIPE: %.*s\n", n, output);
return 0;
}
Note that I changed the definition of output from an array of char * to a simple array of char. With the changes, I got the output:
$ ./pipe3
Hello
PIPE: Hello
Hello
$
That's because I included newlines in the messages written to the pipe, as well as in the format string that ends up on standard error.
Is there a possibility to "reenable" stdout?
Yes; simply preserve a copy of the original file descriptor for standard output before using dup2(), and then reinstate the copy once you've done with the pipe.
I've removed the two leading fflush() calls, and the sample output demonstrates the difference between terminal and file output:
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int out_pipe[2];
char output[101];
int old_stdout;
if (pipe(out_pipe) != 0) {
perror("pipe()");
exit(1);
}
printf("Hello\n");
old_stdout = dup(STDOUT_FILENO);
dup2(out_pipe[1], STDOUT_FILENO);
printf("Hello\n");
close(out_pipe[1]);
printf("Hello\n");
fflush(stdout);
int n = read(out_pipe[0], output, sizeof(output));
close(out_pipe[0]);
dup2(old_stdout, STDOUT_FILENO);
printf("PIPE: %d <<%.*s>>\n", n, n, output);
return 0;
}
Sample outputs:
$ ./pipe3Hello
PIPE: 12 <<Hello
Hello
>>
$./pipe3 > output
'pipe3' is up to date.
$ cat output
PIPE: 18 <<Hello
Hello
Hello
>>
$
If you remove the remaining fflush(), the program hangs. There is nothing in the pipe (because standard I/O hasn't flushed its buffer because it isn't full and the output isn't a terminal any more), but the pipe is open for writing, so the kernel considers that input could appear on it — if only the program that has the pipe open for writing wasn't waiting on the read end of the pipe for the input to appear. The program has deadlocked on itself.

How to write redirected output to file inside loop in C?

I am writing a C program on unix which should redirect it's output to the file, and write to it some text every second in infinite loop:
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main(void) {
int outDes = open("./output.txt", O_APPEND | O_WRONLY);
dup2(outDes, 1);
while(1) {
printf("output text\n");
sleep(1);
}
}
But it writes nothing to the output file. I tried to change the 'while' loop for 'for' with 10 loops, and I found that it writes all 10 lines to the file at once after the series ends. It is not very good for me, while I need to have an infinite loop.
When I'm not redirecting output, it is all ok, and new line appears every second on terminal.
I also tried to put one
printf("text\n");
before redirecting output to the file. Then the program wrote the lines to the file in real time, which is good, but wrote there the first (non redirected) line too. I don't want this first line in my output file, I don't understand how it could be written into file when output was not redirected yet (maybe redirect remained there since last run?), and how it could cause that the lines are suddenly written in real time.
Can anyone explain me how does it work?
You are not checking the return value of open() and dup2(). If either open() or dup2() failed, it won't write anything in output.txt.
if (outDes < -1) {
perror("open");
return 1;
}
if (dup2(outDes, 1) == -1) {
perror("dup2");
return 1;
}
stdio streams are buffered, and the writes happen in memory before being done on the real file description.
Try adding a fflush(stdout) after printf().
You're running afoul of a poorly documented DWIMmy feature in many Unix C libraries. The first time you write to stdout or stderr, the library probes the underlying file descriptor (with isatty(3)). If it's a (pseudo-)terminal, the library puts the FILE in "line buffered" mode, meaning that it'll buffer input until a newline is written and then flush it all to the OS. But if the file descriptor is not a terminal, it puts the FILE in "fully buffered" mode, where it'll buffer something like BUFSIZ bytes of output before flushing them, and pays no attention to line breaks.
This is normally the behavior you want, but if you don't want it (as in this case), you can change it with setvbuf(3). This function (although not the behavior I described above) is ISO standard C. Here's how to use it in your case.
#include <stdio.h>
#include <unistd.h>
int
main(void)
{
if (freopen("output.txt", "a", stdout)) {
perror("freopen");
return 1;
}
if (setvbuf(stdout, 0, _IOLBF, 0)) {
perror("setvbuf");
return 1;
}
for (;;) {
puts("output text");
sleep(1);
}
/* not reached */
}

Resources