what will be the output of the following snippet and why?
#include<stdio.h>
void main(){
printf("Before while");
while(1);
printf("After while");
}
If you had just taken the effort to compile and run, you probably would have seen nothing.
That's because:
Even though you're printing Before while, you do not send a newline character at the end. Since standard output is generally line buffered for an interactive device(a), it should cache this until you send a newline, flush the stream, or exit the program.
You do none of those things because while(1); (note the semicolon) is an infinite loop that does nothing in its body, meaning it will never reach the second printf.
(a) If you're interested in this behaviour, there are basically three buffering strategies for output.
Unbuffered means output is passed straight through;
Line-buffered means output is cached until a newline is output (although it may also be output without a newline if, for example, the buffer fills up);
Fully-buffered means output is cached until the buffer is full at which point it's output and the buffer starts afresh.
The rules in the ISO C standard (C11 7.21.3 Files /7) for the three standard streams are:
At program startup, three text streams are predefined and need not be opened explicitly - standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output).
As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
Related
I have a simple C file, that writes to the standard output using printf and write(1,..) functions.
File: main.c
#include<stdio.h>
#include<unistd.h>
int main()
{
printf("Hello\n");
printf("Mars\n");
fsync(1);
write(1,"Ola\n",4);
write(1,"Earth\n",6);
}
> gcc main.c -o test
When I redirect the output to a file, I see different order than the output in the terminal. It seems, that when I am using redirection write(..) writes before than printf(..), even tough, printf() is earlier in code and there is a fsync(..) in between.
Is there some variant of the redirection operator that will ensure order, or am I doing something wrong.
Output in terminal:
> ./test
Hello
Mars
Ola
Earth
Output in out file:
> ./test >out
# or ./test|tee out
> cat out
Ola
Earth
Hello
Mars
printf writes to the C stream stdout, which is buffered by the C standard library routines and ultimately written to Unix file descriptor 1.
fsync and write operate on Unix file descriptor 1. fsync flushes data from file descriptor 1, but it has no effect on the buffers of the streams of the C standard library.
The C standard library I/O routines operate differently depending on whether standard output is going to a terminal or to a file. This is detected during program start-up. When output is going to a terminal, it is deemed important to get it to the user, and the buffer is flushed when \n is written to a stream. This is called line buffered, and output to a terminal is set to line buffered by default.
When output is going to a file, it is not deemed important to flush it promptly, since a user is not expected to see it right away and performance is better if output is fully buffered. So the buffer is flushed only when it is fully, the stream is closed, or an explicit flush request is made. This is called fully buffered, and output to a file is set to fully buffered by default.
This is generally addressed in C 2018 7.21.3, in which paragraph 7 says:
… the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
(When the output is a terminal, the C implementation may also use unbuffered mode instead of line buffered, in which case all characters are sent to the terminal as soon as they are written. This is uncommon.)
You can explicitly requesting flushing of stdout with fflush(stdout).
After stdout is opened but before any other operation is performed on it, you can change its buffer method with setvbuf(stdout, NULL, mode, 0), where mode is _IOFBF, _IOLBF, or _IONBF for full, line, or no buffering, respectively. (You can also pass a char array and its size in place of NULL and 0, and the library will use that array for the buffer, after which you should not use the array for any other purpose. If you pass NULL and a size, the library may use that size to perform its own buffer allocation, but that is not guaranteed.) setvbuf returns an int that may be non-zero if it cannot honor the request.
have a look at the dup2 system call: https://www.geeksforgeeks.org/dup-dup2-linux-system-call/
you can redirect stdoutput / input to a file
I'm just curious which conditions should be satisfied to flush stdout buffer automatically.
First of all I was confused that this pseudo code doesn't print output every iteration:
while (1) {
printf("Any text");
sleep(1);
}
But if I add newline character it will.
After few experiments i found that on my machine stdout buffer is flushed:
When I put to stdout 1025 characters or more;
When I read stdin;
When I put newline character to stdout;
The first condition is totally clear - when the buffer is full it should be flushed. The second one is also reasonable. But why newline character causes flushing? What are the others implicit conditions for this?
Rules of automatic flushing stdout buffer is implementation-defined (ID). It is ID when the stream is unbuffered, fully buffered, or line buffered.
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block.
When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered. Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment.
Support for these characteristics is implementation-defined, ... C11dr §7.21.3 3
I'm just curious which conditions should be satisfied to flush stdout buffer automatically.
If code wants to insure output is certainly flushed, use fflush(). Other conditions that may automatically flush the stream are implementation defined.
A output stream which is line-buffered shall be flushed whenever a newline is output.
An implementation may (but is not required to) flush all line-buffered output streams whenever a read is attempted from any line-buffered input stream.
Implementations are not allowed to make streams fully-buffered by default unless it can be determined that they are not associated with an "interactive device". So when stdin/stdout are terminals they can't be fully-buffered, only line-buffered (or unbuffered).
If you only need flushing when the output is to a terminal, it suffices to assume that writing a newline results in flushing. Otherwise you should explicitly call fflush wherever you need flushing.
See the man page for setbuf(3). By default, stdout is set to line buffering mode.
printf() and its variants work with buffered output, and delegate to write(). So this buffering is controlled by the C library implementation of printf, with the buffer and buffer settings located in the FILE structure.
It's also worth noting the difference between section 3 and section 2 of the unix man pages. Section 2 is made up of function calls that directly talk to the operating system and do things that it would otherwise be impossible to do from a pure user program. Section 3 is made up of function calls that a user could reproduce on their own, which often delegate to section 2 calls. Section 2 functions contain the low-level "magic" that allow C programs to interact with the outside world and perform I/O. Section 3 functions can provide a more convenient interface to the section 2 functions.
printf, scanf, getchar, fputs, and other FILE * functions all are section 3 functions that delegate to write() and read(), which are section 2 functions. read() and write() do not buffer. printf() interacts with the buffer in the FILE structure, and occasionally decides to send the contents of that buffer out through write().
There are many circumstances when buffered output on a stream is flushed automatically:
When you try to do output and the output buffer is full.
When the stream is closed.
When the program terminates by calling exit.
When a newline is written, if the stream is line buffered.
Whenever an input operation on any stream actually reads data from its file.
stdout is line buffered by default.
If you want to flush the buffered output at another time,You can call fflush.
Online C2011 standard
7.21.3 Files
...
3 When a stream is unbuffered, characters are intended to appear from the source or at the
destination as soon as possible. Otherwise characters may be accumulated and
transmitted to or from the host environment as a block. When a stream is fully buffered,
characters are intended to be transmitted to or from the host environment as a block when
a buffer is filled. When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and setvbuf functions.
...
7 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.
So, a line buffered stream will flush on a newline. On most systems I have experience with, stdout is line buffered in an interactive session.
I know this is because of buffering of stdout but when can i expect output of stdout in the following proggram. If i run, I am always getting "stderr" as output. If i add '\n' or fflush(stdout) then only i am getting both statements. If i don't add '\n' or fflush(stdout),i am not getting "stdout" as output. when will i get all buffered "stdout"s as output if i don't add '\n' or fflush(stdout).
#include <stdio.h>
#include <unistd.h>
int main()
{
for(;;)
{
fprintf(stdout,"stdout");
fprintf(stderr,"stderr");
sleep(1);
}
return 0;
}
Indeed, a new line or a flush because by default stdout is line buffered when it refers a terminal device.
To be more precise: standard input and standard output are fully buffered, if and only if they do not refer to an interactive device. Standard error is never fully buffered.
About line buffering, quoting from APUE:
Line buffering comes with two caveats. First, the size of the buffer that the standard I/O library is using to collect each line is fixed, so I/O might take place if we fill this buffer before writing a newline. Second, whenever input is requested through the standard I/O library from either (a) an unbuffered stream or (b) a line-buffered stream (that requiresdata to be requested from the kernel), all line-buffered output streams are flushed. The reason for the qualifier on (b) is that the requested data may alreadybe
in the buffer, which doesn't require data to be read from the kernel. Obviously, any input from an unbuffered stream, item (a), requires data to be obtained from the kernel.
To change it to unbuffered, use setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);
output sent to stderr usually prints immediately. It is the standard error output. Errors, of course, should be displayed immediately, and so any buffer that exists is flushed immediately.
stdout, on the other hand, is buffered. Getting that buffer to print out varies by language, but usually at least one of two things are required: /n, or some other type of newline character, or .flush();. If you do not provide these options, then usually you have to wait for the buffer to fill. This can take any amount of time, as it pretty much entirely depends on how large the buffer is.
By convention, the way streams are handled is that you shove as much information into them as you want, and then call .flush(). If the buffer happens to fill up in the meantime, then the data is sent to wherever it's going and the buffer starts filling up again.
Here, though, you're filling up a buffer that takes in text. Text does not really take up a whole lot of space compared to binary information (this is a VERY subjective statement and should be interpreted loosely), and so filling up the buffer could take quite a while.
The best practice is for you to manually flush the stream (stdout) once you have "fed" it all of the text you want printed.
fprintf is by default line buffered. You can alter the behavior by calling setvbuf. It allows you to set it to 'unbuffered', 'line buffered', or 'fully buffered'.
So I wrote a test program and here is the code
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int i;
printf("%s", "entering\n");
fflush(stdout);
for (i = 0 ; i < 3 ; i++)
{
fork();
fflush(stdout);
}
printf("%s", "exiting\n");
}
When I compile and run it in the terminal it displays what I expect it to: "entering" once, and "exiting" some times. When I run it and redirect the output to a file it displays entering for every exiting.
1) Why does it not output the same thing to terminal and to a file every time?
2) Why does it display entering in the file 8 times but not entering in in the terminal only once (once is what I would expect it to do).
3) Do the fflush() statements make a difference when my output goes to a file?
It has to do with the buffering on the standard file handles. From ISO C99 7.19.3/7 Files:
As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
When you're writing to the terminal, the output is most likely (a) line-buffered, meaning it will be flushed whenever a newline is sent.
With redirection, it's fully buffered meaning it will only flush when the buffer is full.
You can use setvbuf before operating on stdout, to set it to unbuffered or line buffered, and you won't have to worry about flushing.
setvbuf (stdout, NULL, _IONBF, BUFSIZ); // _IONBF for no buffering.
// _IOFBF for full buffering.
// _IOLBF for line buffering.
Just keep in mind that unbuffered output may affect performance quite a bit if you're sending the output to a file. Also keep in mind that support for this is implementation-defined, not guaranteed by the standard.
C99 section 7.19.3/3 is the relevant bit:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block.
When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered.
Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment.
Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
(a) Whether standard input and output are unbuffered or line buffered where the underlying file is possibly interactive is not specified by the standard (see here). Line buffering is the most common (by far) from what I've encountered.
As to why you're getting multiple entering messages even though you flush it before forking, that's a trickier one. You haven't listed your environment so this is supposition at best but I see a few possibilities (although there may well be more).
Firstly, the fflush may be failing for some reason. This is actually possible but easy to check since it behaves similarly to write (because it usually calls write under the covers).
In other words, check errno after the flush to see if there's been a problem. If so, the C runtime buffers will still be unflushed in all children so they'll all write entering.
Secondly, even if the buffers are flushed, there may be more buffering going on below (at the write level or within the terminal drivers themselves) which is duplicated by the fork, resulting in multiple outputs to the terminal. I consider this unlikely.
Thirdly, this may just be a weird platform-specific issue. When I run your code on my Ubuntu 11.04 box, I see no difference between the terminal output and the file output variants. They both output one entering and eight exiting messages.
If that third one is the case then you really have no recourse: ISO C doesn't mandate what happens in this case because ISO C knows nothing of fork. I can't find anything in POSIX.1 that seems to indicate one way or another but it may be there.
For what it's worth, if I comment out only the first fflush, I get entering twice followed by eight exiting messages. If I comment out only the second, it asts the same as if they're both there, one entering followed by eight exiting.
If I comment out both of them, I get eight entering/exiting pairs.
Regarding the multiple "entering" printed, it may be a bug(?) due to your environment.
I have already noticed the following strange behavior when running a program on zsh/Cygwin/WinXP compiled with MSVC98. If the program crashes, the stdout buffer is discarded and the program is rerun a small number of time.
For instance:
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("hello\n");
fflush(stdout);
*(char*) 0 = 1;
}
will give on my PC:
> cl.exe crash.c
[etc...]
> ./crash.exe
hello
hello
hello
hello
hello
hello
Without the flush(), nothing is printed.
The same program compiled with gcc has a more "expected" behavior, even without the flush():
hello
zsh: segmentation fault (core dumped) ./a.exe
No idea why. I tried to investigate a bit, but without success.
I have 3 different processes that all print out single characters using printf. But I can't see them in the terminal. When I add a newline, printf("\n H") so each character is on a new line, I can see them. Why doesn't it work without the newline character?
Its a matter of flushing. If you flush the buffers after each printf, you should get output closer to what you want. To flush the standard output simply do fflush( stdout ).
The C standard defines 3 types of buffering for output streams:
Unbuffered → no buffering done
Line-buffered → buffer until newline seen
Fully-bufferd → buffer up to the buffer size
An output stream's buffering type can be changed via the setvbuf(3) and setbuf(3) functions.
The C standard requires stderr to not be fully-buffered at startup (it is usually unbuffered on many implementations, so as to see errors as soon as posible); and stdout to be fully-buffered only if it can be determined to not refer to a terminal (when it refers to a terminal, many implementations initialize it as line-buffered, which is what you are seeing).
use'write(1,&c,1)' system call, or
fprintf(stderr,'%c', c);