I know the the messages to be displayed is buffered in stdout. But consider the following code:
#include <stdio.h>
int main()
{
while(true)
{
printf("Buffered, will be flushed");
}
}
I think that the message doesn't display until the newline is reached. But in this case we have that the pack of the messages will be displayed.
Buffers aren't infinite in size. If you output more than the buffer's size, it will be flushed automatically, and you'll see a bunch of messages all at once (repeatedly here).
Here's how the C standard describes the various forms of buffering (C11 draft n1570, §7.21.3/3):
When a stream is unbuffered, characters are intended to appear from the source or at the
destination as soon as possible. Otherwise characters may be accumulated and
transmitted to or from the host environment as a block. When a stream is fully buffered,
characters are intended to be transmitted to or from the host environment as a block when
a buffer is filled. When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and setvbuf functions.
Emphasis added. POSIX has essentially the same definition, which you can browse online in the Standard I/O Streams chapter.
Standard output is usually line buffered.
Related
I am writing a program in C on Linux where various things will be written to stdout via printf. Naturally, I would try to minimize the IO calls and buffer all the information and then pass it to a single print call. However, through testing, I have discovered that printf does buffering of its own until it reaches a '\n'.
My question is, can I be certain that all printf implementations do this, or is glibc just optimized? Is it reliable to trust printf to do the buffering for me?
The C standard allows both unbuffered and buffered streams. The relevant part is C17 7.21.3/3:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled. When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered.
This is typically a decision depending on the OS rather than the standard library implementation. Most hosted console-based OS use the line buffered implementation where \n will "flush the buffer". Otherwise an explicit call to fflush(stdout) will always do that (and it's strictly speaking more portable).
An example of an unbuffered system is limited "bare metal" microcontroller one, where stdout is an UART and there's no hardware buffers to store a lot of characters.
In this code:
#include<stdio.h>
int main()
{
printf("I");
printf("am");
printf("a");
printf("good");
printf("boy");
return 0;
}
Why I don't have to use fflush(stdout) every time? I mean how the strings are getting printed in the console if there is no new line(since new line flushes the output buffer).Or is it the buffer is getting full so I don't to do it explicitly?If so how?
Note that the stdout stream is buffered, so as Michael commented you will see the output(mostly) when the program quits. If you want to immediatel print the output the you can use fprintf like
fprintf(stderr, "my Text");
See the ISO C99 section 7.19.3/3:
When a stream is unbuffered, characters are intended to appear from
the source or at the destination as soon as possible. Otherwise
characters may be accumulated and transmitted to or from the host
environment as a block.
When a stream is fully buffered, characters are intended to be
transmitted to or from the host environment as a block when a buffer
is filled.
When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line
character is encountered.
Furthermore, characters are intended to be transmitted as a block to
the host environment when a buffer is filled, when input is requested
on an unbuffered stream, or when input is requested on a line buffered
stream that requires the transmission of characters from the host
environment.
Support for these characteristics is implementation-defined, and may
be affected via the setbuf and setvbuf functions.
I'm confused the interpretation of printf's return value and buffered stream in Standard C Library.
In C99:TC3 Standard, 7.19.6.3/p3 defines that printf function returns non-negative "number of characters transmitted" in success.
Also, 7.19.3/p3 describes the behaviors of fully/line buffered stream with "transmitted to or from the host environment", and p7 says stdout can be fully buffered stream.
Quote section 7.19.3 with emphasis added:
7.19.3 Files
3 When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered. Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment. Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
7 [...] As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
These definitions lead the following behavior is legal.
But it's counterintuitive and unacceptable result (at least for me).
#include <stdio.h>
#include <assert.h>
// PRECONDITION: `stdout` is fully buffered
int main()
{
int n = printf("abc"); // "abc" is accumulated, and no transmission.
assert(n == 0); // so, return value can be equal to 0 ??
}
What am I wrong in my interpretation?
Or it's only one of "implementation-defined" behavior?
The return value of printf and fprintf are the number of bytes transmitted to the stream -- which is an object inside of the host environment -- not to the final destination. When and how those bytes are transmitted by the stream to the file or device ("transmitted from the host environment") is irrelevant. Buffering does not affect how the stream accepts bytes from the program; just whether and until when it holds onto them until sending them on to the associated file or device.
The word "transmitted" implies that the bytes cross some interface. I submit that the interface that 7.19.6.3 refers to is the interface between printf and the device driver, whereas the interface that 7.19.3 refers to is the output of the device driver.
Furthermore, I submit that the interpretation you propose would make the return value from printf arbitrary and capricious.
Hence, I conclude that the printf in your sample code will always return 3.
int main()
{
printf("Hello"); // doesn't display anything on the screen
printf("\n"); // hello is display on the screen
return 0;
}
All characters(candidate of printing) are buffered until a new line is received? Correct?
Q1 - Why does it wait before printing on terminal until a newline char?
Q2 - Where are the characters of first printf (i.e. "Hello") buffered?
Q3 - What is the flow of printing printf()->puts()->putchar() -> now where? driver? Does the driver has a control to wait until \n?
Q4 - What is the role stdout that is attached to a process?
Looking for a in-depth picture. Feel free to edit the question, if something doesn't makes sense.
printf is not writing directly to the screen, instead it writes to the output stream, which is by default buffered. The reason for this is, that there may not even be a screen attached and the output can go to a file as well. For performance reasons, it is better for a system if access to disc is buffered and then executed in one step with appropriately sized chunks, rather than writing every time.
You can even change the size of the buffer and set it to 0, which means that all output goes directly to the target, which may be usefull for logging purposes.
setbuf(stdout, NULL);
The buffer is flushed either when it is full, or if certain criterions are fullfilled, like printing a newline. So when you would execute the printf in a loop, you would notice that it will write out in chunks unless you have a newline inbetween.
I'll start with some definitions and then go on to answer your questions.
File: It is an ordered sequence of bytes. It can be a disk file, a stream of bytes generated by a program (such as a pipeline), a TCP/IP socket, a stream of bytes received from or sent to a peripheral device (such as the keyboard or the display) etc. The latter two are interactive files. Files are typically the principal means by which a program communicates with its environment.
Stream: It is a representation of flow of data from one place to another, e.g., from disk to memory, memory to disk, one program to another etc. A stream is a source of data where data can be put into (write) or taken data out of (read). Thus, it's an interface for writing data into or reading data from a file which can be any type as stated above. Before you can perform any operation on a file, the file must be opened. Opening a file associates it with a stream. Streams are represented by FILE data type defined in stdio.h header. A FILE object (it's a structure) holds all of the internal state information about the connection to the associated file, including such things as the file position indicator and buffering information. FILE objects are allocated and managed internally by the input/output library functions and you should not try to create your own objects of FILE type, the library does it for us. The programs should deal only with pointers to these objects (FILE *) rather than the objects themselves.
Buffer: Buffer is a block of memory which belongs to a stream and is used to hold stream data temporarily. When the first I/O operation occurs on a file, malloc is called and a buffer is obtained. Characters that are written to a stream are normally accumulated in the buffer (before being transmitted to the file in chunks), instead of appearing as soon as they are output by the application program. Similarly, streams retrieve input from the host environment in blocks rather than on a character-by-character basis. This is done to increase efficiency, as file and console I/O is slow in comparison to memory operations.
The C library provides three predefined text streams (FILE *) open and available for use at program start-up. These are stdin (the standard input stream, which is the normal source of input for the program), stdout (the standard output stream, which is used for normal output from the program), and stderr (the standard error stream, which is used for error messages and diagnostics issued by the program). Whether these streams are buffered or unbuffered is implementation-defined and not required by the standard.
GCC provides three types of buffering - unbuffered, block buffered, and line buffered. Unbuffered means that characters appear on the destination file as soon as written (for an output stream), or input is read from a file on a character-by-character basis instead of reading in blocks (for input streams). Block buffered means that characters are saved up in the buffer and written or read as a block. Line buffered means that characters are saved up only till a newline is written into or read from the buffer.
stdin and stdout are block buffered if and only if they can be determined not to refer to an interactive device else they are line buffered (this is true of any stream). stderr is always unbuffered by default.
The standard library provides functions to alter the default behaviour of streams. You can use fflush to force the data out of the output stream buffer (fflush is undefined for input streams). You can make the stream unbuffered using the setbuf function.
Now, let's come to your questions.
Unmarked question: Yes, becausestdout normally refers to a display terminal unless you have output redirection using > operator.
Q1: It waits because stdout is newline buffered when it refers to a terminal.
Q2: The characters are buffered, well, in the buffer allocated to the stdout stream.
Q3: Flow of the printing is: memory --> stdout buffer --> display terminal. There are kernel buffers as well controlled by the OS which the data pass through before appearing on the terminal.
Q4: stdout refers to the standard output stream which is usually a terminal.
Finally, here's a sample code to experiment things before I finish my answer.
#include <stdio.h>
#include <limits.h>
int main(void) {
// setbuf(stdout, NULL); // make stdout unbuffered
printf("Hello, World!"); // no newline
// printf("Hello, World!"); // with a newline
// only for demonstrating that stdout is line buffered
for(size_t i = 0; i < UINT_MAX; i++)
; // null statement
printf("\n"); // flush the buffer
return 0;
}
Yes, by default, standard output is line buffered when it's connected to a terminal. The buffer is managed by the operating system, normally you don't have to worry about it.
You can change this behavior using setbuf() or setvbuf(), for example, to change it to no buffer:
setbuf(stdout, NULL);
All the functions of printf, puts, putchar outputs to the standard output, so they use the same buffer.
If you wish, you can flush out the characters before the new line by calling
fflush(stdout);
This can be handy if you're slowly printing something like a progress bar where each character gets printed without a newline.
int main()
{
printf("Hello"); // Doesn't display anything on the screen
fflush(stdout); // Now, hello appears on the screen
printf("\n"); // The new line gets printed
return 0;
}
#include <stdio.h>
int main()
{
int age;
printf("Enter Your age: ");
scanf("%lf", &age);
printf("Your age is %d\n", age);
return 0;
}
If we run the program the prompt will be displayed on the terminal screen, even though there is no newline character at the end of the prompt. How can this happen?
That most likely happens because your C standard library chooses to flush the output buffers in its implementation of scanf. That's a good idea because
this is a common problem and you usually want the prompt at the same line as the cursor and
it's not a performance problem because waiting for user input is way slower than flushing the buffer anyways
You can run an experiment:
printf("Enter a number: "); /* the prompt */
sleep(10); /* Add this. */
scanf("%lf", &number);
You'll see the prompt popping up after 10 seconds. It's not documented - that I know of - but it appears in this case scanf causes stdout to be flushed.
You shouldn't count on this behavior however and you should fflush(stdout) when printing a prompt string.
As noted in a comment, the C standard (ISO/IEC 9899:2011) says:
§5.1.2.3 Program execution
¶6 The least requirements on a conforming implementation are:
Accesses to volatile objects are evaluated strictly according to the rules of the abstract
machine.
At program termination, all data written into files shall be identical to the result that
execution of the program according to the abstract semantics would have produced.
The input and output dynamics of interactive devices shall take place as specified in
7.21.3. The intent of these requirements is that unbuffered or line-buffered output
appear as soon as possible, to ensure that prompting messages actually appear prior to
a program waiting for input.
This is the observable behavior of the program.
¶7 What constitutes an interactive device is implementation-defined.
¶8 More stringent correspondences between abstract and actual semantics may be defined by
each implementation.
And the library section describing the functions in <stdio.h> says:
§7.21.3 Files
¶3 When a stream is unbuffered, characters are intended to appear from the source or at the
destination as soon as possible. Otherwise characters may be accumulated and
transmitted to or from the host environment as a block. When a stream is fully buffered,
characters are intended to be transmitted to or from the host environment as a block when
a buffer is filled. When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and setvbuf functions.
¶7 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.
Note the 'intent' comment in §5.1.2.3 ¶6.
The specification in §7.21.3 ¶7 means that standard error is either unbuffered or line buffered at program startup, and standard input and standard output are either unbuffered or line buffered (usually line buffered) unless the output is not an interactive device. On Unix, disk files, pipes, FIFOs, sockets (amongst others) are not interactive. Usually tty and pty devices are deemed interactive.
So, if you run your program with output going to a pipe (e.g. to cat or less or more), you probably* won't see a prompt, but if you type the number, you'll probably get the prompt followed by the first line of output with a newline all on a single line (and the number you typed won't appear). If you redirect the input from a file, you won't see the entered value at all. If you redirect to cat, you'll likely see your input on one line, then the prompt and the output. If you redirect to more, you may not see your input at all (separately from the response printing). If you redirect to less, you may get all sorts of interesting screen effects, and still not see the input you type.
You also need to fix the revised code. The %lf format is not appropriate for reading an integer (it is a hangover from a previous version of the code).
* Since the behaviour is implementation defined, what actually happens depends on your implementation. An implementation might do the equivalent of fflush(0) before any input, for example; this would send the prompt to the device. But no implementation is required to do that, and most aren't that thorough (partly because such thoroughness has a runtime cost associated with it).