Is it just best to flush stdout/stderr every single time? - c

From this stack overflow post: Is stdout line buffered, unbuffered or indeterminate by default?
From that post, it states that "The C99 standard does not specify if the three standard streams are unbuffered or line buffered: It is up to the implementation."
#include <stdio.h>
int main(void) {
printf("Enter name: ");
fflush(stdout);
printf("Another name: ");
fflush(stdout);
return 0;
}
So, does that mean every single time we print, and we want to make sure it is visible to the user, that it's the safest bet to just flush it every single time? Since buffering is up to the implementation?
#include <stdio.h>
int main(void) {
puts("Enter name:");
fflush(stdout);
puts("Another name: ");
fflush(stdout);
return 0;
}
Even on a newline, is it still best to just flush every single time?
I want to 100% make sure the user sees the output, therefore is it best to flush every single time? Even on stderr? Since buffering is up to the implementation?

Looking at the exact examples you show, it's unlikely there will be a visible difference from calling fflush. In particular, you just write two lines of data, and then exit. Normal exit from a program (either by calling exit or returning from main) requires:
Next, all open streams with unwritten buffered data are flushed, all open streams are closed, and all files created by the tmpfile function are removed.
So, the only possibility here would be if the system crashed (or something on that order) immediately after the first fflush call, but before your program exited.
I'd guess, however, that the real intent was when you print out the "Enter name: " prompt, you intended for the program to stop and read a name from the user:
printf("Enter name: ");
char name[256];
fgets(name, sizeof(name), stdin);
For cases like this, flushing the output stream is generally unnecessary.
In particular, the standard says:
When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment.
Now, you can make a little bit of an argument that this only talks about what's intended, not what's required. What that basically comes down to is fairly simple: the operating system could do some buffering over which your program has little or no control. So, what it basically comes down to is pretty simple: when you read from stdin, if anything has been written to stdout, but not flushed yet, it'll be flushed automatically before the system waits for input from stdin (unless you've done something like redirecting one or both to a file).
I'm reasonably certain that essentially every argument for doing that flushing explicitly is purely theoretical. In theory, there could be a system that doesn't do it automatically, and that might not quite violate any strict requirement of the standard.
In reality, however, actual systems fall into two categories: those on which the flushing is automatic, so you gain nothing by doing it explicitly, and those on which flushing simply won't do any good, so you gain nothing by doing it explicitly.
Just for an example of the latter, some old terminals for IBM mainframes worked in what you might call a screen-buffered mode. Basically, you'd send a form to the terminal. The user would then edit data into the form. Then when they had filled in the entire form, they pressed a "send" button, and all of the data for the whole form was sent to the CPU at once. On a system like this, calling fflush won't be (nearly) enough to make code work.
Summary
There are systems where calling fflush is unnecessary.
There are systems where calling fflush is insufficient.
I'm reasonably certain there is no system where calling fflush is both necessary and sufficient.

I want to 100% make sure the user sees the output, therefore is it best to flush every single time?
For "100% make sure" best practice is not based on flushing after output, but flushing stdout before input. This is especially true when the output does not end with a '\n'.
It is best to fflush(stdout); when
Right before input from stdin and ...
There has been some output since the prior input from stdin.
Calling fflush(stdout); more often than that does not impact functionality, but may unnecessarily slow code.
There is a 2nd case for fflush(stdout); even without input: Debugging. Sometimes a fatal error occurs with info buffered in stdout. So a fflush(stdout); is warranted prior to the suspected bad code for better analysis.
Even on a newline, is it still best to just flush every single time?
For "100% make sure", the rules for flushing stdout are soft enough to even oblige fflush(stdout); before reading in this case too.
Yet stdout should be unbuffered or line buffered on start-up (see next) and usually not require a flush in this case.
Even on stderr?
C spec has "As initially opened, the standard error stream is not fully
buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device."
As long as the write to stderr ends with a '\n' and buffering mode not changed, then the output should be seen.
Interesting to see how C++ handles end-of-line.
What is the difference between endl and \n in C++?
#paxdiablo nicely answers Why does printf not flush after the call unless a newline is in the format string?

Not really, no. I flush at logical chunks so I can tell more or less where the program crashed and don't have an extra hard debugging time because it crashed with output in the buffer.
The idea is I'm going to call fflush() just before going and doing something else besides produce more output.
Flush too much = slow code.
Calling fflush() right before reading input is harmless (the only question is whether it will do it implicitly or not), so if you're doing that for some reason there's no reason to stop doing it.

Related

Trying to create a typing effect in C [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?
The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);
No, it's not POSIX behaviour, it's ISO behaviour (well, it is POSIX behaviour but only insofar as they conform to ISO).
Standard output is line buffered if it can be detected to refer to an interactive device, otherwise it's fully buffered. So there are situations where printf won't flush, even if it gets a newline to send out, such as:
myprog >myfile.txt
This makes sense for efficiency since, if you're interacting with a user, they probably want to see every line. If you're sending the output to a file, it's most likely that there's not a user at the other end (though not impossible, they could be tailing the file). Now you could argue that the user wants to see every character but there are two problems with that.
The first is that it's not very efficient. The second is that the original ANSI C mandate was to primarily codify existing behaviour, rather than invent new behaviour, and those design decisions were made long before ANSI started the process. Even ISO nowadays treads very carefully when changing existing rules in the standards.
As to how to deal with that, if you fflush (stdout) after every output call that you want to see immediately, that will solve the problem.
Alternatively, you can use setvbuf before operating on stdout, to set it to unbuffered and you won't have to worry about adding all those fflush lines to your code:
setvbuf (stdout, NULL, _IONBF, BUFSIZ);
Just keep in mind that may affect performance quite a bit if you are sending the output to a file. Also keep in mind that support for this is implementation-defined, not guaranteed by the standard.
ISO C99 section 7.19.3/3 is the relevant bit:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block.
When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered.
Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment.
Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.
To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).
stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.
Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf
by default, stdout is line buffered, stderr is none buffered and file is completely buffered.
You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.
Use setbuf(stdout, NULL); to disable buffering.
There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

How come printf() only executes after a loop that delays execution on WSL2(Ubuntu)? [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?
The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);
No, it's not POSIX behaviour, it's ISO behaviour (well, it is POSIX behaviour but only insofar as they conform to ISO).
Standard output is line buffered if it can be detected to refer to an interactive device, otherwise it's fully buffered. So there are situations where printf won't flush, even if it gets a newline to send out, such as:
myprog >myfile.txt
This makes sense for efficiency since, if you're interacting with a user, they probably want to see every line. If you're sending the output to a file, it's most likely that there's not a user at the other end (though not impossible, they could be tailing the file). Now you could argue that the user wants to see every character but there are two problems with that.
The first is that it's not very efficient. The second is that the original ANSI C mandate was to primarily codify existing behaviour, rather than invent new behaviour, and those design decisions were made long before ANSI started the process. Even ISO nowadays treads very carefully when changing existing rules in the standards.
As to how to deal with that, if you fflush (stdout) after every output call that you want to see immediately, that will solve the problem.
Alternatively, you can use setvbuf before operating on stdout, to set it to unbuffered and you won't have to worry about adding all those fflush lines to your code:
setvbuf (stdout, NULL, _IONBF, BUFSIZ);
Just keep in mind that may affect performance quite a bit if you are sending the output to a file. Also keep in mind that support for this is implementation-defined, not guaranteed by the standard.
ISO C99 section 7.19.3/3 is the relevant bit:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block.
When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered.
Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment.
Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.
To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).
stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.
Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf
by default, stdout is line buffered, stderr is none buffered and file is completely buffered.
You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.
Use setbuf(stdout, NULL); to disable buffering.
There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

Going back to normal terminal I/O after closing ncurses window

In a game I made I create an ncurses window using initscr(), then the user plays the game and when the game ends the program closes the window using endwin() and then prints some statements and gets input from the user using printf and scanf, respectively. My problem is that whenever the game ends scanf always executes before printf. So it waits for a response, and once the user enters the response it prints everything that was before and after the scanf statement. Here is some example code:
...
endwin();
system("clear"); /* Clears terminal window */
printf("New high score!\nPlease enter your first name: ");
scanf("%s",name);
... /* file I/O stuff */
printf("Congratulations, %s!",name);
As you can see, the printf statement is before the scanf statement but for some reason scanf executes first. I tested the code without an ncurses window and I get the desired result. Does anyone know what is causing this?
Nothing to do with curses.
This statement is obviously wrong since using curses makes a difference, as the question states:
As you can see, the printf statement is before the scanf statement but
for some reason scanf executes first. I tested the code without an
ncurses window and I get the desired result. Does anyone know what is
causing this?
While it is right that fflush(stdout) gives the desired result, that does not answer the question.
What is causing this is presumably the implementation-defined standard I/O characteristic to flush a line buffered stdout when input is requested (sometimes documented in man setbuf), and curses setting stdout to fully buffered, thereby disabling the automatic flush.
You can read about it in the manual page where it discusses NCURSES_NO_SETBUF:
to get good performance, most curses implementations change the output buffering from line-oriented to buffer-oriented (so that the library can make larger, more efficient writes)
because that meant that all writes to the standard output would be buffered, it was a nuisance. ncurses provided an option (the environment variable) to override the buffering.
that changed in 2012, by eliminating the stdout buffering altogether to solve a problem with signal handling.
however, "eliminating" the buffering means that ncurses uses its own buffer. It'll flush that in endwin, but doesn't flush stdout when switching to/from curses mode.
a few applications required modification to flush stdout.
The 2012 change for ncurses was part of the ncurses6 release in August 2015. The OP's question is in terms of ncurses5.9 (or earlier), since it describes an application where the standard output is buffered.

When can i expect "stdout" printed to console

I know this is because of buffering of stdout but when can i expect output of stdout in the following proggram. If i run, I am always getting "stderr" as output. If i add '\n' or fflush(stdout) then only i am getting both statements. If i don't add '\n' or fflush(stdout),i am not getting "stdout" as output. when will i get all buffered "stdout"s as output if i don't add '\n' or fflush(stdout).
#include <stdio.h>
#include <unistd.h>
int main()
{
for(;;)
{
fprintf(stdout,"stdout");
fprintf(stderr,"stderr");
sleep(1);
}
return 0;
}
Indeed, a new line or a flush because by default stdout is line buffered when it refers a terminal device.
To be more precise: standard input and standard output are fully buffered, if and only if they do not refer to an interactive device. Standard error is never fully buffered.
About line buffering, quoting from APUE:
Line buffering comes with two caveats. First, the size of the buffer that the standard I/O library is using to collect each line is fixed, so I/O might take place if we fill this buffer before writing a newline. Second, whenever input is requested through the standard I/O library from either (a) an unbuffered stream or (b) a line-buffered stream (that requiresdata to be requested from the kernel), all line-buffered output streams are flushed. The reason for the qualifier on (b) is that the requested data may alreadybe
in the buffer, which doesn't require data to be read from the kernel. Obviously, any input from an unbuffered stream, item (a), requires data to be obtained from the kernel.
To change it to unbuffered, use setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);
output sent to stderr usually prints immediately. It is the standard error output. Errors, of course, should be displayed immediately, and so any buffer that exists is flushed immediately.
stdout, on the other hand, is buffered. Getting that buffer to print out varies by language, but usually at least one of two things are required: /n, or some other type of newline character, or .flush();. If you do not provide these options, then usually you have to wait for the buffer to fill. This can take any amount of time, as it pretty much entirely depends on how large the buffer is.
By convention, the way streams are handled is that you shove as much information into them as you want, and then call .flush(). If the buffer happens to fill up in the meantime, then the data is sent to wherever it's going and the buffer starts filling up again.
Here, though, you're filling up a buffer that takes in text. Text does not really take up a whole lot of space compared to binary information (this is a VERY subjective statement and should be interpreted loosely), and so filling up the buffer could take quite a while.
The best practice is for you to manually flush the stream (stdout) once you have "fed" it all of the text you want printed.
fprintf is by default line buffered. You can alter the behavior by calling setvbuf. It allows you to set it to 'unbuffered', 'line buffered', or 'fully buffered'.

Flushing buffers in C

Should fflush() not be used to flush a buffer even if it is an output stream?
What is it useful for? How do we flush a buffer in general?
Flushing the output buffers:
printf("Buffered, will be flushed");
fflush(stdout); // Prints to screen or whatever your standard out is
or
fprintf(fd, "Buffered, will be flushed");
fflush(fd); //Prints to a file
Can be a very helpful technique. Why would you want to flush an output buffer? Usually when I do it, it's because the code is crashing and I'm trying to debug something. The standard buffer will not print everytime you call printf() it waits until it's full then dumps a bunch at once. So if you're trying to check if you're making it to a function call before a crash, it's helpful to printf something like "got here!", and sometimes the buffer hasn't been flushed before the crash happens and you can't tell how far you've really gotten.
Another time that it's helpful, is in multi-process or multi-thread code. Again, the buffer doesn't always flush on a call to a printf(), so if you want to know the true order of execution of multiple processes you should fflush the buffer after every print.
I make a habit to do it, it saves me a lot of headache in debugging. The only downside I can think of to doing so is that printf() is an expensive operation (which is why it doesn't by default flush the buffer).
As far as flushing the input buffer (stdin), you should not do that. Flushing stdin is undefined behavior according to the C11 standard ยง7.21.5.2 part 2:
If stream points to an output stream ... the fflush function causes any unwritten data for that stream ... to be written to the file; otherwise, the behavior is undefined.
On some systems, Linux being one as you can see in the man page for fflush(), there's a defined behavior but it's system dependent so your code will not be portable.
Now if you're worried about garbage "stuck" in the input buffer you can use fpurge() on that.
See here for more on fflush() and fpurge()

Resources