Flushing buffers in C

Flushing buffers in C - c

Should fflush() not be used to flush a buffer even if it is an output stream?
What is it useful for? How do we flush a buffer in general?

Flushing the output buffers:
printf("Buffered, will be flushed");
fflush(stdout); // Prints to screen or whatever your standard out is
or
fprintf(fd, "Buffered, will be flushed");
fflush(fd); //Prints to a file
Can be a very helpful technique. Why would you want to flush an output buffer? Usually when I do it, it's because the code is crashing and I'm trying to debug something. The standard buffer will not print everytime you call printf() it waits until it's full then dumps a bunch at once. So if you're trying to check if you're making it to a function call before a crash, it's helpful to printf something like "got here!", and sometimes the buffer hasn't been flushed before the crash happens and you can't tell how far you've really gotten.
Another time that it's helpful, is in multi-process or multi-thread code. Again, the buffer doesn't always flush on a call to a printf(), so if you want to know the true order of execution of multiple processes you should fflush the buffer after every print.
I make a habit to do it, it saves me a lot of headache in debugging. The only downside I can think of to doing so is that printf() is an expensive operation (which is why it doesn't by default flush the buffer).
As far as flushing the input buffer (stdin), you should not do that. Flushing stdin is undefined behavior according to the C11 standard §7.21.5.2 part 2:
If stream points to an output stream ... the fflush function causes any unwritten data for that stream ... to be written to the file; otherwise, the behavior is undefined.
On some systems, Linux being one as you can see in the man page for fflush(), there's a defined behavior but it's system dependent so your code will not be portable.
Now if you're worried about garbage "stuck" in the input buffer you can use fpurge() on that.
See here for more on fflush() and fpurge()

Related

Why does my C code skip over the printf(); function? [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?

The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);

No, it's not POSIX behaviour, it's ISO behaviour (well, it is POSIX behaviour but only insofar as they conform to ISO).
Standard output is line buffered if it can be detected to refer to an interactive device, otherwise it's fully buffered. So there are situations where printf won't flush, even if it gets a newline to send out, such as:
myprog >myfile.txt
This makes sense for efficiency since, if you're interacting with a user, they probably want to see every line. If you're sending the output to a file, it's most likely that there's not a user at the other end (though not impossible, they could be tailing the file). Now you could argue that the user wants to see every character but there are two problems with that.
The first is that it's not very efficient. The second is that the original ANSI C mandate was to primarily codify existing behaviour, rather than invent new behaviour, and those design decisions were made long before ANSI started the process. Even ISO nowadays treads very carefully when changing existing rules in the standards.
As to how to deal with that, if you fflush (stdout) after every output call that you want to see immediately, that will solve the problem.
Alternatively, you can use setvbuf before operating on stdout, to set it to unbuffered and you won't have to worry about adding all those fflush lines to your code:
setvbuf (stdout, NULL, _IONBF, BUFSIZ);
Just keep in mind that may affect performance quite a bit if you are sending the output to a file. Also keep in mind that support for this is implementation-defined, not guaranteed by the standard.
ISO C99 section 7.19.3/3 is the relevant bit:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block.
When a stream is fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled.
When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered.
Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment.
Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.

It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.

To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).

stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.

Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf

by default, stdout is line buffered, stderr is none buffered and file is completely buffered.

You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.

Use setbuf(stdout, NULL); to disable buffering.

There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

Bell in C '\a' written twice rings only once why? [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?

The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);

It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.

To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).

stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.

Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf

by default, stdout is line buffered, stderr is none buffered and file is completely buffered.

You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.

Use setbuf(stdout, NULL); to disable buffering.

There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

Is it just best to flush stdout/stderr every single time?

From this stack overflow post: Is stdout line buffered, unbuffered or indeterminate by default?
From that post, it states that "The C99 standard does not specify if the three standard streams are unbuffered or line buffered: It is up to the implementation."
#include <stdio.h>
int main(void) {
printf("Enter name: ");
fflush(stdout);
printf("Another name: ");
fflush(stdout);
return 0;
}
So, does that mean every single time we print, and we want to make sure it is visible to the user, that it's the safest bet to just flush it every single time? Since buffering is up to the implementation?
#include <stdio.h>
int main(void) {
puts("Enter name:");
fflush(stdout);
puts("Another name: ");
fflush(stdout);
return 0;
}
Even on a newline, is it still best to just flush every single time?
I want to 100% make sure the user sees the output, therefore is it best to flush every single time? Even on stderr? Since buffering is up to the implementation?

Looking at the exact examples you show, it's unlikely there will be a visible difference from calling fflush. In particular, you just write two lines of data, and then exit. Normal exit from a program (either by calling exit or returning from main) requires:
Next, all open streams with unwritten buffered data are flushed, all open streams are closed, and all files created by the tmpfile function are removed.
So, the only possibility here would be if the system crashed (or something on that order) immediately after the first fflush call, but before your program exited.
I'd guess, however, that the real intent was when you print out the "Enter name: " prompt, you intended for the program to stop and read a name from the user:
printf("Enter name: ");
char name[256];
fgets(name, sizeof(name), stdin);
For cases like this, flushing the output stream is generally unnecessary.
In particular, the standard says:
When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment.
Now, you can make a little bit of an argument that this only talks about what's intended, not what's required. What that basically comes down to is fairly simple: the operating system could do some buffering over which your program has little or no control. So, what it basically comes down to is pretty simple: when you read from stdin, if anything has been written to stdout, but not flushed yet, it'll be flushed automatically before the system waits for input from stdin (unless you've done something like redirecting one or both to a file).
I'm reasonably certain that essentially every argument for doing that flushing explicitly is purely theoretical. In theory, there could be a system that doesn't do it automatically, and that might not quite violate any strict requirement of the standard.
In reality, however, actual systems fall into two categories: those on which the flushing is automatic, so you gain nothing by doing it explicitly, and those on which flushing simply won't do any good, so you gain nothing by doing it explicitly.
Just for an example of the latter, some old terminals for IBM mainframes worked in what you might call a screen-buffered mode. Basically, you'd send a form to the terminal. The user would then edit data into the form. Then when they had filled in the entire form, they pressed a "send" button, and all of the data for the whole form was sent to the CPU at once. On a system like this, calling fflush won't be (nearly) enough to make code work.
Summary
There are systems where calling fflush is unnecessary.
There are systems where calling fflush is insufficient.
I'm reasonably certain there is no system where calling fflush is both necessary and sufficient.

I want to 100% make sure the user sees the output, therefore is it best to flush every single time?
For "100% make sure" best practice is not based on flushing after output, but flushing stdout before input. This is especially true when the output does not end with a '\n'.
It is best to fflush(stdout); when
Right before input from stdin and ...
There has been some output since the prior input from stdin.
Calling fflush(stdout); more often than that does not impact functionality, but may unnecessarily slow code.
There is a 2nd case for fflush(stdout); even without input: Debugging. Sometimes a fatal error occurs with info buffered in stdout. So a fflush(stdout); is warranted prior to the suspected bad code for better analysis.
Even on a newline, is it still best to just flush every single time?
For "100% make sure", the rules for flushing stdout are soft enough to even oblige fflush(stdout); before reading in this case too.
Yet stdout should be unbuffered or line buffered on start-up (see next) and usually not require a flush in this case.
Even on stderr?
C spec has "As initially opened, the standard error stream is not fully
buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device."
As long as the write to stderr ends with a '\n' and buffering mode not changed, then the output should be seen.
Interesting to see how C++ handles end-of-line.
What is the difference between endl and \n in C++?
#paxdiablo nicely answers Why does printf not flush after the call unless a newline is in the format string?

Not really, no. I flush at logical chunks so I can tell more or less where the program crashed and don't have an extra hard debugging time because it crashed with output in the buffer.
The idea is I'm going to call fflush() just before going and doing something else besides produce more output.
Flush too much = slow code.
Calling fflush() right before reading input is harmless (the only question is whether it will do it implicitly or not), so if you're doing that for some reason there's no reason to stop doing it.

How come printf() only executes after a loop that delays execution on WSL2(Ubuntu)? [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?

The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);

It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.

To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).

stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.

Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf

by default, stdout is line buffered, stderr is none buffered and file is completely buffered.

You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.

Use setbuf(stdout, NULL); to disable buffering.

There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

Loop in child process doesn't work without printf (pipes) [duplicate]

Why does printf not flush after the call unless a newline is in the format string? Is this POSIX behavior? How might I have printf immediately flush every time?

The stdout stream is line buffered by default, so will only display what's in the buffer after it reaches a newline (or when it's told to). You have a few options to print immediately:
Print to stderrinstead using fprintf (stderr is unbuffered by default):
fprintf(stderr, "I will be printed immediately");
Flush stdout whenever you need it to using fflush:
printf("Buffered, will be flushed");
fflush(stdout); // Will now print everything in the stdout buffer
Disable buffering on stdout by using setbuf:
setbuf(stdout, NULL);
Or use the more flexible setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);

It's probably like that because of efficiency and because if you have multiple programs writing to a single TTY, this way you don't get characters on a line interlaced. So if program A and B are outputting, you'll usually get:
program A output
program B output
program B output
program A output
program B output
This stinks, but it's better than
proprogrgraam m AB ououtputputt
prproogrgram amB A ououtputtput
program B output
Note that it isn't even guaranteed to flush on a newline, so you should flush explicitly if flushing matters to you.

To immediately flush call fflush(stdout) or fflush(NULL) (NULL means flush everything).

stdout is buffered, so will only output after a newline is printed.
To get immediate output, either:
Print to stderr.
Make stdout unbuffered.

Note: Microsoft runtime libraries do not support line buffering, so printf("will print immediately to terminal"):
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf

by default, stdout is line buffered, stderr is none buffered and file is completely buffered.

You can fprintf to stderr, which is unbuffered, instead. Or you can flush stdout when you want to. Or you can set stdout to unbuffered.

Use setbuf(stdout, NULL); to disable buffering.

There are generally 2 levels of buffering-
1. Kernel buffer Cache (makes read/write faster)
2. Buffering in I/O library (reduces no. of system calls)
Let's take example of fprintf and write().
When you call fprintf(), it doesn't wirte directly to the file. It first goes to stdio buffer in the program's memory. From there it is written to the kernel buffer cache by using write system call. So one way to skip I/O buffer is directly using write(). Other ways are by using setbuff(stream,NULL). This sets the buffering mode to no buffering and data is directly written to kernel buffer.
To forcefully make the data to be shifted to kernel buffer, we can use "\n", which in case of default buffering mode of 'line buffering', will flush I/O buffer.
Or we can use fflush(FILE *stream).
Now we are in kernel buffer. Kernel(/OS) wants to minimise disk access time and hence it reads/writes only blocks of disk. So when a read() is issued, which is a system call and can be invoked directly or through fscanf(), kernel reads the disk block from disk and stores it in a buffer. After that data is copied from here to user space.
Similarly that fprintf() data recieved from I/O buffer is written to the disk by the kernel. This makes read() write() faster.
Now to force the kernel to initiate a write(), after which data transfer is controlled by hardware controllers, there are also some ways. We can use O_SYNC or similar flags during write calls. Or we could use other functions like fsync(),fdatasync(),sync() to make the kernel initiate writes as soon as data is available in the kernel buffer.

Develop Reference

c reactjs sql-server angularjs arrays wpf database batch-file google-app-engine silverlight