I am writing a C program where I am printing to stderr and also using putchar() within the code. I want the output on the console to show all of the stderr and then finally flush the stdout before the program ends. Does anyone know of a method that will stop stdout from flushing when a putchar('\n') occurs?
I suppose i could just do an if statement to make sure it doesn't putchar any newlines but I would prefer some line or lines of code to put at the top of the program to stop all flushing until i say fflush(stdout) at the bottom of the program
What you're trying to do is horribly fragile. C provides no obligation for an implementation of stdio not to flush output, under any circumstances. Even if you get it to work for you, this behavior will be dependent on not exceeding the buffer size. If you really need this behavior, you should probably buffer the output yourself (possibly writing it to a tmpfile() rather than stdout) then copying it all to stdout as the final step before your program exits.
Run your command from the console using pipeling:
my_command >output.txt
All output witten to stderr will appear immediately. The stuff written to stdout will go to output.txt.
Windows only. I'm still looking for the Unix solution myself if anyone has it!
Here is a minimal working example for Windows that sends a buffer to stdout without flushing. You can adjust the maximum buffer size before a flush occurs by changing max_buffer, though I imagine there's some upper limit!
#include <windows.h>
#include <string.h>
int main()
{
const char* my_buffer = "hello, world!";
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
int max_buffer = 1000000;
int num_remaining = strlen(my_buffer);
while (num_remaining)
{
DWORD num_written = 0;
int buffer_size = num_remaining < max_buffer ? num_remaining : max_buffer;
int retval = WriteConsoleA(hStdout, my_buffer, buffer_size, &num_written, 0);
if (retval == 0 || num_written == 0)
{
// Handle error
}
num_remaining -= num_written;
if (num_remaining == 0)
{
break;
}
my_buffer += num_written;
}
}
You can use setvbuf() to fully buffer output to stdout and provide a large enough buffer size for your purpose:
#include <stdio.h>
int main() {
// issue this call before any output
setvbuf(stdout, NULL, _IOFBF, 16384);
...
return 0;
}
Output to stderr is unbuffered by default, so it should go to the console immediately.
Output to stdout is line buffered by default when attached to the terminal. Setting it to _IOFBF (fully buffered) should prevent putchar('\n') from flushing the pending output.
Related
I'm following a tutorial for making a text editor .
So far it's been tinkering with raw mode . The following code is supposed to turn off canonical mode , and output each keypress.
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <termios.h>
#include <unistd.h>
struct termios orig_termios;
void disableRawMode() { … }
void enableRawMode() { … }
int main() {
enableRawMode();
char c;
while (read(STDIN_FILENO, &c, 1) == 1 && c != 'q') {
if (iscntrl(c)) {
printf("%d\n", c);
} else {
printf("%d ('%c')\n", c, c);
}
}
return 0;
}
I originally forgot to add "\n" after the printf() statements, and the result was that I only got the outputted characters after the program terminates , ie after pressing q in this example .
However after adding "\n", the terminal outputs each letter as pressed.
Could anyone be so kind and explain why is it behaving this way?
Raw-mode is the concern of the terminal but buffer management of stdout occurs before reaching the terminal.
By default, when file-descriptor 1 (STDOUT_FILENO) is link to a terminal, then stdout uses a line-buffering policy.
This means that the output buffer of stdout is flushed to the file-descriptor 1 when a \n is written (or when it is full).
Only at this moment the characters can reach the terminal which can react in different ways depending on its configuration.
In your example, the characters just stay in memory until the process terminates (stdout is flushed at this moment).
Commonly, when a C program starts with the standard output stream connected to a terminal, the stream is line buffered. This means characters printed with printf or standard library methods are kept in a buffer until \n is printed (ending the line, hence “line buffered”), the buffer is full, when the stream is manually flushed (as with fflush), or when input is solicited on stream that is unbuffered or that is line buffered but requires characters from “the host environment” (notably a human).
The terminal setting is irrelevant as the characters are kept in an internal buffer of the standard C library implementation and are not sent to the terminal until one of the above events.
You can set the stream to unbuffered by calling setvbuf(stdout, NULL, _IONBF, 0) before performing any other operation on stdout.
I have a sample program that takes in an input from the terminal and executes it in a cloned child in a subshell.
#define _GNU_SOURCE
#include <stdlib.h>
#include <sys/wait.h>
#include <sched.h>
#include <unistd.h>
#include <string.h>
#include <signal.h>
int clone_function(void *arg) {
execl("/bin/sh", "sh", "-c", (char *)arg, (char *)NULL);
}
int main() {
while (1) {
char data[512] = {'\0'};
int n = read(0, data, sizeof(data));
// fgets(data, 512, stdin);
// int n = strlen(data);
if ((strcmp(data, "exit\n") != 0) && n > 1) {
char *line;
char *lines = strdup(data);
while ((line = strsep(&lines, "\n")) != NULL && strcmp(line, "") != 0) {
void *clone_process_stack = malloc(8192);
void *stack_top = clone_process_stack + 8192;
int clone_flags = CLONE_VFORK | CLONE_FS;
clone(clone_function, stack_top, clone_flags | SIGCHLD, (void *)line);
int status;
wait(&status);
free(clone_process_stack);
}
} else {
exit(0);
}
}
return 0;
}
The above code works in an older Linux system (with minimal RAM( but not in a newer one. Not works means that if I type a simple command like "ls" I don't see the output on the console. But with the older system I see it.
Also, if I run the same code on gdb in debugger mode then I see the output printed onto the console in the newer system as well.
In addition, if I use fgets() instead of read() it works as expected in both systems without an issue.
I have been trying to understand the behavior and I couldn't figure it out. I tried doing an strace. The difference I see is that the wait() return has the output of the ls program in the cases it works and nothing for the cases it does not work.
Only thing I can think of is that read(), since its not a library function has undefined behavior across systems. But I can't agree as to how its affecting the output.
Can someone point me out to why I might be observing this behavior?
EDIT
The code is compiled as:
gcc test.c -o test
strace when it's not working as expected is shown below
strace when it's working as expected (only difference is I added a printf("%d\n", n); following the call for read())
Thank you
Shabir
There are multiple problems in your code:
a successful read system call can return any non zero number between 1 and the buffer size depending on the type of handle and available input. It does not stop at newlines like fgets(), so you might get line fragments, multiple lines, or multiple lines and a line fragment.
furthermore, if read fills the whole buffer, as it might when reading from a regular file, there is no trailing null terminator, so passing the buffer to string functions has undefined behavior.
the test if ((strcmp(data, "exit\n") != 0) && n > 1) { is performed in the wrong order: first test if read was successful, and only then test the buffer contents.
you do not set the null terminator after the last byte read by read, relying on buffer initialization, which is wasteful and insufficient if read fills the whole buffer. Instead you should make data one byte longer then the read size argument, and set data[n] = '\0'; if n > 0.
Here are ways to fix the code:
using fgets(), you can remove the line splitting code: just remove initial and trailing white space, ignore empty and comment lines, clone and execute the commands.
using read(), you could just read one byte at a time, collect these into the buffer until you have a complete line, null terminate the buffer and use the same rudimentary parser as above. This approach mimics fgets(), by-passing the buffering performed by the standard streams: it is quite inefficient but avoids reading from handle 0 past the end of the line, thus leaving pending input available for the child process to read.
It looks like 8192 is simply too small a value for stack size on a modern system. execl needs more than that, so you are hitting a stack overflow. Increase the value to 32768 or so and everything should start working again.
#include<stdio.h>
#include <unistd.h>
int main(){
while(1)
{
fprintf(stdout,"hello-out");
fprintf(stderr,"hello-err");
sleep(1);
}
return 0;
}
On compiling this programme in gcc and on executing it only prints hello-err and not hello-out.Why is that so?Can someone please explain the reason behind it?
If you add a '\n' to your message it will (or should), ie. "hello-out\n".
The reason being is that stdout is buffered in order to be more efficient, whereas stderr doesn't buffer it's output and is more appropriate for error messages and things that need to be printed immediately.
stdout will usually be flushed when:
A newline (\n) is to be printed
You read in from stdin
fflush() is called on it
EDIT: The other thing I wanted to add before my computer crashed...twice...was that you can also use setbuf(stdout, NULL); to disable buffering of stdout. I've done that before when I've had to use write() (Unix) and didn't want my output to be buffered.
It doesn't always print out the output to stdout because by design stdout is BUFFERED output, and stderr is unbuffered. In general, the for a buffered output stream, the stream is not dumped until the system is "free" to do so. So data can continue buffering for a long while, before it gets flushed. If you want to force the buffer to flush you can do so by hand using fflush
#include<stdio.h>
#include <unistd.h>
int main(){
while(1)
{
fprintf(stdout,"hello-out");
fflush(stdout); /* force flush */
fprintf(stderr,"hello-err");
sleep(1);
}
return 0;
}
Update: stdout is linebuffered when connected to a terminal, and simply buffered otherwise (e.g. a redirect or a pipe)
You forgot newlines (noted \n) in your strings. Or you need to call fflush(NULL); or at least fflush(stdout); before sleep(1);
And fprintf(stdout, ...) is the same as printf(...)
You need to output newlines or to call fflush because (at least on Linux) the stdout FILE buffer is line-buffered. This means that the C library is buffering data, and will really output it (using the write Linux system call) when the buffer is full enough, or when you flush it either with a new line, or by calling fflush. Buffering is needed because system calls are costly (calling write for every byte to be output is really too slow). Read also the man page of setbuf
I am writing a C program on unix which should redirect it's output to the file, and write to it some text every second in infinite loop:
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main(void) {
int outDes = open("./output.txt", O_APPEND | O_WRONLY);
dup2(outDes, 1);
while(1) {
printf("output text\n");
sleep(1);
}
}
But it writes nothing to the output file. I tried to change the 'while' loop for 'for' with 10 loops, and I found that it writes all 10 lines to the file at once after the series ends. It is not very good for me, while I need to have an infinite loop.
When I'm not redirecting output, it is all ok, and new line appears every second on terminal.
I also tried to put one
printf("text\n");
before redirecting output to the file. Then the program wrote the lines to the file in real time, which is good, but wrote there the first (non redirected) line too. I don't want this first line in my output file, I don't understand how it could be written into file when output was not redirected yet (maybe redirect remained there since last run?), and how it could cause that the lines are suddenly written in real time.
Can anyone explain me how does it work?
You are not checking the return value of open() and dup2(). If either open() or dup2() failed, it won't write anything in output.txt.
if (outDes < -1) {
perror("open");
return 1;
}
if (dup2(outDes, 1) == -1) {
perror("dup2");
return 1;
}
stdio streams are buffered, and the writes happen in memory before being done on the real file description.
Try adding a fflush(stdout) after printf().
You're running afoul of a poorly documented DWIMmy feature in many Unix C libraries. The first time you write to stdout or stderr, the library probes the underlying file descriptor (with isatty(3)). If it's a (pseudo-)terminal, the library puts the FILE in "line buffered" mode, meaning that it'll buffer input until a newline is written and then flush it all to the OS. But if the file descriptor is not a terminal, it puts the FILE in "fully buffered" mode, where it'll buffer something like BUFSIZ bytes of output before flushing them, and pays no attention to line breaks.
This is normally the behavior you want, but if you don't want it (as in this case), you can change it with setvbuf(3). This function (although not the behavior I described above) is ISO standard C. Here's how to use it in your case.
#include <stdio.h>
#include <unistd.h>
int
main(void)
{
if (freopen("output.txt", "a", stdout)) {
perror("freopen");
return 1;
}
if (setvbuf(stdout, 0, _IOLBF, 0)) {
perror("setvbuf");
return 1;
}
for (;;) {
puts("output text");
sleep(1);
}
/* not reached */
}
I was just getting familiar with sleep(), i found that
#include<stdio.h>
int main()
{
int i=0;
printf("*********Testing Sleep***********\n");
for(i=0;i<10;i++)
{
printf("%d",i);
sleep(1);
}
return 0;
}
this does not print number per iteration rather dumps all numbers when gets out of loop....
but when i modify printf...
#include<stdio.h>
int main()
{
int i=0;
printf("*********Testing Sleep***********\n");
for(i=0;i<10;i++)
{
printf("%d\n",i);
sleep(1);
}
return 0;
}
and now as i've added '\n' new line it works as expected... why it is behaving strangely in former one...
This is because the output buffer isn't being flushed (in other words, actually committed to the terminal). When you write a newline, the output buffer is more likely to be (but still not always, in some cases) flushed. Many terminal implementations do this to improve performance. To force the behaviour you want, you need to call fflush(stdout); after each printf call, like this:
#include<stdio.h>
int main()
{
int i=0;
printf("*********Testing Sleep***********\n");
for(i=0;i<10;i++)
{
printf("%d",i);
fflush(stdout);
sleep(1);
}
return 0;
}
What you are looking at is line buffered output. Actually writing to output is an expensive operation, so I/O streams are usually buffered. Actually writing the buffer is deferred until a specific event is encountered. In standard C, you have three types of buffering:
fully buffered - the buffer is written when full.
line buffered - the buffer is written when a newline is encountered (your case).
unbuffered - the buffer is written whenever an I/O function is executed. (Good for error logging.)
Writing the buffer is called flushing. That's why there is a stdio function called fflush(). You might also want to check out setvbuf() and its parameter constants, _IOFBF, _IOLBF and _IONBF. I am sure you can figure out what they mean without looking them up. ;-)
Edit: This program delivers as you originally expected:
#include <stdio.h>
// This is the header where sleep() is declared. Don't go without it.
#include <unistd.h>
int main()
{
int i=0;
// setvbuf() can be called on a stream only BEFORE
// you do any I/O on it!
setvbuf( stdout, NULL, _IONBF, 0 );
printf( "*********Testing Sleep***********\n" );
for ( i = 0; i < 10; ++i )
{
printf( "%d", i );
sleep( 1 );
}
return 0;
}
standard output for terminals is line buffered, output is not written unless there is a newline or you manually flush it.
Output is buffered, so that that the OS has an opportunity to optimize output speed. To make sure they are flushed immediately, do fflush (stdout);, but usually, you don't.
This is because printf() uses buffered output for better performance. Buffer is flushed to the console once \n is printed.
Printf is buffered.
You can force printf to 'flush' its buffer using the fflush call.
Or
Simply push the buffer to stdout using \n as in your case .
More detailed discussion is here