Streaming output to stdout, in-place status updates to stderr - c

I have a program that should produce output on stdout, and transient status information on stderr. Programs with similar results might include rsync or apt (though their output is sufficiently complex that reducing it to the essentials I need is daunting).
This program seems like it should work as described, but does not:
#include <stdio.h>
#include <limits.h>
void *work () {
for (long i = 0; i < LONG_MAX; i++) {
if (i % 100000000 == 0) {
fprintf(stdout, "%ld\n", i);
}
fprintf(stderr, "\rrunning %ld", i);
}
return 0;
}
int main() {
work(0);
fprintf(stderr, "\ndone\n");
return 0;
}
The status info is updated in place via stderr as desired, but only a single line of output appears via stdout. If the stderr status fprintf is removed, then all stdout output appears as expected.
What is going on here?

This code is correct (well, the void * return from work() is bad form--void would be far better).
I suspect that the problem is simply that you are not waiting long enough--it takes time to iterate through 100,000,000 iterations, especially with output each time through the loop (remember--you will be blocking on output once you fill the stdout buffer).
I estimate it will take a minimum of 3 minutes before you get the second output to stdout.

Related

Print number for each second

#include <stdio.h>
#include <wiringPi.h>
#include <softPwm.h>
int main(void)
{
wiringPiSetupGpio();
for(int i = 0 ; i <50; i++)
{
if ( i%10 == 1)
{
printf("\n");
}
printf("%d ", i);
delay(1000);
}
return 0;
}
I'm working in rasberry pi environment.
I want to print a number for each 1 second. But this code did not print a number one by one but
print 10 numbers for each 10 seconds. This code gives 10 numbers in line at once. What's the problem??
The stdout channel is line buffered by default. This means that data sent to stdout won't necessarily appear until a newline character is printed.
If you call fflush(stdout), any buffered output will be immediately printed.
printf("%d ", i);
fflush(stdout);
in order to print the values at the running time using printf you need to add \n so try this may work.
#include <stdio.h>
#include <wiringPi.h>
#include <softPwm.h>
int main(void)
{
wiringPiSetupGpio();
for(int i = 0 ; i <50; i++)
{
if ( i%10 == 1)
{
printf("\n");
}
// a `\n` added at the end of the string!
printf("%d \n", i);
delay(1000);
}
return 0;
}
for more info read this question's answer
The problem is probably that the output stream is not being flushed. I suggest that you call fflush( stdout ); before the delay(1000); function call. This will ensure that all printed data actually becomes observable, before the program enters a wait state.
Normally, it is not necessary to explicitly flush an output stream, because the output will become visible sooner or later. For example, the output buffer will usually get implicitly flushed whenever you read input or when the program ends. Also, if your program is writing output to the user's screen (in contrast to, for example, writing output to a file), then the output stream is probably line-buffered, which means that the output stream will get implicitly flushed whenever a newline character is written.
However, in this case, it appears that the implicit flushing mentioned above is not sufficient. Therefore, you will have to revert to explicit flushing using the function fflush, as mentioned above.

read on a pipe blocks until program running at end of pipe terminates - Windows

I have a sample program that outputs a line of text every second. In the test program below, this program writes some text to stdout then waits 1 second and repeats 20 times.
I have another program which uses popen (_popen on Windows) to open a pipe for reading from the program. I then use fgets to read data. The problem I have is that the fgets blocks until the program terminates. Then I get all the output, all 20 lines, in one go. I want to get the output a line at a time, then ok for fgets to block until next line ready. The reason is I plan to use this on a program that will be constantly running, outputting text, e.g. like the use of tail.
If I run this code example on a program that outputs some text all in one go and exits then it works fine.
Why does fgets block? The test program does print some text immediately, so why doesn't fgets read this first line of text immediately?
Here is the code:
#include <stdio.h>
#include <windows.h>
void execute(const char* cmd) {
char buffer[128] = { 0 };
FILE* pipe = _popen(cmd, "r");
if (!pipe) {
printf("popen() failed!\n");
return;
}
while (!feof(pipe)) {
if (fgets(buffer, 128, pipe) != nullptr)
printf("%s", buffer);
}
int rc = _pclose(pipe);
if (rc != EXIT_SUCCESS) { // return code not 0
printf("pclose exit failure: %d\n", rc);
}
}
int main(int argc, char* argv[]) {
if (argc != 2) {
printf("Usage: pipe_test.exe <program>\n");
exit(1);
}
execute(argv[1]);
}
The program run, helloworld.exe:
#include <stdio.h>
#include <windows.h>
int main() {
for (int i = 0; i < 20; i++) {
printf("Hello World %d\n", i);
Sleep(1000);
}
}
Why does fgets block?
Because it's waiting for the children to output something.
The test program does print some text immediately, so why doesn't fgets read this first line of text immediately?
It actually does not print text immediately. The problem here, as #Barmar notices, is that writing to a pipe is buffered (and not line buffered) by the C standard library implementation. This buffering happens in your child program (helloworld), not in your parent program (pipe_test).
From your parent program, you have no control over what the children spawned through popen() will do, therefore if the child output is buffered like in this case, the only thing you can do (without modifying the child's code) is to wait until the buffer is flushed to the pipe.
In order to get the output sooner, you would have to modify the children's code to manually call fflush() or use setvbuf() to disable buffering:
int main() {
setvbuf(stdout, NULL, _IONBF, 0); // Disable buffering on stdout.
for (int i = 0; i < 20; i++) {
printf("Hello World %d\n", i);
Sleep(1000);
}
}
There's really not much else you can do.

C Stop stdout from flushing

I am writing a C program where I am printing to stderr and also using putchar() within the code. I want the output on the console to show all of the stderr and then finally flush the stdout before the program ends. Does anyone know of a method that will stop stdout from flushing when a putchar('\n') occurs?
I suppose i could just do an if statement to make sure it doesn't putchar any newlines but I would prefer some line or lines of code to put at the top of the program to stop all flushing until i say fflush(stdout) at the bottom of the program
What you're trying to do is horribly fragile. C provides no obligation for an implementation of stdio not to flush output, under any circumstances. Even if you get it to work for you, this behavior will be dependent on not exceeding the buffer size. If you really need this behavior, you should probably buffer the output yourself (possibly writing it to a tmpfile() rather than stdout) then copying it all to stdout as the final step before your program exits.
Run your command from the console using pipeling:
my_command >output.txt
All output witten to stderr will appear immediately. The stuff written to stdout will go to output.txt.
Windows only. I'm still looking for the Unix solution myself if anyone has it!
Here is a minimal working example for Windows that sends a buffer to stdout without flushing. You can adjust the maximum buffer size before a flush occurs by changing max_buffer, though I imagine there's some upper limit!
#include <windows.h>
#include <string.h>
int main()
{
const char* my_buffer = "hello, world!";
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
int max_buffer = 1000000;
int num_remaining = strlen(my_buffer);
while (num_remaining)
{
DWORD num_written = 0;
int buffer_size = num_remaining < max_buffer ? num_remaining : max_buffer;
int retval = WriteConsoleA(hStdout, my_buffer, buffer_size, &num_written, 0);
if (retval == 0 || num_written == 0)
{
// Handle error
}
num_remaining -= num_written;
if (num_remaining == 0)
{
break;
}
my_buffer += num_written;
}
}
You can use setvbuf() to fully buffer output to stdout and provide a large enough buffer size for your purpose:
#include <stdio.h>
int main() {
// issue this call before any output
setvbuf(stdout, NULL, _IOFBF, 16384);
...
return 0;
}
Output to stderr is unbuffered by default, so it should go to the console immediately.
Output to stdout is line buffered by default when attached to the terminal. Setting it to _IOFBF (fully buffered) should prevent putchar('\n') from flushing the pending output.

why printing to "stdout" not executing sequentially?

My friend gave me a riddle. I run it. but not getting expected output.
Code is:
#include <stdio.h>
#include <unistd.h>
int main()
{
while(1)
{
fprintf(stdout,"hello-out");
fprintf(stderr,"hello-err");
sleep(1);
}
return 0;
}
the output doesn't printing hello-out.
Instead it's printing like this infinitely:
hello-errhello-errhello-errhello-errhello-errhello-errhello-errhello-errhello-errhello-err
Then I tried like this:
#include <stdio.h>
#include <unistd.h>
int main()
{
int i = 0;
while(i <= 5)
{
fprintf(stdout,"hello-out");
fprintf(stderr,"hello-err");
sleep(1);
i++;
}
return 0;
}
the optput is:
hello-errhello-errhello-errhello-errhello-errhello-errhello-outhello-outhello-outhello-outhello-outhello-out
In C language instructions execute line by line. But why it is not following here?
File IO behavior is determined by the system and if you want to keep that order you must explicitly fflush. See this program below:
while(i <= 5)
{
fprintf(stdout,"hello-out");
fflush(stdout);
fprintf(stderr,"hello-err");
fflush(stderr);
sleep(1);
i++;
}
The reason is output buffering.
By default, stdout is buffered: if it's connected to a terminal it's line-buffered, otherwise it's fully-buffered. When it's line-buffered, that means that nothing is printed until you print a newline, the buffer fills up, or the buffer is flushed explicitly. Since you're not printing newlines, the output doesn't show up until the program exits, because all stdio buffers are flushed at that time.
stderr, on the other hand, is not buffered by default. So anything written to it appears immediately.

How to write redirected output to file inside loop in C?

I am writing a C program on unix which should redirect it's output to the file, and write to it some text every second in infinite loop:
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main(void) {
int outDes = open("./output.txt", O_APPEND | O_WRONLY);
dup2(outDes, 1);
while(1) {
printf("output text\n");
sleep(1);
}
}
But it writes nothing to the output file. I tried to change the 'while' loop for 'for' with 10 loops, and I found that it writes all 10 lines to the file at once after the series ends. It is not very good for me, while I need to have an infinite loop.
When I'm not redirecting output, it is all ok, and new line appears every second on terminal.
I also tried to put one
printf("text\n");
before redirecting output to the file. Then the program wrote the lines to the file in real time, which is good, but wrote there the first (non redirected) line too. I don't want this first line in my output file, I don't understand how it could be written into file when output was not redirected yet (maybe redirect remained there since last run?), and how it could cause that the lines are suddenly written in real time.
Can anyone explain me how does it work?
You are not checking the return value of open() and dup2(). If either open() or dup2() failed, it won't write anything in output.txt.
if (outDes < -1) {
perror("open");
return 1;
}
if (dup2(outDes, 1) == -1) {
perror("dup2");
return 1;
}
stdio streams are buffered, and the writes happen in memory before being done on the real file description.
Try adding a fflush(stdout) after printf().
You're running afoul of a poorly documented DWIMmy feature in many Unix C libraries. The first time you write to stdout or stderr, the library probes the underlying file descriptor (with isatty(3)). If it's a (pseudo-)terminal, the library puts the FILE in "line buffered" mode, meaning that it'll buffer input until a newline is written and then flush it all to the OS. But if the file descriptor is not a terminal, it puts the FILE in "fully buffered" mode, where it'll buffer something like BUFSIZ bytes of output before flushing them, and pays no attention to line breaks.
This is normally the behavior you want, but if you don't want it (as in this case), you can change it with setvbuf(3). This function (although not the behavior I described above) is ISO standard C. Here's how to use it in your case.
#include <stdio.h>
#include <unistd.h>
int
main(void)
{
if (freopen("output.txt", "a", stdout)) {
perror("freopen");
return 1;
}
if (setvbuf(stdout, 0, _IOLBF, 0)) {
perror("setvbuf");
return 1;
}
for (;;) {
puts("output text");
sleep(1);
}
/* not reached */
}

Resources