I got some code that prints to stdout, in pseudo code it looks like
int main(){
//allocate data
while(conditional){
char *string = makedata();
fprintf(stdout,"%s",string);
}
//cleanup
}
This works fine, if the conditional is toggled to zero, but if I pipe the output like
./a.out |head -n10 >dumped
Then the code never reaches the cleanup part, I don't understand how to check if the stdout gets closed.
Thanks
Your stdout hasn't been closed, so checking for that will be useless. Your program has received a SIGPIPE and exited. A SIGPIPE is delivered whenever your program writes to a pipe on which there are no readers. In your example, that happens when head exits, closing its stdin.
You should ignore SIGPIPE if you want your program to continue. This code will ignore SIGPIPE:
(void)signal(SIGPIPE, SIG_IGN);
If you don't want to modify your program, you can arrange that something continues to read from the pipe, even after head closes its input. 1
./a.out | ( head -n10 >dumped ; cat > /dev/null )
1: The shell example is valid for bash, maybe not for csh.
It is closed by killing the process with SIGPIPE.
If you want to go on when your output is being ignored, then set up a SIG_IGN handler for SIGPIPE, and handle the error from the fprintf (which will be delayed by buffering, so you cannot assume data that has been written has actually reached the user).
As Rob points out in comments to his answer, you are not worried that stdout has been closed; rather, you are worried that the other end of the pipe is closed. This may be pedantry, but it leads to the solution of your problem. Namely, you do not care if stdout is closed, but only if your printf succeeds. You should check the return value of printf: if it is -1, then the write has failed.
As Simon Richter points out, you will never get the return value of printf if you do not ignore SIGPIPE, because a result of writing to stdout when the other side of the pipe has been closed is that SIG_PIPE will be sent to the process. So you need to do something like:
signal( SIGPIPE, SIG_IGN ); /* Careful: you now must check the return of *all* writes */
if( fprintf( stdout, ... ) == -1 ) {
/* handle the error */
}
I've not tried this, but provided you were using the standard file descriptor for stdout, you may be able to try performing an fstat(1, &stat_structure) and check the return and error codes.
Related
This question already has answers here:
printf anomaly after "fork()"
(3 answers)
fork() branches more than expected?
(3 answers)
Closed last year.
I have the following C code:
#include <stdio.h>
#include <unistd.h>
int main()
{
int i, pid = 0;
for (i = 0; i < 3; i++)
{
fork();
pid = getpid();
printf("i=%d pid=%d\n", i, pid);
}
return 1;
}
Which is supposed to create a total of 7 new processes after all the iterations in the loop. Analyzing it you can see that 14 lines should be printed before all the processes finish, and that is exactly what you see when you execute it from the command line.
However, when you redirect the output to another file ./main > output.txt; cat output.txt, you get a completely different situation. In total, 24 lines are always printed and some of them are repeated for the same i and pid values, and the amount of repetition seems consistent. I'm attaching a screenshot for clarification here Execution example. The system that I'm using is Ubuntu 20.04.3 in a VirtualBox VM.
I really don't understand why that is happening, I'm guessing it has something to do with race conditions on the output buffer or some other conflict when multiple processes are writing to the file, but that doesn't explain to me why it doesn't happen on the terminal. Can anybody explain this odd behaviour? Thanks!
When the standard output is a terminal, the stream is typically line buffered. The C standard requires it not be fully buffered, meaning it must be line buffered or unbuffered; C 2018 7.21.3 6 says:
… As initially opened, … the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.
When the program executes printf("i=%d pid=%d\n", i, pid);, the output is immediately sent to the terminal, either because the stream is line buffered and the new-line character causes the output to be sent or because the stream is unbuffered and the output is always sent in each printf. Then, when the program forks, there is no pending output, because it has already been sent to the terminal. Each forked instance of the program prints only its own output.
When the standard output is redirected to a file, the stream is fully buffered. Then, when the program executes printf("i=%d pid=%d\n", i, pid);, the data is held in a buffer inside the program. It is not sent to the terminal immediately. (It will be sent when the buffer is full or when a flush is requested, which occurs automatically at normal program termination.) When the program forks, the buffer is copied along with the rest of the program state. Each forked instance of the program accumulates output in the buffer.
When each forked instance of the program exits, pending data in its buffers are flushed. Thus includes both data added by that particular instance and data that was put into the buffer in parent processes and copied by the fork. Thus multiple copies of data are printed.
To resolve this, execute fflush(stdout); immediately before fork();. This flushes the buffer before forking. Alternately, request that the stream be line-buffered by executing setvbuf(stdout, NULL, _IOLBF, 0); at the start of main.
I am trying to implement a basic shell in C that handles multiple pipes. It waits for input and execs the commands in a for loop. When it receives EOF, it stops waiting for input and exits.
Right now, my shell outputs the correct output when I input a pipelined command, e.g. ls | wc | grep ... but it stops waiting for input and exits the outer while loop instead of waiting for the next line of input.
I found that this happens because the fgets in my while loop is returning null (stdin is getting EOF somehow?). I do not get any errors while creating forking, creating a pipe, or execing.
However, if I enter one command at a time without any pipes e.g. ls, it successfully prints out the correct output and waits for the next line of input, as it should.
My program parses each line of input into a struct before trying to execute each command (omitted below). The struct is designed such that I can easily pass the parsed arguments into execvp, which I will not describe here.
This is a heavily simplified version of my code with most of the error-checking omitted:
FILE* input;
char line[MAX_LINE];
input = stdin;
printf("> ");
fflush(stdout);
while (fgets(line, sizeof(line), input)) {
int i;
struct cmdLine;
/* struct defined elsewhere
** commands = # of commands in parsed input
** start = index where a command and its args start
** args[] = array holding each command/arg
*/
/* parse input line into cmdLine */
...
/* exec all commands in pipeline except the last */
for (i = 0; i < cmdLine.commands-1; ++i) {
int pd[2];
pipe(pd);
if (fork() == 0) {
dup2(pd[1], 1);
execvp(cmdLine.args[cmdLine.start[i]], &(cmdLine.args[cmdLine.start[i]]));
} else {
wait(NULL);
}
dup2(pd[0], 0);
close(pd[1]);
}
/* exec last command */
if (fork() == 0) {
execvp(cmdLine.args[cmdLine.start[i]], &(cmdLine.args[cmdLine.start[i]]));
} else {
wait(NULL);
}
if (stdin == input) {
printf("> "); /* print shell prompt */
fflush(stdout);
}
}
I am almost certain I messed up somewhere with my duping, but I've been trying for hours and I don't understand what I'm doing wrong. Is EOF somehow being sent to stdin so the enclosing fgets returns NULL?
By calling dup2 with 0 (=stdin) as the second argument, you are closing the original stdin at the end of each iteration of your for loop, so you can no longer actually talk to your program via the original stdin.
The problem in your code is that you are trying to hand off connecting all of the pipes together to someone else; that is not going to work. Here's what should work:
For n programs, you need at least (n-1) pipes.
Record all of the pipe FDs in arrays: one for the input side of the pipe (that is written to), one for the output side (that is read from).
For each process you are forking, connect the previous pipe's output (if any) to its stdin, and the next pipe's input to its stdout (or your main program's stdout if you're handling the last process in your chain of pipes).
Once you've forked everything: in a loop, poll() on the output FDs of your pipes, read from any that have activity, and write to the input of the next pipe (your own stdout at the end). If you get EOF on one of the pipes, close the next pipe's input (and remove the EOF'd pipe output from your output array). Once all of the FDs are closed, exit your loop.
EDIT: I just thought of another, simpler way that requires less code changes but I haven't completely thought it through. :) The problem is that you are destroying your own stdin. If you do all of this (i.e. the whole "process one line of commands) in a forked child, replacing stdin between processes doesn't affect the parent process at all. Still, this would require a lot of buffering in the kernel and so it probably won't scale.
I'm following this guide about fork() but something isn't clear for me.
Both processes will start their execution at the next statement following the fork() call. In this case, both processes will start their execution at the assignment statement as shown below:
According to this sentence, this script
printf("before ");
fork();
printf("after ");
should print this: (Because child process will start from printf("after"))
before after after
but it is printing this instead:
before after before after
So did the child process start from the 1st line of the file? Can you tell me what's wrong with my code? Did I misunderstood that sentence?
EDIT
Script compiled and executed on OS X
When you create a new process, it 'inherits' all the variables of the original process - thus all the buffers as well. Since "before" wasn't flushed yet and is still in the buffer, the child process will as well contain this string in the buffer and print it. Therefore you have to call fflush(stdout); before forking the process.
You understood the sentence correctly, but...
When you call fork it takes a snapshot of the process, and creates an exact duplicate. So if there is data buffered in stdout waiting to be written to the console, then that data will be in the child's output buffer after the fork, as well as the parent's buffer.
There are two ways to clear the output buffer before the fork. You can either add a newline \n at the end of the printf
printf( "before\n" );
fork();
printf( "after\n" );
or you can use the fflush function
printf( "before " );
fflush( stdout );
fork();
printf( "after " );
It is because the "before" is buffered and only outputted when that buffer is flushed. This occurs in when the two processes terminates.
If you call fflush on stdout prior to fork, you will see the output you expect. In general, C buffered IO won't play nicely when you do operations at the OS level.
The memory buffer associated with the standard output is cloned on fork including any previously buffered output. That's why you see the "before" twice.
I am writing some C code that involves the use of pipes. To make a child process use my pipe instead of STDOUT for output, I used the following lines:
close(STDOUT);
dup2(leftup[1], STDOUT);
However, it seems to go into some sort of infinite loop or hang on those lines. When I get rid of close, it hangs on dup2.
Curiously, the same idea works in the immediately preceding line for STDIN:
close(STDIN);
dup2(leftdown[0], STDIN);
What could be causing this behavior?
Edit: Just to be clear...
#define STDIN 0
#define STDOUT 1
Edit 2: Here is a stripped-down example:
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#define STDIN 0
#define STDOUT 1
main(){
pid_t child1 = 0;
int leftdown[2];
if (pipe(leftdown) != 0)
printf("ERROR");
int leftup[2];
if (pipe(leftup) != 0)
printf("ERROR");
printf("MADE PIPES");
child1 = fork();
if (child1 == 0){
close(STDOUT);
printf("TEST 1");
dup2(leftup[1], STDOUT);
printf("TEST 2");
exit(0);
}
return(0);
}
The "TEST 1" line is never reached. The only output is "MADE PIPES".
At a minimum, you should ensure that the dup2 function returns the new file descriptor rather than -1.
There's always a possibility that it will give you an error (for example, if the pipe() call failed previously). In addition, be absolutely certain that you're using the right indexes (0 and 1) - I've been bitten by that before and it depends on whether you're in the parent or child process.
Based on your edit, I'm not the least bit surprised that MADE PIPES is the last thing printed.
When you try to print TEST 1, you have already closed the STDOUT descriptor so that will go nowhere.
When you try to print TEST 2, you have duped the STDOUT descriptor so that will go to the parent but your parent doesn't read it.
If you change your forking code to:
child1 = fork();
if (child1 == 0){
int count;
close(STDOUT);
count = printf("TEST 1\n");
dup2(leftup[1], STDOUT);
printf("TEST 2 (%d)\n", count);
exit(0);
} else {
char buff[80];
read (leftup[0], buff, 80);
printf ("%s\n", buff);
sleep (2);
}
you'll see that the TEST 2 (-1) line is output by the parent because it read it via the pipe. The -1 in there is the return code from the printf you attempted in the child after you closed the STDOUT descriptor (but before you duped it), meaning that it failed.
From ISO C11 7.20.6.3 The printf function:
The printf function returns the number of characters transmitted, or a negative value if an output or encoding error occurred.
Multiple thing to mention,
When you use fork, it causes almost a complete copy of parent process. That also includes the buffer that is set up for stdout standard output stream as well. The stdout stream will hold the data till buffer is full or explicitly requested to flush the data from buffer/stream. Now because of this , now you have "MADE PIPES" sitting in buffer. When you close the STDOUT fd and use printf for writing data out to terminal, it does nothing but transfers your "TEST 1" and "TEST 2" into the stdout buffer and doesn't cause any error or crash (due to enough buffer). Thus even after duplicating pipe fd on STDOUT, due to buffered output printf hasn't even touched pipe write end. Most important, please use only one set of APIs i.e. either *NIX or standard C lib functions. Make sure you understand the libraries well, as they often play tricks for some sort of optimization.
Now, another thing to mention, make sure that you close the appropriate ends of pipe in appropriate process. Meaning that if say, pipe-1 is used to communicate from parent to child then make sure that you close the read end in parent and write end in child. Otherwise, your program may hung, due to reference counts associated with file descriptors you may think that closing read end in child means pipe-read end is closed. But as when you don't close the read end in parent, then you have extra reference count for read end of pipe and ultimately the pipe will never close.
There are many other things about your coding style, better you should get hold on it :)
Sooner you learn it better it will save your time. :)
Error checking is absolutely important, use at least assert to ensure that your assumptions are correct.
While using printf statements to log the error or as method of debugging and you are changing terminal FD's (STDOUT / STDIN / STDERR) its better you open a log file with *NIX open and write errors/ log entries to it.
At last, using strace utility will be a great help for you. This utility will allow you to track the system calls executed while executing your code. It is very straight forward and simple. You can even attach this to executing process, provided you have right permissions.
This question already has answers here:
printf anomaly after "fork()"
(3 answers)
Closed 8 years ago.
I was experimenting with fork() and re-direction to check whether the re-directions done in the parent apply to the child too. I wrote the following simple program
#include<stdio.h>
#include<unistd.h>
#include<stdlib.h>
int main ()
{
freopen( "error.txt", "w+t", stdout ); // From now on, stdout = error.txt
printf (" ERROR! WHY DONT U UNDERSTAND?\n");
if ( fork() == 0 )
{
printf(" I AM CHILD\n");
exit(0);
}
else-
{
printf (" EITHER I AM A PARENT OR SOMETHING GOT SCREWED\n");
}
return 0;
}
The output ( error.txt ) I got is
ERROR! WHY DONT U UNDERSTAND?
EITHER I AM A PARENT OR SOMETHING GOT SCREWED
ERROR! WHY DONT U UNDERSTAND?
I AM CHILD
Surprisingly, ERROR! WHY DONT U UNDERSTAND? is printing twice even though it appears much before the fork() is called and should only be printed once by the parent.
Can anyone shed some light on this?
Since after reopen the stream is non-interactive, it's fully buffered and doesn't flush on '\n'. Before fork is called the buffer still contains the message, and after fork this buffered message was duplicated (because both processes got their own copies of stdout) and then flushed by both the parent and the child. See part 7.19.3 of C standard.
You can avoid such behavior by calling fflush just before fork.
It's because of buffering. Do a fflush right after printf.
Both processes end up with the same copy of stdio's internal stuff and both proceed to flush it at exit. You might also prevent it from happening if you call _exit in the child.
flushing the buffer will solve the problem.
use fflush just after the print statement.
It seems that the ERROR! WHY DONT U UNDERSTAND is still buffered after forking and gets written by both processes.
If you add
fflush(stdout);
right after your first printf() the internal buffer is flushed and it only appears once in your file.