I am currently writing a piece of code whose intended usage is this:
program input.txt output.txt
or
program input.txt
in which case it defaults to stdout.
This is the code I have now (within main()):
FILE *outFile;
if (argc < 3) {
outFile = stdout;
} else {
fprintf(stdout, "Will output to file %s\n", argv[2]);
outFile = fopen(argv[2], "w");
if (outFile == NULL) {
fprintf(stderr, "ERR: Could not open file %s. Defaulting to stdout\n", argv[2]);
outFile = stdout;
}
}
/* ... write stuff to outFile... */
if (argc < 3 && outFile != stdout) {
fclose(outFile);
}
These are my concerns: first of all, will this successfully open and close outFile when provided? Also, will this successfully not close stdout? Can anything bad happen if I close stdout?
Also, is this portable? I compile with gcc but this project will be evaluated by a professor using Windows.
Apologies if this is a bit of a mess of a question. I come from Python and am not a CS major (I'm studying mathematics).
Yes, it's portable and it's okay.
Yes, it's portable. You assigned outfile = stdout, so they will be equal as long as you don't reassign either of them elsewhere in the program.
You don't really need the argc < 3 test as well -- the two conditions should always be the same, since you only do the assignment when that's true.
In any program that writes significant data to stdout, you should close stdout immediately before exiting, so that you can check for and report delayed write errors. (Delayed write errors are a design mistake; it ought to be impossible for fclose or close to fail. But we are stuck with them.)
The usual construct is, at the very end of main,
if (ferror(stdout) || fclose(stdout)) {
perror("stdout: write error");
return 1;
}
return 0;
Some programs stick an fflush in there too, but ISO C requires fclose to perform a fflush, so it shouldn't be necessary. This construct is entirely portable.
It's important for this to be the very last thing you do before exiting. It is relatively common for libraries to assume that stdout is never closed, so they may malfunction if you call into them after closing stdout. stdin and stderr are also troublesome that way, but I've yet to encounter a situation where one wanted to close those.
It does sometimes happen that you want to close stdout before your program is completely done. In that case you should actually leave the FILE open but close the underlying "file descriptor" and replace it with a dummy.
int rfd = open("/dev/null", O_WRONLY);
if (rfd == -1) perror_exit("/dev/null");
if (fflush(stdout) || close(1)) perror_exit("stdout: write error");
dup2(rfd, 1);
close(rfd);
This construct is NOT portable to Windows. There is an equivalent, but I don't know what it is. It's also not thread-safe: another thread could call open in between the close and dup2 operations and be assigned fd 1, or it could attempt to write something to stdout in that window and get a spurious write error. For thread safety you have to duplicate the old fd 1 and close it via that handle:
// These allocate new fds, which can always fail, e.g. because
// the program already has too many files open.
int new_stdout = open("/dev/null", O_WRONLY);
if (new_stdout == -1) perror_exit("/dev/null");
int old_stdout = dup(1);
if (old_stdout == -1) perror_exit("dup(1)");
flockfile(stdout);
if (fflush(stdout)) perror_exit("stdout: write error");
dup2 (new_stdout, 1); // cannot fail, atomically replaces fd 1
funlockfile(stdout);
// this close may receive delayed write errors from previous writes
// to stdout
if (close (old_stdout)) perror_exit("stdout: write error");
// this close cannot fail, because it only drops an alternative
// reference to the open file description now installed as fd 1
close (new_stdout);
Order of operations is critical: the open, dup and fflush calls must happen before the dup2 call, both close calls must happen after the dup2 call, and stdout must be locked from before the fflush call until after the dup2 call.
Additional possible complications, dealing with which is left as an exercise:
Cleaning up temporary fds and locks on error, when you don't want to stop the whole program on error
If the thread might be canceled mid-operation
If a concurrent thread might call fork and execve mid-operation
Related
I have 2 functions. The first function is opening a file in write mode and writing some contents to it and then closing it.
FILE *fp = fopen("file.txt", "w");
//writing itnot file using fwrite
fclose(fp);
The second function opens the file in read mode, parses the content and then closes the file.
FILE *fp = fopen("file.txt", "r");
//parsing logic
fclose(fp);
In main, I am calling function1 and function2 sequentially.
int main()
{
function1();
function2();
return 1;
}
Sometimes, function1 fopen fails with error number 13 i.e. Permission Denied. I am observing this only sometimes. I introduced a sleep in function1 after fclose for 2 seconds and it started working fine without any issues.
So I am suspecting file is not immediately released after fclose. Sleep is not the right solution. Can anyone suggest how to resolve this problem? The example I have given here is a use case and the actual code is running in a thread environment.
Draft N1570 for C11 says as 7.21.5.1 The fclose function
A successful call to the fclose function causes the stream pointed to by stream to be
flushed and the associated file to be closed. Any unwritten buffered data for the stream
are delivered to the host environment to be written to the file; any unread buffered data
are discarded. Whether or not the call succeeds, the stream is disassociated from the file
and any buffer set by the setbuf or setvbuf function is disassociated from the stream
(and deallocated if it was automatically allocated).
It makes no assumption on what happens at the host environment level, that is does the function returns only when the whole operation is finished, or does it returns as soon as a request has been queued.
As race conditions can happen in your environment, you should retry a failed open a number of times, eventually with a delay between them. If portability is not a problem and if your system supports the POSIX sync function, you can also force a disk synchronisation of the file after closing it:
Close part:
...
fclose(fp)
sync(); // forces synchronization of io buffers to disk
Re-open part
ntries = ...; // number of open tries
while (ntries-- > 0) {
fp = fopen(...);
if (fp != NULL) break; // open was successful
// optionaly add a delay
}
In an environment and with a C implementation where you must accommodate such behavior, the best approach is probably to implement some fault tolerance around the fopen()s. Although an unconditional sleep() is not the right answer, short, conditional delays via sleep() or a similar function may indeed be part of such an strategy. For example, you might do something along these lines:
#include <stdio.h>
#include <errno.h>
#define MAX_ATTEMPTS 3
FILE *tolerant_fopen(const char *path, const char *mode) {
FILE *result;
int attempts = 0;
while (!(result = fopen(path, mode))) {
if (errno != EACCES || attempts >= MAX_ATTEMPTS) {
break;
}
if (sleep(1) == 0) {
attempts += 1;
}
}
return result;
}
That attempts to open the file immediately, and in the event that that fails on account of access permissions, it waits a short time and then makes another attempt. Overall it may make three or more attempts to open the file, spaced up to a second apart or perhaps slightly more. (Note that sleep() can be interrupted early; in that case it returns the number of seconds left to sleep.)
You can of course implement a different strategy for the timing and duration of the retries if you prefer, retry more error conditions, etc..
If a child process is spawned (even from another thread) while a file is open, then the child process will inherit the file handle and the file will not be fully closed until after the child process has terminated.
To prevent this behavior pass the "N" flag to fopen:
FILE *fp = fopen("file.txt", "wN");
Here is a code snippet.
int saved_stdout = dup(1);
int fd = open("file.txt", O_WRONLY | O_CREAT, 0640);
close(1);
dup(fd);
close(fd);
printf("This text should go into the file\n");
//restore stdout
dup2(saved_stdout, 1);
printf("stdout restore");
I am trying to learn about dup and dup2. So I initially connected my file.txt to stdout. So whenever I use printf, I should be writing to file.txt instead of stdout. But I want to restore it back as well once I am done with this usage, so I use dup2 at the end as well.
The problem is that the text "This text should go into the file\n" never actually goes into the file, but gets printed on stdout. Why so? I straced for it, only to find that dup2 call occurs before that printf("This text..."); statement, why so?
The problem may be due to output buffering. stdout is fully buffered if it's not writing to a terminal, so when you redirect it to the file with dup() it will be buffered. Try flushing the output after the printf().
printf("This text should go into the file\n");
fflush(stdout);
I removed my prior answer as it's wrong.... But when you use printf() you're using the FILE * for stdout, inside of which there's a descriptor or something pointing to the terminal. Changing fd 1 is apparently not changing that.
I'm getting inconsistent results with testing, so giving up here. I just want to add that putting this line just after the open() makes it work for me, which highlights the obscure behaviour:
printf("fileno is %i\n", fileno(stdout));
Don't mix FILE * operations such as printf() with file descriptor manipulation. You should use write() with the descriptor.
I am working with a multi-thread program.
First I redirect my stdout to a certain file. No problem there (I used dup2(fd, 1) where fd is the file descriptor for the file).
Afterwards, I need to redirect my stdout to the terminal again.
My first approach:
/*Declaration*/
fpost_t stream_sdout;
/*code*/
if ( fgetpos( stdout, &stream_sdout) == -1 )
perror(Error:);
It says illegal seek.
No idea why this is happening.
But if I get this to work, then I only need to use fsetpos(stdout, &stream_stdout) and it should work.
My second idea, was to to copy the stdout using dup2(stdout, 4) to the file descriptor table, at position 4. But that ain't working either.
How can I switch the standard output back to its original destination (terminal, pipe, file, whatever)?
#include <unistd.h>
...
int saved_stdout;
...
/* Save current stdout for use later */
saved_stdout = dup(1);
dup2(my_temporary_stdout_fd, 1);
... do some work on your new stdout ...
/* Restore stdout */
dup2(saved_stdout, 1);
close(saved_stdout);
Before you do the dup2(fd, STDOUT_FILENO), you should save the current open file descriptor for standard output by doing int saved_stdout = dup(STDOUT_FILENO); (letting dup() choose an available file descriptor number for you). Then, after you've finished with the output redirected to a file, you can do dup2(saved_stdout, STDOUT_FILENO) to restore standard output to where it was before you started all this (and you should close saved_stdout too).
You do need to worry about flushing standard I/O streams (fflush(stdout)) at appropriate times as you mess around with this. That means 'before you switch stdout over'.
If the program runs on a Linux environment, you can freopen ("/dev/stdout", "a", stdout).
But if you know that stdout was the terminal, freopen ("/dev/tty", "a", stdout) or the equivalent for other OSs—even Windows.
Is it alright for multiple processes to access (write) to the same file at the same time? Using the following code, it seems to work, but I have my doubts.
Use case in the instance is an executable that gets called every time an email is received and logs it's output to a central file.
if (freopen(console_logfile, "a+", stdout) == NULL || freopen(error_logfile, "a+", stderr) == NULL) {
perror("freopen");
}
printf("Hello World!");
This is running on CentOS and compiled as C.
Using the C standard IO facility introduces a new layer of complexity; the file is modified solely via write(2)-family of system calls (or memory mappings, but that's not used in this case) -- the C standard IO wrappers may postpone writing to the file for a while and may not submit complete requests in one system call.
The write(2) call itself should behave well:
[...] If the file was
open(2)ed with O_APPEND, the file offset is first set to the
end of the file before writing. The adjustment of the file
offset and the write operation are performed as an atomic
step.
POSIX requires that a read(2) which can be proved to occur
after a write() has returned returns the new data. Note that
not all file systems are POSIX conforming.
Thus your underlying write(2) calls will behave properly.
For the higher-level C standard IO streams, you'll also need to take care of the buffering. The setvbuf(3) function can be used to request unbuffered output, line-buffered output, or block-buffered output. The default behavior changes from stream to stream -- if standard output and standard error are writing to the terminal, then they are line-buffered and unbuffered by default. Otherwise, block-buffering is the default.
You might wish to manually select line-buffered if your data is naturally line-oriented, to prevent interleaved data. If your data is not line-oriented, you might wish to use un-buffered or leave it block-buffered but manually flush the data whenever you've accumulated a single "unit" of output.
If you are writing more than BUFSIZ bytes at a time, your writes might become interleaved. The setvbuf(3) function can help prevent the interleaving.
It might be premature to talk about performance, but line-buffering is going to be slower than block buffering. If you're logging near the speed of the disk, you might wish to take another approach entirely to ensure your writes aren't interleaved.
This answer was incorrect. It does work:
So the race condition would be:
process 1 opens it for append, then
later process 2 opens it for append, then
later still 1 writes and closes, then
finally 2 writes and closes.
I'd be impressed if that 'worked' because it isn't clear to me what
working should mean. I assume 'working' means all of the bytes written
by the two processes are inthe log file? I'd expect that they both
write starting at the same byte offset, so one will replace the others
bytes. It will all be okay upto and including step 3. and only show up
as a problem at step 4, Seems like an easy test to write: open getchar
... write close.
Is it critical that they can have the file open simultaneously? A
more obvious solution if the write is quick, is to open exclusive.
For a quick check on your system, try:
/* write the first command line argument to a file called foo
* stackoverflow topic 9880935
*/
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int main (int argc, const char * argv[]) {
if (argc <2) {
fprintf(stderr, "Error: need some text to write to the file Foo\n");
exit(1);
}
FILE* fp = freopen("foo", "a+", stdout);
if (fp == NULL) {
perror("Error failed to open file\n");
exit(1);
}
fprintf(stderr, "Press a key to continue\n");
(void) getchar(); /* Yes, I really mean to ignore the character */
if (printf("%s\n", argv[1]) < 0) {
perror("Error failed to write to file: ");
exit(1);
}
fclose(fp);
return 0;
}
Is it required to check the return value of fclose? If we have successfully opened a file, what are the chances that it may fail to close?
When you fwrite to a file, it may not actually write anything, it may stay in a buffer (inside the FILE object). Calling fflush would actually write it to disk. That operation may fail, for example if you just ran out of disk space, or there is some other I/O error.
fclose flushes the buffers implicitly too, so it may fail for the same reasons.
From comp.lang.c:
The fclose() call can fail, and should
be error-checked just as assiduously
as all the other file operations.
Sounds pedantic, right? Wrong. In a
former life, my company's product
managed to destroy a customer's data
by omitting a check for failure when
closing a file. The sequence went
something like (paraphrased):
stream = fopen(tempfile, "w");
if (stream == NULL) ...
while (more_to_write)
if (fwrite(buffer, 1, buflen, stream) != buflen) ...
fclose (stream);
/* The new version has been written successfully. Delete
* the old one and rename.
*/
remove (realfile);
rename (tempfile, realfile);
Of course, what happened was that
fclose() ran out of disk space trying
to write the last couple blocks of
data, so the `tempfile' was truncated
and unusable. And since the fclose()
failure wasn't detected, the program
went right ahead and destroyed the
best extant version of the data in
favor of the damaged version. And, as
Murphy would have it, the victim in
this particular incident was the
person in charge of the customer's
department, the person with authority
to buy more of our product or replace
it with a competitor's product --
and, natch, a person who was already
unhappy with us for other reasons.
It'd be a stretch to ascribe all of
the ensuing misery to this single
omission, but it may be worth pointing
out that both the customer and my
former company have since vanished
from the corporate ecology.
CHECK THOSE FAILURE CODES!
You could (and should) report the error, but in a sense, the stream is still closed:
After the call to fclose(), any use of stream results in undefined behavior.
fclose() will flush any unwritten output (via fflush()) before returning, so the error results from the underlying write() won't be reported at fwrite() or fprintf() time, but when you do the fclose(). As a result, any error that write() or fflush() can generate can be generated by fclose().
fclose() will also call close() which can generate errors on NFS clients, where the changed file isn't actually uploaded to the remote server until close() time. If the NFS server crashed, then the close() will fail, and thus fclose() will fail as well. This might be true of other networked filesystems.
You must ALWAYS check result of fclose()
Let's say you're generating data. You have old data that you fread() from a file, and then do some processing on the data, generate more data, and then write it to a new file. You are careful to not overwrite the old file because you know that trying to create new file might fail, and you would like to keep your old data in that case (some data is better than no data). After finishing all the fwrite()s, which all succeed (because you meticulously checked the return value from fwrite()), you fclose() the file. Then, you rename() your just-written file and overwrite the old file.
If fclose() failed because of write error (disk full?), you just overwrote your last good file with something that might be junk. Oops.
So, if it is critical, you should check the return value of fclose().
In terms of code:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *ifp = fopen("in.dat", "rb");
FILE *ofp = fopen("out.dat", "wb");
char buf[BUFSIZ];
size_t n;
int success = 1;
if (ifp == NULL) {
fprintf(stderr, "error opening in.dat\n");
perror("in.dat");
return EXIT_FAILURE;
}
if (ofp == NULL) {
fclose(ifp);
fprintf(stderr, "error opening out.dat\n");
perror("out.dat");
return EXIT_FAILURE;
}
while ((n = fread(buf, 1, sizeof buf, ifp)) > 0) {
size_t nw;
if ((nw = fwrite(buf, 1, n, ofp)) != n) {
fprintf(stderr, "error writing, wrote %lu bytes instead of %lu\n",
(unsigned long)n,
(unsigned long)nw);
fclose(ifp);
fclose(ofp);
return EXIT_FAILURE;
}
}
if (ferror(ifp)) {
fprintf(stderr, "ferror on ifp\n");
fclose(ofp);
fclose(ifp);
return EXIT_FAILURE;
}
#ifdef MAYLOSE_DATA
fclose(ofp);
fclose(ifp);
rename("out.dat", "in.dat"); /* Oops, may lose data */
#else
if (fclose(ofp) == EOF) {
perror("out.dat");
success = 0;
}
if (fclose(ifp) == EOF) {
perror("in.dat");
success = 0;
}
if (success) {
rename("out.dat", "in.dat"); /* Good */
}
#endif
return EXIT_SUCCESS;
}
In the above code, we have been careful about fopen(), fwrite(), and fread(), but even then, not checking fclose() may result in data loss (when compiled with MAYLOSE_DATA defined).
One reason fclose can fail is if there is any data still buffered and the implicit fflush fails. What I recommend is to always call fflush and do any error handling there.
I've seen many times fclose() returning non-zero.
And on careful examination found out that the actual problem was with the write and not fclose.
Since the stuff being written are being buffered before actual write happens and when an fclose() is called all the buffer is flushed. So any problem in writing of the buffered suff, say like disk full, appears during fclose(). As David Yell says, to write a bullet proof application you need to consider the return value of fclose().
The fclose man page cites that it may fail for any of the reasons that close or fflush may fail.
Quoting:
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an
input/output error.
fflush can fail for reasons that write() would fail, basically in the event that you can't actually write to/save the file.
In a sense, closing a file never fails: errors are returned if a pending write operation failed, but the stream will be closed.
To avoid problems and ensure (as far as you can from a C program), I suggest you:
Properly handle errors returned by fwrite().
Call fflush() before closing the stream. Do remember to check for errors returned by fflush().
If, in the context of your application, you can think of something useful to do if fclose() fails, then test the return value. If you can't, don't.