Related
I've boiled down my entire program to a short main that replicates the issue, so forgive me for it not making any sense.
input.txt is a text file that has a couple lines of text in it. This boiled down program should print those lines. However, if fork is called, the program enters an infinite loop where it prints the contents of the file over and over again.
As far as I understand fork, the way I use it in this snippet is essentially a no-op. It forks, the parent waits for the child before continuing, and the child is immediately killed.
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(){
freopen("input.txt", "r", stdin);
char s[MAX];
int i = 0;
char* ret = fgets(s, MAX, stdin);
while (ret != NULL) {
//Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0) {
exit(0);
} else {
waitpid(pid, &status, 0);
}
//End region
printf("%s", s);
ret = fgets(s, MAX, stdin);
}
}
Edit: Further investigation has only made my issue stranger. If the file contains <4 blank lines or <3 lines of text, it does not break. However, if there are more than that, it loops infinitely.
Edit2: If the file contains numbers 3 lines of numbers it will infinitely loop, but if it contains 3 lines of words it will not.
I am surprised that there is a problem, but it does seem to be a problem on Linux (I tested on Ubuntu 16.04 LTS running in a VMWare Fusion VM on my Mac) — but it was not a problem on my Mac running macOS 10.13.4 (High Sierra), and I wouldn't expect it to be a problem on other variants of Unix either.
As I noted in a comment:
There's an open file description and an open file descriptor behind each stream. When the process forks, the child has its own set of open file descriptors (and file streams), but each file descriptor in the child shares the open file description with the parent. IF (and that's a big 'if') the child process closing the file descriptors first did the equivalent of lseek(fd, 0, SEEK_SET), then that would also position the file descriptor for the parent process, and that could lead to an infinite loop. However, I've never heard of a library that does that seek; there's no reason to do it.
See POSIX open() and fork() for more information about open file descriptors and open file descriptions.
The open file descriptors are private to a process; the open file descriptions are shared by all copies of the file descriptor created by an initial 'open file' operation. One of the key properties of the open file description is the current seek position. That means that a child process can change the current seek position for a parent — because it is in the shared open file description.
neof97.c
I used the following code — a mildly adapted version of the original that compiles cleanly with rigorous compilation options:
#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(void)
{
if (freopen("input.txt", "r", stdin) == 0)
return 1;
char s[MAX];
for (int i = 0; i < 30 && fgets(s, MAX, stdin) != NULL; i++)
{
// Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0)
{
exit(0);
}
else
{
waitpid(pid, &status, 0);
}
// End region
printf("%s", s);
}
return 0;
}
One of the modifications limits the number of cycles (children) to just 30.
I used a data file with 4 lines of 20 random letters plus a newline (84 bytes total):
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
I ran the command under strace on Ubuntu:
$ strace -ff -o st-out -- neof97
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
…
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
$
There were 31 files with names of the form st-out.808## where the hashes were 2-digit numbers. The main process file was quite large; the others were small, with one of the sizes 66, 110, 111, or 137:
$ cat st-out.80833
lseek(0, -63, SEEK_CUR) = 21
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80834
lseek(0, -42, SEEK_CUR) = -1 EINVAL (Invalid argument)
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80835
lseek(0, -21, SEEK_CUR) = 0
exit_group(0) = ?
+++ exited with 0 +++
$ cat st-out.80836
exit_group(0) = ?
+++ exited with 0 +++
$
It just so happened that the first 4 children each exhibited one of the four behaviours — and each further set of 4 children exhibited the same pattern.
This shows that three out of four of the children were indeed doing an lseek() on standard input before exiting. Obviously, I have now seen a library do it. I have no idea why it is thought to be a good idea, though, but empirically, that is what is happening.
neof67.c
This version of the code, using a separate file stream (and file descriptor) and fopen() instead of freopen() also runs into the problem.
#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX = 100 };
int main(void)
{
FILE *fp = fopen("input.txt", "r");
if (fp == 0)
return 1;
char s[MAX];
for (int i = 0; i < 30 && fgets(s, MAX, fp) != NULL; i++)
{
// Commenting out this region fixes the issue
int status;
pid_t pid = fork();
if (pid == 0)
{
exit(0);
}
else
{
waitpid(pid, &status, 0);
}
// End region
printf("%s", s);
}
return 0;
}
This also exhibits the same behaviour, except that the file descriptor on which the seek occurs is 3 instead of 0. So, two of my hypotheses are disproven — it's related to freopen() and stdin; both are shown incorrect by the second test code.
Preliminary diagnosis
IMO, this is a bug. You should not be able to run into this problem.
It is most likely a bug in the Linux (GNU C) library rather than the kernel. It is caused by the lseek() in the child processes. It is not clear (because I've not gone to look at the source code) what the library is doing or why.
GLIBC Bug 23151
GLIBC Bug 23151 - A forked process with unclosed file does lseek before exit and can cause infinite loop in parent I/O.
The bug was created 2018-05-08 US/Pacific, and was closed as INVALID by 2018-05-09. The reason given was:
Please read
http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01,
especially this paragraph:
Note that after a fork(), two handles exist where one existed before. […]
POSIX
The complete section of POSIX referred to (apart from verbiage noting that this is not covered by the C standard) is this:
2.5.1 Interaction of File Descriptors and Standard I/O Streams
An open file description may be accessed through a file descriptor, which is created using functions such as open() or pipe(), or through a stream, which is created using functions such as fopen() or popen(). Either a file descriptor or a stream is called a "handle" on the open file description to which it refers; an open file description may have several handles.
Handles can be created or destroyed by explicit user action, without affecting the underlying open file description. Some of the ways to create them include fcntl(), dup(), fdopen(), fileno(), and fork(). They can be destroyed by at least fclose(), close(), and the exec functions.
A file descriptor that is never used in an operation that could affect the file offset (for example, read(), write(), or lseek()) is not considered a handle for this discussion, but could give rise to one (for example, as a consequence of fdopen(), dup(), or fork()). This exception does not include the file descriptor underlying a stream, whether created with fopen() or fdopen(), so long as it is not used directly by the application to affect the file offset. The read() and write() functions implicitly affect the file offset; lseek() explicitly affects it.
The result of function calls involving any one handle (the "active handle") is defined elsewhere in this volume of POSIX.1-2017, but if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined.
A handle which is a stream is considered to be closed when either an fclose(), or freopen() with non-full(1) filename, is executed on it (for freopen() with a null filename, it is implementation-defined whether a new handle is created or the existing one reused), or when the process owning that stream terminates with exit(), abort(), or due to a signal. A file descriptor is closed by close(), _exit(), or the exec() functions when FD_CLOEXEC is set on that file descriptor.
(1) [sic] Using 'non-full' is probably a typo for 'non-null'.
For a handle to become the active handle, the application shall ensure that the actions below are performed between the last use of the handle (the current active handle) and the first use of the second handle (the future active handle). The second handle then becomes the active handle. All activity by the application affecting the file offset on the first handle shall be suspended until it again becomes the active file handle. (If a stream function has as an underlying function one that affects the file offset, the stream function shall be considered to affect the file offset.)
The handles need not be in the same process for these rules to apply.
Note that after a fork(), two handles exist where one existed before. The application shall ensure that, if both handles can ever be accessed, they are both in a state where the other could become the active handle first. The application shall prepare for a fork() exactly as if it were a change of active handle. (If the only action performed by one of the processes is one of the exec() functions or _exit() (not exit()), the handle is never accessed in that process.)
For the first handle, the first applicable condition below applies. After the actions required below are taken, if the handle is still open, the application can close it.
If it is a file descriptor, no action is required.
If the only further action to be performed on any handle to this open file descriptor is to close it, no action need be taken.
If it is a stream which is unbuffered, no action need be taken.
If it is a stream which is line buffered, and the last byte written to the stream was a <newline> (that is, as if a
putc('\n')
was the most recent operation on that stream), no action need be taken.
If it is a stream which is open for writing or appending (but not also open for reading), the application shall either perform an fflush(), or the stream shall be closed.
If the stream is open for reading and it is at the end of the file (feof() is true), no action need be taken.
If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.
For the second handle:
If any previous active handle has been used by a function that explicitly changed the file offset, except as required above for the first handle, the application shall perform an lseek() or fseek() (as appropriate to the type of handle) to an appropriate location.
If the active handle ceases to be accessible before the requirements on the first handle, above, have been met, the state of the open file description becomes undefined. This might occur during functions such as a fork() or _exit().
The exec() functions make inaccessible all streams that are open at the time they are called, independent of which streams or file descriptors may be available to the new process image.
When these rules are followed, regardless of the sequence of handles used, implementations shall ensure that an application, even one consisting of several processes, shall yield correct results: no data shall be lost or duplicated when writing, and all data shall be written in order, except as requested by seeks. It is implementation-defined whether, and under what conditions, all input is seen exactly once.
Each function that operates on a stream is said to have zero or more "underlying functions". This means that the stream function shares certain traits with the underlying functions, but does not require that there be any relation between the implementations of the stream function and its underlying functions.
Exegesis
That is hard reading! If you're not clear on the distinction between open file descriptor and open file description, read the specification of open() and fork() (and dup() or dup2()). The definitions for file descriptor and open file description are also relevant, if terse.
In the context of the code in this question (and also for Unwanted child processes being created while file reading), we have a file stream handle open for reading only which has not yet encountered EOF (so feof() would not return true, even though the read position is at the end of the file).
One of the crucial parts of the specification is: The application shall prepare for a fork() exactly as if it were a change of active handle.
This means that the steps outlined for 'first file handle' are relevant, and stepping through them, the first applicable condition is the last:
If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.
If you look at the definition for fflush(), you find:
If stream points to an output stream or an update stream in which the most recent operation was not input, fflush() shall cause any unwritten data for that stream to be written to the file, [CX] ⌦ and the last data modification and last file status change timestamps of the underlying file shall be marked for update.
For a stream open for reading with an underlying file description, if the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream, and any characters pushed back onto the stream by ungetc() or ungetwc() that have not subsequently been read from the stream shall be discarded (without further changing the file offset). ⌫
It isn't exactly clear what happens if you apply fflush() to an input stream associated with a non-seekable file, but that isn't our immediate concern. However, if you're writing generic library code, then you might need to know whether the underlying file descriptor is seekable before doing a fflush() on the stream. Alternatively, use fflush(NULL) to have the system do whatever is necessary for all I/O streams, noting that this will lose any pushed-back characters (via ungetc() etc).
The lseek() operations shown in the strace output seem to be implementing the fflush() semantics associating the file offset of the open file description with the file position of the stream.
So, for the code in this question, it seems that fflush(stdin) is necessary before the fork() to ensure consistency. Not doing that leads to undefined behaviour ('if this is not done, the result is undefined') — such as looping indefinitely.
The exit() call closes all open file handles. After the fork, the child and parent have identical copies of the execution stack, including the FileHandle pointer. When the child exits, it closes the file and resets the pointer.
int main(){
freopen("input.txt", "r", stdin);
char s[MAX];
prompt(s);
int i = 0;
char* ret = fgets(s, MAX, stdin);
while (ret != NULL) {
//Commenting out this region fixes the issue
int status;
pid_t pid = fork(); // At this point both processes has a copy of the filehandle
if (pid == 0) {
exit(0); // At this point the child closes the filehandle
} else {
waitpid(pid, &status, 0);
}
//End region
printf("%s", s);
ret = fgets(s, MAX, stdin);
}
}
As /u/visibleman pointed out, the child thread is closing the file and messing things up in main.
I was able to work around it by checking if the program is in terminal mode with
!isatty(fileno(stdin))
And if stdin has been redirected, then it will read all of it into a linkedlist before doing any processing or forking.
Replace exit(0) with _exit(0), and all is fine. This is an old unix tradition, if you are using stdio, your forked image must use _exit(), not exit().
Is it required to check the return value of fclose? If we have successfully opened a file, what are the chances that it may fail to close?
When you fwrite to a file, it may not actually write anything, it may stay in a buffer (inside the FILE object). Calling fflush would actually write it to disk. That operation may fail, for example if you just ran out of disk space, or there is some other I/O error.
fclose flushes the buffers implicitly too, so it may fail for the same reasons.
From comp.lang.c:
The fclose() call can fail, and should
be error-checked just as assiduously
as all the other file operations.
Sounds pedantic, right? Wrong. In a
former life, my company's product
managed to destroy a customer's data
by omitting a check for failure when
closing a file. The sequence went
something like (paraphrased):
stream = fopen(tempfile, "w");
if (stream == NULL) ...
while (more_to_write)
if (fwrite(buffer, 1, buflen, stream) != buflen) ...
fclose (stream);
/* The new version has been written successfully. Delete
* the old one and rename.
*/
remove (realfile);
rename (tempfile, realfile);
Of course, what happened was that
fclose() ran out of disk space trying
to write the last couple blocks of
data, so the `tempfile' was truncated
and unusable. And since the fclose()
failure wasn't detected, the program
went right ahead and destroyed the
best extant version of the data in
favor of the damaged version. And, as
Murphy would have it, the victim in
this particular incident was the
person in charge of the customer's
department, the person with authority
to buy more of our product or replace
it with a competitor's product --
and, natch, a person who was already
unhappy with us for other reasons.
It'd be a stretch to ascribe all of
the ensuing misery to this single
omission, but it may be worth pointing
out that both the customer and my
former company have since vanished
from the corporate ecology.
CHECK THOSE FAILURE CODES!
You could (and should) report the error, but in a sense, the stream is still closed:
After the call to fclose(), any use of stream results in undefined behavior.
fclose() will flush any unwritten output (via fflush()) before returning, so the error results from the underlying write() won't be reported at fwrite() or fprintf() time, but when you do the fclose(). As a result, any error that write() or fflush() can generate can be generated by fclose().
fclose() will also call close() which can generate errors on NFS clients, where the changed file isn't actually uploaded to the remote server until close() time. If the NFS server crashed, then the close() will fail, and thus fclose() will fail as well. This might be true of other networked filesystems.
You must ALWAYS check result of fclose()
Let's say you're generating data. You have old data that you fread() from a file, and then do some processing on the data, generate more data, and then write it to a new file. You are careful to not overwrite the old file because you know that trying to create new file might fail, and you would like to keep your old data in that case (some data is better than no data). After finishing all the fwrite()s, which all succeed (because you meticulously checked the return value from fwrite()), you fclose() the file. Then, you rename() your just-written file and overwrite the old file.
If fclose() failed because of write error (disk full?), you just overwrote your last good file with something that might be junk. Oops.
So, if it is critical, you should check the return value of fclose().
In terms of code:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *ifp = fopen("in.dat", "rb");
FILE *ofp = fopen("out.dat", "wb");
char buf[BUFSIZ];
size_t n;
int success = 1;
if (ifp == NULL) {
fprintf(stderr, "error opening in.dat\n");
perror("in.dat");
return EXIT_FAILURE;
}
if (ofp == NULL) {
fclose(ifp);
fprintf(stderr, "error opening out.dat\n");
perror("out.dat");
return EXIT_FAILURE;
}
while ((n = fread(buf, 1, sizeof buf, ifp)) > 0) {
size_t nw;
if ((nw = fwrite(buf, 1, n, ofp)) != n) {
fprintf(stderr, "error writing, wrote %lu bytes instead of %lu\n",
(unsigned long)n,
(unsigned long)nw);
fclose(ifp);
fclose(ofp);
return EXIT_FAILURE;
}
}
if (ferror(ifp)) {
fprintf(stderr, "ferror on ifp\n");
fclose(ofp);
fclose(ifp);
return EXIT_FAILURE;
}
#ifdef MAYLOSE_DATA
fclose(ofp);
fclose(ifp);
rename("out.dat", "in.dat"); /* Oops, may lose data */
#else
if (fclose(ofp) == EOF) {
perror("out.dat");
success = 0;
}
if (fclose(ifp) == EOF) {
perror("in.dat");
success = 0;
}
if (success) {
rename("out.dat", "in.dat"); /* Good */
}
#endif
return EXIT_SUCCESS;
}
In the above code, we have been careful about fopen(), fwrite(), and fread(), but even then, not checking fclose() may result in data loss (when compiled with MAYLOSE_DATA defined).
One reason fclose can fail is if there is any data still buffered and the implicit fflush fails. What I recommend is to always call fflush and do any error handling there.
I've seen many times fclose() returning non-zero.
And on careful examination found out that the actual problem was with the write and not fclose.
Since the stuff being written are being buffered before actual write happens and when an fclose() is called all the buffer is flushed. So any problem in writing of the buffered suff, say like disk full, appears during fclose(). As David Yell says, to write a bullet proof application you need to consider the return value of fclose().
The fclose man page cites that it may fail for any of the reasons that close or fflush may fail.
Quoting:
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an
input/output error.
fflush can fail for reasons that write() would fail, basically in the event that you can't actually write to/save the file.
In a sense, closing a file never fails: errors are returned if a pending write operation failed, but the stream will be closed.
To avoid problems and ensure (as far as you can from a C program), I suggest you:
Properly handle errors returned by fwrite().
Call fflush() before closing the stream. Do remember to check for errors returned by fflush().
If, in the context of your application, you can think of something useful to do if fclose() fails, then test the return value. If you can't, don't.
This is my program:
#include <stdio.h>
int main() {
FILE *logh;
logh = fopen("/home/user1/data.txt", "a+");
if (logh == NULL)
{
printf("error creating file \n");
return -1;
}
// write some data to the log handle and check if it gets written..
int result = fprintf(logh, "this is some test data \n");
if (result > 0)
printf("write successful \n");
else
printf("couldn't write the data to filesystem \n");
while (1) {
};
fclose(logh);
return 0;
}
When i run this program, i see that the file is getting created but it does not contain any data. what i understand i that there is data caching in memory before the data is actually written to the filesystem to avoid multiple IOs to increase performance. and I also know that i can call fsync/fdatasync inside the program to force a sync. but can i force the sync from outside without having to change the program?
I tried running sync command from Linux shell but it does not make the data to appear on the file. :(
Please help if anybody knows any alternative to do the same.
One useful information: I was researching some more on this and finally found this, to remove internal buffering altogether, the FILE mode can be set to _IONBF using int setvbuf(FILE *stream, char *buf, int mode, size_t size)
The IO functions usingFILE pointers cache the data to be written in an internal buffer within the program's memory until they decide to perform a system call to 'really' write it (which is for normal files usually when the size of the data cached reaches BUFSIZ).
Until then, there is no way to force writing from outside the progam.
The problem is that your program does not close the file because of your while statement. Remove these lines:
while (1) {
};
If the intent is to wait forever, then close the file with fclose before executing the while statement.
Is it alright for multiple processes to access (write) to the same file at the same time? Using the following code, it seems to work, but I have my doubts.
Use case in the instance is an executable that gets called every time an email is received and logs it's output to a central file.
if (freopen(console_logfile, "a+", stdout) == NULL || freopen(error_logfile, "a+", stderr) == NULL) {
perror("freopen");
}
printf("Hello World!");
This is running on CentOS and compiled as C.
Using the C standard IO facility introduces a new layer of complexity; the file is modified solely via write(2)-family of system calls (or memory mappings, but that's not used in this case) -- the C standard IO wrappers may postpone writing to the file for a while and may not submit complete requests in one system call.
The write(2) call itself should behave well:
[...] If the file was
open(2)ed with O_APPEND, the file offset is first set to the
end of the file before writing. The adjustment of the file
offset and the write operation are performed as an atomic
step.
POSIX requires that a read(2) which can be proved to occur
after a write() has returned returns the new data. Note that
not all file systems are POSIX conforming.
Thus your underlying write(2) calls will behave properly.
For the higher-level C standard IO streams, you'll also need to take care of the buffering. The setvbuf(3) function can be used to request unbuffered output, line-buffered output, or block-buffered output. The default behavior changes from stream to stream -- if standard output and standard error are writing to the terminal, then they are line-buffered and unbuffered by default. Otherwise, block-buffering is the default.
You might wish to manually select line-buffered if your data is naturally line-oriented, to prevent interleaved data. If your data is not line-oriented, you might wish to use un-buffered or leave it block-buffered but manually flush the data whenever you've accumulated a single "unit" of output.
If you are writing more than BUFSIZ bytes at a time, your writes might become interleaved. The setvbuf(3) function can help prevent the interleaving.
It might be premature to talk about performance, but line-buffering is going to be slower than block buffering. If you're logging near the speed of the disk, you might wish to take another approach entirely to ensure your writes aren't interleaved.
This answer was incorrect. It does work:
So the race condition would be:
process 1 opens it for append, then
later process 2 opens it for append, then
later still 1 writes and closes, then
finally 2 writes and closes.
I'd be impressed if that 'worked' because it isn't clear to me what
working should mean. I assume 'working' means all of the bytes written
by the two processes are inthe log file? I'd expect that they both
write starting at the same byte offset, so one will replace the others
bytes. It will all be okay upto and including step 3. and only show up
as a problem at step 4, Seems like an easy test to write: open getchar
... write close.
Is it critical that they can have the file open simultaneously? A
more obvious solution if the write is quick, is to open exclusive.
For a quick check on your system, try:
/* write the first command line argument to a file called foo
* stackoverflow topic 9880935
*/
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int main (int argc, const char * argv[]) {
if (argc <2) {
fprintf(stderr, "Error: need some text to write to the file Foo\n");
exit(1);
}
FILE* fp = freopen("foo", "a+", stdout);
if (fp == NULL) {
perror("Error failed to open file\n");
exit(1);
}
fprintf(stderr, "Press a key to continue\n");
(void) getchar(); /* Yes, I really mean to ignore the character */
if (printf("%s\n", argv[1]) < 0) {
perror("Error failed to write to file: ");
exit(1);
}
fclose(fp);
return 0;
}
Is it required to check the return value of fclose? If we have successfully opened a file, what are the chances that it may fail to close?
When you fwrite to a file, it may not actually write anything, it may stay in a buffer (inside the FILE object). Calling fflush would actually write it to disk. That operation may fail, for example if you just ran out of disk space, or there is some other I/O error.
fclose flushes the buffers implicitly too, so it may fail for the same reasons.
From comp.lang.c:
The fclose() call can fail, and should
be error-checked just as assiduously
as all the other file operations.
Sounds pedantic, right? Wrong. In a
former life, my company's product
managed to destroy a customer's data
by omitting a check for failure when
closing a file. The sequence went
something like (paraphrased):
stream = fopen(tempfile, "w");
if (stream == NULL) ...
while (more_to_write)
if (fwrite(buffer, 1, buflen, stream) != buflen) ...
fclose (stream);
/* The new version has been written successfully. Delete
* the old one and rename.
*/
remove (realfile);
rename (tempfile, realfile);
Of course, what happened was that
fclose() ran out of disk space trying
to write the last couple blocks of
data, so the `tempfile' was truncated
and unusable. And since the fclose()
failure wasn't detected, the program
went right ahead and destroyed the
best extant version of the data in
favor of the damaged version. And, as
Murphy would have it, the victim in
this particular incident was the
person in charge of the customer's
department, the person with authority
to buy more of our product or replace
it with a competitor's product --
and, natch, a person who was already
unhappy with us for other reasons.
It'd be a stretch to ascribe all of
the ensuing misery to this single
omission, but it may be worth pointing
out that both the customer and my
former company have since vanished
from the corporate ecology.
CHECK THOSE FAILURE CODES!
You could (and should) report the error, but in a sense, the stream is still closed:
After the call to fclose(), any use of stream results in undefined behavior.
fclose() will flush any unwritten output (via fflush()) before returning, so the error results from the underlying write() won't be reported at fwrite() or fprintf() time, but when you do the fclose(). As a result, any error that write() or fflush() can generate can be generated by fclose().
fclose() will also call close() which can generate errors on NFS clients, where the changed file isn't actually uploaded to the remote server until close() time. If the NFS server crashed, then the close() will fail, and thus fclose() will fail as well. This might be true of other networked filesystems.
You must ALWAYS check result of fclose()
Let's say you're generating data. You have old data that you fread() from a file, and then do some processing on the data, generate more data, and then write it to a new file. You are careful to not overwrite the old file because you know that trying to create new file might fail, and you would like to keep your old data in that case (some data is better than no data). After finishing all the fwrite()s, which all succeed (because you meticulously checked the return value from fwrite()), you fclose() the file. Then, you rename() your just-written file and overwrite the old file.
If fclose() failed because of write error (disk full?), you just overwrote your last good file with something that might be junk. Oops.
So, if it is critical, you should check the return value of fclose().
In terms of code:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *ifp = fopen("in.dat", "rb");
FILE *ofp = fopen("out.dat", "wb");
char buf[BUFSIZ];
size_t n;
int success = 1;
if (ifp == NULL) {
fprintf(stderr, "error opening in.dat\n");
perror("in.dat");
return EXIT_FAILURE;
}
if (ofp == NULL) {
fclose(ifp);
fprintf(stderr, "error opening out.dat\n");
perror("out.dat");
return EXIT_FAILURE;
}
while ((n = fread(buf, 1, sizeof buf, ifp)) > 0) {
size_t nw;
if ((nw = fwrite(buf, 1, n, ofp)) != n) {
fprintf(stderr, "error writing, wrote %lu bytes instead of %lu\n",
(unsigned long)n,
(unsigned long)nw);
fclose(ifp);
fclose(ofp);
return EXIT_FAILURE;
}
}
if (ferror(ifp)) {
fprintf(stderr, "ferror on ifp\n");
fclose(ofp);
fclose(ifp);
return EXIT_FAILURE;
}
#ifdef MAYLOSE_DATA
fclose(ofp);
fclose(ifp);
rename("out.dat", "in.dat"); /* Oops, may lose data */
#else
if (fclose(ofp) == EOF) {
perror("out.dat");
success = 0;
}
if (fclose(ifp) == EOF) {
perror("in.dat");
success = 0;
}
if (success) {
rename("out.dat", "in.dat"); /* Good */
}
#endif
return EXIT_SUCCESS;
}
In the above code, we have been careful about fopen(), fwrite(), and fread(), but even then, not checking fclose() may result in data loss (when compiled with MAYLOSE_DATA defined).
One reason fclose can fail is if there is any data still buffered and the implicit fflush fails. What I recommend is to always call fflush and do any error handling there.
I've seen many times fclose() returning non-zero.
And on careful examination found out that the actual problem was with the write and not fclose.
Since the stuff being written are being buffered before actual write happens and when an fclose() is called all the buffer is flushed. So any problem in writing of the buffered suff, say like disk full, appears during fclose(). As David Yell says, to write a bullet proof application you need to consider the return value of fclose().
The fclose man page cites that it may fail for any of the reasons that close or fflush may fail.
Quoting:
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an
input/output error.
fflush can fail for reasons that write() would fail, basically in the event that you can't actually write to/save the file.
In a sense, closing a file never fails: errors are returned if a pending write operation failed, but the stream will be closed.
To avoid problems and ensure (as far as you can from a C program), I suggest you:
Properly handle errors returned by fwrite().
Call fflush() before closing the stream. Do remember to check for errors returned by fflush().
If, in the context of your application, you can think of something useful to do if fclose() fails, then test the return value. If you can't, don't.