Is it required to check the return value of fclose? If we have successfully opened a file, what are the chances that it may fail to close?
When you fwrite to a file, it may not actually write anything, it may stay in a buffer (inside the FILE object). Calling fflush would actually write it to disk. That operation may fail, for example if you just ran out of disk space, or there is some other I/O error.
fclose flushes the buffers implicitly too, so it may fail for the same reasons.
From comp.lang.c:
The fclose() call can fail, and should
be error-checked just as assiduously
as all the other file operations.
Sounds pedantic, right? Wrong. In a
former life, my company's product
managed to destroy a customer's data
by omitting a check for failure when
closing a file. The sequence went
something like (paraphrased):
stream = fopen(tempfile, "w");
if (stream == NULL) ...
while (more_to_write)
if (fwrite(buffer, 1, buflen, stream) != buflen) ...
fclose (stream);
/* The new version has been written successfully. Delete
* the old one and rename.
*/
remove (realfile);
rename (tempfile, realfile);
Of course, what happened was that
fclose() ran out of disk space trying
to write the last couple blocks of
data, so the `tempfile' was truncated
and unusable. And since the fclose()
failure wasn't detected, the program
went right ahead and destroyed the
best extant version of the data in
favor of the damaged version. And, as
Murphy would have it, the victim in
this particular incident was the
person in charge of the customer's
department, the person with authority
to buy more of our product or replace
it with a competitor's product --
and, natch, a person who was already
unhappy with us for other reasons.
It'd be a stretch to ascribe all of
the ensuing misery to this single
omission, but it may be worth pointing
out that both the customer and my
former company have since vanished
from the corporate ecology.
CHECK THOSE FAILURE CODES!
You could (and should) report the error, but in a sense, the stream is still closed:
After the call to fclose(), any use of stream results in undefined behavior.
fclose() will flush any unwritten output (via fflush()) before returning, so the error results from the underlying write() won't be reported at fwrite() or fprintf() time, but when you do the fclose(). As a result, any error that write() or fflush() can generate can be generated by fclose().
fclose() will also call close() which can generate errors on NFS clients, where the changed file isn't actually uploaded to the remote server until close() time. If the NFS server crashed, then the close() will fail, and thus fclose() will fail as well. This might be true of other networked filesystems.
You must ALWAYS check result of fclose()
Let's say you're generating data. You have old data that you fread() from a file, and then do some processing on the data, generate more data, and then write it to a new file. You are careful to not overwrite the old file because you know that trying to create new file might fail, and you would like to keep your old data in that case (some data is better than no data). After finishing all the fwrite()s, which all succeed (because you meticulously checked the return value from fwrite()), you fclose() the file. Then, you rename() your just-written file and overwrite the old file.
If fclose() failed because of write error (disk full?), you just overwrote your last good file with something that might be junk. Oops.
So, if it is critical, you should check the return value of fclose().
In terms of code:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *ifp = fopen("in.dat", "rb");
FILE *ofp = fopen("out.dat", "wb");
char buf[BUFSIZ];
size_t n;
int success = 1;
if (ifp == NULL) {
fprintf(stderr, "error opening in.dat\n");
perror("in.dat");
return EXIT_FAILURE;
}
if (ofp == NULL) {
fclose(ifp);
fprintf(stderr, "error opening out.dat\n");
perror("out.dat");
return EXIT_FAILURE;
}
while ((n = fread(buf, 1, sizeof buf, ifp)) > 0) {
size_t nw;
if ((nw = fwrite(buf, 1, n, ofp)) != n) {
fprintf(stderr, "error writing, wrote %lu bytes instead of %lu\n",
(unsigned long)n,
(unsigned long)nw);
fclose(ifp);
fclose(ofp);
return EXIT_FAILURE;
}
}
if (ferror(ifp)) {
fprintf(stderr, "ferror on ifp\n");
fclose(ofp);
fclose(ifp);
return EXIT_FAILURE;
}
#ifdef MAYLOSE_DATA
fclose(ofp);
fclose(ifp);
rename("out.dat", "in.dat"); /* Oops, may lose data */
#else
if (fclose(ofp) == EOF) {
perror("out.dat");
success = 0;
}
if (fclose(ifp) == EOF) {
perror("in.dat");
success = 0;
}
if (success) {
rename("out.dat", "in.dat"); /* Good */
}
#endif
return EXIT_SUCCESS;
}
In the above code, we have been careful about fopen(), fwrite(), and fread(), but even then, not checking fclose() may result in data loss (when compiled with MAYLOSE_DATA defined).
One reason fclose can fail is if there is any data still buffered and the implicit fflush fails. What I recommend is to always call fflush and do any error handling there.
I've seen many times fclose() returning non-zero.
And on careful examination found out that the actual problem was with the write and not fclose.
Since the stuff being written are being buffered before actual write happens and when an fclose() is called all the buffer is flushed. So any problem in writing of the buffered suff, say like disk full, appears during fclose(). As David Yell says, to write a bullet proof application you need to consider the return value of fclose().
The fclose man page cites that it may fail for any of the reasons that close or fflush may fail.
Quoting:
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an
input/output error.
fflush can fail for reasons that write() would fail, basically in the event that you can't actually write to/save the file.
In a sense, closing a file never fails: errors are returned if a pending write operation failed, but the stream will be closed.
To avoid problems and ensure (as far as you can from a C program), I suggest you:
Properly handle errors returned by fwrite().
Call fflush() before closing the stream. Do remember to check for errors returned by fflush().
If, in the context of your application, you can think of something useful to do if fclose() fails, then test the return value. If you can't, don't.
Related
I have 2 functions. The first function is opening a file in write mode and writing some contents to it and then closing it.
FILE *fp = fopen("file.txt", "w");
//writing itnot file using fwrite
fclose(fp);
The second function opens the file in read mode, parses the content and then closes the file.
FILE *fp = fopen("file.txt", "r");
//parsing logic
fclose(fp);
In main, I am calling function1 and function2 sequentially.
int main()
{
function1();
function2();
return 1;
}
Sometimes, function1 fopen fails with error number 13 i.e. Permission Denied. I am observing this only sometimes. I introduced a sleep in function1 after fclose for 2 seconds and it started working fine without any issues.
So I am suspecting file is not immediately released after fclose. Sleep is not the right solution. Can anyone suggest how to resolve this problem? The example I have given here is a use case and the actual code is running in a thread environment.
Draft N1570 for C11 says as 7.21.5.1 The fclose function
A successful call to the fclose function causes the stream pointed to by stream to be
flushed and the associated file to be closed. Any unwritten buffered data for the stream
are delivered to the host environment to be written to the file; any unread buffered data
are discarded. Whether or not the call succeeds, the stream is disassociated from the file
and any buffer set by the setbuf or setvbuf function is disassociated from the stream
(and deallocated if it was automatically allocated).
It makes no assumption on what happens at the host environment level, that is does the function returns only when the whole operation is finished, or does it returns as soon as a request has been queued.
As race conditions can happen in your environment, you should retry a failed open a number of times, eventually with a delay between them. If portability is not a problem and if your system supports the POSIX sync function, you can also force a disk synchronisation of the file after closing it:
Close part:
...
fclose(fp)
sync(); // forces synchronization of io buffers to disk
Re-open part
ntries = ...; // number of open tries
while (ntries-- > 0) {
fp = fopen(...);
if (fp != NULL) break; // open was successful
// optionaly add a delay
}
In an environment and with a C implementation where you must accommodate such behavior, the best approach is probably to implement some fault tolerance around the fopen()s. Although an unconditional sleep() is not the right answer, short, conditional delays via sleep() or a similar function may indeed be part of such an strategy. For example, you might do something along these lines:
#include <stdio.h>
#include <errno.h>
#define MAX_ATTEMPTS 3
FILE *tolerant_fopen(const char *path, const char *mode) {
FILE *result;
int attempts = 0;
while (!(result = fopen(path, mode))) {
if (errno != EACCES || attempts >= MAX_ATTEMPTS) {
break;
}
if (sleep(1) == 0) {
attempts += 1;
}
}
return result;
}
That attempts to open the file immediately, and in the event that that fails on account of access permissions, it waits a short time and then makes another attempt. Overall it may make three or more attempts to open the file, spaced up to a second apart or perhaps slightly more. (Note that sleep() can be interrupted early; in that case it returns the number of seconds left to sleep.)
You can of course implement a different strategy for the timing and duration of the retries if you prefer, retry more error conditions, etc..
If a child process is spawned (even from another thread) while a file is open, then the child process will inherit the file handle and the file will not be fully closed until after the child process has terminated.
To prevent this behavior pass the "N" flag to fopen:
FILE *fp = fopen("file.txt", "wN");
From man page of fwrite in linux,
DESCRIPTION
The function fread() reads nmemb elements of data, each size bytes long, from the stream pointed to by stream, storing them at the location given by ptr.
The function fwrite() writes nmemb elements of data, each size bytes long, to the stream pointed to by stream, obtaining them from the location given by ptr.
For nonlocking counterparts, see unlocked_stdio(3).
RETURN VALUE
On success, fread() and fwrite() return the number of items read or written. This number equals the number of bytes transferred only when size is 1. If an error occurs, or the end of the file is reached,
the return value is a short item count (or zero).
fread() does not distinguish between end-of-file and error, and callers must use feof(3) and ferror(3) to determine which occurred.
Now my question is:
Suppose fwrite() system call failure occurred due to No space on the drive. How can we get the errno for it. As fwrite is not setting the errno in case of failure.
Just in case errno is:
The header file defines the integer variable errno, which
is set by system calls and some library functions in the event of an
error to indicate what went wrong.
Talking in terms of Programming language: C
fwrite() is not a direct system call, and in general all I/O done via a FILE * pointer is buffered. There is no guarantee that a call to fwrite() will actually perform a write at that time. Errors can be returned immediately, or by later calls to functions using FILE *. However, fclose() will write out any remaining buffered data, and will return an error if that write did not succeed.
In summary, you need to check for errors during every call to fwrite(), fread() and fclose(). If you want to avoid doing error checking everywhere, you can call ferror() at any time to check the error status of a FILE *.
The variable errno is set when errors occur, because the C library internally calls read() and write(), and those set errno in case of errors.
fwrite() cannot fail due to EOF, but fread() can.
For fwrite(), you discover an error if it did not write the same amount of items you told it to, so you check it like this:
size_t written = fwrite(buf, element_size, num_elements, file);
if (written != num_elements) {
//error , here errno do get set and you can inspect it/print it etc.
}
Now, a FILE* have an internal buffer, so a fwrite() call might not actually write the data to the operating system or file unless that internal buffer gets full.
This means you also either have to fflush() the FILE* and also check for errors after each fwrite() call like so:
if (fflush(file) == EOF) {
//error , here errno do get set and you can inspect it/print it etc.
}
Or you have to accept that a potential write failure can be given to you on any future fwrite() calls you perform if e.g. those calls happens to flush() the FILE* automatically.
And you'd also have to check for errors when calling fclose():
if (fclose(file) == EOF) {
//error , here errno do get set and you can inspect it/print it etc.
}
For fread() you can inspect errno if it failed. After you have handled any data an fread() call have read, you can do
if (ferror(file)) {
//error , here errno do get set and you can inspect it/print it etc.
}
As fwrite is not setting the errno in case of failure: not exactly. The only possible error for fwrite is an error that would occur during the call of the underlying write system call. In that case the errno variable will be set but the write call and as such will be available, and you will be able to directly use the strerror or perror functions to get a textual description of the error.
BTW, fwrite is not a system call but a standard library function: is does not directly call a kernel function.
Is it alright for multiple processes to access (write) to the same file at the same time? Using the following code, it seems to work, but I have my doubts.
Use case in the instance is an executable that gets called every time an email is received and logs it's output to a central file.
if (freopen(console_logfile, "a+", stdout) == NULL || freopen(error_logfile, "a+", stderr) == NULL) {
perror("freopen");
}
printf("Hello World!");
This is running on CentOS and compiled as C.
Using the C standard IO facility introduces a new layer of complexity; the file is modified solely via write(2)-family of system calls (or memory mappings, but that's not used in this case) -- the C standard IO wrappers may postpone writing to the file for a while and may not submit complete requests in one system call.
The write(2) call itself should behave well:
[...] If the file was
open(2)ed with O_APPEND, the file offset is first set to the
end of the file before writing. The adjustment of the file
offset and the write operation are performed as an atomic
step.
POSIX requires that a read(2) which can be proved to occur
after a write() has returned returns the new data. Note that
not all file systems are POSIX conforming.
Thus your underlying write(2) calls will behave properly.
For the higher-level C standard IO streams, you'll also need to take care of the buffering. The setvbuf(3) function can be used to request unbuffered output, line-buffered output, or block-buffered output. The default behavior changes from stream to stream -- if standard output and standard error are writing to the terminal, then they are line-buffered and unbuffered by default. Otherwise, block-buffering is the default.
You might wish to manually select line-buffered if your data is naturally line-oriented, to prevent interleaved data. If your data is not line-oriented, you might wish to use un-buffered or leave it block-buffered but manually flush the data whenever you've accumulated a single "unit" of output.
If you are writing more than BUFSIZ bytes at a time, your writes might become interleaved. The setvbuf(3) function can help prevent the interleaving.
It might be premature to talk about performance, but line-buffering is going to be slower than block buffering. If you're logging near the speed of the disk, you might wish to take another approach entirely to ensure your writes aren't interleaved.
This answer was incorrect. It does work:
So the race condition would be:
process 1 opens it for append, then
later process 2 opens it for append, then
later still 1 writes and closes, then
finally 2 writes and closes.
I'd be impressed if that 'worked' because it isn't clear to me what
working should mean. I assume 'working' means all of the bytes written
by the two processes are inthe log file? I'd expect that they both
write starting at the same byte offset, so one will replace the others
bytes. It will all be okay upto and including step 3. and only show up
as a problem at step 4, Seems like an easy test to write: open getchar
... write close.
Is it critical that they can have the file open simultaneously? A
more obvious solution if the write is quick, is to open exclusive.
For a quick check on your system, try:
/* write the first command line argument to a file called foo
* stackoverflow topic 9880935
*/
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int main (int argc, const char * argv[]) {
if (argc <2) {
fprintf(stderr, "Error: need some text to write to the file Foo\n");
exit(1);
}
FILE* fp = freopen("foo", "a+", stdout);
if (fp == NULL) {
perror("Error failed to open file\n");
exit(1);
}
fprintf(stderr, "Press a key to continue\n");
(void) getchar(); /* Yes, I really mean to ignore the character */
if (printf("%s\n", argv[1]) < 0) {
perror("Error failed to write to file: ");
exit(1);
}
fclose(fp);
return 0;
}
I am read from a file like this:
#include <stdio.h>
int main() {
FILE *fp = fopen("sorted_hits", "r+");
while(!feof(fp)) {
int item_read;
int *buffer = (int *)malloc(sizeof(int));
item_read = fread(buffer, sizeof(int), 1, fp);
if(item_read == 0) {
printf("at file %ld\n", ftell(fp));
perror("read error:");
}
}
}
This file is big and I got the "Bad file descriptor" error sometimes. "ftell" indicates that the file position stopped when error occurred.
I don't know why it is "sometimes", is that normal? does the problem lie in my code or in my hard disk? How to handle this?
perror prints whatever is in errno as a descriptive string. errno gets set to an error code whenever a system call has an error return. But, if a system call DOESN'T fail, errno doesn't get modified and will continue to contain whatever it contained before. Now if fread returns 0, that means that either there was an error OR you reached the end of the file. In the latter case, errno is not set and might contain any random garbage from before.
So in this case, the "Bad file descriptor" message you're getting probably just means there hasn't been an error at all. You should be checking ferror(fp) to see if an error has occurred.
You seem to be mixing text and binary modes when reading the file.
Normally when you use fread you read from a binary file i.e. fread reads a number of bytes matching the buffer size but you seem to be opening the file in text mode (r+). ftell doesn't work reliably on files opened in text mode because newlines are treated differently than other characters.
Open the file in binary mode (untranslated) instead:
FILE *fp = fopen("sorted_hits", "rb+");
If that's really what your loop looks like, my guess would be that you're probably getting a more or less spurious error because your process is just running out of memory because your loop is leaking it so badly (calling malloc every iteration of your loop, but no matching call to free anywhere).
It's also possible (but a lot less likely) that you're running into a little problem from your (common but nearly always incorrect) use of while (!feof(fp)).
Your all to printf also gives undefined behavior because you've mismatched the conversion and the type (though on many current systems it's irrelevant because long and int are the same size).
Fixing those may or may not remove the problem you've observed, but at least if you still see it, you'll have narrowed down the possibilities of what may be causing the problem.
int main() {
FILE *fp = fopen("sorted_hits", "r+");
int buffer;
while(0 != fread(&buffer, sizeof(int), 1, fp))
; // read file but ignore contents.
if (ferror(fp)) {
printf("At file: %ld\n", ftell(fp));
perror("read error: ");
}
}
Is it required to check the return value of fclose? If we have successfully opened a file, what are the chances that it may fail to close?
When you fwrite to a file, it may not actually write anything, it may stay in a buffer (inside the FILE object). Calling fflush would actually write it to disk. That operation may fail, for example if you just ran out of disk space, or there is some other I/O error.
fclose flushes the buffers implicitly too, so it may fail for the same reasons.
From comp.lang.c:
The fclose() call can fail, and should
be error-checked just as assiduously
as all the other file operations.
Sounds pedantic, right? Wrong. In a
former life, my company's product
managed to destroy a customer's data
by omitting a check for failure when
closing a file. The sequence went
something like (paraphrased):
stream = fopen(tempfile, "w");
if (stream == NULL) ...
while (more_to_write)
if (fwrite(buffer, 1, buflen, stream) != buflen) ...
fclose (stream);
/* The new version has been written successfully. Delete
* the old one and rename.
*/
remove (realfile);
rename (tempfile, realfile);
Of course, what happened was that
fclose() ran out of disk space trying
to write the last couple blocks of
data, so the `tempfile' was truncated
and unusable. And since the fclose()
failure wasn't detected, the program
went right ahead and destroyed the
best extant version of the data in
favor of the damaged version. And, as
Murphy would have it, the victim in
this particular incident was the
person in charge of the customer's
department, the person with authority
to buy more of our product or replace
it with a competitor's product --
and, natch, a person who was already
unhappy with us for other reasons.
It'd be a stretch to ascribe all of
the ensuing misery to this single
omission, but it may be worth pointing
out that both the customer and my
former company have since vanished
from the corporate ecology.
CHECK THOSE FAILURE CODES!
You could (and should) report the error, but in a sense, the stream is still closed:
After the call to fclose(), any use of stream results in undefined behavior.
fclose() will flush any unwritten output (via fflush()) before returning, so the error results from the underlying write() won't be reported at fwrite() or fprintf() time, but when you do the fclose(). As a result, any error that write() or fflush() can generate can be generated by fclose().
fclose() will also call close() which can generate errors on NFS clients, where the changed file isn't actually uploaded to the remote server until close() time. If the NFS server crashed, then the close() will fail, and thus fclose() will fail as well. This might be true of other networked filesystems.
You must ALWAYS check result of fclose()
Let's say you're generating data. You have old data that you fread() from a file, and then do some processing on the data, generate more data, and then write it to a new file. You are careful to not overwrite the old file because you know that trying to create new file might fail, and you would like to keep your old data in that case (some data is better than no data). After finishing all the fwrite()s, which all succeed (because you meticulously checked the return value from fwrite()), you fclose() the file. Then, you rename() your just-written file and overwrite the old file.
If fclose() failed because of write error (disk full?), you just overwrote your last good file with something that might be junk. Oops.
So, if it is critical, you should check the return value of fclose().
In terms of code:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *ifp = fopen("in.dat", "rb");
FILE *ofp = fopen("out.dat", "wb");
char buf[BUFSIZ];
size_t n;
int success = 1;
if (ifp == NULL) {
fprintf(stderr, "error opening in.dat\n");
perror("in.dat");
return EXIT_FAILURE;
}
if (ofp == NULL) {
fclose(ifp);
fprintf(stderr, "error opening out.dat\n");
perror("out.dat");
return EXIT_FAILURE;
}
while ((n = fread(buf, 1, sizeof buf, ifp)) > 0) {
size_t nw;
if ((nw = fwrite(buf, 1, n, ofp)) != n) {
fprintf(stderr, "error writing, wrote %lu bytes instead of %lu\n",
(unsigned long)n,
(unsigned long)nw);
fclose(ifp);
fclose(ofp);
return EXIT_FAILURE;
}
}
if (ferror(ifp)) {
fprintf(stderr, "ferror on ifp\n");
fclose(ofp);
fclose(ifp);
return EXIT_FAILURE;
}
#ifdef MAYLOSE_DATA
fclose(ofp);
fclose(ifp);
rename("out.dat", "in.dat"); /* Oops, may lose data */
#else
if (fclose(ofp) == EOF) {
perror("out.dat");
success = 0;
}
if (fclose(ifp) == EOF) {
perror("in.dat");
success = 0;
}
if (success) {
rename("out.dat", "in.dat"); /* Good */
}
#endif
return EXIT_SUCCESS;
}
In the above code, we have been careful about fopen(), fwrite(), and fread(), but even then, not checking fclose() may result in data loss (when compiled with MAYLOSE_DATA defined).
One reason fclose can fail is if there is any data still buffered and the implicit fflush fails. What I recommend is to always call fflush and do any error handling there.
I've seen many times fclose() returning non-zero.
And on careful examination found out that the actual problem was with the write and not fclose.
Since the stuff being written are being buffered before actual write happens and when an fclose() is called all the buffer is flushed. So any problem in writing of the buffered suff, say like disk full, appears during fclose(). As David Yell says, to write a bullet proof application you need to consider the return value of fclose().
The fclose man page cites that it may fail for any of the reasons that close or fflush may fail.
Quoting:
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an
input/output error.
fflush can fail for reasons that write() would fail, basically in the event that you can't actually write to/save the file.
In a sense, closing a file never fails: errors are returned if a pending write operation failed, but the stream will be closed.
To avoid problems and ensure (as far as you can from a C program), I suggest you:
Properly handle errors returned by fwrite().
Call fflush() before closing the stream. Do remember to check for errors returned by fflush().
If, in the context of your application, you can think of something useful to do if fclose() fails, then test the return value. If you can't, don't.