I have written a C program in which I am logging the results to a file. There is an infinite while loop - this is a requirement. To debug the code, I need to look at the log file, but as the program is running, I don't see anything written there. Closing the program forcibly using ctrl+C does not help either. I see nothing written on the file.
I am using simple fopen and fprintf functions to read the file in write mode and write to it.
FILE *fp = fopen("filename.txt", "w");
fprintf(fp, "this wants itself to be written the moment this statement is executed\n");
PS: There is no bug in the code. If I put a terminating condition in while loop and program exits gracefully, I do see things written in the log file.
A difference between printing to a console and printing to a file is that streams are line buffered by default when attached to the console, but block buffered when attached to a file. Change your code to:
FILE *fp = fopen("filename.txt", "w");
setvbuf(fp,0,_IOLBF,0);
fprintf(fp, "this wants itself to be written the moment this statement is executed\n");
and your output will be line buffered even though the stream is attached to a file. You can also do unbuffered streams.
[EDIT: ]
Ref C11 7.21.5.6:
Synopsis
#include <stdio.h>
int setvbuf(FILE * restrict stream,
char * restrict buf,
int mode, size_t size);
Description
The setvbuf function may be used only after the stream pointed to by
stream has been associated with an open file and before any other
operation (other than an unsuccessful call to setvbuf) is performed on
the stream. The argument mode determines how stream will be buffered,
as follows: _IOFBF causes input/output to be fully buffered; _IOLBF
causes input/output to be line buffered; _IONBF causes input/output to
be unbuffered. If buf is not a null pointer, the array it points to
may be used instead of a buffer allocated by the setvbuf function
and the argument size specifies the size of the array; otherwise, size
may determine the size of a buffer allocated by the setvbuf function.
The contents of the array at any time are indeterminate.
Returns
The setvbuf function returns zero on success, or nonzero if an invalid
value is given for mode or if the request cannot be honored.
You should to see the the function fopen(),if you fopen a file with "w" mode,it means if this file exist,clear this file and then write.I think you should use "a+" mode to append data in the end.
Related
I would like to write a simple API which
allows the user to open a file.
let the user write data to the file
track the write calls and sanity check the written data after each write call.
prevents the data from beeing written to disk if it is not valid -> discard(file)
As a starting point i wrote the test program below, which opens a file in fully buffered "rb+" mode using fopen and setvbuf.
The stream is opened in fully buffered mode for the following reason:
http://www.cplusplus.com/reference/cstdio/setvbuf/
mode
Specifies a mode for file buffering.
Three special macro constants [...]:
_IOFBF Full buffering: On output, data is written once the buffer is full (or flushed). On Input, the buffer is filled when an input
operation is requested and the buffer is empty.
My testprogram contains comments where a validity check could be placed and where the buffer contents should be discarded.
My question is how do i accomplish the discard(file) operation which means the step of getting rid of invalid buffer contents ?
The idea behind this is to assemble some data in the buffer, do a regular validity check after each or several write operations and write the data to disk only, if the data is valid.
Therefore i would need to discard the buffer, if the validity check fails.
When the validity check passes, the whole buffer contents should be written to the file.
My code draft looks like in the following. This is a simplified example:
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
int main(void)
{
static uint8_t buffer[10000];
/* The following would be part of mylib_init */
FILE *file = fopen("test", "wb+");
if (file == NULL){
print ("open error!");
exit(-1);
}
if ( 0 != setvbuf(file , buffer, _IOFBF , sizeof(buffer) ) ){
print("Could not set buffer!");
fclose(file);
exit (-2);
}
/* The following would be part of mylib_write_data.
Each write and check resembles one func call */
// Pretend the user writes some data into the file
// ...
// fwrite(x)
if (data_in_buffer_not_valid(buffer)){
discard(file);
}
// ...
// fwrite(y)
//
if (data_in_buffer_not_valid(buffer)){
discard(file);
}
// ...
// fwrite(z)
// ...
// The following would be part of mylib_exit
// Cleanup stuff
fclose(file)
return 0;
}
If you want to have some like "scratch" temporary file that you want to write your data into and then retrieve them later, then the portable interface would be tmpfile() - it's an interface created just for that. Write to that file, rewind if you want, and when you're ready, rewind it and read from it block by block to another file.
On linux you may use fmemopen and fopencookie to write to a buffer via FILE* - these functions are not available on windows.
I would also strongly consider just creating your own interface that would store the result in memory. Writing an interface like struct mystream; mystream_init(struct mystream *); mystream_printf(struct mystream *, const char *fmt, ...); etc. is some of the tasks you sometimes do in C when fopencookie is not available. And consider writing the interface for storing data, so that instead of calling fwrite you would actually call the function that would check the data and write them and process them along the way.
As for setvbuf, note the standard. From C11 7.21.3p3:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered, [...]. When a stream is line buffered, [...] Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
And these buffering modes may just be not supported at all. And from C11 7.21.5.6:
The setvbuf function may be used only after the stream pointed to by stream has been associated with an open file and before any other operation (other than an unsuccessful call to setvbuf) is performed on the stream. [...] The contents of the array at any time are indeterminate.
You can't count on anything what will be the content of the buffer. Do not expecting any data there.
I have a function like this which aims to read a file:
int foo(FILE* f)
I want to use flock in order to prevent TOCTTOU. flock requires a file descriptor as an integer. I can get this using fileno(file). The implementation of foo therefore might look like this:
int foo(FILE* f) {
if(!f) return -1;
int fd = fileno(f);
if(fd < 0) return -1;
flock(fd, LOCK_EX);
//do all the reading stuff and so on.
}
However, the evil user might do something like this:
FILE* test;
test = fopen("someexistingfile.txt", "r");
fclose(test);
foo(test);
Then I have a problem because fileno will do invalid reads according to valgrind because it assumes that the file is open.
Any ideas on how to check whether the file is closed?
C11 n1570 7.21.3p4
A file may be disassociated from a controlling stream by closing the file. Output streams are flushed (any unwritten buffer contents are transmitted to the host environment) before the stream is disassociated from the file. The value of a pointer to a FILE object is indeterminate after the associated file is closed (including the standard text streams). Whether a file of zero length (on which no characters have been written by an output stream) actually exists is implementation-defined.
After fclose the use of the value of a FILE * in library functions leads to undefined behaviour. The value of the pointer cannot be used safely for anything at all until reassigned.
In other words, you cannot do really anything at all to discern whether the FILE * value you've given refers to a valid open file or not... well except for testing against NULL - if the value of the pointer is NULL it certainly cannot point to an open stream.
I'm basically trying to create a simple program that simulates a screen whose value is generated randomly and are outputted to a bmp file. However, there are some trouble in the file operation.. when the fwrite is called, the file stays at zero bytes, meaning nothing was written to that file.. I've tried changing the fopen modes (e.g "w+b", "wb+", "wb") but still no luck.. here is the code anyway
char resultFileName[BUFSIZ];
char currentDirectory[BUFSIZ];
//char generatedFileFolder[] = "\\generatedFile";
FILE *resultFile;
getcwd(currentDirectory, sizeof(currentDirectory));
snprintf(resultFileName, sizeof(resultFileName), "%s%s", currentDirectory, "\\result.bmp");
resultFile = fopen(resultFileName, "w+b");
BITMAPFILEHEADER bmfh;
BITMAPINFOHEADER bmih;
fwrite(&bmfh, sizeof(bmfh), 1, resultFile);
fwrite(&bmih, sizeof(bmih), 1, resultFile);
what am I doing wrong here?
You need to call fclose.
The I/O functions starting with f is buffered, which means they only write periodically (when reaching the maximum buffer size), or when you close it.
Since C is not a garbage-collected language (or language with the concept of destructor), your buffers are not flushed unless you flush it or close the file.
I am trying to learn the libuv api and wrote the following test:
#include <stdio.h>
#include <stdlib.h>
#include <uv.h>
void timer_cb(uv_timer_t* timer) {
int* i = timer->data;
--*i;
if(*i == 0) {
uv_timer_stop(timer);
}
printf("timer %d\n", *i);
//fflush(stdout);
}
int main() {
uv_loop_t* loop = uv_default_loop();
uv_timer_t* timer = malloc(sizeof(uv_timer_t));
uv_timer_init(loop, timer);
int i = 5;
timer->data = &i;
uv_timer_start(timer, timer_cb, 1000, 2000);
uv_run(loop, UV_RUN_DEFAULT);
printf("Now quitting.\n");
uv_close(timer, 0);
uv_loop_close(loop);
return 0;
}
When run it, no output is displayed until the program finishes running, and then all the output is displayed at once. If I uncomment the fflush line it works as expected, writing every 2 seconds.
Can someone please explain this to me? Why is stdout not flushed after the newline, as is explained here and in other places? Why do I need tomanually flush it?
Stream buffering is implementation-defined.
Per 7.21.3 Files, paragraph 3 of the C Standard:
When a stream is
unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters
may be accumulated and transmitted to or from the host
environment as a block. When a stream is
fully buffered, characters are intended to be transmitted to or from the host environment as a block when a buffer is filled. When a
stream is
line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line
character is encountered. Furthermore, characters are intended to be
transmitted as a block to the host environment when a buffer is
filled, when input is requested on an unbuffered stream, or when
input is requested on a line buffered stream that requires
the transmission of characters from the host
environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and
setvbuf functions.
The type of buffering is dependent on your implementation, and your implementation apparently isn't line-buffering in your example.
There is no strict requirement, that stdout is line buffered. It may be fully buffered as well (or not buffered at all), in which case \n does not trigger to flush the stream.
C11 (N1570) 7.21.3/7 Files:
As initially opened, the standard error stream is not fully buffered;
the standard input and standard output streams are fully buffered if
and only if the stream can be determined not to refer to an
interactive device.
C11 (N1570) 5.1.2.3/7 Program execution:
What constitutes an interactive device is implementation-defined.
You could try to force specific type of buffering by setvbuf standard function. For instance, to set line buffering for stdout, you may try with:
setvbuf(stdout, buff, _IOLBF, size);
where buff is declared as character array of size elements (e.g. 1024).
Note that setvbuf has to be called before any other I/O operation, that is performed to the stream.
For some reason, your system is deciding that your stdout is not interactive. Are you doing some strange redirect of stdout or doing something weird with your terminal? You should be able to override using setbuf or you can use stderr instead of stdout.
#include <unistd.h>
int main(int argc, char* argv[])
{
char buf[500];
read(0, buf, 5);
return 0;
}
The above read 5 characters from stdin,but if I input more than 5:
12345morethan5
[root# test]# morethan5
-bash: morethan5: command not found
The remaining characters will be executed as shell commands.
Is this kind of behavior defined by standard?
Sort of :-)
Your program reads 5 characters, and that's it. Not less, not more. The rest remain in the terminal buffer and get sent to your shell once your C program terminates.
Since you are using read(), which is a raw system call, instead of any of the C stdio buffering alternatives this behaviour is not just expected, but required.
From the POSIX standard on read():
The read() function shall attempt to
read nbyte bytes from the file
associated with the open file
descriptor, fildes, into the buffer
pointed to by buf.
...
Upon successful completion, where
nbyte is greater than 0, read() shall
mark for update the st_atime field of
the file, and shall return the number
of bytes read. This number shall never
be greater than nbyte.
...
Upon successful completion, read()
[XSI] [Option Start] and pread()
[Option End] shall return a
non-negative integer indicating the
number of bytes actually read.
I.e. read() should never read more bytes from the file descriptor than requested.
From the related part on terminals:
It is not, however, necessary to read
a whole line at once; any number of
bytes, even one, may be requested in a
read() without losing information.
...
The last process to close a terminal device file shall cause any output to be sent to the device and any input to be discarded.
Note: normally your shell will still have an open file descriptor for the terminal, until you end the session.
That has nothing to do with any standard, it's up to your runtime what to write to stdin. Your runtime makes the standard input available to your program, which reads some bytes from it and quits, and then the remaining bytes are processed by the runtime itself -- if you can configure it to clear all the file descriptors after forking a process, you could maybe prevent this behaviour, but that would seriously impede most of the standard command line workflows which rely on attaching one process's input to another process's output...