I have a doubt dealing with pointers in C.
During my program execution I must open and close a configuration file but since a different program updates its content i must do this several times while the program runs. So i made a little function which reads the file and closes it and i call it every time I need it.
int readConfigFile()
{
FILE* pfile = fopen("path to file", "r");
fseek(pFile, 0, SEEK_END);
lSize = ftell(pFile);
rewind(pFile);
text = (char*)malloc(sizeof(char) * lSize);
if (text == NULL) {
fputs("Memory error", stderr);
return (0);
}
if (fread(text, 1, lSize, pFile) <= 0) {
printf("Nothing read");
return 0;
}
text[lSize] = '\0';
//Do the actual reading
}
If i am not mistaken this functions creates a new FILE* pointer every time it runs however I was wondering if using a FILE* pointer with a larger scope and recycling it every time the function runs would be better. Meaning:
FILE* globalVaraible;
int readConfigFile(FILE* pFile)
{
*pfile = fopen("path to file", "r");
//Do some stuff here
}
I am worried that, since there is no garbage collector in C++ as far as i know, the first function will create too many FILE* pointers ,if runs for long enough, and overflow my memory. On the other hand having a global pointer, that will open and close in the context of a single function, seems rather wasteful and not very modular, at least for my point of view.
So should I recycle my pointer, or any reference for that matter, or is it okay to declare new ones as needed?
Thanks in advance.
You could use freopen().
http://en.cppreference.com/w/cpp/io/c/freopen
There is no guarantee that it will actually reuse dynamic memory allocated for the "old" file. This function may as well be a wrapper for fclose() and fopen() - it's all implementation defined.
From what I see, newlib (a popular libc for embedded targets) reuses the FILE storage, so there's a small gain here. Whether or not it makes a noticeable difference is another matter.
https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=newlib/libc/stdio/freopen.c;h=fb1f6c4db3f294f672d7e6998e8c69215b7482c1;hb=HEAD
Related
I debuged this many times. The program can create new file in new loation but the new file doesn't have content from the source file. The pointer s have no value. Please tell me why i got this problem and help me fix it.
So here is my function in source.cpp:
#include "Header.h"
void changeFileLoc(char filename[],char location[])
{
char newname[255];
sprintf(newname, "%s\\%s\0", location, filename);
FILE* fp = fopen(filename, "rb");
if (fp!=NULL)
{
char*s = (char*)malloc(sizeof(char)*sizeofFile(fp));
FILE* f = fopen(newname, "wb");
fread(s, sizeof(char)*sizeofFile(fp), 1, fp);
fwrite(s, sizeof(char)*sizeofFile(fp), 1, f);
fclose(fp);
fclose(f);
free(s);
}
}
long sizeofFile(FILE *f)
{
fseek(f, 0, SEEK_END);
long kq = ftell(f);
rewind(f);
return kq;
}
Here is my main:
#define _CRT_SECURE_NO_WARNINGS
#include "Header.h"
void main()
{
changeFileLoc("randombinary.txt", "D:\\Game");
_getch();
}
This is my header:
#ifndef __HEADER_H__
#define __HEADER_H__
#include <stdio.h>
#include <string.h>
#include <conio.h>
#include <malloc.h>
long sizeofFile(FILE *f);
int FileExist(char filename[]);
void changeFileLoc(char filename[], char location[]);
#endif
Though fragile, the code posted works on my machine. That said, it should be made more robust with error checking. Those checks would likely help you in quickly determining how it's going wrong in your case. Some other best practices would also make the code easier to understand (and thus easier to debug).
First, be aware that the method you're using to determine the size of the file isn't guaranteed to be portable by the language standards--the value returned by ftell is really only useful for seeking. That said, it will do the right thing on Linux. It will also work on Windows if the file is opened in binary mode (as it is in your program).
Next, naming things more clearly will help avoid mistakes. fp and f aren't very useful here. Names like source and target, src and dest, or oldfile and newfile would go a long way to avoiding mistakes. The variable kq seems to be meaningless unless it's an abbreviation for file size in some other language. Even the name changeFileLoc is misleading; copyFileToLoc would be more appropriate.
The guts of changeFileLoc are confusing because all the operations are interleaved: first it constructs the new file's name, then it opens the old file and creates a buffer to hold it, and then it opens the new file, then it reads the old file, then it writes to the new file. It's pretty scrambled. And it results in unnecessarily overlapping variable lifetimes, which makes it easier to write bugs.
There's another principle called "don't repeat yourself," which is often abbreviated to DRY. Things like computing the file size three times is not just inefficient, but it clutters the code, making it harder to see what's going on. Computing it once and saving it to a well-named variable will make this easier to understand.
Additional clutter includes an unnecessary cast, surplus string termination, and multiplication by 1 (which wouldn't be so bad if it happened in just one place).
So let's apply these principles to the code:
// Returns the size of the file in bytes, or 0 if the size cannot
// be determined.
long sizeofFile(FILE *f) {
if (fseek(f, 0, SEEK_END) != 0) {
return 0;
}
long size = ftell(f);
rewind(f);
return size;
}
void copyFileToLoc(char filename[], char location[]) {
FILE *source = fopen(filename, "rb");
if (source != NULL) {
size_t size = sizeofFile(source);
char *s = malloc(size);
if (s != NULL) {
fread(s, 1, size, source);
fclose(source);
char newname[255];
sprintf(newname, "%s\\%s", location, filename);
FILE *target = fopen(newname, "wb");
if (target != NULL) {
fwrite(s, 1, size, target);
fclose(target);
}
free(s);
}
}
}
This is better, but still not great. We've checked for some errors but not others (e.g., fread or fwrite could fail). Even so, we're already on our way to the arrow code anti-pattern (where the code gets heavily indented). It might be worth decomposing into smaller functions.
the posted code is opening a file for reading.
Then opening the SAME file for writing.
Suggest re-thinking the calls to fopen()
Also, any library function that you used (except rewind()) can fail. Strongly suggest adding the necessary code to ALWAYS check the returned values to look for errors
I have a legacy function accepting a FILE* pointer in a library. The contents I would like to parse is actually in memory, not on disk.
So I came up with the following steps to work around this issue:
the data is in memory at this point
fopen a temporary file (using tmpnam or tmpfile) on disk for writing
fclose the file
fopen the same file again for reading - guaranteed to exist
change the buffer using setvbuf(buffer, size)
do the legacy FILE* stuff
close the file
remove the temporary file
the data can be discarded
On windows, it looks like this:
int bufferSize;
char buffer[bufferSize];
// set up the buffer here
// temporary file name
char tempName [L_tmpnam_s];
tmpnam_s(tempName, L_tmpnam_s);
// open/close/reopen
fopen_s(&fp, tempName,"wb");
fclose(fp);
freopen_s(&fp, tempName,"rb", fp);
// replace the internal buffer
setvbuf(fp, buffer, _IONBF, bufferSize);
fp->_ptr = buffer;
fp->_cnt = bufferSize;
// do the FILE* reading here
// close and remove tmp file
fclose(fp);
remove(tempName);
Works, but quite cumbersome. The main problem, aside from the backwardness of this approach, are:
the temporary name needs to be determined
the temporary file is actually written to disk
the temporary file needs to be removed afterwards
I'd like to keep things portable, so using Windows memory-mapped functions or boost's facilities is not an option. The problem is mainly that, while it is possible to convert a FILE* to an std::fstream, the reverse seems to be impossible, or at least not supported on C++99.
All suggestions welcome!
Update 1
Using a pipe/fdopen/setvbuf as suggested by Speed8ump and a bit of twiddling seems to work. It does no longer create files on disk nor does it consume extra memory. One step closer, except, for some reason, setvbuf is not working as expected. Manually fixing it up is possible, but of course not portable.
// create a pipe for reading, do not allocate memory
int pipefd[2];
_pipe(pipefd, 0, _O_RDONLY | _O_BINARY);
// open the read pipe for binary reading as a file
fp = _fdopen(pipefd[0], "rb");
// try to switch the buffer ptr and size to our buffer, (no buffering)
setvbuf(fp, buffer, _IONBF, bufferSize);
// for some reason, setvbuf does not set the correct ptr/sizes
fp->_ptr = buffer;
fp->_charbuf = fp->_bufsiz = fp->_cnt = bufferSize;
Update 2
Wow. So it seems that unless I dive into the MS-specific implementation CreateNamedPipe / CreateFileMapping, POSIX portability costs us an entire memcopy (of any size!), be it to file or into a pipe. Hopefully the compiler understands that this is just a temporary and optimizes this. Hopefully.
Still, we eliminated the silly device writing intermediate. Yay!
int pipefd[2];
pipe(pipefd, bufferSize, _O_BINARY); // setting internal buffer size
FILE* in = fdopen(pipefd[0], "rb");
FILE* out = fdopen(pipefd[1], "wb");
// the actual copy
fwrite(buffer, 1, bufferSize, out);
fclose(out);
// fread(in), fseek(in), etc..
fclose(in);
You might try using a pipe and fdopen, that seems to be portable, is in-memory, and you might still be able to do the setvbuf trick you are using.
Your setvbuf hack is a nice idea, but not portable. C11 (n1570):
7.21.5.6 The setvbuf function
Synopsis
#include <stdio.h>
int setvbuf(FILE * restrict stream,
char * restrict buf,
int mode, size_t size);
Description
[...] If buf is not a null pointer, the array it points to may be used instead of a buffer allocated by the setvbuf function [...] and the argument size specifies the size of the array; otherwise, size may determine the size of a buffer allocated by the setvbuf function. The contents of the array at any time are indeterminate.
There is neither a guarantee that the provided buffer is used at all, nor about what it contains at any point after the setvbuf call until the file is closed or setvbuf is called again (POSIX doesn't give more guarantees).
The easiest portable solution, I think, is using tmpfile, fwrite the data into that file, fseek to the beginning (I'm not sure if temporary files are guaranteed to be seekable, on my Linux system, it appears they are, and I'd expect them to be elsewhere), and pass the FILE pointer to the function. This still requires copying in memory, but I guess usually no writing of the data to the disk (POSIX, unfortunately, implicitly requires a real file to exist). A file obtained by tmpfile is deleted after closing.
I have a code where I'm accessing a binary file several times. Each time I call the function, it opens the file for reading and it reads out only the required number of bytes (say n bytes each time).
The binary file contains time series data, and what I'd like it to be able to do is run the function call through a loop, and every time I call the function to open the same file, it needs to read out the next chunk of data, i.e, I don't want the file pointer to be reset every time. Is there a way to do this?
The function looks like so:
int readBinary(float *binImage, int gelements) {
imageFile = fopen("tmpImageFile", "r");
if (imageFile == NULL) {
fprintf(stderr, "Error opening file\n");
return (1);
}
fread(binImage, sizeof(float), gelements, imageFile);
return 0;
}
And in my main code, I'd like to run it through a loop, giving it the array binImage of size gelements every time. I'd rather not give it an array of size gelements * nLoop if that's avoidable.
The quick rules of thumb are
"Life gets easier if you properly separate responsibilities between functions"
and
"Life gets complicated if you use static or global variables".
In this case, giving the readBinary function both the responsibility of managing the FILE and reading the data is too much.
Note that the function doesn't close the handle.
If the handle is a local static in the function then it will be impossible to close it.
It also means that the function will be forever locked into using only "tmpImageFile" (which is not immediately apparent from the function's signature or missing documentation)
If the handle is a global then it may be possible to close it prematurely.
Notice that if you remove the "open file" responsibility the readBinary function is just a call to fread.
The best way to handle this is to skip the readBinary function altogether.
You say you have a caller which reads data in a loop.
Make this caller responsible for opening the FILE, use fread directly in the loop and close the file when you are done.
Now, this might give the caller too much responsibility. Simply have the caller accept the FILE* as a parameter and give the file management responsibility to its caller.
Or the caller's caller's caller, depending on where it makes sense to manage the file's lifetime.
Use a static variable so you retain the file pointer:
int readBinary(float *binImage, int gelements) {
static FILE *imageFile = NULL;
if (imageFile == NULL) {
imageFile = fopen("tmpImageFile", "r");
if (imageFile == NULL) {
perror("Error opening file: ");
return (1);
}
}
fread(binImage, sizeof(float), gelements, imageFile);
return 0;
}
I would pass the FILE* as parameter to function:
int readBinary(float *binImage, int gelements, FILE *imageFile) {
int bytes = fread(binImage, sizeof(float), gelements, imageFile);
return bytes != -1 ? 0 : 1;
}
I also added simple check for fread return value and converting it to your return value convention. Though this function seems so simple, you could just call fread directly, unless you want to add stuff like error print to it.
You can make the file pointer static and initialize it to NULL. Then next time onwards if it is not NULL, then it contains the opened file. Also while closing the file, make sure you make it to NULL again. Same can be done with a global pointer also
int readBinary(float *binImage, int gelements) {
static imageFile = NULL;
if(imageFile == NULL ) imageFile = fopen("tmpImageFile", "r");
if (imageFile == NULL) {
fprintf(stderr, "Error opening file\n");
return (1);
}
fread(binImage, sizeof(float), gelements, imageFile);
return 0;
}
All,
I'm using MapViewOfFile to hold part of a file in memory. There is a stream that points to this file and writes to it, and then is rewound. I use the pointer to the beginning of the mapped file, and read until I get to the null char I write as the final character.
int fd;
yyout = tmpfile();
fd = fileno(yyout);
#ifdef WIN32
HANDLE fm;
HANDLE h = (HANDLE) _get_osfhandle (fd);
fm = CreateFileMapping(
h,
NULL,
PAGE_READWRITE|SEC_RESERVE,
0,
4096,
NULL);
if (fm == NULL) {
fprintf (stderr, "%s: Couldn't access memory space! %s\n", argv[0], strerror (GetLastError()));
exit(GetLastError());
}
bp = (char*)MapViewOfFile(
fm,
FILE_MAP_ALL_ACCESS,
0,
0,
0);
if (bp == NULL) {
fprintf (stderr, "%s: Couldn't fill memory space! %s\n", argv[0], strerror (GetLastError()));
exit(GetLastError());
}
Data is sent to the yyout stream, until flushData() is called. This writes a null to the stream, flushes, and then rewinds the stream. Then I start from the beginning of the mapped memory, and read chars until I get to the null.
void flushData(void) {
/* write out data in the stream and reset */
fprintf(yyout, "%c%c%c", 13, 10, '\0');
fflush(yyout);
rewind(yyout);
if (faqLine == 1) {
faqLine = 0; /* don't print faq's to the data file */
}
else {
char * ps = bp;
while (*ps != '\0') {
fprintf(outstream, "%c%c", *ps, blank);
ps++;
}
fflush(outfile);
}
fflush(yyout);
rewind(yyout);
}
After flushing, more data is written to the stream, which should be set to the start of the memory area. As near as I can determine with gdb, the stream is not getting rewound, and eventually fills up the allocated space.
Since the stream points to the underlying file, this does not cause a problem initially. But, when I attempt to walk the memory, I never find the null. This leads to a SIGSEV. If you want more details of why I need this, see here.
Why am I not reusing the memory space as expected?
I think this line from the MSDN documentation for CreateFileMapping might be the clue.
A mapped file and a file that is accessed by using the input and output (I/O) functions (ReadFile and WriteFile) are not necessarily coherent.
You're not apparently using Read/WriteFile, but the documentation should be understood in terms of mapped views versus explicit I/O calls. In any case, the C RTL is surely implemented using the Win32 API.
In short, this approach is problematic.
I don't know why changing the view/file size helps; perhaps it just shifts the undefined behaviour in a direction that happens to be beneficial.
Well, after working on this for a while, I have a working solution. I don't know why this succeeds, so if someone comes up with something better, I'll be happy to accept their answer instead.
fm = CreateFileMapping(
h,
NULL,
PAGE_READWRITE|SEC_RESERVE,
0,
16384,
NULL);
As you can see, the only change is to the size declared from 4096 to 16384. Why this works when the total chars input at a time is no more than 1200, I don't know. If someone could provide details on this, I would appreciate it.
When you're done with the map, simply un-map it.
UnmapViewOfFile(bp);
I'm looking at some legacy Linux code which uses pthreads.
In one thread a file is read via fgets(). The FILE variable is a global variable shared across all threads. (Hey, I didn't write this...)
In another thread every now and again the FILE is closed and reopened with another filename.
For several seconds after this has happened, the thread fgets() acts as if it is continuing to read the last record it read from the previous file: almost as if there was an error but fgets() was not returning NULL. Then it sorts itself out and starts reading from the new file.
The code looks a bit like this (snipped for brevity so I hope it's still intelligible):
In one thread:
while(gRunState != S_EXIT){
nanosleep(&timer_delay,0);
flag = fgets(buff, sizeof(buff), gFile);
if (flag != NULL){
// do something with buff...
}
}
In the other thread:
fclose(gFile);
gFile = fopen(newFileName,"r");
There's no lock to make sure that the fgets() is not called at the same time as the fclose()/fopen().
Any thoughts as to failure modes which might cause fgets() to fail but not return NULL?
How the described code goes wrong
The stdio library buffers data, allocating memory to store the buffered data. The GNU C library dynamically allocates file structures (some libraries, notably on Solaris, use pointers to statically allocated file structures, but the buffer is still dynamically allocated unless you set the buffering otherwise).
If your thread works with a copy of a pointer to the global file pointer (because you passed the file pointer to the function as an argument), then it is conceivable that the code would continue to access the data structure that was orginally allocated (even though it was freed by the close), and would read data from the buffer that was already present. It would only be when you exit the function, or read beyond the contents of the buffer, that things start going wrong - or the space that was previously allocated to the file structure is reallocated for a new use.
FILE *global_fp;
void somefunc(FILE *fp, ...)
{
...
while (fgets(buffer, sizeof(buffer), fp) != 0)
...
}
void another_function(...)
{
...
/* Pass global file pointer by value */
somefunc(global_fp, ...);
...
}
Proof of Concept Code
Tested on MacOS X 10.5.8 (Leopard) with GCC 4.0.1:
#include <stdio.h>
#include <stdlib.h>
FILE *global_fp;
const char etc_passwd[] = "/etc/passwd";
static void error(const char *fmt, const char *str)
{
fprintf(stderr, fmt, str);
exit(1);
}
static void abuse(FILE *fp, const char *filename)
{
char buffer1[1024];
char buffer2[1024];
if (fgets(buffer1, sizeof(buffer1), fp) == 0)
error("Failed to read buffer1 from %s\n", filename);
printf("buffer1: %s", buffer1);
/* Dangerous!!! */
fclose(global_fp);
if ((global_fp = fopen(etc_passwd, "r")) == 0)
error("Failed to open file %s\n", etc_passwd);
if (fgets(buffer2, sizeof(buffer2), fp) == 0)
error("Failed to read buffer2 from %s\n", filename);
printf("buffer2: %s", buffer2);
}
int main(int argc, char **argv)
{
if (argc != 2)
error("Usage: %s file\n", argv[0]);
if ((global_fp = fopen(argv[1], "r")) == 0)
error("Failed to open file %s\n", argv[1]);
abuse(global_fp, argv[1]);
return(0);
}
When run on its own source code, the output was:
Osiris JL: ./xx xx.c
buffer1: #include <stdio.h>
buffer2: ##
Osiris JL:
So, empirical proof that on some systems, the scenario I outlined can occur.
How to fix the code
The fix to the code is discussed well in other answers. If you avoid the problem I illustrated (for example, by avoiding global file pointers), that is simplest. Assuming that is not possible, it may be sufficient to compile with the appropriate flags (on many Unix-like systems, the compiler flag '-D_REENTRANT' does the job), and you will end up using thread-safe versions of the basic standard I/O functions. Failing that, you may need to put explicit thread-safe management policies around the access to the file pointers; a mutex or something similar (and modify the code to ensure that the threads use the mutex before using the corresponding file pointer).
A FILE * is just a pointer to the various resources. If the fclose does not zero out those resource, it's possible that the values may make enough sense that fgets does not immediately notice it.
That said, until you add some locking, I would consider this code completely broken.
Umm, you really need to control access to the FILE stream with a mutex, at the minimum. You aren't looking at some clever implementation of lock free methods, you are looking at really bad (and dusty) code.
Using thread local FILE streams is the obvious and most elegant fix, just use locks appropriately to ensure no two threads operate on the same offset of the same file at once. Or, more simply, ensure that threads block (or do other work) while waiting for the file lock to clear. POSIX advisory locks would be best for this, or your dealing with dynamically growing a tree of mutexes... or initializing a file lock mutex per thread and making each thread check the other's lock (yuck!) (since files can be re-named).
I think you are staring down the barrel of some major fixes .. unfortunately (from what you have indicated) there is no choice but to make them. In this case, its actually easier to debug a threaded program written in this manner than it would be to debug something using forks, consider yourself lucky :)
You can also put some condition-wait (pthread_cond_wait) instead of just some nanosleep which will get signaled when intended e.g. when a new file gets fopened.