I often use the website www.cplusplus.com as a reference when writing C code.
I was reading the example cited on the page for fread and had a question.
As an example they post:
/* fread example: read a complete file */
#include <stdio.h>
#include <stdlib.h>
int main () {
FILE * pFile;
long lSize;
char * buffer;
size_t result;
pFile = fopen ( "myfile.bin" , "rb" );
if (pFile==NULL) {fputs ("File error",stderr); exit (1);}
// obtain file size:
fseek (pFile , 0 , SEEK_END);
lSize = ftell (pFile);
rewind (pFile);
// allocate memory to contain the whole file:
buffer = (char*) malloc (sizeof(char)*lSize);
if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}
// copy the file into the buffer:
result = fread (buffer,1,lSize,pFile);
if (result != lSize) {fputs ("Reading error",stderr); exit (3);}
/* the whole file is now loaded in the memory buffer. */
// terminate
fclose (pFile);
free (buffer);
return 0;
}
It seems to me that that if result != lSize, then free(buffer) will never get called. Would this be a memory leak in this example?
I have always thought the examples on their site are of a very high quality. Perhaps I am not understanding correctly?
It wouldn't be a memory leak in this example, because terminating the program (by calling exit()) frees all memory associated with it.
However, it would be a memory leak if you used this piece of code as a subroutine and called something like return 1; in place of exit().
Technically, yes it is a memory leak. But any memory allocated by a process is automatically freed when that process terminates, so in this example the calls to free (and fclose) are not really required.
In a more complex program, this would probably be a real problem. The missing free would create a memory leak and the missing fclose would cause a resource leak.
The operating system cleans up any unfreed memory by a process when that process closes. At least, modern operating systems do.
If the program were not exiting at the point result != lSize, that is, it continued with some other path of execution, then yes - it is a guaranteed memory leak.
There are two possible paths.
(1) result != lSize - in this case, exit(0) is called. This kills the process and the operating system will clean up the memory.
(2) result == lsize - in this case, the buffer is explicitly freed, but return is called right afterwards so the free is mostly just good style because this also kills the process and the operating system will, again, clean up the memory.
So in this simple case, there is no memory leak. But it is probably a good practice to just make sure you're freeing any memory you've allocated in any application you write. Getting into this habit will prevent many headaches for you in the future.
As to possible memory leakage, other's have already answered that question. A while ago, I posted a variation of the given code which should handle all possible error conditions correctly:
fsize()
fget_contents()
Related
I have use the code like below
char *es_data;
fp_input = fopen(inp_path, "rb");
fseek(fp_input, 0, SEEK_END);
file_size = ftell(fp_input);
fseek(fp_input, 0, SEEK_SET);
es_data = (char*)malloc(file_size);
fread(es_data, 1, file_size, fp_input);
I have a file of 185mb, i.e., file_size = 190108334 bytes. For this file, malloc is crashing, and program is getting struck at this stage.
If i use any other file of lower size, it works fine.
What can I do ?
You should test at least that fopen succeeds:
fp_input = fopen(inp_path, "rb");
if (!fp_input) { popen(inp_path); exit(EXIT_FAILURE); };
Your malloc is probably not crashing, but failing (by returning NULL): read malloc(3) so code at least:
es_data = malloc(file_size);
if (!es_data) { perror("malloc"); exit(EXIT_FAILURE); }
BTW, you probably want to memory map a file. If you are on Linux or a Posix system, learn about mmap(2) (and use fstat(2) to query the size of an open(2)-ed file descriptor). Windows can also memory map a file with CreateFileMapping
If your malloc is indeed crashing this probably means that you have memory corruption (before that malloc call) so some internal invariant of your system malloc is violated. Use some memory debugger tool (like valgrind on Linux, or purify on Windows) to detect it.
I'm writing a buffer into a binary file. Code is as in the following :
FILE *outwav = fopen(outwav_path, "wb");
if(!outwav)
{
fprintf(stderr, "Can't open file %s for writing.\n", outwav_path);
exit(1);
}
[...]
//Create sample buffer
short *samples = malloc((loopcount*(blockamount-looppos)+looppos) << 5);
if(!samples)
{
fprintf(stderr, "Error : Can't allocate memory.\n");
exit(1);
}
[...]
fwrite(samples, 2, 16*k, outwav); //write samplebuffer to file
fflush(outwav);
fclose(outwav);
free(samples);
The last free() call causes me random segfaults.
After several headaches I thought it was probably because the fwrite call would execute only after a delay, and then it would read freed memory. So I added the fflush call, yet, the problem STILL occurs.
The only way to get rid of it is to not free the memory and let the OS do it for me. This is supposed to be bad practice though, so I'd rather ask if there is no better solution.
Before anyone asks, yes I check that the file is opened correctly, and yes I test that the memory is allocated properly, and no, I don't touch the returned pointers in any way.
Once fwrite returns you are free to do whatever you want with the buffer. You can remove the fflush call.
It sounds like a buffer overflow error in a totally unrelated part of the program is writing over the book-keeping information that free needs to do its work. Run your program under a tool like valgrind to find out if this is the problem and to find the part of the program that has a buffer overflow.
All,
I'm using MapViewOfFile to hold part of a file in memory. There is a stream that points to this file and writes to it, and then is rewound. I use the pointer to the beginning of the mapped file, and read until I get to the null char I write as the final character.
int fd;
yyout = tmpfile();
fd = fileno(yyout);
#ifdef WIN32
HANDLE fm;
HANDLE h = (HANDLE) _get_osfhandle (fd);
fm = CreateFileMapping(
h,
NULL,
PAGE_READWRITE|SEC_RESERVE,
0,
4096,
NULL);
if (fm == NULL) {
fprintf (stderr, "%s: Couldn't access memory space! %s\n", argv[0], strerror (GetLastError()));
exit(GetLastError());
}
bp = (char*)MapViewOfFile(
fm,
FILE_MAP_ALL_ACCESS,
0,
0,
0);
if (bp == NULL) {
fprintf (stderr, "%s: Couldn't fill memory space! %s\n", argv[0], strerror (GetLastError()));
exit(GetLastError());
}
Data is sent to the yyout stream, until flushData() is called. This writes a null to the stream, flushes, and then rewinds the stream. Then I start from the beginning of the mapped memory, and read chars until I get to the null.
void flushData(void) {
/* write out data in the stream and reset */
fprintf(yyout, "%c%c%c", 13, 10, '\0');
fflush(yyout);
rewind(yyout);
if (faqLine == 1) {
faqLine = 0; /* don't print faq's to the data file */
}
else {
char * ps = bp;
while (*ps != '\0') {
fprintf(outstream, "%c%c", *ps, blank);
ps++;
}
fflush(outfile);
}
fflush(yyout);
rewind(yyout);
}
After flushing, more data is written to the stream, which should be set to the start of the memory area. As near as I can determine with gdb, the stream is not getting rewound, and eventually fills up the allocated space.
Since the stream points to the underlying file, this does not cause a problem initially. But, when I attempt to walk the memory, I never find the null. This leads to a SIGSEV. If you want more details of why I need this, see here.
Why am I not reusing the memory space as expected?
I think this line from the MSDN documentation for CreateFileMapping might be the clue.
A mapped file and a file that is accessed by using the input and output (I/O) functions (ReadFile and WriteFile) are not necessarily coherent.
You're not apparently using Read/WriteFile, but the documentation should be understood in terms of mapped views versus explicit I/O calls. In any case, the C RTL is surely implemented using the Win32 API.
In short, this approach is problematic.
I don't know why changing the view/file size helps; perhaps it just shifts the undefined behaviour in a direction that happens to be beneficial.
Well, after working on this for a while, I have a working solution. I don't know why this succeeds, so if someone comes up with something better, I'll be happy to accept their answer instead.
fm = CreateFileMapping(
h,
NULL,
PAGE_READWRITE|SEC_RESERVE,
0,
16384,
NULL);
As you can see, the only change is to the size declared from 4096 to 16384. Why this works when the total chars input at a time is no more than 1200, I don't know. If someone could provide details on this, I would appreciate it.
When you're done with the map, simply un-map it.
UnmapViewOfFile(bp);
Whenever I run my program with fclose(outputFile); at the very end, I get an error. glibc detected...corrupted double-linked list
The confusing thing about this though, is that I have fclose(inputFile); directly above it and it works fine. Any suggestions?
FILE* inputFile = fopen(fileName, "r");
if (inputFile == NULL)
{
printf("inputFile did not open correctly.\n");
exit(0);
}
FILE* outputFile = fopen("output.txt", "wb");
if (outputFile == NULL)
{
printf("outputFile did not open correctly.\n");
exit(0);
}
/* ... read in inputFile ... */
/* ... some fprintf's to outputFile ... */
fclose(inputFile);
fclose(outputFile);
The problem is likely located in this section:
/* ... read in inputFile ... */
You've got some code there that corrupts the heap. Overflowing an array is the typical cause. Heap corruption is rarely detected at the time the corruption happens. Only later, when some code allocates or releases memory and has some basic heap health verification built in. Like fclose() does.
To detect exactly where your code is corrupting the heap, if you are running on Linux, you should use valgrind. It's easy to use:
valgrind ./myprog arguments ...
and will give you a stack trace from the exact point where a bad read or write occurs.
Valgrind is available from major Linux distributions or you can build from source.
I'm looking at some legacy Linux code which uses pthreads.
In one thread a file is read via fgets(). The FILE variable is a global variable shared across all threads. (Hey, I didn't write this...)
In another thread every now and again the FILE is closed and reopened with another filename.
For several seconds after this has happened, the thread fgets() acts as if it is continuing to read the last record it read from the previous file: almost as if there was an error but fgets() was not returning NULL. Then it sorts itself out and starts reading from the new file.
The code looks a bit like this (snipped for brevity so I hope it's still intelligible):
In one thread:
while(gRunState != S_EXIT){
nanosleep(&timer_delay,0);
flag = fgets(buff, sizeof(buff), gFile);
if (flag != NULL){
// do something with buff...
}
}
In the other thread:
fclose(gFile);
gFile = fopen(newFileName,"r");
There's no lock to make sure that the fgets() is not called at the same time as the fclose()/fopen().
Any thoughts as to failure modes which might cause fgets() to fail but not return NULL?
How the described code goes wrong
The stdio library buffers data, allocating memory to store the buffered data. The GNU C library dynamically allocates file structures (some libraries, notably on Solaris, use pointers to statically allocated file structures, but the buffer is still dynamically allocated unless you set the buffering otherwise).
If your thread works with a copy of a pointer to the global file pointer (because you passed the file pointer to the function as an argument), then it is conceivable that the code would continue to access the data structure that was orginally allocated (even though it was freed by the close), and would read data from the buffer that was already present. It would only be when you exit the function, or read beyond the contents of the buffer, that things start going wrong - or the space that was previously allocated to the file structure is reallocated for a new use.
FILE *global_fp;
void somefunc(FILE *fp, ...)
{
...
while (fgets(buffer, sizeof(buffer), fp) != 0)
...
}
void another_function(...)
{
...
/* Pass global file pointer by value */
somefunc(global_fp, ...);
...
}
Proof of Concept Code
Tested on MacOS X 10.5.8 (Leopard) with GCC 4.0.1:
#include <stdio.h>
#include <stdlib.h>
FILE *global_fp;
const char etc_passwd[] = "/etc/passwd";
static void error(const char *fmt, const char *str)
{
fprintf(stderr, fmt, str);
exit(1);
}
static void abuse(FILE *fp, const char *filename)
{
char buffer1[1024];
char buffer2[1024];
if (fgets(buffer1, sizeof(buffer1), fp) == 0)
error("Failed to read buffer1 from %s\n", filename);
printf("buffer1: %s", buffer1);
/* Dangerous!!! */
fclose(global_fp);
if ((global_fp = fopen(etc_passwd, "r")) == 0)
error("Failed to open file %s\n", etc_passwd);
if (fgets(buffer2, sizeof(buffer2), fp) == 0)
error("Failed to read buffer2 from %s\n", filename);
printf("buffer2: %s", buffer2);
}
int main(int argc, char **argv)
{
if (argc != 2)
error("Usage: %s file\n", argv[0]);
if ((global_fp = fopen(argv[1], "r")) == 0)
error("Failed to open file %s\n", argv[1]);
abuse(global_fp, argv[1]);
return(0);
}
When run on its own source code, the output was:
Osiris JL: ./xx xx.c
buffer1: #include <stdio.h>
buffer2: ##
Osiris JL:
So, empirical proof that on some systems, the scenario I outlined can occur.
How to fix the code
The fix to the code is discussed well in other answers. If you avoid the problem I illustrated (for example, by avoiding global file pointers), that is simplest. Assuming that is not possible, it may be sufficient to compile with the appropriate flags (on many Unix-like systems, the compiler flag '-D_REENTRANT' does the job), and you will end up using thread-safe versions of the basic standard I/O functions. Failing that, you may need to put explicit thread-safe management policies around the access to the file pointers; a mutex or something similar (and modify the code to ensure that the threads use the mutex before using the corresponding file pointer).
A FILE * is just a pointer to the various resources. If the fclose does not zero out those resource, it's possible that the values may make enough sense that fgets does not immediately notice it.
That said, until you add some locking, I would consider this code completely broken.
Umm, you really need to control access to the FILE stream with a mutex, at the minimum. You aren't looking at some clever implementation of lock free methods, you are looking at really bad (and dusty) code.
Using thread local FILE streams is the obvious and most elegant fix, just use locks appropriately to ensure no two threads operate on the same offset of the same file at once. Or, more simply, ensure that threads block (or do other work) while waiting for the file lock to clear. POSIX advisory locks would be best for this, or your dealing with dynamically growing a tree of mutexes... or initializing a file lock mutex per thread and making each thread check the other's lock (yuck!) (since files can be re-named).
I think you are staring down the barrel of some major fixes .. unfortunately (from what you have indicated) there is no choice but to make them. In this case, its actually easier to debug a threaded program written in this manner than it would be to debug something using forks, consider yourself lucky :)
You can also put some condition-wait (pthread_cond_wait) instead of just some nanosleep which will get signaled when intended e.g. when a new file gets fopened.