I have the following code that is attempting to read a text file, make a backup, and pass the read file string on to further processing routines. The behaviour I am seeing is very unexpected.
01: rewind(PPLFile);
02: fseek(PPLFile, 0, SEEK_END);
03: unsigned long fsize = ftell(PPLFile);
04: char *string = (char*)calloc(fsize + 1, sizeof(char));
05: rewind(PPLFile);
06: fread(string, sizeof(char), fsize, PPLFile);
07: FILE* PPLBackup;
08: char* fileSuffix = ".backup";
09: char* PPLBackupLocation = (char*)calloc(strlen(basePath) + strlen(fileSuffix) + 1, sizeof(char));
10: strcpy(PPLBackupLocation, basePath);
11: strcat(PPLBackupLocation, fileSuffix);
12: PPLBackup = fopen(PPLBackupLocation, "w");
13: fprintf(PPLBackup, fileContent);
14: fclose(PPLBackup);
15: free(PPLBackupLocation);
The file handle *PPLFile has been opened previously with the w+ or a+ flag depending on certain cases in the previous code that wrote the file initially.
The concept I thought was quite simple:
Move the pointer past the end of the file and sense its location to establish file size in bytes
Create null string of the byte size plus one (for the null terminator)
Rewind and read the whole file content into the string, up to the sensed file size
Create backup file name and file pointer
Write the whole string read from the original file to the new backup file.
Close the backup file and continue by processing the read string
There are a two disturbing symptoms:
The *char string containing the original file has many more zeroes (thousands in a text file ~=20MB), so the last non-zero value might be 3000 bytes from the end. Seemingly smaller than the original file.
When writing the backup file, the resulting file is a small amount larger than the original file, with a seemingly random extra sample of the original file attached to the end, again a few thousand bytes.
Quite simply, what on earth is going on?
This row not good:
13: fprintf(PPLBackup, fileContent);
If the file contains character sequences like %s then fprintf will look for additional data on the stack.
You should use fwrite instead:
13: fwrite(fileContent, fsize, 1, PPLBackup);
or at least do this:
13: fprintf(PPLBackup, "%s", fileContent);
Why are you reading with fread() and writing with fprintf() instead of fwrite()? And the way you're using fprintf(), you're at the mercy of the first % sign in the file (format string vulnerability).
Also, make sure both files are open in the same mode (text mode vs. binary mode). Usually, fread()/fwrite() tend to be used with binary mode.
This complete program works for me on Linux. It has the alteration by Klas. Note I had to use fsize as a global because passing around the char * contents of the file to be copied broke if it was later strlen'd - for obvious reasons
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
unsigned long fsize=0L;
char *get_file(FILE *PPLFile) {
rewind(PPLFile);
fseek(PPLFile, 0, SEEK_END);
fsize = ftell(PPLFile);
char *string = (char*)calloc(fsize + 1, sizeof(char));
rewind(PPLFile);
fread(string, sizeof(char), fsize, PPLFile);
return string;
}
void write_file(char *basePath, char *fileContent) {
FILE* PPLBackup;
char* fileSuffix = ".backup";
char* PPLBackupLocation = (char*)calloc(strlen(basePath) + strlen(fileSuffix) + 1, sizeof(char));
strcpy(PPLBackupLocation, basePath);
strcat(PPLBackupLocation, fileSuffix);
PPLBackup = fopen(PPLBackupLocation, "w");
fwrite(fileContent, fsize, 1, PPLBackup);
fclose(PPLBackup);
free(PPLBackupLocation);
}
Multiple issues
As mentioned by #Klas Lindbäck & #Medinoc, do not use fprintf(). fwrite() is much better.
#Klas Lindbäck says, insure that you open both the file read and write in binary mode. as in fopen(PPLBackupLocation, "wb").
The return value from fread() should be assessed and if good, used for your fwrite().
calloc(fsize + 1, sizeof(char)) does not need to add 1.
Related
We have library which accepts FILE* (CImg). For performance reason we wish to handle data already in memory without accessing a disk. Target platform is windows which unfortunately does not support fmemopen (and funopen)
char* buf = new char[sz];
FILE *fp = fopen("C:\\test.dat", "wb");
int r = setvbuf(fp, buf, _IOFBF, sz);
r = fwrite(src_buf, 1, sz, fp); // Here r contains right size
fseek(fp, 0, SEEK_END);
size_t sz2 = ftell(fp); // sz2 contains right size as well
rewind(fp);
// Test (something like this actually is somewhere deep inside library)
char* read_buf = new char[sz];
r = fread(read_buf, 1, sz, fp); // Zero!
Final fread() can't read anything... Any suggestions?
Possibly because you opened with "wb" (write-only) instead of "wb+" (read + write)
fopen() function
To help further, you can #include errno.h and print out the error code and string.
printf( "errno: %d, error:'%s'\n", errno, strerror( errno ) );
For our specific case we found CImg plugin doing the thing.
As of a sample in an original question - at least MSVC runtime flushes written content to disk itself. So it isn't a valid replace for fmemopen()
https://github.com/Snaipe/fmem is a wrapper for different platform/version specific implementations of in-memory files
It tries in sequence the following implementations:
open_memstream.
fopencookie, with growing dynamic buffer.
funopen, with growing dynamic buffer.
WinAPI temporary memory-backed file.
When no other mean is available, fmem falls back to tmpfile()
I am working on an assignment in socket programming in which I have to send a file between sparc and linux machine. Before sending the file in char stream I have to get the file size and tell the client. Here are some of the ways I tried to get the size but I am not sure which one is the proper one.
For testing purpose, I created a file with content " test" (space + (string)test)
Method 1 - Using fseeko() and ftello()
This is a method I found on https://www.securecoding.cert.org/confluence/display/c/FIO19-C.+Do+not+use+fseek()+and+ftell()+to+compute+the+size+of+a+regular+file
While the fssek() has a problem of "Setting the file position indicator to end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior for a binary stream", fseeko() is said to have tackled this problem but it only works on POSIX system (which is fine because the environment I am using is sparc and linux)
fd = open(file_path, O_RDONLY);
fp = fopen(file_path, "rb");
/* Ensure that the file is a regular file */
if ((fstat(fd, &st) != 0) || (!S_ISREG(st.st_mode))) {
/* Handle error */
}
if (fseeko(fp, 0 , SEEK_END) != 0) {
/* Handle error */
}
file_size = ftello(fp);
fseeko(fp, 0, SEEK_SET);
printf("file size %zu\n", file_size);
This method works fine and get the size correctly. However, it is limited to regular files only. I tried to google the term "regular file" but I still not quite understand it thoroughly. And I do not know if this function is reliable for my project.
Method 2 - Using strlen()
Since the max. size of a file in my project is 4MB, so I can just calloc a 4MB buffer. After that, the file is read into the buffer, and I tried to use the strlen to get the file size (or more correctly the length of content). Since strlen() is portable, can I use this method instead? The code snippet is like this
fp = fopen(file_path, "rb");
fread(file_buffer, 1024*1024*4, 1, fp);
printf("strlen %zu\n", strlen(file_buffer));
This method works too and returns
strlen 8
However, I couldn't see any similar approach on the Internet using this method. So I am thinking maybe I have missed something or there are some limitations of this approach which I haven't realized.
Regular file means that it is nothing special like device, socket, pipe etc. but "normal" file.
It seems that by your task description before sending you must retrieve size of normal file.
So your way is right:
FILE* fp = fopen(...);
if(fp) {
fseek(fp, 0 , SEEK_END);
long fileSize = ftell(fp);
fseek(fp, 0 , SEEK_SET);// needed for next read from beginning of file
...
fclose(fp);
}
but you can do it without opening file:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
struct stat buffer;
int status;
status = stat("path to file", &buffer);
if(status == 0) {
// size of file is in member buffer.st_size;
}
OP can do it the easy way as "max. size of a file in my project is 4MB".
Rather than using strlen(), use the return value from fread(). stlen() stops on the first null character, so may report too small a value. #Sami Kuhmonen Also we do not know the data read contains any null character, so it may not be a string. Append a null character (and allocate +1) if code needs to use data as a string. But in that case, I'd expect the file needed to be open in text mode.
Note that many OS's do not even use allocated memory until it is written.
Why is malloc not "using up" the memory on my computer?
fp = fopen(file_path, "rb");
if (fp) {
#define MAX_FILE_SIZE 4194304
char *buf = malloc(MAX_FILE_SIZE);
if (buf) {
size_t numread = fread(buf, sizeof *buf, MAX_FILE_SIZE, fp);
// shrink if desired
char *tmp = realloc(buf, numread);
if (tmp) {
buf = tmp;
// Use buf with numread char
}
free(buf);
}
fclose(fp);
}
Note: Reading the entire file into memory may not be the best idea to begin with.
I am currently working on a project in which I have to read from a binary file and send it through sockets and I am having a hard time trying to send the whole file.
Here is what I wrote so far:
FILE *f = fopen(line,"rt");
//size = lseek(f, 0, SEEK_END)+1;
fseek(f, 0L, SEEK_END);
int size = ftell(f);
unsigned char buffer[MSGSIZE];
FILE *file = fopen(line,"rb");
while(fgets(buffer,MSGSIZE,file)){
sprintf(r.payload,"%s",buffer);
r.len = strlen(r.payload)+1;
res = send_message(&r);
if (res < 0) {
perror("[RECEIVER] Send ACK error. Exiting.\n");
return -1;
}
}
I think it has something to do with the size of the buffer that I read into,but I don't know what it's the correct formula for it.
One more thing,is the sprintf done correctly?
If you are reading binary files, a NUL character may appear anywhere in the file.
Thus, using string functions like sprintf and strlen is a bad idea.
If you really need to use a second buffer (buffer), you could use memcpy.
You could also directly read into r.payload (if r.payload is already allocated with sufficient size).
You are looking for fread for a binary file.
The return value of fread tells you how many bytes were read into your buffer.
You may also consider to call fseek again.
See here How can I get a file's size in C?
Maybe your code could look like this:
#include <stdint.h>
#include <stdio.h>
#define MSGSIZE 512
struct r_t {
uint8_t payload[MSGSIZE];
int len;
};
int send_message(struct r_t *t);
int main() {
struct r_t r;
FILE *f = fopen("test.bin","rb");
fseek(f, 0L, SEEK_END);
size_t size = ftell(f);
fseek(f, 0L, SEEK_SET);
do {
r.len = fread(r.payload, 1, sizeof(r.payload), f);
if (r.len > 0) {
int res = send_message(&r);
if (res < 0) {
perror("[RECEIVER] Send ACK error. Exiting.\n");
fclose(f);
return -1;
}
}
} while (r.len > 0);
fclose(f);
return 0;
}
No, the sprintf is not done correctly. It is prone to buffer overflow, a very serious security problem.
I would consider sending the file as e.g. 1024-byte chunks instead of as line-by-line, so I would replace the fgets call with an fread call.
Why are you opening the file twice? Apparently to get its size, but you could open it only once and jump back to the beginning of the file. And, you're not using the size you read for anything.
Is it a binary file or a text file? fgets() assumes you are reading a text file -- it stops on a line break -- but you say it's a binary file and open it with "rb" (actually, the first time you opened it with "rt", I assume that was a typo).
IMO you should never ever use sprintf. The number of characters written to the buffer depends on the parameters that are passed in, and in this case if there is no '\0' in buffer then you cannot predict how many bytes will be copied to r.payload, and there is a very good chance you will overflow that buffer.
I think sprintf() would be the first thing to fix. Use memcpy() and you can tell it exactly how many bytes to copy.
I have a wav file named "file.wav", and I want to read some specification of the file with a C program. I found the structure of wav file here. According to this document, in a normal wav file, the AudioFormat bytes should be 0x1 and 0x0, or the NumChannels bytes should be (if mono) 0x1 and 0x0; there are lots of similar sections that should look like these.
Now, my C program is very easy: I open the file in binary mode, I seek to the AudioFormat section (the 20th byte), I read two bytes and I put them in a buffer; then I print to stdin the buffer content.
#include <stdio.h>
main()
{
void *buf[2];
FILE *f;
f=fopen("file.wav", "rb");
fseek(f, 20, SEEK_SET);
fread(buf, 1, 2, f);
printf("example: %#hx %#hx\n", buf[0], buf[1]);
/*the '#' flag stands for the 0x[···] format output,
the 'h' says that the number is a short integer (2 bytes).*/
return 0;
}
My problem is that the second byte instead of being 0x0 is always 0x4e0, and this for every byte which should be 0 (for example for NumChannels or BitPerSamples bytes too). What is the problem?
If you were lucky, you would have got compilation warning or crash at runtime. Although I think for this particular case you won't, I don't think what you are doing is actually undefined behavior. The core of your problem is that when you declare this:
void *buf[2];
you are declaring buf as array 2 of pointer to void. When you access buf[0] you are not reading bytes but pointers. There is a lot more that could be said to explain what is going on, but I think this would be enough for what you are trying to do:
char buf[2];
As others has mentioned, you are reading into a pointer array.
Second is the audio format field is 2 bytes.
FILE *f;
short afmt;
if((f = fopen("file.wav", "rb")) == NULL)
err(1, "fopen");
fseek(f, 20, SEEK_SET);
fread(&afmt, sizeof(short), 1, f);
printf("audio format: 0x%hx\n", afmt);
fclose(f);
I'm using fmemopen to create a variable FILE* fid to pass it to a function the reads data from an open file.
Somewhere in that function it uses the following code to find out the size of the file:
fseek(fid, 0, SEEK_END);
file_size = ftell(fid);
this works well in case of regular files, but in case of file ids created by fmemopen I always get file_size = 8192
Any ideas why this happens?
Is there a method to get the correct file size that works for both regular files and files created with fmemopen?
EDIT:
my call to fmemopen:
fid = fmemopen(ptr, memSize, "r");
where memSize != 8192
EDIT2:
I created a minimal example:
#include <cstdlib>
#include <stdio.h>
#include <string.h>
using namespace std;
int main(int argc, char** argv)
{
const long unsigned int memsize = 1000000;
void * ptr = malloc(memsize);
FILE *fid = fmemopen(ptr, memsize, "r");
fseek(fid, 0, SEEK_END);
long int file_size = ftell(fid);
printf("file_size = %ld\n", file_size);
free(ptr);
return 0;
}
btw, I am currently working on another computer, and here I get file_size=0
In case of fmemopen , if you open using the option b then SEEK_END measures the size of the memory buffer. The value you see must be the default buffer size.
OK, I have got this mystery solved by myself. The documentation says:
If the opentype specifies append mode, then the initial file position is set to the first null character in the buffer
and later:
For a stream open for reading, null characters (zero bytes) in the buffer do not count as "end of file". Read operations indicate end of file only when the file position advances past size bytes.
It seems that fseek(fid, 0, SEEK_END) goes to the first zero byte in the buffer, and not to the end of the buffer.
Still looking for a method that will work on both standard and fmemopen files.