I consider reading file of unknown size that I know doesn't change size in the meantime. So I intend to use fstat() function and struct stat. Now I am considering what the st_size field really means and how should I use it.
If I get the file size's in this way, then allocate a buffer of that size and read exactly that size of bytes there seems to be one byte left over. I come to this conclusion when I used feof() function to check if there really nothing left in FILE *. It returns false! So I need to read (st_size + 1) and only than all bytes have been read and feof() works correctly. Should I always add this +1 value to this size to read all bytes from binary file or there is some hidden reason that this isn't reading to EOF?
struct stat finfo;
fstat(fileno(fp), &finfo);
data_length = finfo.st_size;
I am asking about this because when I add +1 then the number of bytes read by fread() is really -1 byte less, and as the last byte is inserted 00 byte. I could also before checking with feof() do something like this
fread(NULL, 1, 1, fp);
It is the real code, it is a little odd situation:
// reading png bytes from file
FILE *fp = fopen("./test/resources/RGBA_8bits.png", "rb");
// get file size from file info
struct stat finfo;
fstat(fileno(fp), &finfo);
pngDataLength = finfo.st_size;
pngData = malloc(sizeof(unsigned char)*pngDataLength);
if( fread(pngData, 1, pngDataLength, fp) != pngDataLength) {
fprintf(stderr, "%s: Incorrect number of bytes read from file!\n", __func__);
fclose(fp);
free(pngData);
return;
}
fread(NULL, 1, 1, fp);
if(!feof(fp)) {
fprintf(stderr, "%s: Not the whole binary file has been read.\n", __func__);
fclose(fp);
free(pngData);
return;
}
fclose(fp);
This behaviour is normal.
feof will return true only once you have tried to read beyond the file's end which you don't do as you read exactly the size of the file.
Related
I am trying to read 128KB binary file in chunks of 256 Bytes. The first 20-40 bytes of 256 bytes seems to be always correct. However after that the data gets corrupted. I tried reading the file and writing it into another binary file and compared. More than half of the data is corrupted. Here is my code
uint8_t buffer[256]
read_bin_file = fopen("vtest.bin", "r");
if (read_bin_file == NULL)
{
printf("Unable to open file\n");
return false;
}
test_bin = fopen("test_file.bin", "w");
if (test_file == NULL)
{
printf("Unable to open file\n");
return false;
}
fflush(stdout);
for (i = 0; i <=0x1FF; i++)
{
file_Read_pointer = i * 256;
fseek(read_bin_file, file_Read_pointer, SEEK_SET);
fread(buffer, 256, 1, read_bin_file);
fseek(test_file, file_Read_pointer, SEEK_SET);
fwrite(buffer, 256, 1, test_file);
}
What is that I am missing?
Also when i try to increase the bytes read from 256 to 1024 ( i<0x7F) the error seems to decrease significantly. The file is almost 90% matching
If it is binary data you're reading and writing, then you should open the files in binary mode with read_bin_file = fopen("vtest.bin", "rb");. Note the "b" in the mode. This prevents special handling of new line characters.
Your fseeks are also unnecessary, the fread and fwrite calls will handle that for you.
From here "The file position indicator for the stream is advanced by the number of characters read."
I am working on an assignment in socket programming in which I have to send a file between sparc and linux machine. Before sending the file in char stream I have to get the file size and tell the client. Here are some of the ways I tried to get the size but I am not sure which one is the proper one.
For testing purpose, I created a file with content " test" (space + (string)test)
Method 1 - Using fseeko() and ftello()
This is a method I found on https://www.securecoding.cert.org/confluence/display/c/FIO19-C.+Do+not+use+fseek()+and+ftell()+to+compute+the+size+of+a+regular+file
While the fssek() has a problem of "Setting the file position indicator to end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior for a binary stream", fseeko() is said to have tackled this problem but it only works on POSIX system (which is fine because the environment I am using is sparc and linux)
fd = open(file_path, O_RDONLY);
fp = fopen(file_path, "rb");
/* Ensure that the file is a regular file */
if ((fstat(fd, &st) != 0) || (!S_ISREG(st.st_mode))) {
/* Handle error */
}
if (fseeko(fp, 0 , SEEK_END) != 0) {
/* Handle error */
}
file_size = ftello(fp);
fseeko(fp, 0, SEEK_SET);
printf("file size %zu\n", file_size);
This method works fine and get the size correctly. However, it is limited to regular files only. I tried to google the term "regular file" but I still not quite understand it thoroughly. And I do not know if this function is reliable for my project.
Method 2 - Using strlen()
Since the max. size of a file in my project is 4MB, so I can just calloc a 4MB buffer. After that, the file is read into the buffer, and I tried to use the strlen to get the file size (or more correctly the length of content). Since strlen() is portable, can I use this method instead? The code snippet is like this
fp = fopen(file_path, "rb");
fread(file_buffer, 1024*1024*4, 1, fp);
printf("strlen %zu\n", strlen(file_buffer));
This method works too and returns
strlen 8
However, I couldn't see any similar approach on the Internet using this method. So I am thinking maybe I have missed something or there are some limitations of this approach which I haven't realized.
Regular file means that it is nothing special like device, socket, pipe etc. but "normal" file.
It seems that by your task description before sending you must retrieve size of normal file.
So your way is right:
FILE* fp = fopen(...);
if(fp) {
fseek(fp, 0 , SEEK_END);
long fileSize = ftell(fp);
fseek(fp, 0 , SEEK_SET);// needed for next read from beginning of file
...
fclose(fp);
}
but you can do it without opening file:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
struct stat buffer;
int status;
status = stat("path to file", &buffer);
if(status == 0) {
// size of file is in member buffer.st_size;
}
OP can do it the easy way as "max. size of a file in my project is 4MB".
Rather than using strlen(), use the return value from fread(). stlen() stops on the first null character, so may report too small a value. #Sami Kuhmonen Also we do not know the data read contains any null character, so it may not be a string. Append a null character (and allocate +1) if code needs to use data as a string. But in that case, I'd expect the file needed to be open in text mode.
Note that many OS's do not even use allocated memory until it is written.
Why is malloc not "using up" the memory on my computer?
fp = fopen(file_path, "rb");
if (fp) {
#define MAX_FILE_SIZE 4194304
char *buf = malloc(MAX_FILE_SIZE);
if (buf) {
size_t numread = fread(buf, sizeof *buf, MAX_FILE_SIZE, fp);
// shrink if desired
char *tmp = realloc(buf, numread);
if (tmp) {
buf = tmp;
// Use buf with numread char
}
free(buf);
}
fclose(fp);
}
Note: Reading the entire file into memory may not be the best idea to begin with.
I am currently working on a project in which I have to read from a binary file and send it through sockets and I am having a hard time trying to send the whole file.
Here is what I wrote so far:
FILE *f = fopen(line,"rt");
//size = lseek(f, 0, SEEK_END)+1;
fseek(f, 0L, SEEK_END);
int size = ftell(f);
unsigned char buffer[MSGSIZE];
FILE *file = fopen(line,"rb");
while(fgets(buffer,MSGSIZE,file)){
sprintf(r.payload,"%s",buffer);
r.len = strlen(r.payload)+1;
res = send_message(&r);
if (res < 0) {
perror("[RECEIVER] Send ACK error. Exiting.\n");
return -1;
}
}
I think it has something to do with the size of the buffer that I read into,but I don't know what it's the correct formula for it.
One more thing,is the sprintf done correctly?
If you are reading binary files, a NUL character may appear anywhere in the file.
Thus, using string functions like sprintf and strlen is a bad idea.
If you really need to use a second buffer (buffer), you could use memcpy.
You could also directly read into r.payload (if r.payload is already allocated with sufficient size).
You are looking for fread for a binary file.
The return value of fread tells you how many bytes were read into your buffer.
You may also consider to call fseek again.
See here How can I get a file's size in C?
Maybe your code could look like this:
#include <stdint.h>
#include <stdio.h>
#define MSGSIZE 512
struct r_t {
uint8_t payload[MSGSIZE];
int len;
};
int send_message(struct r_t *t);
int main() {
struct r_t r;
FILE *f = fopen("test.bin","rb");
fseek(f, 0L, SEEK_END);
size_t size = ftell(f);
fseek(f, 0L, SEEK_SET);
do {
r.len = fread(r.payload, 1, sizeof(r.payload), f);
if (r.len > 0) {
int res = send_message(&r);
if (res < 0) {
perror("[RECEIVER] Send ACK error. Exiting.\n");
fclose(f);
return -1;
}
}
} while (r.len > 0);
fclose(f);
return 0;
}
No, the sprintf is not done correctly. It is prone to buffer overflow, a very serious security problem.
I would consider sending the file as e.g. 1024-byte chunks instead of as line-by-line, so I would replace the fgets call with an fread call.
Why are you opening the file twice? Apparently to get its size, but you could open it only once and jump back to the beginning of the file. And, you're not using the size you read for anything.
Is it a binary file or a text file? fgets() assumes you are reading a text file -- it stops on a line break -- but you say it's a binary file and open it with "rb" (actually, the first time you opened it with "rt", I assume that was a typo).
IMO you should never ever use sprintf. The number of characters written to the buffer depends on the parameters that are passed in, and in this case if there is no '\0' in buffer then you cannot predict how many bytes will be copied to r.payload, and there is a very good chance you will overflow that buffer.
I think sprintf() would be the first thing to fix. Use memcpy() and you can tell it exactly how many bytes to copy.
I am dealing with a code which reading data from a binary file. The code is given here. Would anyone please make clear to me the role of fseek and fread here.
fc = fopen(CLOUDS_FILE, "rb");
if (fc == NULL){ fputs("File open error.\n", stderr); exit(1); }
crs = aux[CLRS];
fpos = (int) (pixel[2]*crs*crs + pixel[1]*crs + pixel[0]);
flsz = sizeof(fd);
fseek(fc, fpos*flsz, 0);
rd = fread((void *) &fd, flsz, 1, fc);
if (rd != 1){ fputs("Read error.\n", stderr); exit(1); }
fclose(fc);
fseek() changes the file offset. fread() reads data starting from the current offset, incrementing the offset by the number of elements read.
(Or is the question something else entirely? I mean, the above is something one can trivially figure by reading the manpages)
The binary file reading is done with an internal 'pointer', just like text editors have a cursor position when editing something. When opening the file in reading mode (using fopen) the pointer will be at the beginning of the file. Read operations (like fread, which will read a specified number of bytes from the stream) start reading at the pointer position and usually advance the pointer when they're done. If it is only necessary to read a specific part of the file, it is possible to manually set the pointer to a certain (relative or absolute) position, this is what fseek is used for.
#include <stdio.h>
int fseek(FILE *stream, long offset, int whence);
The fseek() function sets the file position indicator for the stream
pointed to by stream. The new position, measured in bytes, is obtained
by adding offset bytes to the position specified by whence. If whence
is set to SEEK_SET, SEEK_CUR, or SEEK_END, the offset is relative to
the start of the file, the current position indicator, or end-of-file,
respectively.
#include <stdio.h>
size_t fread(void *ptr, size_t size, size_t nmemb, FILE *stream);
The function fread() reads nmemb elements of data, each size bytes
long, from the stream pointed to by stream, storing them at the loca‐
tion given by ptr.
Sure, fseek is forwarding the "read from" index in the file to a calculated offset in CLOUDS_FILE, while fread is reading one object of size sizeof(fd) (whatever fd is, as that's not in your pasted code) into fd.
I am reaing from a file, and when i read, it takes it line by line, and prints it
what i want exactly is i want an array of char holding all the chars in the file and print it once,
this is the code i have
if(strcmp(str[0],"#")==0)
{
FILE *filecomand;
//char fname[40];
char line[100];
int lcount;
///* Read in the filename */
//printf("Enter the name of a ascii file: ");
//fgets(History.txt, sizeof(fname), stdin);
/* Open the file. If NULL is returned there was an error */
if((filecomand = fopen(str[1], "r")) == NULL)
{
printf("Error Opening File.\n");
//exit(1);
}
lcount=0;
int i=0;
while( fgets(line, sizeof(line), filecomand) != NULL ) {
/* Get each line from the infile */
//lcount++;
/* print the line number and data */
//printf("%s", line);
}
fclose(filecomand); /* Close the file */
You need to determine the size of the file. Once you have that, you can allocate an array large enough and read it in a single go.
There are two ways to determine the size of the file.
Using fstat:
struct stat stbuffer;
if (fstat(fileno(filecommand), &stbuffer) != -1)
{
// file size is in stbuffer.st_size;
}
With fseek and ftell:
if (fseek(fp, 0, SEEK_END) == 0)
{
long size = ftell(fp)
if (size != -1)
{
// succesfully got size
}
// Go back to start of file
fseek(fp, 0, SEEK_SET);
}
Another solution would be to map the entire file to the memory and then treat it as a char array.
Under windows MapViewOfFile, and under unix mmap.
Once you mapped the file (plenty of examples), you get a pointer to the file's beginning in the memory. Cast it to char[].
Since you can't assume how big the file is, you need to determine the size and then dynamically allocate a buffer.
I won't post the code, but here's the general scheme. Use fseek() to navigate to the end of file, ftell() to get size of the file, and fseek() again to move the start of the file. Allocate a char buffer with malloc() using the size you found. The use fread() to read the file into the buffer. When you are done with the buffer, free() it.
Use a different open. i.e.
fd = open(str[1], O_RDONLY|O_BINARY) /* O_BINARY for MS */
The read statement would be for a buffer of bytes.
count = read(fd,buf, bytecount)
This will do a binary open and read on the file.