I have the following struct:
struct entry
{
int next;
int EOC;
} entry;
After creating an instance of the struct and setting its 'next' and 'EOC' values accordingly, I wrote the struct to a file via the C fwrite function. I'd now like to read the values of 'next' and 'EOC' from that file. My question is this: how are these two int variables stored in memory? Once I call fseek to move the file pointer to the correct byte in the file, how many bytes do I read to get the value of 'next' or 'EOC'?
If the file is written on the same machine as you read it, then:
You write with fwrite(&entry, sizeof(entry), 1, fp).
You read with fread(&entry, sizeof(entry), 1, fp).
The size you write is the size of the structure; the size you read is the size of the structure. You test the return values; they'll be either 0 or 1 given what I wrote, and 0 indicates a problem and 1 success in this context.
If your file is open for reading and writing, after writing, you'd seek backwards by sizeof(entry) and then read.
If your file is written on a different (type of) machine, then you may have to worry about the difference in the sizes of the basic data types, and in the endianness of the two systems, and with more complex structures, you might have to worry about packing and alignment rules too. There are ways of dealing with these issues (and they're more complex than single fwrite() and fread() calls).
Related
I'm writing down a function that should save 3 structures (2 of them are arrays of structs) in a binary file. Here's my function:
void saveFile(Struct1 *s1, Struct2 *s2, Struct3 s3) {
FILE *fp = NULL;
fp = fopen("save.bin", "w+b");
if (fp == NULL) {
printf("Save failed.\n");
}
fwrite(s1, sizeof(Struct1), struct3.nElements, fp);
fwrite(s2, sizeof(Struct2), NELEMENTS, fp);
fwrite(&s3, sizeof(Struct3), 1, fp);
printf("Save done.\n");
}
s1 have struct3.nElements, s2 have NELEMENTS (that's a constant) and s3 is just one struct and not an array. When I try to open the save.bin with HexEditor it gives very different results from the ones I was expecting, I'm wondering if I used correctly the fwrite function, especially for array of structs.
There are small issues with you function that might cause problems:
you define the function as taking s3 by value. Why not pass a pointer to the third struct? Is the saveFile function properly declared before the calling code? Are you sure the calling code passes the struct by value?
You forget to close the stream. The handle gets lost, and the contents is not flushed to disk until the program exits.
You open the file in "w+b" mode: write with read. It is correct to use binary mode, but unnecessary to add the + for read. Just use "wb".
If fopen fails, you output a diagnostic message, but you do not return from the function. You will invoke undefined behavior when trying to write to a NULL stream pointer.
Regarding your question, the dump of the file does not correspond to what you expect... give us more information, such as the definitions of the different structures and the hex dump. Here are some ideas:
Some of the fields in the structures might need a specific aligned and thus be separated from the previous field by padding bytes. The values of those padding bytes is not necessarily 0: if the structures are in automatic storage or allocated with malloc, their initial state is undefined and can change as a side effect of storing other fields.
Integers can have different sizes and be stored in little endian or big endian order in the file, depending on the specific architecture your program is compiled for. For this reason, values stored by your program should only be read back with the appropriate, but reasonably similar code, running on the same architecture and OS.
If your structures contain pointers, you cannot really make sense from the values stored in the output file.
When reading K&R, I became interested in how the file position is determined. By file position, I mean where in the file the stream is currently reading or writing. I think it must have something to do with the file pointer, or the piece of data it's pointing to. So I checked stack overflow, and find the following answer:
Does fread move the file pointer?
The answer indicates that file pointer will change with the change of file position. This makes me very confused, because in my understanding, a file pointer for a certain file should always point to the same address, where information about this file is stored. So I wrote a small piece of code, trying to find the answer:
#include<stdio.h>
int main(void)
{
char s[1000];
FILE *fp,*fp1,*fp2;
fp = fopen("input","r");
fp1 = fp; /* File poiter before fread */
fread(s,sizeof(char),100,fp);
fp2 = fp; /* File pointer after fread */
printf("%d\n",(fp1 == fp2) ? 1 : -1);
}
It gives the output 1, which I believe indicates that the file pointer actually doesn't move and is still pointing to the same address. I have also changed the fread line to be a fseek, which gave the same output. So does file pointer move with the change of file position, or where am I wrong in the verifying process?
Thanks!
I think you are confusing the general concept of pointers in C, vs. the nomenclature of a "file pointer". FILE is just a structure that contains most of the "housekeeping" attributes that the C stdio runtime library needs to interact with when using the stdio functions such as, fopen(), fread(), etc. Here is an example of the structure:
typedef struct {
char *fpos; /* Current position of file pointer (absolute address) */
void *base; /* Pointer to the base of the file */
unsigned short handle; /* File handle */
short flags; /* Flags (see FileFlags) */
short unget; /* 1-byte buffer for ungetc (b15=1 if non-empty) */
unsigned long alloc; /* Number of currently allocated bytes for the file */
unsigned short buffincrement; /* Number of bytes allocated at once */
} FILE;
Note that this may be somewhat platform-dependent, so don't take it as gospel. So when you call fopen(), the underlying library function interacts with the O/S's file system APIs and caches relevant information about the file, buffer allocation, etc, in this structure. The fopen() function allocates memory for this structure, and then returns the address of that memory back to the caller in the form of a C Pointer.
Assigning the pointers values to another pointer has no effect on the attributes inside the FILE structure. However, the FILE structure, internally, may have indexes or "pointers" to the underlying O/S file. Hence, the confusion in terminology. Hope that helps.
You are right fp is never changed by fread, fseekor other f... functions. Except, of course, if you do fp = fopen(...), but then you are assigning the return value of fopen to fp and then fp changes of course.
Remember, in C parameters are passed by value, so fread cannot change it's value.
But fread does change the internal structure fp points to.
You made some confusion between a file pointer, under common definition, and the pointer in the file.
Normally with the term file pointer we refer to a pointer to a FILE structure. That structure contains all variables necessary to manage file access. This structure is created upon a successful opening of a file, and remains the same (same address) for all the time until you fclose() the file (when became undefined).
Inside the FILE structure there are many pointers that points to the file block on disk and to the position inside the current record. These pointers, managed by file I/O routines, changes when file is accessed (read or write).
And these pointers are that to which the answer you cited refers.
So, I was trying to read a BMP image using fread and structs. I create the following struct to read the header
struct head{
char sigBM[2];//This will get the 'B' and 'M' chars
int fileSize;
int reserved;
int offset
...
};
And in the main function I used
fread(pointerToStruct,sizeof(struct head),1,image);
And I just got some weird results. But then I decided to take the char sigBM[2] from the struct and read it with a different fread. Something like:
char sigBM[2];
struct head *p = malloc(sizeof(struct head));/*
Without the char sigBM[2]
*/
fread(sigBM,sizeof(char),2,image);
fread(p,sizeof(struct head),1,image);
And it worked!
I already got it working, I just want to know why it worked like that
Your data seem to be written to disk without padding. That is; that integer fileSize comes directly after the two chars. This is normally not how structs are kept in memory.
the size of
struct head{
char sigBM[2];//This will get the 'B' and 'M' chars
// two padding bytes hide here
int fileSize;
}
is 8 on my machine. not 2+4 as you may expect. If you read/write with the same compiler options on the same platform you can expect the struct to be read in correctly. If not, you need to be in control of details like these.
Most architectures require (or prefer) that numeric types start at specific multipliers of two [for example the size of the type itself].
In a 16-bit int, 32-bit long world with
struct head{
char sigBM[2];
long fileSize;
long reserved;
long offset
...
};
all works well. But try to read with a struct that has padding (extra space between fields) causes OP's issue.
A solution is to read each field, one at a time - like OP's approach. Another approach is to use a compiler specific option or keyword to "pack" the structure.
In general, given C variant int size, better to use think of these fields as int8_t, int16_t, int32_t rather than char, int, etc.
the OP has workable idea. It does need some extension.
Of some note is that several fields within the .bmp file
have non-fixed offsets from the start of the file.
so a single struct will not properly handle the whole file
Of critical interest is the number of 'special' color entries
as that changes the offsets for all the rest of the file.
the actual image, depending on the pixel width and the image width
can have from 0 to 3 bytes of filler for each pixel line of the image
certain fields are optional in the second section of the file.
and some fields within the first section of the file contain
offsets into the file for other key areas.
in general, it is best to treat the file as a long string of bytes
and use offsets (and field length) to access specific fields
this has been demonstrated, with full C code,
on several answers in stackoverflow.com
suggest performing a search.
So, I have an assignment that basically is an exercise in comparing the speed of system calls opposed to library functions. We're sorting a string we grab from a file through stdin. We're supposed to determine if the file is a regular file or not using fstat. I've read through the manual page and while I know WHAT it does, I'm having problem figuring out how to use it.
I know it returns a stat struct, so can I literally just make a variable and store it like that? Ex:
int n;
struct stat *val;
n = fstat(0, val);
Is that how you get the struct? Is it returned somewhere else? I need access to the off_t st_size variable so I know how many bytes the file is. And can that be cast to an int?
Also, apparently you can use the st_mode field to check if the file is regular (using the macros S_ISREG), but...how? Does it return true of false or something? It's annoying because I can find all these documents telling me what the fields are, but not how to use them.
If I have a regular file, I'm supposed to allocate enough memory to store it before any function calls. If not, then I reallocate memory as I read. I've got the second part done, I just have no idea how to use fstat correctly.
fstat doesn't allocate memory, you need to give it the address of pre-allocated space. For example:
int n;
struct stat myStat;
n = fstat(0, &myStat);
Note the lack of pointer.
IMPORTANT EDIT:
Sorry everyone, i made a big mistake in the structure.
char *name; is meant to be outside of the structure, written to the file after the structure.
This way, you read the structure, find out the size of the name, then read in the string. Also explains why there is no need for a null terminator.
However, i feel somewhere, my actual question has been answered. If someone would like to edit their responses so i can choose one which is the best fitting i'd appreciate it.
Again, the question I was asking is "If you read in a structure, are you also reading in the data it holds, or do you need to access it some other way".
Sorry for the confusion
For an assignment, I've been tasked with a program which writes and reads structures to a disk (using fread and fwrite).
I'm having trouble grasping the concept.
Lets say we have this structure:
typedef struct {
short nameLength;
char* name;
}attendenceList;
attendenceList names;
now assume we give it this data:
names.name = "John Doe\0";
names.nameLength = strlen(names.name); /*potentially -1?*/
and then we use fwrite... given a file pointer fp.
fwrite(&names,sizeof(names),1,fp);
now we close the file, and open it later to read in the structure.
the question is this: when we read in the structure, are we also reading in the variables it stores?
Can we then now do something like:
if(names.nameLength < 10)
{
...
}
Or do we have to fread something more then just the structure, or assign them somehow?
Assuming the fread is:
fread(&names,sizeof(names),1,fp);
Also assuming we've defined the structure in our current function, as above.
Thanks for the help!
You have a problem here:
fwrite(&names,sizeof(names),1,fp);
Since attendenceList saves the name as a char * this will just write out the pointer, not the actual text. When you read that back in, the memory the pointer is referencing will most likely have something else in it.
You have two choices:
Put a character array (char names[MAXSIZE]) in attendenceList.
Don't write the raw data structure, but write the necessary fields.
You're writing the memory layout of the structure, which includes its members.
You'll get them back if you read the structure back in again - atleast if you do it on the same platform, with a program compiled with the same compiler and compiler settings.
Your name member is declared just as a char, so you can't store a string in it.
If name was a pointer like this:
typedef struct {
short nameLength;
char *name;
}attendenceList;
You really should not read/write the struct to a file. You will write the structure as it's laid out in memory, and that includes the value if the name pointer.
fwrite knows nothing about pointers inside your structure, it will not follow pointers and also write whatever they point to.
when you read the structure back again, you'll read in the address in the name pointer, and that might not point to anything sensible anymore.
If you declare name as an array, you'll be ok, as the array and its content is part of the structure.
typedef struct {
short nameLength;
char name[32];
}attendenceList;
As always, make sure you don't try to copy a string - including its nul terminator- to name that's larger than 32. And when you read it back again. set yourstruct.name[31] = 0; so you are sure the buffer is null terminated.
To write a structure, you'd do
attendenceList my_list;
//initialize my_list
if(fwrite(&my_list,sizeof my_list,1,f) != 1) {
//handle error
}
And to read it back again:
attendenceList my_list;
//initialize my_list
if(fread(&my_list,sizeof my_list,1,f) != 1) {
//handle error
}
}
I'm assuming you meant char* name instead of char name.
Also sizeof(name) will return 4 because you are getting the size of a char* not the length of the char array. So you should write strlen(name) not sizeof(name) inside your fwrite.
In your above example I would recommend storing the string exact size without the null termination. You don't need to store the string length as you can get that after.
If you are reading just a string from a file, and you wrote the exact size without the null termination. Then you need to manually null terminate your buffer after you read the data in.
So make sure you allocate at least the size of your data you are reading in plus 1.
Then you can set the last byte of that array to '\0'.
If you write a whole struct at a time to the buffer, you should be careful because of padding. The padding may not always be the same.
when we read in the structure, are we also reading in the variables it stores?
Yes you are, but the problem you have is that as I mentioned above you will be storing the pointer char* (4 bytes) and not the actual char array. I would recommend storing the struct elements individually.
You ask:
now we close the file, and open it later to read in the structure. the question is this: when we read in the structure, are we also reading in the variables it stores?
No. sizeof(names) is a constant value defined at compile time. It will be the same as
sizeof(short) + sizeof(void*) + some_amount_of_padding_to_align_things
it will NOT include the size of what names.name points to, it will only include the size of the pointer itself.
So you have two problems when writing this to a file.
you aren't actually writing the name string to the file
you are writing a pointer value to the file that will have no meaning when you read it back.
As your code is currently written, When you read back the names, names.name will point to somewhere, but it won't point to "John Doe\0".
What you need to do is to write the string pointed to by names.name instead of the pointer value.
What you need to do is sometimes called "flattening" the structure, You make a structure in memory that contains no pointers, but holds the same data as the structure you want to use, then you write the flattened structure to disk. This is one way to do that.
typedef struct {
short nameLength;
char name[1]; // this will be variable sized at runtime.
}attendenceListFlat;
int cbFlat = sizeof(attendenceListFlat) + strlen(names.name);
attendenceListFlat * pflat = malloc(cbFlat);
pflat->nameLength = names.nameLength;
strcpy(pflat->name, names.name);
fwrite(pflat, cbFlat, 1, fp);
The flattened structure ends with an array that has a minimum size of 1, but when we malloc, we add strlen(names.name) so we can treat that as an array of strlen(names.name)+1 size.
A few things.
Structures are just chunks of memory. It's just taking a bunch of bytes and drawing boundaries on them. Accessing structure elements is just a convenient way of getting a particular memory offset cast as a particular type of data
You are attempting to assign a string to a char type. This will not work. In C, strings are arrays of characters with a NULL byte at the end of them. The easiest way to get this to work is to set a side a fixed buffer for the name. When you create your structure you'll have to copy the name into the buffer (being very careful not to write more bytes than the buffer contains). You can then write/read the buffer from the file in one step.
struct attendanceList {
int namelen;
char name[256]; //fixed size buffer for name
}
Another way you could do it is by having the name be a pointer to a string. This makes what you're trying to do more complicated, because in order to write/read the struct to/from a file, you will have to take into account that the name is stored in a different place in memory. This means two writes and two reads (depending on how you do it) as well as correctly assigning the name pointer to wherever you read the data for the name.
struct attendanceList {
int namelen;
char* name; //the * means "this is a pointer to a char somewhere else in memory"
}
There's a third way you could do it, with a dynamically sized struct using a trick with a zero length array at the end of a struct. Once you know how long the name is, you allocate the correct amount (sizeof(struct attendanceList) + length of string). Then you have it in one contiguous buffer. You just need to remember that sizeof(struct attendanceList) is not the size you need to write/read. This might be a little confusing as a beginning. It is also kind of a hack that's not supported under all compilers.
struct attendanceList {
int namelen;
char name[0]; //this just allows easy access to the data following the struct. Be careful!
}