I have a program which uses stdio for reading and writing a binary file. It caches the current stream position and will not seek if the read/write offset is already at the desired position.
However, an interesting problem appears, that when a byte is read and the following byte is written, it doesn't actually get written!
Here is a program to reproduce the problem:
#include <cstdio>
int main() {
FILE *f = fopen("test.bin", "wb");
unsigned char d[1024] = { 0 };
fwrite(d, 1, 1024, f);
fclose(f);
f = fopen("test.bin", "rb+");
for (size_t i = 0; i < 1024; i++) {
unsigned char a[1] = { 255 - (unsigned char)(i) };
fflush(f);
fwrite(a, 1, 1, f);
fflush(f);
fseek(f, i, SEEK_SET);
fread(a, 1, 1, f);
printf("%02X ", a[0]);
}
fclose(f);
return 0;
}
You are supposed to see it write the bytes FF down to 00, however only the first byte is written because it does not follow a fread immediately.
If it seeks before fwrite, it acts correctly.
The problem happens on Visual Studio 2010/2012 and TDM-GCC 4.7.1 (Windows), however it works on codepad which I guess is due to it being executed on Linux.
Any idea why this happens?
C99 §7.18.5.3/6 (quoted from N869 final draft):
“When a file is opened with update mode (’+’ as the second or third character in the
above list of mode argument values) […] input shall not be directly followed by output without an
intervening call to a file positioning function, unless the input operation encounters end-
of-file.”
Related
I am unable to find a documented reason why I need the fflush at the end of the for loop in the following code on windows but not on linux. This is to xor one file with another. without the fflush on windows it seems to read only to the second position in the file I am updating and never reads any further filling the file with that character xor'd with the second file. The fflush makes it work as I expect but runs slowly. On linux the fflush makes no difference, it works either way. I am using gcc on both systems but the windows compiler does the same thing.
#include < stdio.h>
#include < string.h>
#include < stdlib.h>
#include < limits.h>
int main(int argc, char *argv[])
{
FILE *pf , *kf;
int index, fsize, ksize, fpos;
unsigned char t_char, k_char, c_char;
pf = fopen(argv[1],"rb+"); /* open the file for I/O */
kf = fopen(argv[2],"rb+"); /* open key file */
fseek(pf, 0, SEEK_END);
fsize = ftell(pf);
printf(" Input size = %d\n", fsize);
rewind(pf);
fseek(kf, 0, SEEK_END);
ksize = ftell(kf);
printf(" keyfile size = %d\n", ksize);
rewind(kf);
if (ksize < fsize) {
printf(" Keyfile must be equal to or longer than crypt file\n");
exit(0);
}
for(index=0; index<fsize; index++) {
t_char = fgetc(pf);
k_char = fgetc(kf);
c_char = t_char ^ k_char;
fseek(pf, -1, SEEK_CUR);
fputc(c_char, pf);
/*fflush(pf);*/
}
fclose(pf);
fclose(kf);
return(0);
}
Quoted from: https://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html
When a file is opened with update mode ( '+' as the second or third
character in the mode argument), both input and output may be
performed on the associated stream. However, the application shall
ensure that output is not directly followed by input without an
intervening call to fflush() or to a file positioning function (
fseek(), fsetpos(), or rewind()), and input is not directly followed
by output without an intervening call to a file positioning function,
unless the input operation encounters end-of-file.
So this is necessary no matter which OS you are using, according to the documentation.
However, you could use a file positioning function, instead of fflush, and see if that is faster.
Alternatively you could copy the file content to memory and do the xor operations there, and write everything back to the file once its finished.
I have following problem:
void edit(FILE *f, int cnt)
{
int i = 0;
int offset = 0;
rewind(f);
schedule todo;
schedule *p = &todo;
fprintf(stdout, "\n%s\n", "------------------------------------------------");
fread(&todo, sizeof(schedule), 1, f);
while (!feof(f)) {
fprintf(stdout, "%6d%18s\n",
++i, todo.title);
fread(&todo, sizeof(schedule), 1, f);
}
fprintf(stdout, "%s\n\n", "-------------------------------------------------");
fprintf(stdout, "%s\n\n", "Number: ");
scanf("%d", &i);
getchar();
rewind(f);
offset = (long) sizeof(schedule);
fseek(f, (i - 1)*offset, SEEK_CUR);
fread(&todo, sizeof(schedule), 1, f);
printf("Edit: %s\n", todo.title);
fprintf(stdout, "%6d%18s%8s%10s%8d\n",
todo.number, todo.title, todo.where, todo.details, todo.importance);
scanf("%s", todo.title);
fwrite(&todo, (long)sizeof(todo.title), 1, f);
}
It's part of editing data codes.
This is what I expected.
If a user put a number(i in the code), the program will find the location (in binary file).
Then, the user put todo.title by (scanf("%s", todo.title);) and the program will edit it by using
fwrite(&todo, (long)sizeof(todo.title), 1, f);
I got a warning like
Expression: ("Flush between consecutive read and write.", !stream.has_any_of(_IOREAD))
I think there's problems with buffer, but I can't fix this.
If you have a file open for update (read and write) then the C11 standard requires:
§7.21.5.3 The fopen function
¶7 When a file is opened with update mode ('+' as the second or third character in the above list of mode argument values), both input and output may be performed on the associated stream. However, output shall not be directly followed by input without an intervening call to the fflush function or to a file positioning function (fseek, fsetpos, or rewind), and input shall not be directly followed by output without an intervening call to a file positioning function, unless the input operation encounters end-of-file. Opening (or creating) a text file with update mode may instead open (or create) a binary stream in some implementations.
The quote comes from the C11 specification, but the wording is essentially unchanged in all versions of the standard.
Note that you must do a seek operation, even if it is just fseek(f, 0, SEEK_CUR), between a read and a write operation, and between a write and a read operation.
Your code has an fread() followed by an fwrite() with no intervening fseek(). That's probably an oversight since you change the record and overwrite the next record in the file with the updated information. You probably need an fseek(f, -(long)sizeof(schedule), SEEK_CUR) or thereabouts to move back and overwrite the record just read and changed.
I have a small example program written in C. I have a main that call a function writeFile that writes some numbers in a binary file. Then I call overwrite to replace 0 with 1 and finally I print the result.
This is the code:
#include <stdio.h>
/* Print the content of the file */
void printFile(){
printf("Read test.dat:\n");
int r;
FILE* fp = fopen("test.dat", "rb+");
if(fp) {
while(fread(&r,sizeof(int),1,fp)){
printf("%d\n", r);
}
}
fclose(fp);
}
/* Replace 0 with 1 */
void overwrite(){
int r;
FILE *fp = fopen("test.dat", "rb+");
if (fp) {
int i=0;
while (i < 4 && fread(&r, sizeof(int), 1, fp)) {
i++;
if (r == 0) {
r = 1;
fseek(fp,-sizeof(int),SEEK_CUR);
fwrite(&r,sizeof(int),1,fp);
}
}
}
fclose(fp);
}
/* Create original file */
void writeFile() {
int b, b1, b2, b3, b4;
b = 3;
b1 = 2;
b2 = 0;
b3 = 4;
FILE *fp = fopen("test.dat", "wb");
if (fp) {
fwrite(&b, sizeof(int), 1, fp);
fwrite(&b1, sizeof(int), 1, fp);
fwrite(&b2, sizeof(int), 1, fp);
fwrite(&b3, sizeof(int), 1, fp);
}
fclose(fp);
}
int main() {
writeFile();
printf("---------BEFORE--------\n");
printFile();
printf("-----------------------\n");
printf("Overwriting...\n");
overwrite();
printf("---------AFTER---------\n");
printFile();
return 0;
}
This code works with Linux, but when I run the same code on Windows the output is this:
---------BEFORE--------
Read test.dat:
3
2
0
4
-----------------------
Overwriting...
---------AFTER---------
Read test.dat:
3
2
1
2
Not only 0 was replaced by 1 but also the last number changed. Someone can help me to understand why this happens?
Another problem is that in the overwrite I must use i to stop the while because without the i<4 I get an infinite loop (only with Windows).
I tested this code on Windows 8.1 compiled with gcc 4.8.1 (from MinGW).
On my Linux machine I tested the code with gcc 5.1.1.
Thank you all,
It's because you need to fflush() after the fwrite(), since you should not call fread() after calling fwrite() without an intervening call to fflush(),
This is the section of the standard that is relevant in this case
7.21.5.3 The fopen function
When a file is opened with update mode ('+' as the second or third character in the above list of mode argument values), both input and output may be performed on the associated stream. However, output shall not be directly followed by input without an
intervening call to the fflush function or to a file positioning function (fseek, fsetpos, or rewind), and input shall not be directly followed by output without an
intervening call to a file positioning function, unless the input operation encounters end-of-file. Opening (or creating) a text file with update mode may instead open (or create) a binary stream in some implementations.
It's weird that this section is within fopen() function since it involves fread() and fwrite(), which is where I was looking for an answer.
You can also see that my previous answer worked but not for the reason I stated in it, instead the explanation is found in the paragraph above.
MSDN states for Stream I/O:
Input can follow output directly only with an intervening call to fflush or to a file-positioning function (fseek, fsetpos, or rewind). Output can follow input without an intervening call to a file-positioning function if the input operation encounters the end of the file.
So you'd have to call fflush() after calling fwrite:
void overwrite()
{
FILE *fp = fopen("test.dat", "rb+");
if (fp) {
int r;
while (fread(&r, sizeof(int), 1, fp) == 1) {
if (r == 0) {
r = 1;
fseek(fp,-sizeof(int),SEEK_CUR);
fwrite(&r,sizeof(int),1,fp);
fflush(fp); /* Flush here */
}
}
fclose(fp); /* Also: call fclose() only when fopen() succeeded */
}
}
I using some C code that writes binary data to a file. In the process, it seeks around to different positions and then finally seeks to the end with fseeko(fp, 0, SEEK_END);.
However, in some cases, I want to work on a stream in memory instead. I use open_memstream for this, but seeking to the end pads the buffer with zeros and it ends up being twice as big as it should be.
An example just to demonstrate the effect of the fseek to the end of the stream is below. In the actual code, we also fseek to different parts of the stream, patching and editing bits of it, etc., as the stream is processed. Note also that writing the file at the end to the filesystem is just for demonstration to show the contents of the buffer – otherwise I wouldn't need the memory stream.
#include <stdio.h>
#include <stdlib.h>
#if (defined(BSD) || __APPLE__)
#include "open_memstream.h"
#endif
int main(void) {
FILE *stream;
FILE *outfile;
char *buf;
size_t buf_len;
int i;
stream = open_memstream(&buf, &buf_len);
for(i = 0; i < 1000; i++) {
fprintf(stream, "%d\n", i);
}
fseeko(stream, 0, SEEK_END);
fclose(stream);
outfile = fopen("out.txt", "w");
fwrite(buf, buf_len, 1, outfile);
fclose(outfile);
return 0;
}
I was testing this out on Mac OS X with this implementation of open_memstream and it worked as I expected, but when I run this on Linux the file is twice the size with zeros at the end.
What's the best way to deal with this? I'm not sure if it's reliable to divide the buffer length by two and truncate it.
I've just ran into the same problem on Linux.
// It seams that SEEK_END does not work with open_memstream()
fseek(stream, 0, SEEK_END);
I've ended up doing this:
off_t o = ftell(stream);
/* do some things with the stream */
fseek(stream, o, SEEK_SET);
I write this C code so that I could test whether fwrite could update some values in a text file. I tested on Linux and it works fine. In Windows (vista 32bits), however, it simply does not work. The file remains unchanged after I write a different byte using: cont = fwrite(&newfield, sizeof(char), 1, fp);
The registers are written on the file using a "#" separator, in the format:
Reg1FirstField#Reg1SecondField#Reg2FirstField#Reg2SecondField...
The final file should be: First#1#Second#9#Third#1#
I also tried putc and fprintf, all with no result. Can someone please help me with this?
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct test {
char field1[20];
char field2;
} TEST;
int main(void) {
FILE *fp;
TEST reg, regread;
char regwrite[22];
int i, cont, charwritten;
fp=fopen("testupdate.txt","w+");
strcpy(reg.field1,"First");
reg.field2 = '1';
sprintf(regwrite,"%s#%c#", reg.field1, reg.field2);
cont = (int)strlen(regwrite);
charwritten = fwrite(regwrite,cont,1,fp);
fflush(fp);
strcpy(reg.field1,"Second");
reg.field2 = '1';
sprintf(regwrite,"%s#%c#", reg.field1, reg.field2);
cont = (int)strlen(regwrite);
charwritten = fwrite(regwrite,cont,1,fp);
fflush(fp);
strcpy(reg.field1,"Third");
reg.field2 = '1';
sprintf(regwrite,"%s#%c#", reg.field1, reg.field2);
cont = (int)strlen(regwrite);
charwritten = fwrite(regwrite,cont,1,fp);
fflush(fp);
fclose(fp);
// open file to update
fp=fopen("testupdate.txt","r+");
printf("\nUpdate field 2 on the second register:\n");
char aux[22];
// search for second register and update field 2
for (i = 0; i < 3; i ++) {
fscanf(fp,"%22[^#]#", aux);
printf("%d-1: %s\n", i, aux);
if (strcmp(aux, "Second") == 0) {
char newfield = '9';
cont = fwrite(&newfield, sizeof(char), 1, fp);
printf("written: %d bytes, char: %c\n", cont, newfield);
// goes back one byte in order to read properly
// on the next fscanf
fseek(fp,-1,SEEK_CUR);
}
fscanf(fp,"%22[^#]#", aux);
printf("%d-2: %s\n",i, aux);
aux[0] = '\0';
}
fflush(fp);
fclose(fp);
// open file to see if the update was made
fp=fopen("testupdate.txt","r");
for (i = 0; i < 3; i ++) {
fscanf(fp,"%22[^#]#", aux);
printf("%d-1: %s\n", i, aux);
fscanf(fp,"%22[^#]#",aux);
printf("%d-2: %s\n",i, aux);
aux[0] = '\0';
}
fclose(fp);
getchar();
return 0;
}
You're missing a file positioning function between the read and write. The Standard says:
7.19.5.3/6
When a file is opened with update mode, both input and output may be performed on the associated stream. However, ... input shall not be directly followed by output without an intervening call to a file positioning function, unless the input operation encounters end-of-file. ...
for (i = 0; i < 3; i ++) {
fscanf(fp,"%22[^#]#", aux); /* read */
printf("%d-1: %s\n", i, aux);
if (strcmp(aux, "Second") == 0) {
char newfield = '9';
/* added a file positioning function */
fseek(fp, 0, SEEK_CUR); /* don't move */
cont = fwrite(&newfield, sizeof(char), 1, fp); /* write */
I didn't know it but here they explain it:
why fseek or fflush is always required between reading and writing in the read/write "+" modes
Conclusion: You must either fflush or fseek before every write when you use "+".
fseek(fp, 0, SEEK_CUR);
// or
fflush(fp);
cont = fwrite(&newfield, sizeof(char), 1, fp);
Fix verified on Cygwin.
You're not checking any return values for errors. I'm guessing the file is read-only and is not even opening properly.
At least here on OSX, your value 9 is begin appended to the end of the file ... so you're not updating the actual register value for Second at it's position in the file. For some reason after the scan for the appropriate point to modify the values, your stream pointer is actually at the end of the file. For instance, running and compiling your code on OSX produced the following output in the actual text file:
First#1#Second#1#Third#1#9
The reason your initial read-back is working is because the data is being written, but it's at the end of the file. So when you write the value and then back-up the stream and re-read the value, that works, but it's not being written in the location you're assuming.
Update: I've added some calls to ftell to see what's happening to the stream pointer, and it seems that your calls to fscanf are working as you'd assume, but the call to fwrite is jumping to the end of the file. Here's the modified output:
Update field 2 on the second register:
**Stream position: 0
0-1: First
0-2: 1
**Stream position: 8
1-1: Second
**Stream position before write: 15
**Stream position after write: 26
written: 1 bytes, char: 9
1-2: 9
**Stream position after read-back: 26
Update-2: It seems by simply saving the position of the stream-pointer, and then setting the position of the stream-pointer, the call to 'fwrite` worked without skipping to the end of the file. So I added:
fpos_t position;
fgetpos(fp, &position);
fsetpos(fp, &position);
right before the call to fwrite. Again, this is on OSX, you may see something different on Windows.
With this:
fp=fopen("testupdate.txt","w+");
^------ Notice the + sign
You opened the file in "append" mode -- that is what the plus sign does in this parameter. As a result, all of your fwrite() calls will be relative to the end of the file.
Using "r+" for the fopen() mode doesn't make sense -- the + means nothing in this case.
This and other issues with fopen() are why I prefer to use the POSIX-defined open().
To fix your particular case, get rid of the + characters from the fopen() modes, and consider that you might need to specify binary format on Windows ("wb" and "rb" modes).