I'm losing binary data when using fseek in c - c

I wrote a function that is responsible for moving to a binary file and editing its bytes
int replace(FILE *binaryFile, long offset, unsigned char *replaced, int length) {
if (binaryFile != NULL) {
for (int i = 0; i < length; i++) {
fseek(binaryFile, offset + i, SEEK_SET);
fwrite(&replaced[i], sizeof(*replaced), 1, binaryFile);
}
fclose(binaryFile);
return 0;
}
else return -1;
}
When I use this function, I encounter a strange problem
All data in the file is filled with NULL bytes
And only one of the addresses in the file changes
Example:
FILE* fp = fopen("target.bin", "wb");
replace(fp, 0x57d8b0, "\x1E\xFF\x2F\xE1", 4);
replace(fp, 0x57c770, "\x01\x00\xA0\xE3\x1E\xFF\x2F\xE1", 8);
Result:
0x57c770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
0x57d8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x57d8b0: 1e ff 2f e1
Correct result:
0x57c770: 01 00 A0 E3 1E FF 2F E1 ...
...
0x57d8a0: 9c c4 0a ea 70 d2 68 00 44 d4 68 00 10 d1 68 00
0x57d8b0: 1e ff 2f e1 18 b0 8d e2 02 8b 2d ed 18 d0 4d e2
Please help me to solve the function problem or other problems.

wb is not correct. It clobbers the file.
From here,
Mode
Meaning
Explanation
If already exists
If does not exist
"r"
read
Open a file for reading
read from start
failure to open
"w"
write
Create a file for writing
destroy contents
create new
"a"
append
Append to a file
write to end
create new
"r+"
read extended
Open a file for read/write
read from start
error
"w+"
write extended
Create a file for read/write
destroy contents
create new
"a+"
append extended
Open a file for read/write
write to end
create new
You want r+b.
Also, you close the file handle in replace, which is premature. This should be done outside of replace, after you're done with the handle.
As an aside, you shouldn't be doing a number of seeks and writes of length one equal to length; you should be doing one seek and one write of length length.

Related

C: Memory allocation issues

I'm having an issue with (I think) memory reallocation in C. The program is meant to run such that when fopen(array, &num); is called, it will first retrieve the number of elements in the array from file and place that in num, reallocate memory for the array pointer given to give it enough room to store the contents of the file proper, then copy the values over into that array. This seems to work while still in the fopen function (shown by 'mark 1'), but does not work outside of this (shown by 'mark 2') instead seeming to spew out random memory garbage. Any help appreciated (both with code and formatting my poorly laid out question).
//main.c
void Rtest(){
char num;
struct individual *array;
array = (struct individual *) malloc(sizeof(struct individual));
openf(array, &num);
printf("%d\n", num);
for (int i = 0; i < num; i++) {printf("%s\n", array[i].name);} //mark 2
free(array);
}
//fil.h
struct individual {
char name[32];
char stats[7];
char role;
char roles[13];
};
void openf(struct individual *array, char *num){
FILE *fp;
fp = fopen("save.bin", "rb");
fread(num, 1, sizeof(char), fp);
array = (struct individual *)realloc(array, *num * sizeof(struct individual));
printf("%d\n", sizeof(*array));
fread(array, *num, sizeof(struct individual), fp);
for (int i = 0; i < *num; i++) {printf("%s\n", array[i].name);} //mark 1
fclose(fp);
}
File contents:
03 43 61 72 6C 73 6F 6E 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 02 03 04 05 06 08 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 43 61 72 6C 73 6F 6E 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 02 03 04 05 06 08 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 43 61 72 6C 73 6F 6E 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 02 03 04 05 06 08 00 01 02 03 04 05 06 07 08 09 0A 0B 0C
When you want to change the argument inside a function, you pass a pointer to it.
For example, inside Rtest you declared a char called num. It has no value, and you sent it to openf, But you actually sent the pointer to num since you wanted to change its value, you did it correctly and indeed openf changed num value successfully.
But how about array? Well, you declared it on Rtest and allocated space in memory for it, which is all correct. Then, you wanted to send it to Rtest as a pointer so the function could change it.
array is a variable of the type "pointer to struct individual". This is okay, but if you wanted to change it inside Rtest, well you need to send a pointer for that variables.. hence, you needed a "POINTER TO pointer to struct individual". Note that the variable name was copied from before and I just added "POINTER TO"
I'm sure you know what pointer to pointer is, and what you needed to do is use:
openf(&array, &num);
And of course modift openf as well so it will use the new "pointer to pointer", something like that:
void openf(struct individual **array, char *num){
FILE *fp;
fp = fopen("save.bin", "rb");
fread(num, 1, sizeof(char), fp);
*array = (struct individual **)realloc(*array, *num * sizeof(struct individual));
printf("%d\n", sizeof(**array));
fread(*array, *num, sizeof(struct individual), fp);
for (int i = 0; i < *num; i++) {printf("%s\n", (*array)[i].name);} //mark 1
fclose(fp);
}
When I run this code on my machine, along with Rtest and provided save.bin I get the following output:
53
Carlson
Carlson
Carlson
3
Carlson
Carlson
Carlson
EDIT:
As #WhozCraig mentioned in the comments, You could use the unused return value for the function and return the pointer for the "new" array, which might be a slightly better way of doing things here instead of the "pointer to pointer" stuff, but its up to you.

Opening and Reading from a "large" gzip compressed file in C

I have been trying to open and read a gzip compressed file using gzip-based file IO functions in C. The compressed file that I have with me is quite large of size 12 GB. The uncompressed file was ~260 GB and hence I am not prepared to uncompress the file using gunzip and go ahead from there.
I am specifically using the below code to read and write into the buffers available to us-
#define windowBits 15
#define ENABLE_ZLIB_GZIP 32
#define CHUNK 0x4000
#define CALL_ZLIB(x) { \
int status; \
status = x; \
if (status < 0) \
{ \
fprintf(stderr, "%s:%d: %s returned a bad status of %d.\n", __FILE__, __LINE__, #x, status); \
exit(EXIT_FAILURE);\
} \
} \
int main ()
{
const char * file_name = "test.gz";
FILE * file;
z_stream strm = {0};
unsigned char in[CHUNK];
unsigned char out[CHUNK];
strm.zalloc = Z_NULL;
strm.zfree = Z_NULL;
strm.opaque = Z_NULL;
strm.next_in = in;
strm.avail_in = 0;
CALL_ZLIB (inflateInit2 (& strm, windowBits | ENABLE_ZLIB_GZIP));
/* Open the file. */
file = fopen (file_name, "rb");
while (1) {
int bytes_read;
bytes_read = fread (in, sizeof (char), sizeof (in), file);
strm.avail_in = bytes_read;
do {
unsigned have;
strm.avail_out = CHUNK;
strm.next_out = out;
CALL_ZLIB (inflate (& strm, Z_NO_FLUSH));
have = CHUNK - strm.avail_out;
fwrite (out, sizeof (unsigned char), have, stdout);
}
while (strm.avail_out == 0);
if (feof (file)) {
inflateEnd (& strm);
break;
}
}
return 0;
}
The code accurately reads and writes from the zlib file based on a buffer that you specify initially. The buffer size is being fixed to a certain value (in the above case to 0x4000).
The problem now is that I cannot increase the size of this buffer beyond a certain value ( I can use 3276008 as buffer size, but not 32760008 ). To read a 12 GB compressed value, would need me to use a very big buffer. As specified in my edits, this looks like some kind of a DATA_ERROR not a BUFFERerror... so it is not a buffer error after all!
Is there any way how I can be able to record the whole 12 GB compressed file using the zlib functions above ?
EDIT #1
The error code returned by the function inflate is encapsulated by the CALL_ZLIB function which I am sorry to have not included. So I get the below error code when I run with the buffer size of 0x4000. I have added the CALL_ZLIB function to the code for your reference also.
Error msg :
parser.c:96: inflate(&strm, Z_NO_FLUSH) returned a bad status of -3. This obviously looks like a **DATA_ERROR.
EDIT #2
I have tried adding a negative value of windowBits to InflateInit2() but that did not solve any of my problems. The inflate() function initially reads my file correctly -- displaying all of my data the way I want it to..
0x55b0 [0x40]: event: 3
.
. ... raw event: size 64 bytes
. 0000: 03 00 00 00 00 00 40 00 18 03 00 00 18 03 00 00 ......#.........
. 0010: 4d 6f 64 65 6d 4d 61 6e 61 67 65 72 00 00 00 00 ModemManager....
. 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
. 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0 0 0x55b0 [0x40]: PERF_RECORD_COMM: ModemManager:792/792
0x55f0 [0x40]: event: 7
.
. ... raw event: size 64 bytes
. 0000: 07 00 00 00 00 00 40 00 19 03 00 00 01 00 00 00 ......#.........
. 0010: 19 03 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ................
. 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
. 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0 0 0x55f0 [0x40]: PERF_RECORD_FORK(793:793):(1:1)
0x5630 [0x40]: event: 3
.
But after some time, the displayed output becomes garbled and I cannot read from it anymore..
0x4d68 [0x38]: ........... 001 0..
0 0 00 00 00 0 00 000 00 ze 64s
. 0000: 07 00 00 00 00 00 40 00 19 03 00 00 01 00 00 00 .. 00 0 event: size 64 bytes
. 0000: 03 00 00 00 si sisizsiz4s
. 0000: 07 00 00 00 00 00 40 00 19 0....
. 0030: 00 00 00 00 00 00 00 00 00 00 00 00 ..#.#. 0010: 19 03 00 00 [0x38]: ........... 001 0..
0 0 00 00 00 0 00 000 00 ze 64s
. 0000: 07 00 00 00 00 00 40 00 100 00 00 00 00 ..............0 0 0x4d28 [0x40]: PERF_RECORD_FORK(135:135):(2:62)
0x4d68 [0x38]: ........... 001 0..
0 0 00 00 00 0 00 000 00 00 00 00: PERORD_FORK(135:135):(2:2)
This finally terminates with the error message I described in the Edit #1
I have resolved the problem.
The basic problem was that I was not initializing the strm.next_in member of z_stream in my code inside the loop. Hence after doing 1 iteration, the buffer got corrupted and I was getting the above errors.
I modified my code to --
strm.next_in = in;
strm.avail_in = 0;
CALL_ZLIB(inflateInit2 (&strm, windowBits | ENABLE_ZLIB_GZIP));
file = fopen(filename, "rb");
while(1)
{
int bytes_read;
strm.next_in = in; // added this line
bytes_read = fread(in, sizeof(char), sizeof(in), file);
strm.avail_in = bytes_read;
do
{
unsigned have;
strm.avail_out = CHUNK;
strm.next_out = out;

How can I write a structure to a file ?

I have tried to look this up in multiple places and I cannot understand why fwrite doesn't work.
If I had a structure with 100 fields I would not want to use fprintf with 100 format specifiers.
struct emp
{
char name[15];
int age;
int salary;
char address[30];
};
int main()
{
char str[60];
struct emp emp1[5] = {{"Yoda",23,45000,"Asia"},{"Darth",34,2344,"NAmerica"},{"Jabba",22,5566,"Africa"},{"Luke",33,3399,"SAmerica"},{"Laya",44,6677,"Europe"}};
FILE *fp;
fp = fopen("C:/.../sampleText.txt","w");`
int i=0;
for(i=0; i<5; i++)
{
fwrite(&emp1[i],sizeof(emp1[i]),1,fp);
//fprintf(fp,"%s, %d, %d, %s\n",&emp1[i].name,emp1[i].age,emp1[i].salary,emp1[i].address);
}
fclose(fp);
getch();
}
There are two answers:
It does work, if everything is set correctly and porting the written data to other machines is not an issue.
It doesn't work if you have any of a large number of common features in data structures, or if you need to move the data from one type of machine (say an Intel machine) to another type (say PowerPC or SPARC).
In your example structure, you have no pointers, so you could write the structure verbatim to a file, and then in another invocation of the program running on the same (type of) machine, you could read it back in, and you would see the same data.
However, if your structure contained pointers, you could not meaningfully write the structure to disk. The pointers in one invocation of the program need not have any significance in another invocation of the program. If you needed to port the data between a little-endian (Intel) and big-endian (PowerPC, SPARC) machine, you'd have to use a platform-neutral way of accessing the data; simply writing the data to disk would not work.
So, where portability is not an issue, this code should work — Unix or Windows. It uses the "wb" and "rb" arguments to fopen() because the data is binary data, not plain text. The b is optional but harmless on Unix; it is crucial on Windows. The code also fixes the file name to sampledata.bin so it can be run on either platform, writing in the current directory. It writes the data; it then reads the data; it then compares the read data with the written data, reporting any problems. If the program says nothing, all is OK.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct emp
{
char name[15];
int age;
int salary;
char address[30];
};
int main(void)
{
char const filename[] = "sampledata.bin";
struct emp emp1[5] =
{
{ "Yoda", 23, 45000, "Asia" },
{ "Darth", 34, 2344, "N America" },
{ "Jabba", 22, 5566, "Africa" },
{ "Luke", 33, 3399, "S America" },
{ "Leia", 44, 6677, "Europe" },
};
struct emp emp2[5];
FILE *ifp;
FILE *ofp;
int i;
ofp = fopen(filename, "wb");
if (ofp != 0)
{
if (fwrite(emp1, sizeof(emp1), 1, ofp) != 1)
{
fprintf(stderr, "Failed to write to %s\n", filename);
exit(1);
}
fclose(ofp);
}
ifp = fopen(filename, "rb");
if (ifp != 0)
{
if (fread(emp2, sizeof(emp2), 1, ifp) != 1)
{
fprintf(stderr, "Failed to read from %s\n", filename);
exit(1);
}
fclose(ifp);
}
for (i = 0; i < 5; i++)
{
if (emp1[i].age != emp2[i].age ||
emp1[i].salary != emp2[i].salary ||
strcmp(emp1[i].name, emp2[i].name) != 0 ||
strcmp(emp1[i].address, emp2[i].address) != 0)
printf("Difference in record %d\n", i);
}
return 0;
}
Content of the file sampledata.bin:
0x0000: 59 6F 64 61 00 00 00 00 00 00 00 00 00 00 00 00 Yoda............
0x0010: 17 00 00 00 C8 AF 00 00 41 73 69 61 00 00 00 00 ........Asia....
0x0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0030: 00 00 00 00 00 00 00 00 44 61 72 74 68 00 00 00 ........Darth...
0x0040: 00 00 00 00 00 00 00 00 22 00 00 00 28 09 00 00 ........"...(...
0x0050: 4E 20 41 6D 65 72 69 63 61 00 00 00 00 00 00 00 N America.......
0x0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0070: 4A 61 62 62 61 00 00 00 00 00 00 00 00 00 00 00 Jabba...........
0x0080: 16 00 00 00 BE 15 00 00 41 66 72 69 63 61 00 00 ........Africa..
0x0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00A0: 00 00 00 00 00 00 00 00 4C 75 6B 65 00 00 00 00 ........Luke....
0x00B0: 00 00 00 00 00 00 00 00 21 00 00 00 47 0D 00 00 ........!...G...
0x00C0: 53 20 41 6D 65 72 69 63 61 00 00 00 00 00 00 00 S America.......
0x00D0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00E0: 4C 65 69 61 00 00 00 00 00 00 00 00 00 00 00 00 Leia............
0x00F0: 2C 00 00 00 15 1A 00 00 45 75 72 6F 70 65 00 00 ,.......Europe..
0x0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0110: 00 00 00 00 00 00 00 00 ........
0x0118:
You don't specify what you mean by fwrite doesn't work, but I'll assume you're working on Windows, in which case you need to specify "wb" to fopen. By default on Windows, it's writing in text mode (i.e. "wt").
not a good idea to write struct to file or sockets as it is. It is inviting complex to solve problems. the best approach is to use serialization before writing. Also, as Jim pointed out above, make sure to open the file in binary.
Take a look in this question and the answers. there is a pretty good answer and explanation for your question.
Passing a structure through Sockets in C
Data serialization is a non-trivial task. As some others have pointed out, it is possible in some cases to write the contents of your struct to disk as binary data. It's the simplest to write, but it is unlikely to be stable. Each time you recompile your code, it can potentially change the format the data is written and read in.
Your best option is to use a standard data interchange format, such as CSV, XML, or JSON. There are many existing tools to utilize these formats, so you should look into using one of them.

C: edit file to remove null characters:

I thought this would be an easy task, after a couple of tries I try the tried and true write to a temp than reopen and rewrite:
#include <stdlib.h>
#include <stdio.h>
int main()
{
FILE *f = fopen("main2.c","r");
FILE *t = fopen("temp","w");
int c;
int count = 0;
while((c = fgetc(f))!=EOF)
{
if(c)
{
fputc(c,t);
}
else
{
printf("null found\n");
}
}
fclose(f);
fclose(t);
FILE *n = fopen("main2.c","w");
FILE *w = fopen("temp","r");
while((c=fgetc(w))!=EOF)
{
fputc(c,n);
}
fclose(n);
fclose(w);
return 0;
}
this just spits out a bunch of chinese characters. Could the underlying character encoding be the issue? Or am I just a total noob here?
My hex editor won't let me copy/paste. I don't know how I can get the file up here in its original condition so I have it zipped in google docs let me know immediately if you cant get it:
https://docs.google.com/open?id=0B4UPOuCR5uRGZzJQZUpVaktKYlk
EDIT: wait wait here it is via HxE Edit:
FF FE 23 00 69 00 6E 00 63 00 6C 00 75 00 64 00 65 00 20 00 3C 00 73 00 74 00 64 00
6C 00 69 00 62 00 2E 00 68 00 3E 00 0D 00 0A 00 23 00 69 00 6E 00 63 00 6C 00 75 00 64 00
65 00 20 00 3C 00 61 00 6C 00 6C 00 65 00 67 00 72 00 6F 00 2E 00 68 00 3E 00 0D 00 0A 00
23 00 69 00 6E 00 63 00 6C 00 75 00 64 00 65 00 20 00 22 00 6D 00 6F 00
Open the files in binary mode:
FILE *f = fopen("main2.c","rb");
FILE *t = fopen("temp","wb");
Odds are that you are removing NULL bytes because the input is UTF-16 Unicode. If so, you also must remove the byte-order mark (BOM) at the start of the file. If the first two bytes are 0xFF, 0xFE then you have a little-endian UTF-16 file. Discard them! If you leave them in, every pair of ASCII characters in your source will be treated as a combined 16-bit character code. Strangeness will ensue.
Likewise if the first two bytes are 0xFE, 0xFF, the file is big-endian UTF-16 and you must also delete those two bytes, else the file will be treated as 16-bit codes again, only with high bytes first.

How can I change char array to int using C?

i am studying C now, and I am parsing a raw registry file and read it.
i have some problem now,
000011E0 00 00 00 00 60 01 00 00 B9 01 00 00 00 00 00 00
000011F0 20 C0 26 00 FF FF FF FF 00 00 00 00 FF FF FF FF
00001200 10 FC 00 00 FF FF FF FF 4C 00 01 00 00 00 00 00
this is hex value of REGISTRY file.
fseek(fp,0x11F0,SEEK_SET);
char tmp[4];
int now = ftell(fp);
for(int i = 0 ; i < 4 ; i++){
tmp[i] = fgetc(fp);
}
I made this tmp array, but I need 0x0026c020.
how can I change this array to that value? or please suggest me better algorithm.
Thanks.
If you know for a fact that the value is stored with the same endianness as the host OS architecture, you can just do:
int value = *(int *)tmp;
However, you should not read the bytes in backwards order, as you do here -- that alters the endianness and will result in an incorrect value. Try this:
int value;
if (fread(&value, sizeof(value), 1, fp) != 1) {
/* Could not read, handle error. */
}
/* value is set, inspect it */
To convert a string into integer there are already available functions one such function is
strtoul().
you can use standard strtoul() function to convert string into integer values.

Resources