Strange 0x0D being added to my binary file - c

I have this strange problem:
I write 16 chars to a binary file and then I write 3 integers but when I open my file with some binary file viewer, I see an extra byte is added (which equals 0x0D).
Here's my code:
for(i = 0; i < 16; i++)
{
if(i < strlen(inputStr))
{
myCharBuf[0] = inputStr[i];
}
else
{
myCharBuf[0] = 0;
}
fwrite(myCharBuf, sizeof(char), 1, myFile);
}
myIntBuf[0] = inputNumber1;
fwrite(myIntBuf, sizeof(int), 1 ,myFile);
myIntBuf[0] = inputNumber2;
fwrite(myIntBuf, sizeof(int), 1 ,myFile);
myIntBuf[0] = inputNumber3;
fwrite(myIntBuf, sizeof(int), 1 ,myFile);
I get the following byte-values:
61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0D 0A 00 00 00 05 00 00 00 08 00 00 00
When I expect:
61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0A 00 00 00 05 00 00 00 08 00 00 00
Does anyone have an idea why it might happen?

0A is the line feed character and 0D is the carriage return. These are usually associated with text mode.
Have you opened the file in binary mode? (e.g. fopen("foo.txt", "wb"))

When you open the file, open for writing as binary "wb":
fopen(filename, "wb");
When you open in text mode, translation of Line Feeds (0A) and Carriage Returns (0D) occurs.

fopen the file in binary mode with "wb".
fopen(filename, "wb");
otherwise, the code in the library will do automatic line end translation (on windows you are on Windows, are you not? that means translate '\n' to '\r' '\n').

I believe that your inputStr variable contains a newline character and it is written to the binary file as carriage return and linefeed - binary '0D' followed by '0A'.
For eg, the following program writes 16 characters and 3 numbers as follows.
FILE *fp;
fp = fopen("sample.bin", "wb+");
if(fp == NULL)
{
printf("Cannot create a file\n");
return;
}
int i;
char c[1] = {'A'};
for(i = 0; i < 16; i++)
{
fwrite(c, sizeof(char), 1, fp);
c[0]++;
}
int ip[1] = {1};
fwrite(ip, sizeof(int), 1, fp);
fwrite(ip, sizeof(int), 1, fp);
fwrite(ip, sizeof(int), 1, fp);
fclose(fp);
If the 'sample.bin' file is viewed using a dump program such as 'od', it gives the content as follows.
od -t x1 -c sample.bin
0000000 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50
A B C D E F G H I J K L M N O P
0000020 01 00 00 00 01 00 00 00 01 00 00 00
001 \0 \0 \0 001 \0 \0 \0 001 \0 \0 \0
0000034

MS-DOS (and so today with Windows), when writing a file in text mode, adds an 0x0D before every 0x0A. In other words, it processes arbitrary data streams as they go to and from store and messes with their data - utterly, utterly insane.
Open the file in binary mode for non-insane handling.

This code
#include <stdio.h>
#define SECTORSIZE 512 // bytes per sector
int main(int argc, char *argv[])
{
FILE *fp; // filepointer
size_t rdcnt; // num. read bytes
unsigned char buffer[SECTORSIZE];
if(argc < 2)
{
fprintf(stderr, "usage:\n\t%s device\n", argv[0]);
return 1;
}
fp = fopen(argv[1], "rb");
if(fp == NULL)
{
fprintf(stderr, "unable to open %s\n",argv[1]);
return 1;
}
rdcnt = fread(buffer, 1, SECTORSIZE, fp);
if(rdcnt != SECTORSIZE)
{
fprintf(stderr, "reading %s failed\n", argv[1]);
fclose(fp);
return 1;
}
fwrite(buffer, 1, SECTORSIZE, stdout);
fclose(fp);
return 0;
}
kindly taken from here
https://redeaglesblog.wordpress.com/2011/04/05/sektoren-eines-datentragers-lesen/
reads the boot sector from any given disk
Pasted as it is in your preferred C (ANSI) IDE or editor, it compiles and works either in windows (mingw passing \.\PhysicalDriveX) and linux (gcc passing /dev/sdX)
But it works like a charm ONLY in Linux while it anyway inserts/adds an x0D preceding any x0A despiting the fp = fopen(argv[1], "rb");
I've compiled it from code::blocks with mingw as readsect.exe and run it reading the boot sector of my hard drive
c:\readsect.exe \\.\PhysicalDrive0 > read.bin
The file read.bin results with a lenght of 515 bytes instead than 512.
With an HEX editor able to open physical drives, I've compared my boot sector content with the read.bin one.
Well, every x0A in the physical boot sector (x0A is found 3 times), in the read.bin file is dumped as x0D + X0A. So I have three x0D, three bytes more.
Googleing, it looks like a widely reported problem.
Do any of you have found a fix?
Maybe stdio.h needs a fix for the windows environment?
Thank you

Related

I'm losing binary data when using fseek in c

I wrote a function that is responsible for moving to a binary file and editing its bytes
int replace(FILE *binaryFile, long offset, unsigned char *replaced, int length) {
if (binaryFile != NULL) {
for (int i = 0; i < length; i++) {
fseek(binaryFile, offset + i, SEEK_SET);
fwrite(&replaced[i], sizeof(*replaced), 1, binaryFile);
}
fclose(binaryFile);
return 0;
}
else return -1;
}
When I use this function, I encounter a strange problem
All data in the file is filled with NULL bytes
And only one of the addresses in the file changes
Example:
FILE* fp = fopen("target.bin", "wb");
replace(fp, 0x57d8b0, "\x1E\xFF\x2F\xE1", 4);
replace(fp, 0x57c770, "\x01\x00\xA0\xE3\x1E\xFF\x2F\xE1", 8);
Result:
0x57c770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
0x57d8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x57d8b0: 1e ff 2f e1
Correct result:
0x57c770: 01 00 A0 E3 1E FF 2F E1 ...
...
0x57d8a0: 9c c4 0a ea 70 d2 68 00 44 d4 68 00 10 d1 68 00
0x57d8b0: 1e ff 2f e1 18 b0 8d e2 02 8b 2d ed 18 d0 4d e2
Please help me to solve the function problem or other problems.
wb is not correct. It clobbers the file.
From here,
Mode
Meaning
Explanation
If already exists
If does not exist
"r"
read
Open a file for reading
read from start
failure to open
"w"
write
Create a file for writing
destroy contents
create new
"a"
append
Append to a file
write to end
create new
"r+"
read extended
Open a file for read/write
read from start
error
"w+"
write extended
Create a file for read/write
destroy contents
create new
"a+"
append extended
Open a file for read/write
write to end
create new
You want r+b.
Also, you close the file handle in replace, which is premature. This should be done outside of replace, after you're done with the handle.
As an aside, you shouldn't be doing a number of seeks and writes of length one equal to length; you should be doing one seek and one write of length length.

Why does C use 5 bytes to write/fwrite the number 10, but 4 bytes to write 9 or 11?

GCC on Windows
#include <stdio.h>
#include <stdlib.h>
struct test {
int n1;
int n2;
};
int main() {
FILE *f;
f = fopen("test.dat", "w");
struct test test1 = {10, 10};
fwrite(&test1, sizeof(struct test), 1, f);
fclose(f);
struct test test2;
f = fopen("test.dat", "r");
while(fread(&test2, sizeof(struct test), 1, f))
printf("n1=%d n2=%d\n", test2.n1, test2.n2);
fclose(f);
return 0;
}
If I set test1 to 10,10 then fwrite will write 10 bytes to file: 0D 0A 00 00 00 0D 0A 00 00 00
(each 4-byte int will be padded with a 0D carriage return character before it)
If I set test1 to 11,11 then fwrite will write 8 bytes to file: 0B 00 00 00 0B 00 00 00
(as I would expect)
If I set test1 to 9,9 then fwrite will write 8 bytes to file: 09 00 00 00 09 00 00 00
(as I would expect)
If I set test1 to 9,10 then fwrite will write 9 bytes to file: 09 00 00 00 0D 0A 00 00 00
The 9 gets 4 bytes as expected, but the 10 gets padded with an extra 0D byte, resulting in 5 bytes. What is so special about the number 10 that requires this padding? And why do both smaller and larger numbers (8, 9, 11, 12, 13, 14, etc) not get padded? I thought maybe GCC is confusing the number 10 for a new-line character (a new-line is 10 is acsii), but this does not explain how fread is able to get the number 10 back out correctly.
And how do I write a struct to file without getting this extra padding on the number 10?
You opened the file in text mode, so on Windows every '\n' character gets a carriage return prepended in front of it.
You should write (and read) binary data in binary mode instead (fopen(..., "wb")) -- this will be much faster, and avoids surprises (and also requires only 8 bytes, which is what sizeof(struct test) is).
What is so special about the number 10 that requires this padding?
The number 10 just happens to be the ASCII code for a newline ('\n') character.
You are writing and reading in text mode.
Open the file with the flags "wb" and "rb". This will treat the files as binary files.

Segmentation fault(core dumped ) error while reading a python generated binary array in C [duplicate]

This question already has answers here:
Crash or "segmentation fault" when data is copied/scanned/read to an uninitialized pointer
(5 answers)
Closed 3 years ago.
I am trying to load a 2D array created by numpy and read the elements in C, but I get Segmentation fault(core dumped ) error while running it. The code goes by the lines of
#include <stdio.h>
#include <string.h>
int main(){
char *file;
FILE *input;
int N1, N2, ii, jj;
float element;
strcpy(file, "/home/caesar/Desktop/test.bin");
input = fopen(file, "rb");
fread(&N1, sizeof(int), 1, input);
fread(&N2, sizeof(int), 1, input);
float memoryarray[N1][N2];
for(ii= 0; ii<N1; ii++){
for(jj=0; jj<N2; jj++){
fread(&element, sizeof(float), 1, input);
memoryarray[ii][jj]= element;
}
}
printf("%f", memoryarray[2][3]);
fclose(input);
return 0;
}
This is the starting for a task where I will have to read elements from matrices of the form 400*400*400 or so. The idea is to read all elements from the file and store it in memory and then read from memory index wise when necessary, for example, here i am trying to access and print the element in the second row third column.
P.S: I am quite new to pointers.
Dear all, I tried the methods you said., here is the modified version of the code, the segmentation fault error is gone but the output is either all zeros, or is just plain garbage values.
I ran the executable three times and the outputs I got were
Output1: -0.000000
Output 2: 0.000000
Output 3 : -97341413674450944.000000
My array contains integers btw
Here is the modified version of the code
#include <stdio.h>
#include <string.h>
void main(){
const char file[] ="/home/caesar/Desktop/test.bin";
FILE *input;
int N1, N2, ii, jj;
float element;
//strcpy(file, "/home/caesar/Desktop/test.bin");
input = fopen(file, "r");
fread(&N1, sizeof(int), 1, input);
fread(&N2, sizeof(int), 1, input);
float memoryarray[N1][N2];
for(ii= 0; ii<N1; ii++){
for(jj=0; jj<N2; jj++){
fread(&element, sizeof(float), 1, input);
memoryarray[ii][jj]= element;
}
}
printf("%f", memoryarray[1][2]);
fclose(input);
Also here is the hex dump of the file that i am trying to open. Some of you asked me to verify whether fopen() is working or not, i checked, it is working.
00000000 00 00 40 40 00 00 40 40 01 00 00 00 00 00 00 00 |..##..##........|
00000010 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 |................|
*
00000030 04 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 |................|
00000040 05 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 |................|
00000050
So here is my problem in brief. I have multidimensional arrays of double precision floats written to a file using python. I want to take those files and access the elements whenever necessary by using the index of the elements to get the values. Any C code to do so would solve my problem.
Here is the python code i am using to write the file
with open('/home/caesar/Desktop/test.bin', 'wb') as myfile:
N= np.zeros(2, dtype= np.float32, order= "C")
N[0]= 3
N[1]= 3
a= [[1,2,3],[2,3,4], [4,5,6]]
N.astype(np.float32).tofile(myfile)
b= np.asarray(a)
b.tofile(myfile)
strcpy(file, "/home/caesar/Desktop/test.bin");
This writes to a garbage memory address.
You should either declare file as an array of suitable size, like this:
char file[100];
or
initialize the char pointer directly with the path like this (and get rid of the strcpy):
const char *file = "/home/caesar/Desktop/test.bin";
or the best, as per common consensus (refer comments):
fopen("/home/caesar/Desktop/test.bin", "rb");

fwrite in binary mode of ints not writing proper number of bytes

I am not able to understand the fread fwrite behavior of the following code snippet, exemplified by the code is straightforward:
#include <stdio.h>
int main(void)
{
FILE *fp;
int arr[10] = {1,2,3,4,5,6,7,8,9,10};
int temp[100] = {0};
int i;
fp = fopen("testdata.bin","wb");
if( fp!= NULL ) {
fwrite( arr,sizeof(int), 10, fp);
fclose(fp);
}
fp = fopen("testdata.bin","rb");
if( fp!= NULL ) {
fread( temp,sizeof(int), 10, fp);
fclose(fp);
}
for(i=0;i<100;i++)
printf("%#x,",temp[i]);
printf("\b \n");
return 0;
}
The output on stdout is:
0x1,0x2,0x3,0x4,0x5,0x6,0x7,0x8,0x9,0xa,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
which makes sense. However, when I open the testdata.bin file, I see only two bytes for value (int) where I expect 4 bytes as size of int is 4 on my machine.
Here is the content of testdata.bin:
0x00000001: 01 00 02 00 03 00 04 00 05 00 06 00 07 00 08 00
0x00000010: 09 00 0a 00
I would have expected
0x00000001: 01 00 00 00 02 00 00 00 03 00 00 00 ...
Any ideas?
I think fwrite is working fine. Change the declaration of temp so the type is an array of one-byte unsigned characters:
unsigned char temp [100] = {0} ;
The current version of the code displays a four-byte integer each time in the print statement. This will confirm that the contents of the file are as you expect. On my machine:
0x1,0,0,0,0x2,0,0,0,0x3,0,0,0,0x4,0,0,0,0x5,0,0,0,0x6,0,0,0,0x7,0,0,0,0x8,0,0,0,0x9,0,0,0,0xa,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

How to write file at specific position?

I made a function in C to create a file using Unix command dd:
dd if=/dev/zero of=file.data bs=8 count=32
This works and created file.data with size 256 bytes and if I open the file I can see it's empty.
Now I want to write to a specific position in this file using fseek and fwrite, but whenever I try to write to a position different from 0, it does nothing.
For example, If I want to write to position 2, I must also write to position 0 and 1.
void createFile() {
char command[100];
sprintf(comando, "dd if=/dev/zero of=file.data bs=8 count=32");
system(command);
}
void writeFile(int position, char * data) {
FILE * file = fopen("file.data", "r+");
fseek(file, position, SEEK_SET);
fwrite(data, strlen(data), 1, file);
fclose(file);
}
Some examples
Input:
writeFile(0, "0");
writeFile(1, "1");
writeFile(2, "2");
output > 012
Input:
writeFile(2, "2");
writeFile(1, "1");
writeFile(0, "0");
output > 012
Input:
writeFile(1, "1");
writeFile(2, "2");
output > empty
Is there some way to write into the file without having to write into the previous positions also?
You don't have to do anything special. Your code works, as long as you know how to demonstrate that it works. Here's a mildly extended version of it:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static void createFile(void)
{
char command[100];
sprintf(command, "dd if=/dev/zero of=file.data bs=8 count=32"); // Typo fixed!
system(command);
}
static void writeFile(int position, char *data)
{
FILE *file = fopen("file.data", "r+");
fseek(file, position, SEEK_SET);
fwrite(data, strlen(data), 1, file);
fclose(file);
}
int main(void)
{
createFile();
system("odx file.data");
writeFile(2, "012");
system("odx file.data");
return 0;
}
The odx command is a hex dump program; you could use od -c or xxd -g1 instead.
The sample output is:
32+0 records in
32+0 records out
256 bytes transferred in 0.000109 secs (2349544 bytes/sec)
0x0000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
* (15)
0x0100:
0x0000: 00 00 30 31 32 00 00 00 00 00 00 00 00 00 00 00 ..012...........
0x0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
* (14)
0x0100:
The first three lines are from dd. I'm not convinced that using dd is necessary, but it does no great harm. The next three lines indicate that the first 16 bytes in the file are all zero bytes, and that this pattern repeats for 15 more lines, and then you reach EOF at offset 0x100 (25610). The next four lines show that there are 2 null bytes, then the three digits 012, then all null bytes to the end of file.

Resources