How do I read/write a block device? I heard I read/write like a normal file so I setup a loop device by doing
sudo losetup /dev/loop4 ~/file
Then I ran the app on the file then the loop device
sudo ./a.out file
sudo ./a.out /dev/loop4
The file executed perfectly. The loop device reads 0 bytes. In both cases I got FP==3 and off==0. The file correctly gets the string length and prints the string while the loop gets me 0 and prints nothing
How do I read/write to a block device?
#include <fcntl.h>
#include <cstdio>
#include <unistd.h>
int main(int argc, char *argv[]) {
char str[1000];
if(argc<2){
printf("Error args\n");
return 0;
}
int fp = open(argv[1], O_RDONLY);
printf("FP=%d\n", fp);
if(fp<=0) {
perror("Error opening file");
return(-1);
}
off_t off = lseek(fp, 0, SEEK_SET);
ssize_t len = read(fp, str, sizeof str);
str[len]=0;
printf("%d, %d=%s\n", len, static_cast<int>(off), str);
close(fp);
}
The losetup seems to map file in 512-byte sectors. If file size is not multiples of 512, then the rest will be truncated.
When mapping a file to /dev/loopX with losetup,
for fiile which is smaller than 512 bytes it gives us following warning:
Warning: file is smaller than 512 bytes;
the loop device may be useless or invisible for system tools.
For file which the size cannot be divided by 512:
Warning: file does not fit into a 512-byte sector;
the end of the file will be ignored
This warning was added since util-linux ver 2.22 in this commit
You can not put zeros or random values on the file to get 512 byte alignment. Use the first few byte to store the file size, followed by the file content. Now you know where the file content is ending. You put random data to achieve the 512 alignment.
e.g. File structure:
[File Size] [Data][<padding to get 512 alignment>]
Related
I'm currently developing a program that mimics UNIX file system. I've prepared my disk as file (1 MB) got all data blocks inside it. Now what I'm doing is implementing some simple commands like mkdir, ls etc. In order to work with those commands, I need to read specific offset(no problem with that) and write the modified blocks to specific location.
Simply my goal is:
SIIIDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD (Current Disk)
I wan't to change three blocks with AAA after 16.byte so it will be like:
SIIIDDDDDDDDDDDDAAADDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD (Modified Disk)
I'm not going to provide all of my implementation here I just want to have some ideas about it how can I implement it without buffering all the 1 MB data in my program. In short I know locations of my data blocks so I just want to replace that part of my file not whole file. Can't I simply do this with file stream functions ?
Another example:
fseek(from_disk,superblock.i_node_bit_map_starting_addr , SEEK_SET); //seek to known offset.
read_bit_map(&from_disk); // I can read at specific location without problem
... manipulate bit map ...
fseek(to_disk,superblock.i_node_bit_map_starting_addr , SEEK_SET); //seek to known offset.
write_bit_map(&to_disk); //Write back the data.
//This will destroy the current data of file. (Tried with w+, a modes.)
Note: Not provided in example but I have two file pointers both writing and reading and I'm aware I need to close one before opening another.
I think you are looking for the r+ (potentially rb+ mode). Here is a complete example, afterwards you can run grep -n hello data.txt to verify for yourself the result. You can run it with make prog && ./prog.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(int argc, char const *argv[])
{
FILE *file;
file = fopen("data.txt", "w+");
char dummy_data[] = "This is stackoverflow.com\n";
int dummy_data_length = strlen(dummy_data);
for (int i = 0; i < 1000; ++i)
fwrite(dummy_data, dummy_data_length, 1, file);
fclose(file);
file = fopen("data.txt", "r+");
fseek(file, 500, SEEK_CUR);
fwrite("hello", 5, 1, file);
fclose(file);
return 0;
}
I am using the following program to find out the size of a file and allocate memory dynamically. This program has to be multi-platform functional.
But when I run the program on Linux machine and on a Windows machine using Cygwin, I see different outputs — why?
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
/*
Observation on Linux
When reading text file remember
the content in the text file if arranged in lines like below:
ABCD
EFGH
the size of file is 12, its because each line is ended by \r\n, so add 2 bytes for every line we read.
*/
off_t fsize(char *file) {
struct stat filestat;
if (stat(file, &filestat) == 0) {
return filestat.st_size;
}
return 0;
}
void ReadInfoFromFile(char *path)
{
FILE *fp;
unsigned int size;
char *buffer = NULL;
unsigned int start;
unsigned int buff_size =0;
char ch;
int noc =0;
fp = fopen(path,"r");
start = ftell(fp);
fseek(fp,0,SEEK_END);
size = ftell(fp);
rewind(fp);
printf("file size = %u\n", size);
buffer = (char*) malloc(sizeof(char) * (size + 1) );
if(!buffer) {
printf("malloc failed for buffer \n");
return;
}
buff_size = fread(buffer,sizeof(char),size,fp);
printf(" buff_size = %u\n", buff_size);
if(buff_size == size)
printf("%s \n", buffer);
else
printf("problem in file size \n %s \n", buffer);
fclose(fp);
}
int main(int argc, char *argv[])
{
printf(" using ftell etc..\n");
ReadInfoFromFile(argv[1]);
printf(" using stat\n");
printf("File size = %u\n", fsize(argv[1]));
return 0;
}
The problem is fread reading different sizes depends on compiler.
I have not tried on proper windows compiler yet.
But what would be the portable way to read contents from file?
Output on Linux:
using ftell etc..
file size = 34
buff_size = 34
ABCDEGFH
IJKLMNOP
QRSTUVWX
YX
using stat
File size = 34
Output on Cygwin:
using ftell etc..
file size = 34
buff_size = 30
problem in file size
ABCDEGFH
IJKLMNOP
QRSTUVWX
YX
_ROAMINGPRã9œw
using stat
File size = 34
Transferring comments into an answer.
The trouble is probably that on Windows, the text file has CRLF line endings ("\r\n"). The input processing maps those to "\n" to match Unix because you use "r" in the open mode (open text file for reading) instead of "rb" (open binary file for reading). This leads to a difference in the byte counts — ftell() reports the bytes including the '\r' characters, but fread() doesn't count them.
But how can I allocate memory, if I don't know the actual size? Even in this case also the return value of fread is 30/34, but my content is only of 26 bytes.
Define your content — there's a newline or CRLF at the end of each of 4 lines. When the file is opened on Windows (Cygwin) in text mode (no b), then you will receive 3 lines of 9 bytes (8 letters and a newline) plus one line with 3 bytes (2 letters and a newline), for 30 bytes in total. Compared to the 34 that's reported by ftell() or stat(), the difference is the 4 CR characters ('\r') that are not returned. If you opened the file as a binary file ("rb"), then you'd get all 34 characters — 3 lines with 10 bytes and 1 line with 4 bytes.
The good news is that the size reported by stat() or ftell() is bigger than the final number of bytes returned, so allocating enough space is not too hard. It might become wasteful if you have a gigabyte size file with every line containing 1 byte of data and a CRLF. Then you'd "waste" (not use) one third of the allocated space. You could always shrink the allocation to the required size with realloc().
Note that there is no difference between text and binary mode on Unix-like (POSIX) systems such as Linux. It does not do mapping of CRLF to NL line endings. If the file is copied from Windows to Linux without mapping the line endings, you will get CRLF at the end of each line on Linux If the file is copied and the line endings are mapped, you'll get a smaller size on Linux than under Cygwin. (Using "rb" on Linux does no harm; it doesn't do any good either. Using "rb" on Windows/Cygwin could be important; it depends on the behaviour you want.)
See also the C11 standard §7.21.2 Streams and also §7.21.3 Files.
I want to read a file bytewise into an array and then write the data of the array reversed
in a new file (program takes filename over command line argument). Tried it with an txt-file
and it worked, but if I try it on a jpg-file the new file is bigger than the original!
The determined file size saved in long size; is also correct for jpg-files and write loop
get size-time executed writing one char (char is one byte big, I am right?).
Does anybody know how the output file can get bigger than size*byte?
It doesn't seem logical to me!
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc,char* argv[])
{
FILE *file;
char *buffer;
long size;
char filename[32];
if(argc>1)
{
//determine file size
file=fopen(argv[1],"r");
fseek(file,0,SEEK_END);
size=ftell(file);
rewind(file);
if(size>33554432) //32MB
{
fclose(file);
return 0;
}
//create buffer and read file content
buffer=malloc(33554432);
fread(buffer,1,size,file);
fclose(file);
//create new file name and write new file
strcpy(filename,argv[1]);
strcat(filename,"_");
file=fopen(filename,"w");
{
long i;
for(i=size-1;i>=0;i--)
{
fputc(buffer[i],file);
}
}
fclose(file);
free(buffer);
}
return 0;
}
The comments you're receiving are implying something: the newline character \n works differently in text mode on Windows compared with some other systems.
fputc('\n', file) on Windows actually writes two bytes if file was opened in text mode "w", as if you did fwrite("\r\n", 1, 2, file). This means for any \n byte read by fread, you're writing two bytes back.
If you want to write binary data back, you need to open your output file using the mode "wb" to fopen(). You also need to open it for reading in binary mode, "rb".
I am trying to build my own C program that basically works just like fdisk vdisk 'p' command. I just want to be able to read in the first 512 bytes of the disk, lseek to the start of the partitions at (0x1BE) and then read the partition type, name, size, ect. I am unsure how to actually read these values. I have used the read() linux function to read in 512 bytes but when I try displaying/viewing them in any way, nothing is shown. What am I doing wrong?
int main(int argc, char *argv[]) {
int bytes_read;
char mbr[512];
int file;
if(argc == 1) {
// Print some help info
printf ("Here is some help info: \n\n");
} else if(argc < 3) {
printf("File: %s\n\n", argv[1]);
file = open(argv[1], O_RDONLY);
lseek(bytes_read, 0, 0);
//First get the MBR
bytes_read = read(file, mbr, 512);
printf("MBR=%s\n\nbytes_read=%d\n\n", mbr, bytes_read);
} else {
printf ("Incorrect usage: fdisk <disk>\n\n");
}
}
Don't try to use printf with binary data. If your binary data starts with a NUL (ASCII 0), then printf will assume you've got an empty string. You can use write() to write out arbitrary data (it takes a buffer and length), e.g:
#include <unistd.h>
write(STDOUT_FILENO, mbr, 512)
...but even this won't necessarily display anything useful, because your terminal may try to interpret control characters in the output. You're best bet would then be to pipe the output to something like xxd or od, both of which will produce a hexdump of their input data.
For example, the first 512 bytes of my local drive are all NUL. Using write() in your code (and removing that lseek) results in 512 NUL bytes on output. Try passing something other than disk to your code, e.g.:
myexe /etc/passwd
The structure of a standard DOS MBR is documented here, suggesting that you might start with data structures like this:
struct _partition {
uint8_t status;
uint8_t chs_start[3];
uint8_t part_type;
uint8_t chs_end[3];
uint32_t lba_start;
uint32_t sectors;
};
And populate it something like this:
fd = open(target, O_RDONLY);
lseek(fd, 446, SEEK_SET);
for (i=0; i<4; i++) {
struct _partition p;
bytes_read = read(fd, &p, sizeof(struct _partition));
// assume this prints information to stdout or something.
print_part(i, &p);
}
Get rid of the lseek. Your compiler should be throwing a warning right now, as you're passing it an argument (bytes_read) which has not been initialized.
Once that's done, you'll need to do something to display the contents; right now, you've got nothing that uses the data you read.
I'm using fmemopen to create a variable FILE* fid to pass it to a function the reads data from an open file.
Somewhere in that function it uses the following code to find out the size of the file:
fseek(fid, 0, SEEK_END);
file_size = ftell(fid);
this works well in case of regular files, but in case of file ids created by fmemopen I always get file_size = 8192
Any ideas why this happens?
Is there a method to get the correct file size that works for both regular files and files created with fmemopen?
EDIT:
my call to fmemopen:
fid = fmemopen(ptr, memSize, "r");
where memSize != 8192
EDIT2:
I created a minimal example:
#include <cstdlib>
#include <stdio.h>
#include <string.h>
using namespace std;
int main(int argc, char** argv)
{
const long unsigned int memsize = 1000000;
void * ptr = malloc(memsize);
FILE *fid = fmemopen(ptr, memsize, "r");
fseek(fid, 0, SEEK_END);
long int file_size = ftell(fid);
printf("file_size = %ld\n", file_size);
free(ptr);
return 0;
}
btw, I am currently working on another computer, and here I get file_size=0
In case of fmemopen , if you open using the option b then SEEK_END measures the size of the memory buffer. The value you see must be the default buffer size.
OK, I have got this mystery solved by myself. The documentation says:
If the opentype specifies append mode, then the initial file position is set to the first null character in the buffer
and later:
For a stream open for reading, null characters (zero bytes) in the buffer do not count as "end of file". Read operations indicate end of file only when the file position advances past size bytes.
It seems that fseek(fid, 0, SEEK_END) goes to the first zero byte in the buffer, and not to the end of the buffer.
Still looking for a method that will work on both standard and fmemopen files.