I need to implement the following function as part of a kernel module:
ssize_t write(struct file *filp, const char *buf, size_t count, loff_t *f_pos)
which writes buf into the file filp using f_pos, and then updates f_pos.
my question is how can I write to f_pos and then change it's value?
I understand that it points to the start of the free file space but how can I update it after i'm done writing?
Thank you
I am writing a character device driver, but at the moment it freezes, and i have to reboot to stop it or crashes and the terminal exits
I have a global array
char* array;
On which i use kmalloc(9, GFP_KERNEL) so it should be the size of 9. If i wanted to use file operations .write to set a specific index how would i do that?
This is my current code (which crashes and terminal exits)
ssize_t mydriver_write(struct file *filp, const char* buf, size_t count, loff_t *f_pos)
{
raw_copy_to_user(array[*buf], 'x', 1);
}
EDIT:
I have already tried this version aswell
raw_copy_to_user(&array[3], x, 1);
where x is kmalloc'd to size 1 and x[0]='x'
But in this case my program freezes and i cannot remove the driver and the machine requires a reboot to remove it.
I have an uint8_t array of raw data that I want to write to a file (I have it's length)
The problem is that because I'm dealing with raw data there might be a 0x00 (aka null terminator) somewhere, meaning fputs is not reliable, the obvious alternative is to have a loop to use fputc() but is there a way i can do it without that?
Is there say a function that takes a pointer and a size and writes that amount of data from the pointer's location to the file?
In addition to the problem with null-character, there is problem reading binary data when file is opened in text mode (for example fgets stops when it encounters new line or 0x0A and 0x1A character in Windows)
Open the file in binary mode instead, and use fread/fwrite
FILE *fout = fopen("test.bin", "wb");
And use fwrite and fread
Reference
fread and fwrite are your friends.
uint8_t TheData[NUMBER_OF_ARRAY_ITEMS] = {0};
// ... Transformations to your data ...
// Persist the data
FILE *fHandleOutput = fopen("test.bin", "wb");
if(!fHandleOutput){
printf("Error: Output file handle was NULL!\n");
return;
}
// SIGNATURE: fwrite(const void *restrict ptr, size_t size, size_t nitems, FILE *restrict stream);
fwrite(TheData, sizeof(TheData[0]), NUMBER_OF_ARRAY_ITEMS, fHandleOutput);
fflush(fHandleOutput); // Ensure changes get written to disk before we close
fclose(fHandleOutput);
fHandleOutput = NULL;
// Read the data
// Incoming data buffer
uint8_t TheData[NUMBER_OF_ARRAY_ITEMS] = {0};
// Attempt file open for binary mode
FILE *fHandleInput = fopen("test.bin", "rb");
if(!fHandleInput){
printf("Error: Input file handle was NULL!\n");
return;
}
// SIGNATURE: fread(void *restrict ptr, size_t size, size_t nitems, FILE *restrict stream);
size_t iRead = fread(TheData, sizeof(TheData[0]), NUMBER_OF_ARRAY_ITEMS, fHandleInput);
fclose(fHandleInput);
fHandleInput = NULL;
It's worth noting that the return value of fread can be used to detect End-of-File (EOF) and I/O errors. If iRead < NUMBER_OF_ARRAY_ITEMS, then either an error occurred, or there were only iRead-number of sizeof(TheData[0])-byte segments between the filepointer's position and the EOF. (feof(...) or ferror(...) can be used to determine the cause of a low item read count.)
I need to read a specific set of bytes from a file type for a project I'm working on.
For example, let's say we have the following open file called "image":
int fd = open(image, O_RDWR, S_IRWXU);
Let's say I needed to read a specific section of this file starting from an offset from the beginning, say, a section starting at 1024 bytes and ending at 2048 bytes into the file.
I know we have read(),
int rd = read(fd, &example, sizeof(1024));
and this would read into "example", the first 1024 bytes.
I know fseek exists, which would set a pointer starting where I want to start reading into "example", but only if I have a FILE type, right? I currently only have a name and it's fd,
char *image;
int fd;
Can I use file_name or fd in fseek? Or is there a better way to do this?
On Linux, 'pread()' does the trick:
ssize_t pread(int fd, void *buf, size_t count, off_t offset);
It will seek to 'offset', and then read 'count' bytes int 'buf'.
lseek is the equivalent function to fseek, using file descriptors instead of FILE pointers. You can also use pread as Mahonri Moriancumer noted.
Ok, I have been reading up on fread() [which returns a type size_t]and saw several posts regarding large files and some issues others have been having - but I am still having some issues. This function passes in a file pointer and a long long int. The lld is from main where I use another function to get the actual filesize which is 6448619520 bytes.
char *getBuffer(FILE *fptr, long long size) {
char *bfr;
size_t result;
printf("size of file in allocate buffer: %lld\n", size);
//size here is 6448619520
bfr = (char*) malloc(sizeof(char) * size);
if (bfr == NULL) {
printf("Error, malloc failed..\n");
exit(EXIT_FAILURE);
}
//positions fptr to offset location which is 0 here.
fseek(fptr, 0, SEEK_SET);
//read the entire input file into bfr
result = fread(bfr, sizeof(char), size, fptr);
printf("result = %lld\n", (long long) result);
if(result != size)
{
printf("File failed to read\n");
exit(5);
}
return (bfr);
}
I have tested it on files of around 1-2gb in size and it works fine, however, when I test it on a 6gb file, nothing is read in to the buffer. Ignore the other results, (focus on the bolded for results), the issue lies with reading in the data bfr. Here are some of the results I get.
1st of a file that is 735844352 bytes (700+MB)
root#redbox:/data/projects/C/stubs/# ./testrun -x 45004E00 -i /data/Helix2008R1.iso
Image file is /data/Helix2008R1.iso
hex string = 45004E00
>Total size of file: 735844352
size of file in get buffer: 735844352
result = 735844352**
Begin parsing the command line hex value: 45004E00
Total number of bytes in hex string: 4
Results of hex string search:
Hex string 45004E00 was found at byte location: 37441
Hex string 45004E00 was found at byte location: 524768
....
Run #2 against a 6gb file:
root#redbox:/data/projects/C/stubs/# ./testrun -x BF1B0650 -i /data/images/sixgbimage.img
Image file is /data/images/sixgbimage.img
hex string = BF1B0650
Total size of file: 6448619520
size of file in allocate buffer: 6448619520
result = 0
File failed to read
I am still not sure why it it failing with large files and not smaller ones, is it a >4gb issue. I am using the following:
/* Support Large File Use */
#define _LARGEFILE_SOURCE 1
#define _LARGEFILE64_SOURCE 1
#define _FILE_OFFSET_BITS 64
BTW, I am using an ubuntu 9.10 box (2.6.x kernel). tia.
If you're just going to be reading through the file, not modifying it, I suggest using mmap(2) instead of fread(3). This should be much more efficient, though I haven't tried it on huge files. You'll need to change my very simplistic found/not found to report offsets if that is what you would rather have, but I'm not sure what you want the pointer for. :)
#define _GNU_SOURCE
#include <string.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
int main(int argc, char* argv[]) {
char *base, *found;
off_t len;
struct stat sb;
int ret;
int fd;
unsigned int needle = 0x45004E00;
ret = stat(argv[1], &sb);
if (ret) {
perror("stat");
return 1;
}
len = sb.st_size;
fd = open(argv[1], O_RDONLY);
if (fd < 0) {
perror("open");
return 1;
}
base = mmap(NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
if (!base) {
perror("mmap");
return 1;
}
found = memmem(base, len, &needle, sizeof(unsigned int));
if (found)
printf("Found %X at %p\n", needle, found);
else
printf("Not found");
return 0;
}
Some tests:
$ ./mmap ./mmap
Found 45004E00 at 0x7f8c4c13a6c0
$ ./mmap /etc/passwd
Not found
If this is a 32 bit process, as you say, then size_t is 32 bit and you simply cannot store more than 4GB in your process's address space (actually, in practice, a bit less than 3GB). In this line here:
bfr = (char*) malloc(sizeof(char) * size);
The result of the multiplication will be reduced modulo SIZE_MAX + 1, which means it'll only try and allocate around 2GB. Similarly, the same thing happens to the size parameter in this line:
result = fread(bfr, sizeof(char), size, fptr);
If you wish to work with large files in a 32 bit process, you have to work on only a part of them at a time (eg. read the first 100 MB, process that, read the next 100 MB, ...). You can't read the entire file in one go - there just isn't enough memory available to your process to do that.
When fread fails, it sets errno to indicate why it failed. What is the value of errno after the call to fread that returns zero?
Update:
Are you required to read the entire file in one fell swoop? What happens if you read in the file, say, 512MB at a time?
According to your comment above, you are using a 32-bit OS. In that case, you will be unable to handle 6 GB at a time (for one, size_t won't be able to hold that large of a number). You should, however, be able to read in and process the file in smaller chunks.
I would argue that reading a 6GB file into memory is probably not the best solution to your problem even on a 64-bit OS. What exactly are you trying to accomplish that is requiring you to buffer a 6GB file? There's probably a better way to approach the problem.
After taking the advice of everyone, I broke the 6GB file up into 4K chunks, parsed the hex bytes and was able to get what the byte locations which will help me later when I pull out MBR from a VMFS partition that has been dd imaged. Here was the quick and dirty way of reading it per chunk:
#define DEFAULT_BLOCKSIZE 4096
...
while((bytes_read = fread(chunk, sizeof(unsigned char), sizeof(chunk), fptr)) > 0) {
chunkptr = chunk;
for(z = 0; z < bytes_read; z++) {
if (*chunkptr == pattern_buffer[current_search]) {
current_search++;
if (current_search > (counter - 1)) {
current_search = 0;
printf("Hex string %s was found at starting byte location: %lld\n",
hexstring, (long long int) (offsetctr-1));
matches++;
}
} else {
current_search = 0;
}
chunkptr++;
//printf("[%lld]: %02X\n", offsetctr, chunk[z] & 0xff);
offsetctr++;
}
master_counter += bytes_read;
}
...
and here were the results I got...
root#redbox:~/workspace/bytelocator/Debug# ./bytelocator -x BF1B0650 -i /data/images/sixgbimage.img
Total size of /data/images/sixgbimage.img file: 6448619520 bytes
Parsing the hex string now: BF1B0650
Hex string BF1B0650 was found at starting byte location: 18
Hex string BF1B0650 was found at starting byte location: 193885738
Hex string BF1B0650 was found at starting byte location: 194514442
Hex string BF1B0650 was found at starting byte location: 525033370
Hex string BF1B0650 was found at starting byte location: 1696715251
Hex string BF1B0650 was found at starting byte location: 1774337550
Hex string BF1B0650 was found at starting byte location: 2758859834
Hex string BF1B0650 was found at starting byte location: 3484416018
Hex string BF1B0650 was found at starting byte location: 3909721614
Hex string BF1B0650 was found at starting byte location: 3999533674
Hex string BF1B0650 was found at starting byte location: 4018701866
Hex string BF1B0650 was found at starting byte location: 4077977098
Hex string BF1B0650 was found at starting byte location: 4098838010
Quick stats:
================
Number of bytes that have been read: 6448619520
Number of signature matches found: 13
Total number of bytes in hex string: 4
Have you verified that malloc and fread are actually taking in the right type of parameters? You may want to compile with the -Wall option and check if your 64-bit values are actually being truncated. In this case, malloc won't report an error but would end up allocating far less than what you had asked for.