Is there any way to make pointers read a file as a block of memory in C?
Can access of a file can be faster...?
Treating a file as memory (and letting the OS do the file IO for you) is termed 'memory mapping'.
On POSIX (e.g. Linux), the mmap() function does this.
On Windows, the OpenFileMapping() function and friends do this. Microsoft have excellent description of how this works, why to use it, and particulars on their platform here.
Related
I thought if I have the raw machine code output of an individual function, that I could read it with dread into an block of memory, and just create an function pointer to there, to dynamically include a function at runtime.
Would this work?
Yes it would work, but not in a cross-platform way. Memory returned by malloc is not executable, at least it is not guaranteed to be. On Windows you'll have to use VirtualAlloc, followed by VirtualProtect with the PAGE_EXECUTE_READ flag. On Linux, look into mprotect. There can also be differences based on compiler used and machine architecture, for a Linux+GCC example see this code golf.
Though why not just dynamically load the shared library with LoadLibrary+GetProcAddress (Win) or dlopen+dlsym (Linux)?
Is there anything like a string file in stdio/string/stdlib ? I mean a special way to fopen a FILE stream, which actually directs the writes to an internal buffer and takes care of buffer allocation/reallocation ? After fclose, the text should be available as null-terminated char[] or similar.
I need to interface to legacy code that receives a FILE* as an argument and writes to it, and I'd prefer to avoid writing to a temporary disk file.
Other forms of storage could do instead of char[] (f.i. string), but a FILE* pointer must be available.
I am looking for an alternative to creating a temporary disk file.
fmemopen & open_memstream are in the POSIX 2008 standard, probably inspired by GNU libc string streams, and give in-memory FILE* streams.
See also this question quite similar to yours, and also that answer.
BTW, many operating systems have RAM based or virtual memory based filesystems (à la tmpfs)
If you are coding in C++11 (not in C) and perhaps for some earlier C++ standard you can of course use std::stringstream-s
So you could use open_memstream on Posix, and some other solution on Windows (just with #if _POSIX_C_SOURCE > 200809L per feature_test_macros(7) ...)
The C standard does not provide (yet) any in-memory FILE streams, so if you need them you have to code or use platform-specific functions.
Create the temporary file using CreateFile(... FILE_ATTRIBUTE_TEMPORARY, FILE_FLAG_DELETE_ON_CLOSE ...) and then convert the HANDLE to FILE*.
You said you didn't like a write to a temporary file, so these flags to CreateFile are a strong hint to Windows to keep the file in cache if possible. And if Windows would run of of RAM, even a char[] can end up in a swap file anyway.
So I have just started customizing the FreeBSD kernel, but unfortunately the resources available for FreeBSD development are scarce .
Im writing a systemcall in which should read a file(optionally), read the blocks of physical memory according to input and write the results into another file(generally "filename.results")
my problems are:
Standard C libraries: it seems to be that they are unavailable for kernel module programming so how should I replace the functions such as write and read(and strlen and some others in string.h)?
Malloc function: it seems that it accepts 3 inputs instead of 1, and I have no idea how to fill the 2nd variable even after reading the man page(tried FOO but returns symlink error).
Also I was interested in any other topics u think they are useful for this routine.
In case of malloc, do "man 9 malloc". The "9" here means section describing kernel functions, userland malloc is described in section 3.
Well I've said that I got the answer.
So for future reads I'm just leaving it here.
MALLOC: you need to define your own memory description(or use an existing one) in order to be able to locate it, that's a POSIX standard and its for sanity check purposes.
as for the other things, for the fact that standard c libraries are not available in kernel mode, the kernel variant of them is likely available in libkern (open /sys/libkern), and they will be all available once you implement it(say uprintf, strlen and stuff), if its not there you have to call the relying module by implementing them in your header file(say for FILE interaction you need to include the I/O module located in /sys/(dir)) since you ARE in kernel mode it doesn't create a problem.(also note that those functions are well implemented so you wont likely face a kernel crash.)
As an obvious fact you have to copy the buffer from user memory to kernel memory in order to do modifications on it, and copy it back when you are done.
one last thing, in order to implement your systemcall via sysproto auto build you need to include it as well(and add your syscall to the list). and don't forget to include your file in the source file configuration file (located in /sys/(dir) again).
I am quite new to the FILE family of functions that the standard C library provides.
I recently stumbled across fopen() and the similar functions after researching how stdout, stdin and stderr work alongside functions like printf().
I was wondering, what is needed to use fopen() on an embedded system (which doesn't necessarily have operating system support). After reading more about it, is seems like a cool thing to do on more powerful embedded systems to hook into say, a UART/SPI interface, so that calling printf() would print data out of the UART. Simarly, you could read data from a UART buffer by calling scanf().
This would also increase portability! (code written for say, Linux, would be easier to port if printf() was supported). You could also print debug data to a file if it was running in a production environment, and read from it later.
Can you just use fopen() on a bare-bones embedded system? If so who/where/when is the "FILE" then created (as far as I now, fopen() does not malloc() space for the file, nor do you specify how much)? Or do you need a operating system with FAT file support. If so, would something like http://ultra-embedded.com/?fat_filelib work? Would using FreeRTOS help at all?
Check the documentation for your toolchain's C library - it should have something to say about re-targeting the library.
For example if you are using Newlib you must re-implement some or all of the [syscalls stubs][3] to suit your target. The low level open() syscall in this case will allow fopen() to work as necessary. At its simplest, you might implement open() to support higher-level stdio access to serial ports, but if you are expecting standard file-system access, then you will still need an underlying file-system to map it too.
Another example of re-targeting the Keil/ARM standard library can be found here.
Yes, it's often possible to use fopen() and similar routines in code for embedded systems. The way it often works is that the vendor supplies a C compiler and associated libraries
targeted for their system, which implement some supported subset of the language in a way that's appropriate for that system (e.g. an implementation of printf() that outputs via a UART, or fopen() that uses RAM to simulate some sort of filesystem).
On the Keil compiler, the stdio library is designed to allow the user to define the __FILE structure in any desired fashion. A function like fprintf will perform a sequence of calls to fputc, which will receive a copy of the pointer passed to fprintf. One may define something like fopen to "create" a __FILE and populate its members via any desired means (if there will never be more than one file open at a time, one could simply fill in the fields of a static instance and return that). Variables __stdin, __stdout, and __stderror may likewise be defined as desired (stdin is defined to point to __stdin, and likewise with stdout and stderror).
"Can you just use fopen() on a bare-bones embedded system?"
It depends. Depends on the configuration of your embedded system, the types of memories interfaced, on what memory do you want to implement the file system, the file system library code size (ROM & RAM requirements).
FILE manipulation functions can be used independent of any OS. But a proper file system must be used and FAT is not the only file system (JFFS2, YAFS,...some other proprietary file system)
The file system is generally (but not always) implemented on Flash memories (Nand Flash, Nor Flash). USB device is also a flash (Nand flash). The Nand Flash & Nor Flash may have Parallel interface, I2C interface or SPI interface.
I wrote a custom (VERY basic "Hello world!") bootloader in Assembler and I would like to execute a C program in that. Would the C program work, or fail due to a lost stdio.h file? And how could I bundle the C program along with the bootloader into a single .bin file to dd to a flash drive/CD?
I'm not sure what you mean by "lost stdio.h", but many C runtime functions, including those prototyped in stdio.h, are implemented using system calls. Without an OS running, those system calls won't work.
It is possible to write C code that runs without an OS, for example most common bootloaders have just a tiny amount of assembler and mostly C code. The trick is to avoid using runtime libraries. Alternatives to syscalls, for e.g. display, are BIOS calls and hardware-specific I/O.
To take just one example, in addition to dynamic allocation, fopen in read mode needs the following low-level operations:
Reading a block of data from storage
Reading the file system metadata (often, superblock and root directory)
Processing file system metadata to find out where the file content is stored
Creating a FILE object that contains enough information for fread and fgetc to find the data on disk
You don't have an OS to help with any of that, your C code will need to implement a driver (possibly calling the BIOS) for block read, and implement the behavior of the other steps.