I have a piece of code that tries to mmap some file. If it can mmap the file, it does something, if it can not mmap it does something else.
The code is working in both cases, but I wanted to to do some tests in some filesystem that does not support mmap. The problem is that I did not find any filesystem that were not able to mmap.
Can someone point me out some filesystem that is not mmap-able?
You might simulate a non-mmap-capable system using library interposition. Simply take this C file
#include <errno.h>
#include <sys/mman.h>
void* mmap(void*, size_t, int, int, int, off_t) {
errno = ENODEV;
return NULL;
}
and compile it to a shared library. Name the path of that library in the LD_PRELOAD environment variable, and it should take precedence over the real mmap, thus simulating a system where mmap will always fail. That way, you can test your code without superuser privileges, without creating certain file systems, and without having to have the corresponding kernel modules, userland tools and the likes available.
You might theoretically encounter situations where some library outside your control relies on specific kinds of mmap to always work. A map with MAP_ANONYMOUS is a prime example, since it is not backed by a file system and therefore does not depend on FS types. If you encounter any problems in libraries failing due to violated mmap assumptions, you might have to modify the interposer to have a closer look at its arguments and forward some calls to the libc implementation while rejecting others itself. But I'd only do this if the need actually arises.
Related
So I have just started customizing the FreeBSD kernel, but unfortunately the resources available for FreeBSD development are scarce .
Im writing a systemcall in which should read a file(optionally), read the blocks of physical memory according to input and write the results into another file(generally "filename.results")
my problems are:
Standard C libraries: it seems to be that they are unavailable for kernel module programming so how should I replace the functions such as write and read(and strlen and some others in string.h)?
Malloc function: it seems that it accepts 3 inputs instead of 1, and I have no idea how to fill the 2nd variable even after reading the man page(tried FOO but returns symlink error).
Also I was interested in any other topics u think they are useful for this routine.
In case of malloc, do "man 9 malloc". The "9" here means section describing kernel functions, userland malloc is described in section 3.
Well I've said that I got the answer.
So for future reads I'm just leaving it here.
MALLOC: you need to define your own memory description(or use an existing one) in order to be able to locate it, that's a POSIX standard and its for sanity check purposes.
as for the other things, for the fact that standard c libraries are not available in kernel mode, the kernel variant of them is likely available in libkern (open /sys/libkern), and they will be all available once you implement it(say uprintf, strlen and stuff), if its not there you have to call the relying module by implementing them in your header file(say for FILE interaction you need to include the I/O module located in /sys/(dir)) since you ARE in kernel mode it doesn't create a problem.(also note that those functions are well implemented so you wont likely face a kernel crash.)
As an obvious fact you have to copy the buffer from user memory to kernel memory in order to do modifications on it, and copy it back when you are done.
one last thing, in order to implement your systemcall via sysproto auto build you need to include it as well(and add your syscall to the list). and don't forget to include your file in the source file configuration file (located in /sys/(dir) again).
I am writing a C shared library in Linux in which a function would like to discover the path to the currently running executable. It does NOT have access to argv[0] in main(), and I don't want to require the program accessing the library to pass that in.
How can a function like this, outside main() and in the wild, get to the path of the running executable? So far I've thought of 2 rather unportable, unreliable ways: 1) try to read /proc/getpid()/exe and 2) try to climb the stack to __libc_start_main() and read the stack params. I worry about all machines having /proc mounted.
Can you think of another way? Is there something buried anywhere in dlopen(NULL, 0) ? Can I get a reliable proc image of self from the kernel??
Thanks for any thoughts.
/proc is your best chance, as "path of the executable" is not that well defined concept in Linux (you can even delete it while the program is running).
To get the breakdown of loaded modules (with the main executable usually being the first entry) you should look at /proc/<pid>/maps. It's a text formatted file which will allow you to associate executable and library paths with load addresses (if the former are known and still valid).
Unless you are writing software that may be used very early in system startup, you can safely assume that /proc will always be mounted on a Linux system. It contains quite a bit of data that is not accessible any other way, and thus must be mounted for a system to function properly. As such, you can pretty easily obtain a path to your executable using:
readlink("/proc/self/exe", buf, sizeof(buf));
If for some reason you want to avoid this, it's also possible to read it from the process's auxiliary vector:
#include <sys/auxv.h>
#include <elf.h>
const char *execpath = (const char *) getauxval(AT_EXECFN);
Note that this will require a recent version of glibc (2.16 or later). It'll also return the path that was used to execute your application (e.g, possibly something like ./binary), rather than its absolute path.
I am quite new to the FILE family of functions that the standard C library provides.
I recently stumbled across fopen() and the similar functions after researching how stdout, stdin and stderr work alongside functions like printf().
I was wondering, what is needed to use fopen() on an embedded system (which doesn't necessarily have operating system support). After reading more about it, is seems like a cool thing to do on more powerful embedded systems to hook into say, a UART/SPI interface, so that calling printf() would print data out of the UART. Simarly, you could read data from a UART buffer by calling scanf().
This would also increase portability! (code written for say, Linux, would be easier to port if printf() was supported). You could also print debug data to a file if it was running in a production environment, and read from it later.
Can you just use fopen() on a bare-bones embedded system? If so who/where/when is the "FILE" then created (as far as I now, fopen() does not malloc() space for the file, nor do you specify how much)? Or do you need a operating system with FAT file support. If so, would something like http://ultra-embedded.com/?fat_filelib work? Would using FreeRTOS help at all?
Check the documentation for your toolchain's C library - it should have something to say about re-targeting the library.
For example if you are using Newlib you must re-implement some or all of the [syscalls stubs][3] to suit your target. The low level open() syscall in this case will allow fopen() to work as necessary. At its simplest, you might implement open() to support higher-level stdio access to serial ports, but if you are expecting standard file-system access, then you will still need an underlying file-system to map it too.
Another example of re-targeting the Keil/ARM standard library can be found here.
Yes, it's often possible to use fopen() and similar routines in code for embedded systems. The way it often works is that the vendor supplies a C compiler and associated libraries
targeted for their system, which implement some supported subset of the language in a way that's appropriate for that system (e.g. an implementation of printf() that outputs via a UART, or fopen() that uses RAM to simulate some sort of filesystem).
On the Keil compiler, the stdio library is designed to allow the user to define the __FILE structure in any desired fashion. A function like fprintf will perform a sequence of calls to fputc, which will receive a copy of the pointer passed to fprintf. One may define something like fopen to "create" a __FILE and populate its members via any desired means (if there will never be more than one file open at a time, one could simply fill in the fields of a static instance and return that). Variables __stdin, __stdout, and __stderror may likewise be defined as desired (stdin is defined to point to __stdin, and likewise with stdout and stderror).
"Can you just use fopen() on a bare-bones embedded system?"
It depends. Depends on the configuration of your embedded system, the types of memories interfaced, on what memory do you want to implement the file system, the file system library code size (ROM & RAM requirements).
FILE manipulation functions can be used independent of any OS. But a proper file system must be used and FAT is not the only file system (JFFS2, YAFS,...some other proprietary file system)
The file system is generally (but not always) implemented on Flash memories (Nand Flash, Nor Flash). USB device is also a flash (Nand flash). The Nand Flash & Nor Flash may have Parallel interface, I2C interface or SPI interface.
I'm looking for a way to load generated object code directly from memory.
I understand that if I write it to a file, I can call dlopen to dynamically load its symbols and link them. However, this seems a bit of a roundabout way, considering that it starts off in memory, is written to disk, and then is reloaded in memory by dlopen. I'm wondering if there is some way to dynamically link object code that exists in memory. From what I can tell there might be a few different ways to do this:
Trick dlopen into thinking that your memory location is a file, even though it never leaves memory.
Find some other system call which does what I'm looking for (I don't think this exists).
Find some dynamic linking library which can link code directly in memory. Obviously, this one is a bit hard to google for, as "dynamic linking library" turns up information on how to dynamically link libraries, not on libraries which perform the task of dynamically linking.
Abstract some API from a linker and create a new library out its codebase. (obviously this is the least desirable option for me).
So which ones of these are possible? feasible? Could you point me to any of the things I hypothesized existed? Is there another way I haven't even thought of?
I needed a solution to this because I have a scriptable system that has no filesystem (using blobs from a database) and needs to load binary plugins to support some scripts. This is the solution I came up with which works on FreeBSD but may not be portable.
void *dlblob(const void *blob, size_t len) {
/* Create shared-memory file descriptor */
int fd = shm_open(SHM_ANON, O_RDWR, 0);
ftruncate(fd, len);
/* MemMap file descriptor, and load data */
void *mem = mmap(NULL, len, PROT_WRITE, MAP_SHARED, fd, 0);
memcpy(mem, blob, len);
munmap(mem, len);
/* Open Dynamic Library from SHM file descriptor */
void *so = fdlopen(fd,RTLD_LAZY);
close(fd);
return so;
}
Obviously the code lacks any kind of error checking etc, but this is the core functionality.
ETA: My initial assumption that fdlopen is POSIX was wrong, this appears to be a FreeBSD-ism.
I don't see why you'd be considering dlopen, since that will require a lot more nonportable code to generate the right object format on disk (e.g. ELF) for loading. If you already know how to generate machine code for your architecture, just mmap memory with PROT_READ|PROT_WRITE|PROT_EXEC and put your code there, then assign the address to a function pointer and call it. Very simple.
There is no standard way to do it other than writing out the file and then loading it again with dlopen().
You may find some alternative method on your current specific platform. It will up to you to decide whether that is better than using the 'standard and (relatively) portable' approach.
Since generating the object code in the first place is rather platform specific, additional platform-specific techniques may not matter to you. But it is a judgement call - and in any case depends on there being a non-standard technique, which is relatively improbable.
We implemented a way to do this at Google. Unfortunately upstream glibc has failed to comprehend the need so it was never accepted. The feature request with patches has stalled. It's known as dlopen_from_offset.
The dlopen_with_offset glibc code is available in the glibc google/grte* branches. But nobody should enjoy modifying their own glibc.
You don't need to load the code generated in memory, since it is already in memory!
However, you can -in a non portable way- generate machine code in memory (provided it is in a memory segment mmap-ed with PROT_EXEC flag).
(in that case, no "linking" or relocation step is required, since you generate machine code with definitive absolute or relative addresses, in particular to call external functions)
Some libraries exist which do that: On GNU/Linux under x86 or x86-64, I know of GNU Lightning (which generates quickly machine code which runs slowly), DotGNU LibJIT (which generates medium quality code), and LLVM & GCCJIT (which is able to generate quite optimized code in memory, but takes time to emit it). And LuaJit has some similar facility too. Since 2015 GCC 5 has a gccjit library.
And of course, you can still generate C code in a file, fork a compiler to compile it into a shared object, and dlopen that shared object file. I'm doing that in GCC MELT , a domain specific language to extend GCC. It does work quite well in practice.
addenda
If performance of writing the generated C file is a concern (it should not be, since compiling a C file is much slower than writing it) consider using some tmpfs file system for that (perhaps in /tmp/ which is often a tmpfs filesystem on Linux)
I am trying to include math.h in my Linux kernel module. If I use,
#include '/usr/include/math.h'
It give me theses errors:
error: features.h: No such file or directory
error: bits/huge_val.h: No such file or directory
error: bits/mathdef.h: No such file or directory
error: bits/mathcalls.h: No such file or directory
Why is this?
You cannot use the C library in a kernel module, this is even more true for the math library part.
You can't include a userspace C module in kernel space. Also are you sure that you want to be doing this? This thread may help http://kerneltrap.org/node/16570. You can do math functions inside the kernel, just search around on http://lxr.linux.no/ for the function you need.
Standard libraries are not available in the kernel. This includes libc, libm, etc. Although some of the functions in those libraries are implemented in kernel space, some are not. Without knowing what you're trying to call, it's impossible to say for sure whether or not you should be doing what you're trying to do in kernel space.
I should further note that the kernel does NOT have access to the FPU. This is to save time when switching tasks (since saving the FPU registers would add unnecessary overhead when performing context switches). You can get access to the FPU from kernel space if you really want it, but you need to be really careful not to trash the user space's FPU registers when doing so.
Edit: This summarizes the caveat about the FPU much better than I did.
Floating point operations is not supported in the kernel. This is because when switching from kernel context to user context, registers must be saved. If the kernel would make use of floating point, then also the floating point registers would have to be saved also, which would cause bad performance for each context switch. So because floating point is very rarely needed, especially in the kernel it is not supported.
If you really have to:
maybe you could compile your own kernel with floating point support
you could block context switch within your floating point operations
the best would be to use fixed point arithmetics.
AFAIK kernel space is separated from user space, and so should the source code. /usr/include is for general programming.
This suggests that doing floating point math in the kernel is not as simple is in user-space code. Another instance suggesting that this is hard.
Still looking for a more definitive answer.
well you cannot, you can rewrite functions you need in your module, it's dirty but it should work...
Thanks a lot for your comments
To use math functions
Is it possiable to make a plane C application and pass variables from kernel source file. So the C Application will compute the variables and sends back the information .
Kernel source file (kernel space) ---> C Application (user space)
|
<---|
Kernel source file
So we may include header file in kernel source code. In case of any event, it pass the values to a C application (user space)
Details:
I am trying to modify my HID joystick events(absolute x, y) So It may only move to the improved location, which will be genarated by my application, with some math functions like (pow, tan,etc).
So I used hid-input.c to get raw events, and modify them. which will be used for input subsystem through hid kernel module –
Looking for your comments
Regards.
You cannot (often, without lots of kernel know-how to lock and preserve these registers while not impacting other critical sections) use floating point registers in the kernel, and besides it is of course inappropriate to do "processing" in the kernel. Many others have mentioned this. Performance will be terrible. Thus, math.h is not provided for kernel modules. We accept this and move on...
However, as I am also a victim of crazy requirements and completely insane designs forced on us by others, this is a legitimate question. After reducing the usage of the math.h API to minimize the performance impact, you can use floating point emulation (soft-float) via correct compiler settings to implement your required functions without using floating point registers. Kernel code should already compile with these soft-float settings.
In order to implement math.h functionality, you can look at glibc or uClibc and perhaps others. Both of these libraries have generic "C" implementations of libm which implement math.h without the use of special intrinsics or platform specific types and should therefore compile just fine in the kernel.
uClibc: The above link takes you directly to the libm section of uClibc.
glibc: After "git"-ing glibc, you'll find what you're looking for in glibc/sysdeps/ieee754/flt-32.
glibc may be more difficult to understand because it is more sophisticated and has more inter-dependencies within itself, but uClibc only provides (at the moment) C89 math.h. If you want single precision (read: faster) or complex math functionality as in C99+, you'll have to look at glibc.
Maybe try using double quotes (") instead of single quotes?
In experts view , its NOT a good approach to communicate data between kernel space and user space. Either fully work on kernel space OR only on user space.
But one solution can, use read() and write() command in a kernel module to send the information between user space and kernel space.