Question:
How to (where to find additional information i.e., examples) programatically create a pseudo device-node under /dev from a kernel module?
From your question I'm guessing your talking about Linux (since you are talking about kernel modules). In that case I'd strongly recommend reading Linux Device Driver. I'd recommend looking at chapter 14 to understand better how device work.
It should also be noted that in most current desktop and server distribution of Linux, udev is responsible for creating entries in /dev. You can configure udev with rules that allow you to create your the device node with a specific name and location. In the embedded world it might be mdev with busybox that's responsible for populating /dev or even it could simply be the deprecated devfs.
Linux Device Driver is certainly a must read. However, I would start with chapter 3, since it is a step by step example on how to create a char device driver.
The kernel API is a moving target. More than often, you will discover that some example that used to compile against a previous version of the kernel generates a warning or an error with a newer version. In this situation, being able to browse through the sources without getting lost is very useful.
Related
I have an adaptec 6805e RAID controller and I'm looking to compile the linux driver for ubuntu 20.04 (kernel=v5.4) but there have been a vast number of changes between kernel ~4x => 5x and I'm not a kernel developer.
OS:
Ubuntu LTS 20.04
Kernel:
v5.4
Adaptec 6805e:
https://storage.microsemi.com/en-us/support/raid/sas_raid/sas-6805e/
Any obvious major hurdles, or advice is appreciated, but right now the issue I'm having is trying to find a replacement for the 'pci_set_dma_max_seg_size' function. I have found the 'dma_set_max_seg_size' but that function accepts a generic device structure (struct device), whereas the adaptec code is using the (struct pci_dev) structure that seems to be specific for PCI devices.
aacraid-1.2.1-52011/linit.c:3992:10: error: implicit declaration of function ‘pci_set_dma_max_seg_size’; did you mean ‘dma_set_max_seg_size’? [-Werror=implicit-function-declaration]
3992 | error = pci_set_dma_max_seg_size(pdev,
Like I said any help is appreciated and if this seems like the wrong way to resolve this problem I'm open to ideas -- I'd like to be able to utilize the benefits of more recent linux developments with a slightly older adaptec chipset.
Thanks
Reference materials:
https://elixir.bootlin.com/linux/v5.4.128/source/include/linux
I believe since Linux v4.x you fell victim to (or benefitted from, depending on your perspective) an effort to overhaul the DMA unmapping support. That patchset, and particularly the specific patch linked to here, show what PCI driver maintainers who were obliged to accommodate this rework were expected to do:
[3/3] PCI: remove pci_set_dma_max_seg_size
Fortunately for you, they gave specific examples of how existing drivers got patched and it's not bad. It should be pretty straightforward for you to figure out at least roughly the Linux version this change corresponds to around the 2018-19 date of the patch's acceptance upstream, code a new stanza in the Adaptec driver source to update it accordingly (e.g.
#if ((LINUX_VERSION_CODE > KERNEL_VERSION(5,1,0))
<patch Adaptec's code to use dma_set_max_seg_size() here>
#else
compile the driver and be on your way. Note that the README does not explicitly mention support for the Adaptec 6805e RAID controller yet, so if you manage to make it work and post your patch you should at least add a comment to the effect that you believe this supports the controller now.
I trying to boot an elf microkernel in an UEFI environment. So i compiled a minimal boot loader and created an ESP image. This works fine if I boot via an HDD but I want to direct boot it via the qemu -kernel option (This is some special requirement as I am working with AMD SEV). This doesn't work.
I can boot my kernel like this with grub if I use grub mkimage with a fat image included i.e. like this:
mcopy -i "${basedir}/disk.fat" -- "${basedir}/kernel" ::kernel
mcopy -i "${basedir}/disk.fat" -- "${basedir}/module" ::module
grub-mkimage -O x86_64-efi
-c "${basedir}/grub-bootstrap.cfg"
-m "${basedir}/disk.fat"
-o "${basedir}/grub.efi"
But the goal for my system is minimalism and security hence the microkernel, so grub and it's vulnerabilities is out of question.
So my question is:
How to create a bootable application image similar to grub-mkimage?
I have read about efi stub boot but couldn't really figure out how to build an efi stub image.
Normally I am a bare metal embedded programmer, so the whole uefi boot thing is a bit weird to me. I am glad for any tips or recommendations. Also I figured stack overflow might not be the best place for such low level questions, can you maybe recommend other forums?
I want to direct boot it via the qemu -kernel option
Why? It's a qemu-specific hack that doesn't exist on anything else (including any real computer). By using this hack the only thing you're doing is failing to test anything you'd normally use to boot (and therefore failing to test anything that actually matters).
(This is some special requirement as I am working with AMD SEV)
That doesn't make any sense (it's a little bit like saying "I have a banana in my ear because I'm trying to learn how to play piano").
AMD's SEV is a set of extensions intended to enhance the security of virtual machines that has nothing at all to do with how you boot (or whether you boot from BIOS or UEFI or a qemu-specific hack).
I am glad for any tips or recommendations.
My recommendation is to stop using GRUB specific (multi-boot), Qemu specific (-kernel) and Linux/Unix specific (elf) tools and actually try to use UEFI. This will require you to write your own boot loader using (Microsoft's) PE32+ file format that uses UEFI's services itself. Note that GNU's tools (their "Gnu-EFI" stuff for GCC) is relatively awful (it puts a PE32+ wrapper around an ELF file and does run-time patching to make the resulting Franken-monster work); and there are much better alternatives now (e.g. the Clang/LLVM/lld toolchain).
If you care about security, then it'll also involve learning about UEFI SecureBoot (and key management, and digital signatures). If you care about secure virtual machines I'd also recommend learning about the SKINIT instruction from AMD"s manual (used to create a dynamic root of trust after boot); but don't forget that this is AMD specific and won't work on any Intel CPU, and is mostly obsolete (the "trusted measurement" stuff from BIOS and TPM was mostly superseded by SecureBoot anyway), and (even on Intel CPUs) if you're only the guest then the hyper-visor can emulate it in any way it wants (and it won't guarantee anything is secure).
Finally; note that booting a micro-kernel directly doesn't make much sense either. There's no device drivers in a micro-kernel; so after booting a micro-kernel you end up with a "can't start any device drivers because there are no device drivers" problem. Instead you need to load many files (e.g. maybe an initial RAM disk), then (e.g.) start some kind of "boot log handler" (to display error messages, etc); then find and start the kernel, then start other processes (e.g. "device manager" to detect devices and drivers; "VFS layer" to handle file systems and file IO; etc). For the whole thing; starting the kernel is just one relatively insignificant small step (not much more than starting a global shared library that provides multi-tasking) buried among a significantly larger amount of code that does all the work.
Sadly; booting a monolithic kernel directly can make sense because it can contain all the drivers (or at least, has enough built into the kernel's executable file to handle an initial RAM disk if it's "modular monolithic" with dynamically loaded drivers); and this "monolithic with stuff that doesn't belong in any micro-kernel" idea is what most beginner tutorials assume.
I intend to make some changes to XFS filesystem.
Is there a documentation on the steps involved in doing so? I tried google but found nothing of much use. I have a rough understanding of the steps involved (see below), but I am looking for a detailed description so that I don't get stuck on simple things.
Steps in my mind:
Ensure my OS is not using XFS currently (no directories formatted as
XFS)
Download the source of XFS for my kernel version
Make changes to the source file
compile the modified source code (this step requires some conf files, which I am not sure where I can get from)
rmmod the xfs module and then insmod the xfs module so that changes are reflected in the system.
create a new partition, format it with XFS and test if things are alright after my changes.
Looking forward to some useful pointers.
Its OK if the pointers are for some other FS like ext3 or 4 as I believe the details would not vary from FS to FS.
Thanks
Your steps might work, but if you run into any issues you could wind up with an unbootable system.
The modules for the kernel must be built with the same version of compiler as the kernel itself or you'll have trouble. I've been stymied every time I tried to build a module for the kernel that came with the distro because the distribution maintainers invariably used some customized version of the compiler that I was not able to match.
A safer but longer option is install the kernel source package for your distro, and modify the XFS module source as necessary. Then build the entire kernel, including the customized XFS module, following the instructions for your linux distribution. Google for your distro and 'custom linux kernel', you should find dozens of hits.
Once built, you'll want to install your new kernel alongside your old one, and configure the bootloader to make the kernel selectable at boot time. This way even if something goes horribly wrong, you can still boot your system using your existing kernel.
I am wondering if it's possible to write an application that will access a foreign filesystem, but without needing support for that filesystem from the operating system. For example, I'd like to write an app in C that runs on Mac OS X that can browse / copy files from an ext2/ext3 formatted disk. Of course, you'd have to do all the transfers through the application (not through the system using cp or the Finder), but that would be OK for my purpose. Is this possible?
There are user space libraries that allow you to access file systems.
The Linux-NTFS library (libntfs) allows you to access NTFS file systems and there are user space programs like ntfsfix to do things to the file system.
E2fsprogs does the same for ext2, ext3 and ext4 filesystems.
As Basile mentioned, Mtools is another one that provides access to FAT partitions.
There was even a program that does exactly what you're looking for on Windows. It's called ext2explore and allows you to access ext2 partitions from Windows.
It is possible. For example the GNU mtools utility are doing that (assuming a way to access the raw device or partition) for MS-DOS FAT file systems.
However, file systems inside the kernel are usually very well tested and optimized.
Yes and No. For a regular user Application is usually not possible because access to block devices is restricted to root only. Every block device should give read/write to the needed block device for that effect. This would need at best a server/client approach where a service is started on the machine and configured to give the permissions on a per block device manner.
The somewhat easier alternative would be you to use the MacFUSE implementation.
Look here:
http://code.google.com/p/macfuse/
http://groups.google.com/group/macfuse?pli=1
The MacFuse project seems no longer mantained, but can give you a starting point for your project.
The dirty and quick approach is the following as root chmod 666 /dev/diskN
You can hijack syscalls and library calls from your application and then redirect reads/writes to anything like a KV store or a distributed DB layer (using the regular calls for the "virtual devices" that you do not support).
Then, the possibilities are boundless because you don't have to reach the physical/virtual devices when someone asks for them (resolving privilege issues).
I'm looking to make a custom filesystem for a project I'm working on. Currently I am looking at writing it in Python combined with fusepy, but it got me wondering how a compiled non-userspace filesystem is made in Linux. Are there specific libraries that you need to work with or functions you need to implement for the mount command to work properly. Overall I'm not sure how the entire process works.
Yup you'd be programming to the kernel interfaces, specifically the VFS layer at a minimum. Edit Better link [1]
'Full' documentation is in the kernel tree: http://www.mjmwired.net/kernel/Documentation/filesystems/vfs.txt. Of course, the fuse kernel module is programmed to exactly the same interface
This, however, is not what you'd call a library. It is a kernel component and intrinsically there, so the kernel doesn't have to know how a filesystem is implemented to work with one.
[1] google was wrong: the first hit wasn't the best :)
If you'd like to write it in Python, fuse is a good option. There are lots of tutorials for this, such as the one here: http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FUSE_Python_tutorial
In short: Linux is a monolithic kernel with some module-loading capabilities. That means that every kernel feature (filesystems, scheduler, drivers, memory management, etc.) is part of the one big program called Linux. Loadable modules are just a specialized way of run-time linking, which allows the user to pick those features as needed, but they're all still developed mostly as a single program.
So, to create a new filesystem, you just add new C source code files to the kernel code, defining the operations your filesystem has to perform. Then, create an initialization function that allocates a new instance of the VFS structure, fills it with the appropriate function pointers and registers with the VFS.
Note that FUSE is nothing more than a userlevel accessible API to do the same, so the FUSE hooks correspond (roughly) to the VFS operations.