I'm writing my own operating system, and so far I'm only really able to write it in assembly, because I don't really understand how I would set it up with multiple files/languages. I've written bootloaders with executable code in them before, but what I don't understand is how to make the bootloader aware of other files outside of itself. How would I be able to write a bootloader in assembly and then tell it to load, say, a kernel written in C in a different file? Do I have to bundle the .o files from the compilation of the kernel into the fdd image and tell the bootloader to load/execute them or is it more complicated than that?
Since it looks like you're trying to get the hang of system bring up it might be worthwhile to take a look at some "smaller" embedded systems to get a feel for what goes on once power is applied/chip comes out of reset. Take a look at U-Boot here: http://www.denx.de/wiki/U-Boot
It is a very popular bootloader especially for embedded systems and can launch a variety of OS's. The mainline supports a ton of different configurations as well. I think it is relatively straight forward to follow what happens during power up if you are comfortable with C.
To answer your question more specifically for instance with U-Boot you can either build parameters into the u-boot image as to where you are going to load your code, it can read where you image file is stored from a configuration file on powerup, u-boot can load a configuration automatically from your network somewhere, you can even tell u-boot where and what to load from its command line interface. Take a look and see if you have any further questions.
Related
I trying to boot an elf microkernel in an UEFI environment. So i compiled a minimal boot loader and created an ESP image. This works fine if I boot via an HDD but I want to direct boot it via the qemu -kernel option (This is some special requirement as I am working with AMD SEV). This doesn't work.
I can boot my kernel like this with grub if I use grub mkimage with a fat image included i.e. like this:
mcopy -i "${basedir}/disk.fat" -- "${basedir}/kernel" ::kernel
mcopy -i "${basedir}/disk.fat" -- "${basedir}/module" ::module
grub-mkimage -O x86_64-efi
-c "${basedir}/grub-bootstrap.cfg"
-m "${basedir}/disk.fat"
-o "${basedir}/grub.efi"
But the goal for my system is minimalism and security hence the microkernel, so grub and it's vulnerabilities is out of question.
So my question is:
How to create a bootable application image similar to grub-mkimage?
I have read about efi stub boot but couldn't really figure out how to build an efi stub image.
Normally I am a bare metal embedded programmer, so the whole uefi boot thing is a bit weird to me. I am glad for any tips or recommendations. Also I figured stack overflow might not be the best place for such low level questions, can you maybe recommend other forums?
I want to direct boot it via the qemu -kernel option
Why? It's a qemu-specific hack that doesn't exist on anything else (including any real computer). By using this hack the only thing you're doing is failing to test anything you'd normally use to boot (and therefore failing to test anything that actually matters).
(This is some special requirement as I am working with AMD SEV)
That doesn't make any sense (it's a little bit like saying "I have a banana in my ear because I'm trying to learn how to play piano").
AMD's SEV is a set of extensions intended to enhance the security of virtual machines that has nothing at all to do with how you boot (or whether you boot from BIOS or UEFI or a qemu-specific hack).
I am glad for any tips or recommendations.
My recommendation is to stop using GRUB specific (multi-boot), Qemu specific (-kernel) and Linux/Unix specific (elf) tools and actually try to use UEFI. This will require you to write your own boot loader using (Microsoft's) PE32+ file format that uses UEFI's services itself. Note that GNU's tools (their "Gnu-EFI" stuff for GCC) is relatively awful (it puts a PE32+ wrapper around an ELF file and does run-time patching to make the resulting Franken-monster work); and there are much better alternatives now (e.g. the Clang/LLVM/lld toolchain).
If you care about security, then it'll also involve learning about UEFI SecureBoot (and key management, and digital signatures). If you care about secure virtual machines I'd also recommend learning about the SKINIT instruction from AMD"s manual (used to create a dynamic root of trust after boot); but don't forget that this is AMD specific and won't work on any Intel CPU, and is mostly obsolete (the "trusted measurement" stuff from BIOS and TPM was mostly superseded by SecureBoot anyway), and (even on Intel CPUs) if you're only the guest then the hyper-visor can emulate it in any way it wants (and it won't guarantee anything is secure).
Finally; note that booting a micro-kernel directly doesn't make much sense either. There's no device drivers in a micro-kernel; so after booting a micro-kernel you end up with a "can't start any device drivers because there are no device drivers" problem. Instead you need to load many files (e.g. maybe an initial RAM disk), then (e.g.) start some kind of "boot log handler" (to display error messages, etc); then find and start the kernel, then start other processes (e.g. "device manager" to detect devices and drivers; "VFS layer" to handle file systems and file IO; etc). For the whole thing; starting the kernel is just one relatively insignificant small step (not much more than starting a global shared library that provides multi-tasking) buried among a significantly larger amount of code that does all the work.
Sadly; booting a monolithic kernel directly can make sense because it can contain all the drivers (or at least, has enough built into the kernel's executable file to handle an initial RAM disk if it's "modular monolithic" with dynamically loaded drivers); and this "monolithic with stuff that doesn't belong in any micro-kernel" idea is what most beginner tutorials assume.
I'm a newbie to learning OS development. From the book I read, it said that boot loader will copy first MBR into 0x7c00, and starts from there in real mode.
And, example starts with 16 bit assembly code.
But, when I looked at today's linux kernel, arch/x86/boot has 'header.S' and 'boot.h', but actual code is implemented in main.c.
This seems to be useful by "not writing assembly."
But, how is this done specifically in Linux?
I can roughly imagine that there might be special gcc options and link strategy, but I can't see the detail.
I'm reading this question more as an X-Y problem. It seems to me the question is more about whether you can write a bootloader (boot code) in C for your own OS development. The simple answer is YES, but not recommended. Modern Linux kernels are probably not the best source of information for creating bootloaders written in C unless you have an understanding of what their code is doing.
If using GCC there are restrictions on what you can do with the generated code. In newer versions of GCC there is an -m16 option that is documented this way:
The -m16 option is the same as -m32, except for that it outputs the ".code16gcc" assembly directive at the beginning of the assembly output so that the binary can run in 16-bit mode.
This is a bit deceptive. Although the code can run in 16-bit real mode, the code generated by the back end uses 386 address and operand prefixes to make normally 32-bit code execute in 16-bit real mode. This means the code generated by GCC can't be used on processors earlier than the 386 (like the 8086/80186/80286 etc). This can be a problem if you want a bootloader that can run on the widest array of hardware. If you don't care about pre-386 systems then GCC will work.
Bootloader code that uses GCC has another downside. The address and operand prefixes that get get added to many instructions add up and can make a bootloader bloated. The first stage of a bootloader is usually very constrained in space so this could potentially become a problem.
You will need inline assembly or assembly language objects with functions to interact with the hardware. You don't have access to the Linux C library (printf etc) in bootloader code. For example if you want to write to the video display you have to code that functionality yourself either writing directly to video memory or through BIOS interrupts.
To tie it altogether and place things in the binary file usable as an MBR you will likely need a specially crafted linker script. In most projects these linker scripts have an .ld extension. This drives the process of taking all the object files putting them together in a fashion that is compatible with the legacy BIOS boot process (code that runs in real mode at 0x07c00).
There are so many pitfalls in doing this that I recommend against it. If you are intending to write a 32-bit or 64-bit kernel then I'd suggest not writing your own bootloader and use an existing one like GRUB. In the versions of Linux from the 1990s it had its own bootloader that could be executed from floppy. Modern Linux relies on third party bootloaders to do most of that work now. In particular it supports bootloaders that conform to the Multiboot specification
There are many tutorials on the internet that use GRUB as a bootloader. OS Dev Wiki is an invaluable resource. They have a Bare Bones tutorial that uses the original Multiboot specification (supported by GRUB) to boot strap a basic kernel. The Mulitboot specification can easily be developed for using a minimal of assembly language code. Multiboot compatible bootloaders will automatically place the CPU in protected mode, enable the A20 line, can be used to get a memory map, and can be told to place you in a specific video mode at boot time.
Last year someone on the #Osdev chat asked about writing a 2 stage bootloader located in the first 2 sectors of a floppy disk (or disk image) developed entirely in GCC and inline assembly. I don't recommend this as it is rather complex and inline assembly is very hard to get right. It is very easy to write bad inline assembly that seems to work but isn't correct.
I have made available some sample code that uses a linker script, C with inline assembly to work with the BIOS interrupts to read from the disk and write to the video display. If anything this code should be an example why it's non-trivial to do what you are asking.
1) I learned writing bootloaders and tested it using bochs. Now, I want to add GUI to by bootloader. I have googled but didnt hit on relevant sources for that. I even tried searching github for existing projects. I checked out this question. But is there any way to install some graphics libraries or include x-window-system APIs in my code to give in a GUI env (e.g: Chameleon, GAG) instead of including just a splash image ?
2) Is there any possibility to add python execution environment during bootloader stage, so that when I step in protected mode, i could add some python scripts?
3) How to add standard C/C++ library support?
Thanks in advance.
It might help if you told what kind of bootloader you're talking about. I work with this daily and most of what you're saying seems like crazy talk. LOL.
Usually bootloaders are very small startup program that takes a decision on what to load next. Some bootloaders contains features for "flashing" the main application. (The main app often cannot flash it self.) The bootloader being the first piece of code the processor meets, makes it very critical. You do not want it to fail nor upgrade it. Hence bootloaders tend to strive for smaller is better.
As for your questions.
3) I've never seen a bootloader not written in c. So it got that one covered. (They exist though) C++ you would have to compile the whole project with c++. (Not a problem.)
1) and 2) I'm not sure what "protected mode" is, but to include eg python or x-windows you would have to include the source into your project. Here's the thing. Usually bootloaders are baremetal code and code like X isn't. You'd have to include a whole lot of other code as well. And you'd end up with a bootloader at 500 mb. Usually you strive for something like 5 kb.
The splash image is usually done through low level interaction with the graphic card.
If anyone else is interested in bootloader development, I can recommend to look at uboot. It's a linux (mostly) loader and is rather easy to find your way through. And it already supports a lot.
I was wondering if anyone knows an efficient way to program the FPGA(PL) for a Xilinx Zynq-7 series or related devices,from a host C program (not on the SoC, but from the host PC). Is there an Xilinx API I can use/include in my program. As the only way I can think of doing it at the moment is invoking command line programming via Impact.
Basically I want to put the SDK "Program FPGA" functionality in my host C program where the user selects a prebuilt .bit file (and .elf file if possible) to program the FPGA/(SoC). This is just for a test of concept, later I would like to put this dynamic configuration onto one of the ARM CPU's.
Many Thanks
Sam
At the very least you'll need an intermediate MPU/MCU that can read from USB, as at startup most FPGAs aren't capable of much at all. I'm guessing this'll make it hard to find a MPU/library pair to do so, because there are so many options, each of which would be pretty application-specific. You're better off starting with programming them off an ARM chip, since you'll need some CPU with the FPGA in any case.
This seems somewhat useful.
I've been following the "Mike OS Guide" to make my own kernel, and I got it working. But then I went onto the many guides on the internet for making a boot sector in NASM that loads a main function from a compiled C object. I have tried compiling and linking with all kinds of GCC installations:
x86_64-pc-linux-
arm-uclinux-elf-
arm-agb-elf-
arm-elf-
arm-apple-darwin10-
powerpc-apple-darwin10-
i686-apple-darwin10-
i586-pc-linux-
i386-elf-
All of them fail once I put them onto a floppy like I do with the MikeOS bootstrap. I've tried various tutorials on http://www.osdever.net/ like the one here and I've tried http://wiki.osdev.org/Bare_Bones , but none work when trying to compile on a Mac, yet I have not tired on an actual Linux machine yet. But I was wondering how I could get a bootstrap in assembly the calls the C function and put them together into a working kernel file and then load the onto a floppy file then onto an ISO like in the MikeOS tutorial. Or should I just make the kernel.bin and load it with syslinux? Could anyone give me a tip on how to make this all work on a Mac developement environment? I have tolls via macports and homebrew so that helps. Anyone successively done this?
EDIT
Here's my bootsector so far.
I just wanna know how to jump to an extern function from the C and link it.
There's a few problems with this. First of all, all the compilers you mentioned output either 32-bit or 64-bit code. That's great, but when the boot sector starts, it's running in 16-bit real mode. If you want to be able to run that 32-bit or 64-bit code, you'll need to first switch to the appropriate mode (32-bit protected mode for, well, 32-bit, and long mode for 64-bit).
Then, once you switch to the appropriate mode, you don't even have that much space for code: boot sectors are 512 bytes; two bytes are reserved for the bootable signature, and you'll need some bytes for the code that switches to the appropriate mode. If you want to be able to use partitions on that disk or maybe a FAT filesystem, take away even more usable bytes. You simply won't have enough space for all but the most trivial program.
So how do real operating systems deal with that? Real operating systems tend to use the boot sector to load a bigger bootloader from the disk. Then that bigger bootloader can load the actual kernel and switch to the appropriate mode (although that might be the responsibility of the loaded kernel — it depends).
It can be a lot of work to write a bootloader, so rather than rolling your own, you may want to use GRUB and have your kernel comply to the Multiboot standard. GRUB is a bootloader which will be able to load your kernel from the disk (probably in ELF format) and jump to the entry point in 32-bit protected mode. Helpful, right?
This does not free you from learning assembly, though: The entry point of the kernel must be assembly. Often, all it does is set up a little stack and pass the appropriate registers to a C function with the correct calling convention.
You may think that you can just copy that rather than writing it yourself, and you'd be right, but it doesn't end there. You also need assembly for (at least):
Loading a new global descriptor table.
Handling interrupts.
Using non-memory-mapped I/O ports.
…and so on, not to mention that if you have to debug, you may not have a nice debugger; instead, you'll have to look at disassemblies, register values, and memory dumps. Even if your code is compiled from C, you'll have to know what the underlying assembly does or you won't be able to debug it.
In summary, your main problem is not knowing assembly. As stated before, assembly is essential for operating system development. Once you know assembly thoroughly, then you may be able to start writing an operating system.