rust-osdev/bootloader

An experimental x86_64 bootloader that works on both BIOS and UEFI systems

Rust compiler with the llvm-tools-preview component, which can be installed through rustup component add llvm-tools-preview

bootloader

An experimental x86_64 bootloader that works on both BIOS and UEFI systems. Written in Rust and some inline assembly, buildable on all platforms without additional build-time dependencies (just some rustup components).

Requirements

You need a nightly Rust compiler with the llvm-tools-preview component, which can be installed through rustup component add llvm-tools-preview.

Usage

See our documentation. Note that the bootimage crate is no longer used since version 0.10.0.

Architecture

This project consists of three separate entities:

  • A library with the entry point and boot info definitions that kernels can include as a normal cargo dependency.
  • BIOS and UEFI binaries that contain the actual bootloader implementation.
  • A builder binary to simplify the build process of the BIOS and UEFI binaries.

These three entities are currently all combined in a single crate using cargo feature flags. The reason for this is that the kernel and bootloader must use the exact same version of the BootInfo struct to prevent undefined behavior (we did not stabilize the boot info format yet, so it might change between versions).

Build and Boot

The build and boot process works the following way:

  • The builder binary is a small command line tool that takes the path to the kernel manifest and binary as arguments. Optionally, it allows to override the cargo target and output dirs. It also accepts a --quiet switch and allows to only build the BIOS or UEFI binary instead of both.
  • After parsing the arguments, the builder binary invokes the actual build command for the BIOS/UEFI binaries, which includes the correct --target and --features arguments (and -Zbuild-std). The kernel manifest and binary paths are passed as KERNEL_MANIFEST and KERNEL environment variables.
  • The next step in the build process is the build.rs build script. It only does something when building the BIOS/UEFI binaries (indicated by the binary feature), otherwise it is a no-op.
    • The script first runs some sanity checks, e.g. the kernel manifest and binary should be specified in env variables and should exist, the correct target triple should be used, and the llvm-tools rustup component should be installed.
    • Then it copies the kernel executable and strips the debug symbols from it to make it smaller. This does not affect the original kernel binary. The stripped binary is then converted to a byte array and provided to the BIOS/UEFI binaries, either as a Rust static or through a linker argument.
    • Next, the bootloader configuration is parsed, which can be specified in a package.metadata.bootloader table in the kernel manifest file. This requires some custom string parsing since TOML does not support unsigned 64-bit integers. Parse errors are turned into compile_error! calls to give nicer error messages.
    • After parsing the configuration, it is written as a Rust struct definition into a new bootloader_config.rs file in the cargo OUT_DIR. This file is then included by the UEFI/BIOS binaries.
  • After the build script, the compilation continues with either the bin/uefi.rs or the bin/bios.rs:
    • The bin/uefi.rs specifies an UEFI entry point function called efi_main. It uses the uefi crate to set up a pixel-based framebuffer using the UEFI GOP protocol. Then it exits the UEFI boot services and stores the physical memory map. The final step is to create some page table abstractions and call into load_and_switch_to_kernel function that is shared with the BIOS boot code.
    • The bin/bios.rs function does not provide a direct entry point. Instead it includes several assembly files (asm/stage-*.rs) that implement the CPU initialization (from real mode to long mode), the framebuffer setup (via VESA), and the memory map creation (via a BIOS call). The assembly stages are explained in more detail below. After the assembly stages, the execution jumps to the bootloader_main function in bios.rs. There we set up some additional identity mapping, translate the memory map and framebuffer into Rust structs, detect the RSDP table, and create some page table abstractions. Then we call into the load_and_switch_to_kernel function like the bin/uefi.rs.
  • The common load_and_switch_to_kernel function is defined in src/binary/mod.rs. This is also the file that includes the bootloader_config.rs generated by the build script. The load_and_switch_to_kernel functions performs the following steps:
    • Parse the kernel binary and map it in a new page table. This includes setting up the correct permissions for each page, initializing .bss sections, and allocating a stack with guard page. The relevant functions for these steps are set_up_mappings and load_kernel.
    • Create the BootInfo struct, which abstracts over the differences between BIOS and UEFI booting. This step is implemented in the create_boot_info function.
    • Do a context switch and jump to the kernel entry point function. This involves identity-mapping the context switch function itself in both the kernel and bootloader page tables to prevent a page fault after switching page tables. This switch step is implemented in the switch_to_kernel and context_switch functions.
  • As a last step after a successful build, the builder binary turns the compiled bootloader executable (includes the kernel) into a bootable disk image. For UEFI, this means that a FAT partition and a GPT disk image are created. For BIOS, the llvm-objcopy tool is used to convert the bootloader executable to a flat binary, as it already contains a basic MBR.

BIOS Assembly Stages

When you press the power button the computer loads the BIOS from some flash memory stored on the motherboard. The BIOS initializes and self tests the hardware then loads the first 512 bytes into memory from the media device (i.e. the cdrom or floppy disk). If the last two bytes equal 0xAA55 then the BIOS will jump to location 0x7C00 effectively transferring control to the bootloader.

At this point the CPU is running in 16 bit mode, meaning only the 16 bit registers are available. Also since the BIOS only loads the first 512 bytes this means our bootloader code has to stay below that limit, otherwise we’ll hit uninitialised memory! Using Bios interrupt calls the bootloader prints debug information to the screen.

For more information on how to write a bootloader click here. The assembler files get imported through the global_asm feature. The assembler syntax definition used is the one llvm uses: GNU Assembly.

The purposes of the individual assembly stages in this project are the following:

  • stage_1.s: This stage initializes the stack, enables the A20 line, loads the rest of the bootloader from disk, and jumps to stage_2.
  • stage_2.s: This stage sets the target operating mode, loads the kernel from disk,creates an e820 memory map, enters protected mode, and jumps to the third stage.
  • stage_3.s: This stage performs some checks on the CPU (cpuid, long mode), sets up an initial page table mapping (identity map the bootloader, map the P4 recursively, map the kernel blob to 4MB), enables paging, switches to long mode, and jumps to stage_4.

Future Plans

  • Create a multiboot2 compatible disk image in addition to the BIOS and UEFI disk images. This would make it possible to use it on top of the GRUB bootloader.
  • Rewrite most of the BIOS assembly stages in Rust. This has already started.
  • Instead of linking the kernel bytes directly with the bootloader, use a filesystem (e.g. FAT) and load the kernel as a separate file.
  • Stabilize the boot info format and make it possible to check the version at runtime.
  • Instead of searching the bootloader source in the cargo cache on building, use the upcoming "artifact dependencies" feature of cargo to download the builder binary separately. Requires doing a boot info version check on build time.
  • Transform this "Future Plans" list into issues and a roadmap.

License

Licensed under either of

  • Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
  • MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Issues

Collection of the latest Issues

Veliania

Veliania

Comment Icon0

I'm not sure what to do about this or if this is the right place but whenever I try to run the very barest of what I understand should work in terms of code for a kernel

But no matter what it wont work and it throws an error saying I'm building using x86_64-unkown-none as the target and using the BootImage command to build it

michalusio

michalusio

help wanted
Comment Icon1

Hi! First off, I wanted to thank the creator and the contributors of this package for all the great work.

Now, onto the issue: I have been using the bootloader to run a small kernel I'm writing and I have noticed that while I can boot when the built image is attached as a hard drive to QEMU, I am not able to boot when it's attached as a CD-ROM. I have dug deeper and from what I've read it is because CD-ROMs have a completely separate booting system called El Torito.

As such, I would like to know if there would be a possibility of adding El Torito support and add it so we could build the images directly for booting from a CD?

InfRandomness

InfRandomness

Comment Icon9

Hi, I have started working on trying to implement a way to feed the builder a path to a target JSON file.. This went about as well as I wanted it to, until I ran into this :

Here are the changes I brought to the builder binary :

I was wondering, if I were to dig more into the problem to implement this feature, would this work at all if the chose target is x86_64 UEFI or aarch64 UEFI ? I don't know a whole lot about UEFI so I am assuming that the code from an x86_64 UEFI bootloader and an ARM64 UEFI bootloader is pretty much the same; Maybe I'm wrong.

kennystrawnmusic

kennystrawnmusic

Comment Icon0

Browsing crates.io for file system crates turned up something very interesting: a btrfs driver which, upon snooping the code, was very easy to port to make it work in #![no_std] environments. So I forked it to accomplish exactly that.

Not thoroughly tested of course, but this would definitely be a great choice for integration into the bootloader so that it recognizes btrfs as a file system from which to load a kernel binary (something very notably on the checklist at the bottom of the README).

asoderman

asoderman

Comment Icon0

Hello I've been stepping through the bootloader code to learn more about osdev. Upon entering stage_3 I attempted to add a call vga_printline with a test string I defined in stage_3.s this caused repeated rebooting. I verified it wasn't just me invoking vga_printline incorrectly by adding jmp no_cpuid after call check_cpuid so as to simulate a no_cpuid result and invoke vga_printline as it was intended but received the same rebooting. Running qemu with -d int outputs check_exception old: 0x8 new 0xd which I believe is Double Fault -> GPF.

Some investigation revealed the faulting instruction is mov eax, vga_position line 121 in stage_2.s but I'm not sure why that would be. My method for determining this was not the best as I would mov a dummy value into edi then would check edi after the fault for the dummy value using qemu's monitor. If the dummy value was there I would move the instruction further along in the execution then repeat the process. So maybe double check this 😄 .

Steps to reproduce (using master branch):

  1. add jmp no_cpuid to line 17 in src/asm/stage_3.s
  2. in examples/basic create a disk image with cargo kimage
  3. invoke qemu-system-x86_64 --no-shutdown --no-reboot -drive format=raw,file=target/x86_64-custom/debug/boot-bios-basic_example.img -monitor stdio -d int (I believe the default runner behavior just exits immediately)
vorporeal

vorporeal

Comment Icon1

I'm trying to set up a basic skeleton for a Rust-based OS using this package to create a bootloader and boot image, and am running into the following problem:

I've also tried just setting bootloader v0.10.11 as a dependency for my crate and trying to build a minimal kernel (just a _start that loops forever and a panic handler) and run into the same issue when trying to build it:

I am using the nightly toolchain via rustup, and have installed the llvm-tools-preview component for the nightly toolchain. Did I miss an important step somewhere? Any ideas?

Narasimha1997

Narasimha1997

Comment Icon0

As I've seen, the bootloader can boot in legacy BIOS ams UEFI modes. However, there is no way for the kernel to directly know in which mode it has been booted. This can be useful in different parts of the kernel (For example, during the initialisation of 8259 PIC as trying to enable PIC in UEFI mode double faults immediately).

One way I've seen is to iterate over the memory map and check if the regions are of type UnknownBios or UnknownUefi and take decisions. But this isn't an idiomatic way.

The best way would be to pass a flag in BootInfo struct called is_uefi which is true or false based on the mode the kernel is booted.

vikigenius

vikigenius

Comment Icon0

So in both the examples, -Zbuild-std=core is being passed from .cargo/config.toml. My question is how does that work with the boot/simple_bootsub crates that need std as well?

Or does the build options only apply to the main crate and not the boot sub crates as well?

saki-osive

saki-osive

Comment Icon2

I'm a complete beginner to the project, is there any documentation to get started with the project that possibly covers everything from scratch? My aim is to be able to build and run the bootloader in a virtualized environment. Thanks

ethindp

ethindp

Comment Icon6

So I have no idea where to put this, though I've spoke about it in various places and have gotten no response in a month or so. (I apologize if I seem impatient.) It doesn't appear like the bootloader is mapping all of the required paged memory regions that my kernel uses. This in turn makes it prohibitive to continue my project because I have no idea why this is even happening to begin with. My debugging has lead me -- repeatedly -- to conclude that its a bug when x86_64 goes to zero an entry in one of the paging structures, though which one I am unclear about. Specifically, it occurs when allocating the kernel heap. When my kernel attempts to allocate address 3000h (or, rather, 18000003000h) I get a #PF, but since (for some reason) my IDT is being ignored (presumably it isn't mapped yet) a page fault occurs. Then another occurs because the ISRs aren't mapped, and then a double fault occurs, and then a triple fault. It has gotten to the point where I'm considering just making my own paging structures and not using the bootloader-provided ones. I want to avoid that but if there is truly no solution to this then I won't have much of a choice. My code is available here. Does anyone know why this is happening? Is there something I'm missing? From what I can tell the bootloader sets CR3 and (appears) to map everything but the behavior of my kernel leads me to believe otherwise.

actuallyexeon

actuallyexeon

Comment Icon13

I know it's possible to increase the brightness of pixels but how can you change the color, I see it's possible in logger.rs but way to hard to understand. Does anyone know how to use the framebuffer?

ethindp

ethindp

Comment Icon2

Currently, the bootloader presents an incorrect memory map to the kernel (potentially invalidating the entire setup and making all mapped regions using that memory map incorrect). Specifically, an example (wrong) memory map is provided by me in this OSDev forum topic. According to replies in that topic, UEFI runtime services (both code and data) should be in the memory map, but none exist. Currently the Bootloader passes raw values for memory map region types. This is (not) a good thing because the UEFI specification does not specify these values, and so they may differ from implementation to implementation. Furthermore, BIOS memory types are also not specified, and so there is no point of reference for these values, and these will (undoubtedly) differ from BIOS implementation to BIOS implementation. This is a big problem. A theoretical situation where things could go (very) wrong is if an OS wants to allocate a page that it considers "free". If an OS uses a page that is (really) PAL code or NVRAM storage, but it is marked as "free" because the values are not specified by the bootloader crate in any safe abstraction, such an allocation (and then operations on such areas of RAM) could be disastrous. PAL code, in particular, could be especially bad because PAL code is specifically for (if I understand correctly) programmable array logic devices. (I don't know of implementations that do expose things like PAL code to UEFI applications or drivers, but since its in the specification, an implementation may choose to do so.) Is there a reason why the bootloader exposes memory types without any standard definition of at least the predefined region types (for UEFI) or something like it for BIOS, if it exists? For reference, the code used to generate this map was this (I needed to do manual reading of the data to get this to work):

Andy-Python-Programmer

Andy-Python-Programmer

enhancement
Comment Icon4

It will be nice to pass UnwindInfo in BootInfo so that we can load the kernel ELF in the kernel to do stack unwinding.

MarcoCicognani

MarcoCicognani

Comment Icon3

Hi, as I written in the title it would be a good idea to enable global pages and map the kernel using this flag. Doing this any kernel mapping would be implicitly duplicated across all the address spaces.

Keep it up! Waiting UEFI rework's release

pitust

pitust

enhancement
Comment Icon7

Yeah, stuff gets mapped multiple times. Here is my log output [from an isa-debugcon]:

And it just hangs. So... It seems that sections do NOT have to be 4k-aligned in ELFs. The culprit in my case was .got. Forcing it to stay aligned fixed my problem. But, this should just work and not require custom linker scripts.

Versions

Find the latest versions by id

Information - Updated May 03, 2022

Stars: 756
Forks: 142
Issues: 42

Rust library for Self Organising Maps (SOM)

Add rusticsom as a dependency in Cargo

Rust library for Self Organising Maps (SOM)

Rust library for parsing configuration files

The 'option' can be any string with no whitespace

Rust library for parsing configuration files

Rust library for the Pimoroni Four Letter pHAT

This library aims to port ht16k33 (or rather a fork, as of right now) so credit goes to ht16k33-diet

Rust library for the Pimoroni Four Letter pHAT

Rust library for emulating 32-bit RISC-V

This library can execute instructions against any memory and register file that implements

Rust library for emulating 32-bit RISC-V

Rust library for connecting to the IPFS HTTP API using Hyper/Actix

You can use actix-web as a backend instead of hyper

Rust library for connecting to the IPFS HTTP API using Hyper/Actix

Rust library to manipulate file system access control lists (ACL) on macOS, Linux, and FreeBSD

This module provides two high level functions, getfacl and setfacl

Rust library to manipulate file system access control lists (ACL) on macOS, Linux, and FreeBSD

Rust library translation (rust-src/rust-std/stdlib/rustlib translation)

This is the place to translate Having a documentation in your native language is essential if you don't speak English, and still enjoyable even if...

Rust library translation (rust-src/rust-std/stdlib/rustlib translation)

Rust library for using Infrared hardware decoders (For example a Vishay TSOP* decoder),

enabling remote control support for embedded project

Rust library for using Infrared hardware decoders (For example a Vishay TSOP* decoder),

Rust library for interaction with the OriginTrail Decentralized Knowledge Graph

open up an issue on this repository and let us know

Rust library for interaction with the OriginTrail Decentralized Knowledge Graph

Rust library for parsing COLLADA files

Notice: This library is built around files exported from Blender 2

Rust library for parsing COLLADA files

Rust library for low-level abstraction of MIPS32 processors

This project is licensed under the terms of the MIT license

Rust library for low-level abstraction of MIPS32 processors
Facebook Instagram Twitter GitHub Dribbble
Privacy