New project: memdisk_uefi

[russor]

Hey all, I got inspired and built something like Syslinux's MEMDISK, but for UEFI:

https://github.com/russor/memdisk_uefi

I've only tested on VMWare amd64, but hopefully it works elsewhere too. It uses iPXE's UEFI download protocol to fetch a disk image, mounts it via UEFI Ram Disk Protocol, exposes the image as a NVDIMM, and then boots from the mounted image.

Like with memdisk, early init of the OS will likely use system facilities and be able to access the ramdisk with no problem, but once the OS kernel exit's UEFI boot services, it needs to find and use the image. Thankfully, NVDIMM has been around for a while, so there's some support already. Linux has had a driver since 2015, and FreeBSD has had a driver since 2018; I know the FreeBSD installer needs manual help to load the nvdimm driver, but the one Linux ISO I had on hand (ubuntu-24.04.1-live-server) did automatically load the driver; I don't know if this is common or not.

This might help setup UEFI netbooting for more distributions, without having to deconstruct installer images as is often the case.

[NiKiZe]

This sounds similar to what the iPXE wimboot does, I have used it to for example boot grub with local "on disk" files, and also other efi binaries that had intentional limitations to network boot.
I'm curries to know if your usecase is solvable with wimboot as well? (the upside would be that it would potentially be a solution that could work with secure boot enabled)

[russor]

(sorry for the wall of text!)

I hadn't looked at wimboot before, but if I understand it correctly, there are some similarities. As I understand it, wimboot (in UEFI) works roughly like this:

a) wim image(s) are loaded by pxe as initrds
b) wimboot is started as a kernel
c) wimboot mounts the images from initrd (I believe using filesystem code inside wimboot)
d) wimboot finds the next stage loader from the image
e) (if running in a secure boot environment) wimboot checks if the next stage loader is signed with a valid key
f) wimboot does something so the next state loader can use the image (I didn't look a whole lot into this part)
g) wimboot runs the next stage loader

Also, and this is pretty neat, wimboot is delivered as a BIOS/UEFI hybrid file; so an ipxe script that might run in bios mode or amd64 uefi mode can use the same initrd and kernel lines for either environment (ARM 64 or x86-32 uefi would need a different kernel though)

memdisk_uefi isn't secure boot aware and I'm not really either. I looked around a bit, and I think you'd need a signed iPXE and signed wimboot to do this? I see there might be some signed builds floating around, but I didn't play with them. I think it should be possible to add secure boot logic and sign memdisk_uefi and get similar results, but it's not there now.

What memdisk_uefi does that I didn't see wimboot do is mount a regular iso9660 image; memdisk_uefi uses UEFI's apis to mount the memdisk and that includes support for iso9660 and likely some other filesystems. Otherwise what memdisk_uefi does is relatively similar to wimboot:

a) memdisk_uefi is started as a kernel
b) memdisk_uefi uses the iPXE api to download the image (discussion below)
c) memdisk_uefi calls UEFI apis to mount the image
d) memdisk_uefi (optionally) calls UEFI apis to add ACPI tables to mark the memory disk as a NVDIMM, which some OSes have a driver to use
e) memdisk_uefi finds the next stage bootloader from the image
f) memdisk_uefi runs the next stage bootloader

I had memdisk_uefi download the image, rather than having it as an initrd to avoid the need for multiple copies in memory. If the image is expected to live beyond the loader phase, either the OS loader needs to do something special (which is tricky) or it needs to be loaded into memory allocated with a UEFI type that survives beyond boot services. I don't think I can control the memory allocation of initrd from ipxe, so if it's an initrd, memdisk_uefi would need to copy it, and for large images it's conceivable that you wouldn't have enough memory for two copies. Booting a Windows disc image with memdisk_uefi doesn't really work anyway, because there's no driver to continue access to the image after the end of boot services, but it's a good example of a large installer image: it's 4.7 GB --- if it needed to be in memory twice, you would need at least 10 GB of memory, but there's plenty of people running Windows on 8 GB. Again, I'm not sure how wimboot deals with this, the underlying wim images are the bulk of the disc size --- I know I saw there's a driver for windows to use a loaded wim image, but I didn't look at the details.

I have gotten feedback from FreeBSD people that they're not really comfortable using the NVDIMM driver to load an image; I had been hoping they'd just enable the driver in the installer kernel, but they'd prefer to do something else and I haven't looked at other alternatives yet. When I have some time, I'll explore some other options. As background, I submitted a patch that got cleaned up/rewritten and merged into FreeBSD 15.0 to automatically mount memdisks loaded by syslinux's memdisk in the BIOS environment; that enables using the regular installer image from ipxe for BIOS systems, and I'm trying to find a solution for UEFI too.

[NiKiZe]

wimboot creates a virtual fs, it maps initrd loaded data as files in the virtual fs so that they are available as if they where on a fs, it then kicks of the next stage (which must be named correctly for the platform)

Windows uses winpe (the boot.wim), which is around 500mb and not the full os. As you say, this data is then gone when kernel loads.

Wimboot does not mount isos, or do partial downloads.
Nor does it expose memdisk, which don't work in most cases anyway.

Instead the OS should support loading the bulk of data in some other way once kernel starts, or support it being injected on the native memfs.

wimboot is signed by MS, and iPXE will be available as signed as well. Getting other binaries signed by MS is hard.