Ability to run aarch64 guests on Linux without KVM

[msgilligan]

Description

It should be possible to run aarch64 guest VMs on aarch64 Linux without support for KVM. There should be a way to set the QEMU options to support this. This would allow running Lima on ubuntu-24.04-arm GitHub Actions runners and also inside aarch64 Linux Lima instances on pre-M3 macOS systems.

Here is an example GitHub Actions workflow that should work: https://github.com/nixos-lima/nixos-lima-config-sample/blob/b02a4c9cffaeb646feda5ad8940c9ade7599cb32/.github/workflows/ubuntu-arm.yml
This job fails with the following message in the log:

{"level":"debug","msg":"qemu[stderr]: Could not access KVM kernel module: No such file or directory","time":"2025-01-26T22:10:43Z"}

[afbjorklund]

You want to be able to opt-in to running in "slow mode" (qemu-system), even on the native architecture?

Override the "accel" used

Similar to cpu type et al

Or should there just be more validation on /dev/kvm etc to check if it is actually working, instead of letting qemu test it?

[msgilligan]

I just want to be able to run aarch64 Lima VMs inside GitHub ubuntu-24.04-arm instances in the most efficient manner possible. Currently, I can't get one to run at all.

I'm seeing similar issues trying to run nested aarch64 Lima VMs on an M1 Mac.

Shouldn't it be possible to run these without nested hardware virtualization?

[afbjorklund]

It would be possible to make the accelerator configurable. As a workaround until then, you could run x86_64?

[msgilligan]

As a workaround until then, you could run x86_64?

That's exactly what I'm already doing. (It hurts me to my soul, though. I'm trying to be an aarch64-maximalist!)

[msgilligan]

FYI, with this change I am able to get it to work on GitHub Actions:

nixos-lima/nixos-lima-config-sample@038f6d7

This was based on @jandubois 's suggestion here:

#3173 (comment)

[msgilligan]

I am able to get it to work on GitHub Actions

I may have spoken too soon.

I'm having stability issues with nixos-rebuild inside the aarch64 -machine virt -cpu max VM on a non-KVM aarch64 ubuntu-24.04-arm runner. I am not seeing these issues when emulating aarch64 on x86_64. It could be a resource issue, I'm not sure at this time.

[afbjorklund]

Stability issues with emulation are to be expected, I am using a Raspberry Pi for some of the native builds...

There are cloud providers that offer ARM support, so that you don't need to use virtualization nor emulation.