KVM2KFX

KVM2KFX

Transplanting a KVM VM for fuzzing on Xen.

KVM setup

1. Install build dependencies

sudo apt-get install build-essential bc fakeroot flex bison libelf-dev libssl-dev ncurses-dev libpixman-1-dev pkg-config zlib1g-dev libglib2.0-dev dh-autoreconf libpci-dev libudev-dev autoconf gawk git elfutils libvirt-clients virt-manager ninja-build

2. Patch Linux

git clone https://github.com/torvalds/linux
cd linux
git am ../patches/0001-linux-kfx-cpuid-vmexit.patch
make defconfig
make deb-pkg
sudo dpkg -i ../linux-image-*
cd ..

3. Patch QEMU

git clone https://github.com/qemu/qemu
cd qemu
git am ../patches/0001-qemu-*.patch
git am ../pathces/0002-qemu-*.patch
./configure --target-list=x86_64-softmmu --prefix=/opt/qemu-kfx
make
sudo make install
echo "/opt/qemu-kfx/bin/qemu-system-x86_64 PUx," | sudo tee /etc/apparmor.d/local/usr.sbin.libvirtd
sudo /etc/init.d/apparmor restart
cd ..

4. Reboot into new kernel

sudo reboot

5. Create VM with new QEMU

In your VM's config, change emulator line of VM in XML config:

   <emulator>/opt/qemu-kfx/bin/qemu-system-x86_64</emulator>

For example in virt-manager you can edit the VM's configuration XML after you enable Edit -> Preferences -> Enable XML editing.

Also make sure your VM has only a single CPU! Multi-CPU VMs are not currently supported.

After you change the config make sure to save it, and then start your VM.

6. Execute magic CPUID in VM

Log into the VM and execute:

cpuid -l 0x13371337

The VM state should show as paused in virsh list

7. Save VM

sudo ./scripts/kvm2kfx.sh <vmname>

Several files should now be in your folder: <vmname>-{regmap, memmap, vmcore};

7. Change CPUID that the VM will pause on:

For example to use 0x13371338 instead of the default 0x13371337:

Convert to decimal:

echo $((16#13371338))

Note output: 332376504

Run:

sudo virsh qemu-monitor-command <vmname> '{ "execute": "kfx", "arguments": { "cpuid": 332376504 } }'

Unpause VM

sudo virsh resume <vmname>

Execute new CPUID in VM:

cpuid -l 0x13371338

Create new save file as necessary.

Xen setup

0. Transfer saved files from KVM host

scp <vmname>-* [email protected]

1. Create VM shell for transplant

Use the following as transplant.cfg (change memory to be larger then it was on KVM by at least 1Gb):

arch = 'x86_64'
name = "transplant"
memory = 9000
vcpus = 1
type = "hvm"
hap = 1
vga="none"
vnc=0

Run:

xl create -p -e transplant.cfg
xl list

Note the transplant VMs domainid.

2. Transplant the VM

./xen-transplant <transplant_domainid> <vmname>-regmap <vmname>-memmap <vmname>-vmcore