Xen to KVM migration

I recently had to migrate a Centos (paravitualized) Xen domU to KVM, after some digging and cursing, here's how the magic operated :

First of all, stop the virtual machine (guest) and copy the virtual disk to the new location.
We will use logical volumes for both virtual disks (orig/dest), because of the speed you gain compared to using files, and we will of course assume both are the same size (or at least the destination is bigger).
Also assumed here: you have a huge /tmp or a tiny gzip'ed image of the guest.

[root@xenhost ~] dd if=/dev/vg_xenhost/lv_xen_centos_orig | gzip - > /tmp/centos_orig.iso.gz
[root@kvmhost ~] scp root@xenhost:/tmp/centos_orig.iso.gz /tmp
[root@kvmhost ~] zcat centos_orig.iso.gz | dd of=/dev/vg_kvmhost/lv_kvm_centos_dest

Make the guest's logical volumes known to the kernel:

[root@kvmhost ~] kpartx -a /dev/vg_kvmhost/lv_kvm_centos_dest
[root@kvmhost ~] vgchange -ay VolGroup00

Mount the guest's filesystems (this article will use the utterly stupid(*) centos default's LVM naming scheme):

[root@kvmhost ~] mount /dev/VolGroup00/LogVol00 /mnt/loop/
[root@kvmhost ~] mount /dev/mapper/vg_kvmhost-lv_kvm_centos_dest1 /mnt/loop/boot
[root@kvmhost ~] mount --bind /dev /mnt/loop/dev/
[root@kvmhost ~] mount -t proc none /mnt/loop/proc/

And finally, chroot inside the VM:

[root@kvmhost ~] chroot /mnt/loop

Remove the Xen console for getty:

/etc/inittab
------------
#co:2345:respawn:/sbin/agetty xvc0 9600 vt100-nav
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6

We will need a normal (not Xenified) kernel:

[root@guest ~]# yum install kernel

Make it the default and remove the Xen console:

/boot/grub/menu.lst
-------------------
remove all occurences of: console=xvc0
default=0

Grub will need this symlink to exist for setup (hd0,0) to work:

[root@guest ~]# ln -s /dev/mapper/vg_kvmhost-lv_kvm_centos_dest1 /dev/vg_kvmhost/lv_kvm_centos_dest1

Install grub on the logical volume:

[root@guest ~]# grub
device (hd0) /dev/vg_kvmhost/lv_kvm_centos_dest
root (hd0,0)
setup (hd0)
quit

Remove the symlink and exit the guest:

[root@guest ~]# rm -f /dev/vg_kvmhost/lv_kvm_centos_dest1
[root@guest ~]# exit

Close the guest:

[root@kvmhost ~] umount /mnt/loop/{dev,proc,boot,}
[root@kvmhost ~] vgchange -an VolGroup00
[root@kvmhost ~] kpartx -d /dev/vg_kvmhost/lv_kvm_centos_dest

Here is an example you can use for creating the VM, which will generate an error on the first run, because /dev/zero is not a bootable device ! :)

[root@kvmhost ~] virt-install --connect qemu:///system -n migration -r 512 --vcpus=1 -c /dev/zero -f /dev/vg_kvmhost/lv_kvm_centos_dest --vnc --os-type linux --accelerate --network=bridge:br0 --noreboot --keymap fr

Remember to change /etc/sysconfig/network-scripts/ifcfg-eth0 according to your new MAC address.


Note: On a recent (guest) kernel, you should be able to migrate from Xen paravirt to KVM w/ virtio.

(*) /dev/VolGroup00/LogVol00 ? seriously ?