[Bug 1641532] Re: machine-types trusty and utopic are not unique (depend on the qemu version)

Tue Jan 10 15:46:30 UTC 2017

Sorry it took so long, but I'm happy that I finally found the right
people to talk about it, summarizing the issue once more and then
subscribing a bunch of people to make the right call for the UCA on
this.

Affected:
- Trusty type guests in Liberty/Mitaka/Xenial/Yakkety/Zesty
- Utopic type guests in Liberty/Mitaka/Xenial

Status today - a migration forward fails only in some combinations but not in others.
- Trusty as-is (2.0) to Xenial works even with the bug in place (accidentally/fortunately)
- Trusty + UCA-Liberty (2.3) to Xenial fails due to the issue (reported case).
- other combinations have to be evaluated

Cause:
- the way machine types are defined got changed several times in qemu along the updates Trusty to Xenial
- It seem that there was an error added when porting the machine types to Wily
- This issue got carried forward when the conversion to Xenial was made
- Due to that those types are not non-ambiguous, they inherit the current qemus version
- So instead of Trusty being always 2.0 as it should be, it is 2.x matching the hosts qemu version
- The same applies to the utopic type

Challenges on fixing that:
- Testing showed that fixing on Xenial did not help - especially for the reported case - it expected 2.1 (utopic) instead ot 2.5 after the fix, but the source was Liberty which is 2.3)
- Since it seems it needs fixing on the source of the migration as well it needs
- In general to "pick up" the fix you need to (re-)start the guest
- This has the additional impact that the guest visible virtual hardware will change, so e.g. once we fix Xenial/Liberty to expect trusty to be 2.0 as it should be - any guests that were started as 2.5 type before will restart as 2.0 type and could cause further issues which makes that fix super critical. And especially going down versions might be critical.
- Due to the fact that Trusty->Xenial works fine, we might think on doing only a fix to UCA-Liberty and avoid these new issues by fixing the old one.
- There might be other even more complex approaches, but then equally more error prone, we have to decide on what we need here

Actions that are to be evaluated:
- Create a fix for Liberty (and eventually Kilo)
- Test if that fixes it without restarting a guest (unlikely but might help and would be great)
- Test if that fixes it with restarting the guest (that should work for sure at least)
- After that fix of the more immediate issue we have to think on >=Xenial preparation with the lowest impact to users due to restarting into lower machine type levels or later migration.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1641532

Title:
  machine-types trusty and utopic are not unique (depend on the qemu
  version)

Status in Ubuntu Cloud Archive:
  New
Status in qemu package in Ubuntu:
  Confirmed

Bug description:
  Hi,

  I'm currently live-migrating many VMs from an old server to a new one,
  and some VM can't be live migrated.

  The source host is trusty with qemu-system-x86 1:2.3+dfsg-5ubuntu9.4~cloud2
  The destination host is xenial with qemu-system-x86 1:2.5+dfsg-5ubuntu10.6

  When the issue occurs, the destination host raises an error [1] and
  stop the migration process.

  The only difference I see between VMs where live migration works and
  those were it doesn't work is a different machine type.

  * migration works when VM have been created with pc-i440fx-vivid
  * migration doesn't work when VM have been created with pc-i440fx-utopic

  [1] the qemu error report by libvirt on the destination host

  2016-11-14 08:25:40.774+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.5 (Stefan Bader <stefan.bader at canonical.com> Thu, 06 Oct 2016 13:07:20 +0200), qemu version: 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.6), hostname: n7.tetaneutral.net
  LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=spice /usr/bin/kvm-spice -name a81e7133-9601-4432-86dd-a2401dcad8c2 -S -machine pc-i440fx-utopic,accel=kvm,usb=off -cpu Nehalem -m 256 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid a81e7133-9601-4432-86dd-a2401dcad8c2 -smbios 'type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=2015.1.2,serial=fe2641d2-543a-4d65-b75f-6337bf4b8744,uuid=a81e7133-9601-4432-86dd-a2401dcad8c2' -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-a81e7133-9601-4432-86dd-a2401dcad8c2/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive 'file=rbd:disks/a81e7133-9601-4432-86dd-a2401dcad8c2_disk.config:id=openstack-service:key=XXXXXXXXXXXX==:auth_supported=cephx\;none:mon_host=192.168.99.251\:6789\;192.168.99.252\:6789\;192.168.99.253\:6789,format=raw,if=none,id=drive-ide0-1-1,readonly=on,cache=none,aio=native' -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -drive 'file=rbd:disks/volume-a82bb407-5ccb-4b0e-ba68-a0de1cd58cc3:id=openstack-service:key=XXXXXXXXXXXXXXXXXX:auth_supported=cephx\;none:mon_host=192.168.99.251\:6789\;192.168.99.252\:6789\;192.168.99.253\:6789,format=raw,if=none,id=drive-virtio-disk0,serial=a82bb407-5ccb-4b0e-ba68-a0de1cd58cc3,cache=none,aio=native' -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=43,id=hostnet0,vhost=on,vhostfd=45 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:37:fb:ba,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/a81e7133-9601-4432-86dd-a2401dcad8c2/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -spice port=5916,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -incoming defer -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
  2016-11-14T08:25:44.216364Z qemu-system-x86_64: Unknown ramblock "/rom at etc/acpi/rsdp", cannot accept migration
  2016-11-14T08:25:44.216394Z qemu-system-x86_64: error while loading state for instance 0x0 of device 'ram'
  2016-11-14T08:25:44.216509Z qemu-system-x86_64: load of migration failed: Invalid argument

  Cheers,

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1641532/+subscriptions