[Bug 2020139] Autopkgtest regression report (python-openstacksdk/0.61.0-0ubuntu2)

Thu Jun 19 04:02:43 UTC 2025

All autopkgtests for the newly accepted python-openstacksdk (0.61.0-0ubuntu2) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

heat/1:18.0.1-0ubuntu1.2 (amd64)
heat/unknown (s390x)
neutron/unknown (amd64)
python-os-client-config/unknown (s390x)
python-osc-lib/unknown (s390x)
python-oslo.limit/unknown (s390x)
python-senlinclient/unknown (s390x)
senlin/1:13.0.0-0ubuntu1 (ppc64el)
senlin/unknown (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-
migration/jammy/update_excuses.html#python-openstacksdk

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-glanceclient in Ubuntu.
https://bugs.launchpad.net/bugs/2020139

Title:
  [SRU] Image fetch speed from Glance to Cinder volume service can be
  slow and capped by client CPU

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive bobcat series:
  Fix Released
Status in Ubuntu Cloud Archive caracal series:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  In Progress
Status in Glance Client:
  Fix Committed
Status in python-glanceclient package in Ubuntu:
  Fix Released
Status in python-openstacksdk package in Ubuntu:
  Fix Released
Status in python-glanceclient source package in Jammy:
  Fix Committed
Status in python-openstacksdk source package in Jammy:
  Fix Committed
Status in python-glanceclient source package in Noble:
  Fix Released
Status in python-openstacksdk source package in Noble:
  Fix Released

Bug description:
  *********** SRU TEMPLATE AT THE BOTTOM **************

  When Cinder volume service creates a volume from an image, it will
  download an image from Glance to a cinder-volume unit (to be precise,
  when CoW is not available by the backend between Glance and Cinder).

  The image fetch speed is not super fast as of today:

  2023-05-17 02:52:28.275 158391 DEBUG cinder.image.image_utils [req-e4234307-ae7c-4220-a10a-15760679ac01 a32b8e6ed20f484eba8ddae8f7b6eb1e 08dd29cd8676470c8e0b73d2667f8cf0 - - -] Image fetch details: dest /var/lib/cinder/conversion/image_fetch_741953c3-08f7-43b6-b423-2be938ac8d02_bc1hvgd7cinder at cinder-ceph, size 1907.35 MB, duration 27.80 sec fetch /usr/lib/python3/dist-packages/cinder/image/image_utils.py:637
  2023-05-17 02:52:28.275 158391 INFO cinder.image.image_utils [req-e4234307-ae7c-4220-a10a-15760679ac01 a32b8e6ed20f484eba8ddae8f7b6eb1e 08dd29cd8676470c8e0b73d2667f8cf0 - - -] Image download 1907.35 MB at 68.61 MB/s

  -> ~ 549 Mbps

  By running a profiler, it looks like serious_integrity_iter
  (glanceclient/common/utils.py) is consuming a lot of CPU cycles on the
  client side (in this case the cinder-volume unit).

  Image caching can overcome this kind of challenges but this report is for when we need to download images (new images, updated images, etc.).
  https://docs.openstack.org/cinder/latest/admin/image-volume-cache.html

  ===============
  SRU DESCRIPTION
  ===============

  [Impact]

  This issue causes images to be download much slower than they should,
  causing VM creation to potentially time out if the image is too big
  and takes too long. The fix proposed greatly improves performance and
  prevents timeouts in most cases where it would timeout without the
  fix.

  [Test case]

  1. Deploy OpenStack with Cinder and Glance

  2. Upload a resonable large image, like jammy ~650mb is enough.

  3. Create a volume from the image

  openstack volume create --size 3 --image jammy v1

  4. Check the logs for messages of time to fetch/download the image:

  ... cinder.image.image_utils ... Image fetch details: dest
  /var/lib/cinder/conversion/image_fetch_ed429779-0d11-45cb-8ee5-fee7c2003d86_rubea933juju-056b6e-jy3-0 at LVM-
  default, size 645.16 MB, duration 3.59 sec fetch ...

  ... cinder.image.image_utils ... Image download 645.16 MB at 179.58
  MB/s

  5. Repeat steps (3) and (4) 3 times just to make sure it is consistent
  and to have an average (avoid underlying infra fluctuations)

  6. Install fix and restart all cinder services (such as apache2,
  cinder-scheduler and cinder-volume)

  7. Repeat steps (3), (4) and (5), and make sure the new duration and
  download rate is at least about 50% faster.

  [Where problems could occur]

  If there are issues with the fix, it could impact the ability to
  download images at all, therefore impacting the ability to create
  volumes from images, and create VMs. Reverting the package is enough
  to undo the changes and restore previous functionality.

  [Other Info]

  Both python-glanceclient and openstacksdk fixes listed below are
  necessary.

  https://review.opendev.org/c/openstack/openstacksdk/+/883461

  https://review.opendev.org/c/openstack/python-glanceclient/+/924060

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2020139/+subscriptions