[Bug 2088620] Please test proposed package

[Edward Hope-Morley]

Hello Bryan, or anyone else affected,

Accepted watcher into bobcat-proposed. The package will build now and be
available in the Ubuntu Cloud Archive in a few hours, and then in the
-proposed repository.

Please help us by testing this new package. To enable the -proposed
repository:

  sudo add-apt-repository cloud-archive:bobcat-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-bobcat-needed to verification-bobcat-done. If it does
not fix the bug for you, please add a comment stating that, and change
the tag to verification-bobcat-failed. In either case, details of your
testing will help us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance!

** Changed in: cloud-archive/bobcat
       Status: New => Fix Committed

** Tags added: verification-bobcat-needed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2088620

Title:
  [SRU] Deprecated usage of cpu_util

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive antelope series:
  Fix Committed
Status in Ubuntu Cloud Archive bobcat series:
  Fix Committed
Status in Ubuntu Cloud Archive caracal series:
  Fix Released
Status in Ubuntu Cloud Archive dalmatian series:
  Fix Released
Status in Ubuntu Cloud Archive epoxy series:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  New
Status in Ubuntu Cloud Archive zed series:
  Won't Fix
Status in watcher package in Ubuntu:
  Fix Released
Status in watcher source package in Focal:
  Confirmed
Status in watcher source package in Jammy:
  Confirmed
Status in watcher source package in Noble:
  Fix Released
Status in watcher source package in Oracular:
  Fix Released

Bug description:
  [ Impact ]

    * The watcher releases targeted by this SRU are using a deprecated
  ceilometer metric, cpu_util, which reported cpu utilization as a
  percentage. This metric was deprecated in Openstack Rocky in favor of
  the Gnocchi rate calculation equivalent [1].

    * Upstream Watcher continued to use cpu_util until the commit at [2]
  landed on master for 2024.1. This commit correctly performs the cpu
  calculation and removes the deprecated metric. The calculation is
  summarized in the next bullet point and there is an example
  calculation in the original commit

    * The gnocchi calculation uses the cumulative cpu time in ns
  (reported by the cpu metric), taken as a rate (the difference in
  cumulative time over the last two sampling intervals) to find the
  total cpu time during the previous sampling period. Dividing the cpu
  time in one interval by the duration of the interval multiplied by the
  number of vcpus provides the cpu utilization as a percentage:
  cpu_usage = [cpu_time / (period * 10^9 * nvcpus)] * 100%. A sample
  calculation is provided in the original commit message.

    * I cherry-picked to stable/2023.2 [3], but the other branches have
  gone unmaintained

  [ Test Plan ]

    * Deploy openstack yoga on jammy with watcher and gnocchi services

    * Launch a server and take note of it's resource id. Then find the
  gnocchi cpu metric associated with the instance

    * Create a watcher audit based on a goal that previously depended on instance cpu utilization. For example the workload_balance goal [4]
      Ex. openstack optimize audit create -t CONTINUOUS -i 60 -g workload_balancing -s workload_balance --auto-trigger
      Without the patch instance_cpu_usage appears as None in the audits. With the patch you can observe the correct cpu utilization percentage in the watcher-decision-engine.log

    * Wait for at least one sampling period to elapse and check
  /var/log/watcher/watcher-decision-engine.log for entries showing
  "instance_cpu_usage" - this is the cpu utilization as a percentage.

    * To verify the percentage with a manual calculation, run gnocchi
  measure show <metric uuid> --aggregation "rate:mean" and perform the
  calculation instance_cpu_usage = 100*[<value> / (period * 10^9 *
  nvcpus) using the cpu time from the corresponding sampling period

  [ What can go wrong ]

    * While this is replacing a deprecated methodology and metric and
  should lead to improvements, any custom strategies relying on cpu_util
  may be affected.

  [1] https://docs.openstack.org/releasenotes/ceilometer/rocky.html
  [2] https://review.opendev.org/c/openstack/watcher/+/898791
  [3] https://review.opendev.org/c/openstack/watcher/+/934181
  [4] https://docs.openstack.org/watcher/rocky/strategies/workload_balance.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2088620/+subscriptions