[Bug 2007919] Re: [SRU] Stable point releases for python-ovsdbapp

Wed Aug 30 13:20:40 UTC 2023

Thanks.

I looked at the specific changes being made by upstream in this
microrelease, and have some concerns. I found them broken down by commit
here: https://opendev.org/openstack/ovsdbapp/commits/branch/stable/yoga

https://opendev.org/openstack/ovsdbapp/commit/ab3e0cb0d0865417efbf103f44954573a5ba92ac
- this seems like a deliberate change in behaviour. But it doesn't come
with any tests, and there's no explanation as to why it's needed in a
stable release. Is this even a bugfix? What if the retrying behaviour
*is* working for some user? Won't dropping that behaviour regress them?

https://opendev.org/openstack/ovsdbapp/commit/92cbba4481a7c7acedf24d8250fc7cf4be6b46d7
seems OK as a bugfix, but doesn't come with a test.

https://opendev.org/openstack/ovsdbapp/commit/315c8096c9bb3ad6d09e3b2f09bc1c128bc35497
seems good as a bugfix, and refers to bug 1988457 as what it is fixing,
but doesn't come with a test.

https://opendev.org/openstack/ovsdbapp/commit/a69430a967bfa1096b9258a8445b617b9e122a22
doesn't come with a test.

https://opendev.org/openstack/ovsdbapp/commit/de473cea9defe04a6f904fa63a8cf49746534916
fixes a regression caused by a change in OVS, and does come with a test
- great!

Conclusion

It doesn't seem like upstream microrelease updates come with enough
testing to land them for Ubuntu users in stable releases without further
manual testing. The upstream ab3e0cb commit is also concerning as it is
a deliberate change in behaviour.

NACK for an Ubuntu SRU based on the upstream microrelease exception.
Please justify and test each change you need individually instead.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-ovsdbapp in Ubuntu.
https://bugs.launchpad.net/bugs/2007919

Title:
  [SRU] Stable point releases for python-ovsdbapp

Status in OpenStack Neutron API Charm:
  Invalid
Status in Ubuntu Cloud Archive:
  Invalid
Status in Ubuntu Cloud Archive wallaby series:
  Fix Committed
Status in Ubuntu Cloud Archive xena series:
  Fix Committed
Status in Ubuntu Cloud Archive yoga series:
  Triaged
Status in Ubuntu Cloud Archive zed series:
  Fix Committed
Status in python-ovsdbapp package in Ubuntu:
  Invalid
Status in python-ovsdbapp source package in Jammy:
  New

Bug description:
  [Impact]
  This release sports mostly bug-fixes and we would like to make
  sure all of our supported customers have access to these
  improvements. The update contains the following package updates:

  python-ovsdbapp 2.1.1 (zed)
  python-ovsdbapp 1.15.3 (jammy/yoga)
  python-ovsdbapp 1.12.3 (xena)
  python-ovsdbapp 1.9.4 (wallaby)

  Delta between 2.1.0 and 2.1.1 (zed):

   - [97e738d] Fix TRY_AGAIN handling
   - [bbdc14a] Don't force_reconnect() on unhandled Idl exception
   - [c9e9473] Fix logging exception while handling transaction exception
   - [d90358c] Update TOX_CONSTRAINTS_FILE for stable/zed
   - [b58a340] Update .gitreview for stable/zed

  Delta between 1.15.1 and 1.15.3 (jammy/yoga):

   - [ab3e0cb] Don't force_reconnect() on unhandled Idl exception
   - [92cbba4] Fix logging exception while handling transaction exception
   - [315c809] Fix TRY_AGAIN handling
   - [a69430a] Handle OVS 2.17 change to Idl.tables
   - [de473ce] Fix broken behavior after recent OVS update
   - [10539ed] github no longer accepts git:// connections
   - [7735d5c] Update TOX_CONSTRAINTS_FILE for stable/yoga
   - [253afd4] Update .gitreview for stable/yoga

  Delta between 1.12.0 and 1.12.3 (xena):

   - [8202d53] Don't force_reconnect() on unhandled Idl exception
   - [1b04058] Fix logging exception while handling transaction exception
   - [fc62ae2] Fix TRY_AGAIN handling
   - [4d4402c] Update QoS register in "QoSAddCommand" if exists
   - [62ee947] Add cooperative_yield() to OvsdbIdl
   - [ab571f4] Handle OVS 2.17 change to Idl.tables
   - [a7a1d5c] Fix broken behavior after recent OVS update
   - [a73fbf1] github no longer accepts git:// connections
   - [3a0cd11] Log what exception caused abort of the transaction
   - [8c09568] Update TOX_CONSTRAINTS_FILE for stable/xena
   - [19a4364] Update .gitreview for stable/xena
   - [49ab130] tools: run functional tests against ovn/main branch

  Delta between 1.9.0 and 1.9.4 (wallaby):

   - [65d02f0] Don't force_reconnect() on unhandled Idl exception
   - [87dcab2] Fix logging exception while handling transaction exception
   - [dd7e332] Fix TRY_AGAIN handling
   - [f804411] Update QoS register in "QoSAddCommand" if exists
   - [e1a0d7c] Add cooperative_yield() to OvsdbIdl
   - [c6e2c43] Handle OVS 2.17 change to Idl.tables
   - [9990509] Fix broken behavior after recent OVS update
   - [fb5feee] github no longer accepts git:// connections
   - [fa4a747] Log what exception caused abort of the transaction
   - [eaa6406] tools: run functional tests against ovn/main branch
   - [8304251] Actually close the connection in Connection.stop()
   - [66918f9] Don't spam retries 100s of times a second
   - [d0cc492] Add an active wait in the "Backend.lookup"
   - [007c0f6] Fix docs job
   - [f6a19a3] Update TOX_CONSTRAINTS_FILE for stable/wallaby
   - [875be60] Update .gitreview for stable/wallaby

  [Test Case]

  The microrelease testing documentation at
  https://wiki.ubuntu.com/StableReleaseUpdates requests 4 points of
  testing, shown below, along with my comments on the coverage that we
  currently have in place:

  * a reliable and credible test suite for assuring the quality of every
  commit or release,

  coreycb: Upstream has unit tests and they are run during the package
  build.

  * the tests are covering both functionality and API/ABI stability,

  coreycb: This would be covered by unit tests as well as our planned
  functional testing described at [1].

  * the tests run during package build to cover all architectures,

  coreycb: Unit tests are run during package builds. Since this is a
  pure python package it is only built on amd64.

  * the package has an autopkgtest to run the tests in an Ubuntu
  environment against the actual binary packages"

  coreycb: The autopkgtests only test python imports. Since the
  regression testing described in [1] will be exercising the actual
  binary package functionality, I would like to see if we can get
  approval based on that being an adequate alternative.

  [1]
  The following SRU process was followed:
  https://wiki.ubuntu.com/OpenStackUpdates

  In order to avoid regression of existing consumers, the OpenStack team
  will run their continuous integration test against the packages that
  are in -proposed. A successful run of all available tests will be
  required before the proposed packages can be let into -updates.

  The OpenStack team will be in charge of attaching the output summary
  of the executed tests. The OpenStack team members will not mark
  ‘verification-done’ until this has happened.

  [Regression Potential]
  In order to mitigate the regression potential, the results of the
  aforementioned tests are attached to this bug.

  [Discussion]
  I would like to provide these new stable releases via microrelease SRU updates: https://wiki.ubuntu.com/StableReleaseUpdates#New_upstream_microreleases

  The recommended releases that upstream tests with are listed at:
  https://github.com/openstack/requirements/blob/stable/yoga/upper-constraints.txt
  https://github.com/openstack/requirements/blob/stable/xena/upper-constraints.txt
  https://github.com/openstack/requirements/blob/stable/wallaby/upper-constraints.txt

  [Original Bug Description]

  Subject: ovsdbapp: AttributeError: 'TableSchema' object has no
  attribute 'condition_state'

  After upgrade from victoria to wallaby neutron-api units firing
  errors. Exception is related to
  https://bugs.launchpad.net/ovsdbapp/+bug/1972989, which is fixed in
  ovsdbapp 1.9.3

  https://releases.openstack.org/wallaby/ states that ovsdbapp should be
  1.9.4 (latest for walalby)

  Installed in the unit is:

  root at juju-4c2163-3-lxd-14:~# python3 -m pip list | grep ovsdbapp
  ovsdbapp                1.9.0

  Also:
  root at juju-4c2163-3-lxd-14:~# apt-cache policy python3-ovsdbapp
  python3-ovsdbapp:
    Installed: 1.9.0-0ubuntu1~cloud0
    Candidate: 1.9.0-0ubuntu1~cloud0
    Version table:
   *** 1.9.0-0ubuntu1~cloud0 500
          500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-updates/wallaby/main amd64 Packages
          100 /var/lib/dpkg/status
       1.1.0-0ubuntu2 500
          500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
       1.1.0-0ubuntu1 500
          500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

  So it looks like the charm doesn't upgrade ovsdbapp during openstack
  upgrade, because a newer version with the above fix doesn't exist in
  the UCA:

   python3-ovsdbapp | 1.9.0-0ubuntu1~cloud0  | wallaby           | focal-updates   | all
   python3-ovsdbapp | 1.9.0-0ubuntu1~cloud0  | wallaby-proposed  | focal-proposed  | all

  Full traceback:

  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 81, in start
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 52, in wait_for_change
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     while idl_.change_seqno == seqno and not idl_.run():
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 541, in run
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__send_monitor_request()
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovs/db/idl.py", line 850, in __send_monitor_request
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     not ConditionState.is_true(table.condition_state.acked)):
  2023-02-21 07:39:19.639 431391 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn AttributeError: 'TableSchema' object has no attribute 'condition_state'

  Worked around by:
  juju run -a neutron-api 'sudo python3 -m pip install ovsdbapp==1.9.4'
  juju run -a neutron-api 'sudo service neutron-* restart'

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-api/+bug/2007919/+subscriptions