[Bug 1885264] Re: neutron-ovs-cleanup runs on package upgrade

[Alex Kavanagh]

Some additional information regarding how the bug was found:

During the upgrade the ovs runs, deleting the ports.  They don't come
back, even after restarting the services, which suggests an issue within
the neutron openvswitch agent in terms of re-creating ports that have
been deleted.

from the ovsdb-tool -mm show-log tool on the neutron-gateway AFTER the
upgrade.

```
record 44: 2020-06-25 19:10:48.024
  table Interface row "tap03282320-fb" (74bf6e22):
    delete row
  table Interface row "tap7e15b3c6-e3" (d331a9b6):
    delete row
  table Interface row "tap8e76995d-a6" (f8ca1e54):
    delete row
  table Port row "tap03282320-fb" (d891c2a5):
    delete row
  table Port row "tap7e15b3c6-e3" (8f105882):
    delete row
  table Port row "tap8e76995d-a6" (b5f389e0):
    delete row
  table Bridge row "br-int" (56555638):
    ports=[704ecf79-ff42-44bd-a72c-c586f59f433b, dcfd1c64-2e07-4bf7-8815-a96950384705, f38673c2-87c9-4ce1-b160-236637caf8d2]
  table Open_vSwitch row 70200fb1 (70200fb1):
    next_cfg=111
```

i.e. it was asked to be deleted.


from syslog (during upgrade):

```
Jun 25 19:10:44 juju-ed722c-mojo-14 systemd[1]: Stopping Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:44 juju-ed722c-mojo-14 ovsdb-client: ovs|00001|fatal_signal|WARN|terminating with signal 15 (Terminated)
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopped Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopped OpenStack Neutron OVS cleanup.
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron OVS cleanup...
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Starting OpenStack Neutron OVS cleanup...
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.589055] device tap03282320-fb left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.611240] device tap7e15b3c6-e3 left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.612612] device tap8e76995d-a6 left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Started OpenStack Neutron OVS cleanup.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Starting Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Started Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Reloading.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopping Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopped Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopped OpenStack Neutron OVS cleanup.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron OVS cleanup...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Starting OpenStack Neutron OVS cleanup...
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Started OpenStack Neutron OVS cleanup.
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Starting Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Started Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: Reloading.
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: message repeated 2 times: [ Reloading.]
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron Metering Agent...
```

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron in Ubuntu.
https://bugs.launchpad.net/bugs/1885264

Title:
  neutron-ovs-cleanup runs on package upgrade

Status in Ubuntu Cloud Archive:
  Triaged
Status in Ubuntu Cloud Archive ussuri series:
  Triaged
Status in Ubuntu Cloud Archive victoria series:
  Triaged
Status in neutron package in Ubuntu:
  Triaged
Status in neutron source package in Focal:
  Triaged
Status in neutron source package in Groovy:
  Triaged

Bug description:
  Release: Focal/Bionic+Ussuri:

  As part of the package upgrade for neutron-openvswitch-agent, the
  neutron-ovs-cleanup systemd unit is restarted.

  This unit should only be run on reboot, and not during package
  upgrades.

  It has the side effect of removing all ports from the local OVS
  bridge, impacting networking to services.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1885264/+subscriptions