[Bug 2077348] Re: [SRU] Loadbalacer stuck in status PENDING_DELETE if TLS storage unavailable in cascade deletion

Tue Mar 18 07:19:23 UTC 2025

Joshua,

Some notes about your debdiffs:

- each patch has a trailing whitespace at the start of empty lines, and I had
to remove them all. Your text editor is configured wrong, and you need to adjust
it. The git diff should have no red spaces anywhere.
- I usually like naming the patch lp<number>-<git subject>, e.g. lp2077348-fix-loadbalancer-stuck-in-cascade-delete.patch. For multiple patches I usually like
lp2077348-1-fix-blah.patch, lp2077348-2-fix-blah.patch etc.
- The debian changelog needs to have an actual description in it. Simply having
the patch name is not enough. I usually like the format of:

* Description, a few sentences. (LP: #BUG)
  - d/p/patch1
  - d/p/patch2.

For example, the final one I used was:

  * Fix loadbalancer stuck in PENDING_DELETE in cascade delete
    with TERMINATED_HTTPS listener if TLS storage not available.
    (LP: #2077348)
    - d/p/lp2077348-fix-loadbalancer-stuck-in-cascade-delete.patch

- You can use raw git diffs in the patch, but I usually like making them seem
like they are properly dep3 tagged. An example of what I did is here:

https://git.launchpad.net/~mruffell/ubuntu/+source/octavia/tree/debian/patches/lp2077348-fix-
loadbalancer-stuck-in-cascade-delete.patch?h=stable/2024.1

See how I moved the dep3 tags to the bottom, used the git description as the
Subject Line, spaced 1 whitespace in, and made it a little neater.

Your SRU template is okay, but you really need to consider adding to the
"Where problems could occur" section. You need to consider the impact to users
when / if a regression were to occur, and what workarounds a user might use.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2077348

Title:
  [SRU] Loadbalacer stuck in status PENDING_DELETE if TLS storage
  unavailable in cascade deletion

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive antelope series:
  In Progress
Status in Ubuntu Cloud Archive bobcat series:
  In Progress
Status in Ubuntu Cloud Archive caracal series:
  New
Status in Ubuntu Cloud Archive dalmatian series:
  Fix Released
Status in Ubuntu Cloud Archive epoxy series:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  New
Status in octavia:
  Fix Released
Status in octavia package in Ubuntu:
  Fix Released
Status in octavia source package in Focal:
  Won't Fix
Status in octavia source package in Jammy:
  In Progress
Status in octavia source package in Noble:
  In Progress
Status in octavia source package in Oracular:
  Fix Released
Status in octavia source package in Plucky:
  Fix Released

Bug description:
  [Impact]

  Loadbalacer stuck in status PENDING_DELETE if TLS cert unavailable

  [Test Case]

  Pls refer to [Test steps] section below.

  [Regression Potential]

  The fix is already in the upstream main, stable/2024.1, stable/2023.2,
  stable/2023.1 branches, so it is a clean backport and might be helpful
  for deployments using octavia.

  I also test this fix, it works well -
  https://paste.ubuntu.com/p/s4MsMjV6mP/

  [Others]

  Original Bug Description Below
  ===========

  Loadbalacer stuck in status PENDING_DELETE if TLS cert unavailable

  1. Create load balancer with TERMINATED_HTTPS listener
  2. Disable your TLS storage, or delete cert from storage
  3. Try to delete loadbalancer with cascade flag

  Error on logs:

  ```
  Unable to retrieve certificate(s) due to Could not retrieve certificate: <some id>
  Exception during message handling
  ```

  ```
  Traceback (most recent call last):
  File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming, res = self.dispatcher.dispatch(message),
  File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch, return self._do_dispatch(endpoint, method, ctxt, args),
  File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch, result = func(ctxt, **new_args),
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/queue/v2/endpoints.py", line 56, in delete_load_balancer, self.worker.delete_load_balancer(loadbalancer, cascade),
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/controller_worker.py", line 387, in delete_load_balancer, listeners = flow_utils.get_listeners_on_lb(db_lb),
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/flows/flow_utils.py", line 52, in get_listeners_on_lb, prov_listener = provider_utils.db_listener_to_provider_listener(),
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 182, in db_listener_to_provider_listener, new_listener_dict = listener_dict_to_provider_dict(),
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 261, in listener_dict_to_provider_dict, with excutils.save_and_reraise_exception() as ctxt:,
  File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__, self.force_reraise(),
  File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise, raise self.value,
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 258, in listener_dict_to_provider_dict, cert_dict = cert_parser.load_certificates_data(cert_manager)
  File "/var/lib/openstack/lib/python3.10/site-packages/octavia/common/tls_utils/cert_parser.py", line 381, in load_certificates_data, raise exceptions.CertificateRetrievalException(, octavia.common.exceptions.CertificateRetrievalException: Could not retrieve certificate: ]

  ```

  [Test steps]

  1. Create load balancer with TERMINATED_HTTPS listener, eg:

  secret1_id=$(openstack secret store --name='lb_tls_secret_1' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < www.server1.com.p12)" -f value -c "Secret href")
  octavia_user_id=$(openstack user show octavia --domain service_domain -f value -c id); echo $octavia_user_id;
  openstack acl user add -u $octavia_user_id $secret1_id
  subnetid=$(openstack subnet show private_subnet -f value -c id); echo $subnetid
  lb_id=$(openstack loadbalancer create --name lb1 --vip-subnet-id $subnetid -f value -c id); echo $lb_id
  listener_id=$(openstack loadbalancer listener create $lb_id --name https_listener --protocol-port 80 --protocol TERMINATED_HTTPS --default-tls-container=$secret1_id --sni-container-refs $secret1_id $secret2_id -f value -c id); echo $listener_id

  2. Disable your TLS storage, or delete cert from storage, eg:

  openstack secret delete $secret1_id

  3. Try to delete loadbalancer with cascade flag

  openstack loadbalancer delete lb1 --cascade

  Here are the detailed steps for me to reproduce the problem -
  https://paste.ubuntu.com/p/wh3dJpJR9B/

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2077348/+subscriptions