[Bug 2140579] Re: Sunbeam cannot get openrc because keystone is timing out
David Wilde
2140579 at bugs.launchpad.net
Wed Feb 11 15:34:31 UTC 2026
** Also affects: ubuntu
Importance: Undecided
Status: New
** Package changed: ubuntu => keystone (Ubuntu)
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to keystone in Ubuntu.
https://bugs.launchpad.net/bugs/2140579
Title:
Sunbeam cannot get openrc because keystone is timing out
Status in OpenStack Identity (keystone):
New
Status in keystone package in Ubuntu:
New
Bug description:
# Reproducibility
- `openstack` snap revision 783 (2024.1/stable)
- Installed through MAAS with the following configuration:
- 3 infra nodes
- 3 compute nodes
- Environment has been working for about 2-3 months
- About a dozen VMs have been created and migration tests have been run
Have not found any specific trigger yet to reproduce this, except
time.
# Expected behavior
- Running `sunbeam openrc` should return an openrc file for the admin account.
- Running `juju status keystone -m openstack` units should all be active.
- Running `juju run keystone/leader get-admin-account -m openstack` should return the admin account information.
# Observed behavior
- Running `sunbeam openrc` times out.
- Running `juju status keystone -m openstack` returns:
```
Model Controller Cloud/Region Version SLA Timestamp
openstack foundations-maas cgi-internaldev-k8s/localhost 3.6.12 unsupported 16:08:30Z
SAAS Status Store URL
cinder-volume active local sunbeam/openstack-machines.cinder-volume
grafana-dashboards active foundations-maas sunbeam/cos.grafana-dashboards
loki-logging maintenance foundations-maas sunbeam/cos.loki-logging
microceph active local sunbeam/openstack-machines.microceph
prometheus-receive-remote-write maintenance foundations-maas sunbeam/cos.prometheus-receive-remote-write
App Version Status Scale Charm Channel Rev Address Exposed Message
keystone active 3 keystone-k8s 2024.1/stable 243 10.152.183.211 no
Unit Workload Agent Address Ports Message
keystone/0 active idle 10.1.2.174
keystone/1 active idle 10.1.1.152
keystone/2* active idle 10.1.0.28
Offer Application Charm Rev Connected Endpoint Interface Role
ceilometer ceilometer ceilometer-k8s 86 1/1 ceilometer-service ceilometer provider
cert-distributor keystone keystone-k8s 243 2/2 send-ca-cert certificate_transfer provider
certificate-authority certificate-authority self-signed-certificates 317 1/1 certificates tls-certificates provider
cinder-volume-mysql-router cinder-volume-mysql-router mysql-router-k8s 748 1/1 database mysql_client provider
keystone-credentials keystone keystone-k8s 243 2/2 identity-credentials keystone-credentials provider
keystone-endpoints keystone keystone-k8s 243 1/1 identity-service keystone provider
nova nova nova-k8s 134 1/1 nova-service nova provider
ovn-relay ovn-relay ovn-relay-k8s 121 1/1 ovsdb-cms-relay ovsdb-cms provider
rabbitmq rabbitmq rabbitmq-k8s 54 2/2 amqp rabbitmq provider
traefik-rgw traefik-rgw traefik-k8s 254 1/1 traefik-route traefik_route provider
```
- Running `juju run keystone/leader get-admin-account -m openstack`
times out.
Looking at `juju debug-log -i keystone -m openstack`, I get:
```
unit-keystone-2: 15:55:20 ERROR unit.keystone/2.juju-log Uncaught exception while in charm code:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-keystone-2/charm/./src/charm.py", line 1798, in <module>
ops.main(KeystoneOperatorCharm)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/__init__.py", line 356, in __call__
return _main.main(charm_class=charm_class, use_juju_for_storage=use_juju_for_storage)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/_main.py", line 474, in main
manager.run()
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/_main.py", line 458, in run
self._emit()
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/_main.py", line 402, in _emit
self._emit_charm_event(self.dispatcher.event_name)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/_main.py", line 437, in _emit_charm_event
event_to_emit.emit(*args, **kwargs)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/framework.py", line 351, in emit
framework._emit(event)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/framework.py", line 923, in _emit
self._reemit(event_path)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/framework.py", line 1029, in _reemit
custom_handler(event)
File "/var/lib/juju/agents/unit-keystone-2/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 677, in wrapped_function
return callable(*args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-keystone-2/charm/lib/charms/loki_k8s/v1/loki_push_api.py", line 2556, in _on_pebble_ready
self._update_endpoints(event.workload, loki_endpoints)
File "/var/lib/juju/agents/unit-keystone-2/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 677, in wrapped_function
return callable(*args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-keystone-2/charm/lib/charms/loki_k8s/v1/loki_push_api.py", line 2582, in _update_endpoints
_PebbleLogClient.enable_endpoints(
File "/var/lib/juju/agents/unit-keystone-2/charm/lib/charms/loki_k8s/v1/loki_push_api.py", line 2513, in enable_endpoints
container.add_layer(f"{container.name}-log-forwarding", layer, combine=True)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/model.py", line 2490, in add_layer
self._pebble.add_layer(label, layer, combine=combine)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/pebble.py", line 2543, in add_layer
self._request('POST', '/v1/layers', body=body)
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/pebble.py", line 2155, in _request
response = self._request_raw(method, path, query, headers, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/pebble.py", line 2194, in _request_raw
response = self.opener.open(request, timeout=self.timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 515, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 532, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/var/lib/juju/agents/unit-keystone-2/charm/venv/ops/pebble.py", line 402, in http_open
return self.do_open(
^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open
r = h.getresponse()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse
response.begin()
File "/usr/lib/python3.12/http/client.py", line 331, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 292, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 707, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out
unit-keystone-2: 15:55:21 ERROR juju.worker.uniter.operation hook "keystone-pebble-ready" (via hook dispatching script: dispatch) failed: exit status 1
```
To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/2140579/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list