[Bug 2119024] Re: [SRU] Squid: Ceph new point release 19.2.3

Wed Nov 26 20:21:12 UTC 2025

Our infrastructure wrt Github runners is having issues with
connectivity, so I'm running the verification myself. I've run the
entire functional test suite locally and all the tests passed. I'm
pasting the results of one such run and attaching the full logs as
proof, but I can do the same for all the tests run:

2025-11-26 16:16:23 [INFO] Events:
  Deploy Bundle:
    Start: 1764183921.2315564
    Finish: 1764183934.9559
    Elapsed Time: 13.724343538284302
    PCT Of Run Time: 3
  Prepare Environment:
    Start: 1764183920.6793232
    Finish: 1764183921.2313614
    Elapsed Time: 0.5520381927490234
    PCT Of Run Time: 1
  Test tests.target.CephAuthTest:
    Start: 1764184540.159873
    Finish: 1764184548.451297
    Elapsed Time: 8.291424036026001
    PCT Of Run Time: 2
  Test tests.target.CephLowLevelTest:
    Start: 1764184456.572862
    Finish: 1764184478.533339
    Elapsed Time: 21.960477113723755
    PCT Of Run Time: 4
  Test tests.target.CephMonActionsTest:
    Start: 1764184548.4513416
    Finish: 1764184566.2094848
    Elapsed Time: 17.758143186569214
    PCT Of Run Time: 3
  Test tests.target.CephMonJujuPersistent:
    Start: 1764184573.6686642
    Finish: 1764184583.0324569
    Elapsed Time: 9.363792657852173
    PCT Of Run Time: 2
  Test tests.target.CephMonKeyRotationTests:
    Start: 1764184566.2095222
    Finish: 1764184573.6686409
    Elapsed Time: 7.459118604660034
    PCT Of Run Time: 2
  Test tests.target.CephPrometheusTest:
    Start: 1764184540.158875
    Finish: 1764184540.1598554
    Elapsed Time: 0.000980377197265625
    PCT Of Run Time: 1
  Test tests.target.CephTest:
    Start: 1764184478.5333674
    Finish: 1764184540.158836
    Elapsed Time: 61.625468492507935
    PCT Of Run Time: 10
  Test zaza.charm_tests.lifecycle.refresh.CharmRefreshAll:
    Start: 1764184456.5684538
    Finish: 1764184456.5728464
    Elapsed Time: 0.0043926239013671875
    PCT Of Run Time: 1
  Wait for Deployment:
    Start: 1764183934.9559112
    Finish: 1764184456.5666878
    Elapsed Time: 521.6107766628265
    PCT Of Run Time: 79
Metadata: {}

The Ceph version matches the expectations:

> lmlg at lmlg-pc:~/src/ceph-charms/ceph-mon$ juju ssh ceph-mon/0 -- sudo ceph --version
ceph version 19.2.3 (c92aebb279828e9c3c1f5d24613efca272649e62) squid (stable)

I've additionally repeated the process for testing LTTNG and got similar
(i.e: positive) results.

** Attachment added: "caracal-results.txt"
   https://bugs.launchpad.net/cloud-archive/+bug/2119024/+attachment/5929946/+files/caracal-results.txt

** Tags removed: verification-caracal-needed
** Tags added: verification-caracal-done

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2119024

Title:
  [SRU] Squid: Ceph new point release 19.2.3

Status in Ubuntu Cloud Archive:
  New
Status in Ubuntu Cloud Archive caracal series:
  Fix Committed
Status in ceph package in Ubuntu:
  Fix Released
Status in ceph source package in Noble:
  Fix Committed
Status in ceph source package in Plucky:
  Fix Committed
Status in ceph source package in Questing:
  Fix Committed
Status in ceph source package in Resolute:
  Fix Released

Bug description:
  [Impact]
  This release fixes several bugs. We would like to make sure all of our users have access to these improvements.

  The update contains the following package updates:
     * ceph 19.2.2 https://ceph.io/en/news/blog/2025/v19-2-2-squid-released/
     * ceph 19.2.3 https://ceph.io/en/news/blog/2025/v19-2-3-squid-released/

  Notable changes:
      * This fixes a silent RGW data loss regression when CopyObject is used to copy an object onto itself. The regression results in objects past the original 4mb head object being added to the garbage collection queue and deleted permanently, and are not recoverable.
      * LTTNG has been enabled for easier tracing and debugging.

  Key differences from upstream:
      * libnbd has been disabled from the build system. libnbd is in universe in Ubuntu, and cannot be a dependency for 19.2.3.
      * On questing, adjustments had to be made to build with boost1.88. On plucky and noble, these adjustments have been removed to build with standard boost 1.83 and 1.82.
      * python-cryptography uses PyO3 to call some rust based bcrypt and TLS functions, and has problems being called in parallel and in sub-interpreters, necessary for ceph-dashboard to work correctly. This was fixed by backporting:
    commit 717d0a6f3530ad3e07f4423002810327b2addcf1
    From: "Paulo E. Castro" <pecastro at wormholenet.com>
    Date: Sat, 5 Apr 2025 21:47:55 +0100
    Subject: pybind/mgr: Hack around the 'ImportError: PyO3 modules
     may only be initialized once per interpreter process' issue.
    Link: https://github.com/ceph/ceph/commit/7094a5a44d90e705141dbae9739e6c0835bf7ce3
    See also:
     https://tracker.ceph.com/issues/64213
     https://github.com/bazaah/aur-ceph/issues/20
     https://bugs.launchpad.net/charm-ceph-dashboard/+bug/2073115
      * Noble also receives the bugfix for hardened CFLAGS for python modules. See https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2078244

  [Test Case]

  This SRU will run all functional tests with the packages in 'proposed'
  for every Ceph charm maintained by Canonical's Ceph team, so that
  there's a higher chance of catching bugs associated with any Ceph
  component.

  The charms for which the functional tests will be run are: ceph-mon, ceph-osd,
  ceph-radosgw, ceph-proxy, ceph-nfs, ceph-nvme, ceph-fs and ceph-rbd-mirror.

  The functional test suite includes upgrade testing from the current stable
  release to the proposed (in this case, from 19.2.1 to 19.2.3)

  Since our functional tests do not include testing for LTTNG, this will be
  tested manually. This involves the following steps:

  - Deploy a Ceph cluster with 19.2.3.
  - Enable OSD tracing by adding the option `osd_tracing` to /etc/ceph.conf
  - Run `lttng create test`
  - Run `lttng enable-event --userspace osd:*`
  - At this point, tracing is enabled. Do some writes to an OSD, e.g: By creating an RBD object or via ceph bench.
  - Stop tracing with `lttng stop`
  - View the logs with `lttng view`.
  - We can then do some more writes on an OSD and verify that no more tracing output is generated.

  [Regression Potential]

  ceph is a critical package as it underpins large scale data storage
  and integrity checking, and care must be taken to not introduce any
  data loss regressions. ceph also is a common backing store for
  OpenStack virtual machines through ceph-rbd, and care must be taken to
  not break any API or ABI compatibility with current OpenStack
  releases.

  In order to mitigate the regression potential, the results of the
  aforementioned tests will be attached to this bug.

  [Other info]

  Microrelease exception:
  https://documentation.ubuntu.com/project/SRU/reference/exception-
  OpenStack-Updates/

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2119024/+subscriptions