[Bug 2074309] Re: upgrade 22.04 -> 24.04 won't start due to feature flags

[Hadmut Danisch]

You're welcome. :-)

As I commented in #1, the upgrade using podman was working well for me,
but the question is, how long these docker images of old versions will
be available.

When creating an upgrade strategy, you also should take into
consideration, how long this upgrade should work, i.e. whether the
upgrade path should still work in 2027 when 22.04's support will end.

Another problem is that 24.04 is still stuck to rabbitmq 3.12, which is
out of support and does not receive updates anymore, thus could be a
security problem and require an emergency update 3.12 -> 3.13 if a
security problem is found.

Maybe it would be easier and less complex, to first update 24.04  from
rabbitmq 3.12 to 3.13, which seems to be the last 3.x version (they are
beta-testing 4.x), as a reasonable version for 24.04 LTS, and check
whether there's a similar upgrade problem 3.12->3.13, and incorporate
this into your solution.

And, keep in mind, you probably will have similar problems with future
versions. The same problem will probably reoccur.


Maybe, one clean and simple solution could be to decouple rabbitmq-packages from Ubuntu releases and simply turn them into snaps with a channel for each version, i.e. do no upgrade at all and just replace ubuntu package 3.9 with snap channel 3.9 during update to keep things running and benefit from snap's revert mechanism in case something doesn't start.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/2074309

Title:
  upgrade 22.04 -> 24.04 won't start due to feature flags

Status in Release Notes for Ubuntu:
  New
Status in rabbitmq-server package in Ubuntu:
  Confirmed
Status in ubuntu-release-upgrader package in Ubuntu:
  New
Status in rabbitmq-server source package in Noble:
  Confirmed
Status in ubuntu-release-upgrader source package in Noble:
  In Progress
Status in rabbitmq-server source package in Oracular:
  Confirmed
Status in ubuntu-release-upgrader source package in Oracular:
  New

Bug description:
  [Impact]

  Upgrading systems with rabbitmq-server installed will leave the server
  in a bad state because upgrading directly from the version in Jammy to
  the version in Noble is not supported by upstream rabbitmq-server.

  [Test Plan]

  For now, we add an upgrade quirk to prevent upgrades from Jammy to
  Noble if rabbitmq-server is installed, and provide an brief
  explanation to the user.

  To test:

  1. Create a Jammy container

  $ lxc launch ubuntu:jammy jammy

  2. Install rabbitmq-server

  $ apt install -y rabbitmq-server

  3. Attempt the upgrade

  $ do-release-upgrade -d

  The upgrade should abort with a message explaining why.

  [Impact]

  This quirk uses a common pattern in ubuntu-release-upgrader for
  similar purposes. If the package name was typo'd, it would not work
  correctly.

  [Other information]

  We may eventually need another SRU to revert this change if another
  solution is found.

  [Original Description]

  Hi,

  I was just doing an upgrade from ubuntu 22.04 server to 24.04 server.

  After upgrading, the rabbitmq-server cannot be started anymore.

  Reason:

  See
  https://www.rabbitmq.com/blog/2022/07/20/required-feature-flags-in-rabbitmq-3.11

  Higher versions of rabbitmq require feature flags to be set *before*
  upgrade. In my rabbitmq installation these features had not been set,
  therefore the the rabbitmq server would not start. They ask you to
  downgerade to an earlier version and do the upgrade.

  I therefore used LXD to run a Ubuntu 22.04 machine with rabbitmq 3.9
  to set the missing feature flags and achieved:

  rabbitmqctl  enable_feature_flag all

  rabbitmqctl list_feature_flags
  Listing feature flags ...
  name    state
  implicit_default_bindings       enabled
  maintenance_mode_status enabled
  quorum_queue    enabled
  stream_queue    enabled
  user_limits     enabled
  virtual_host_metadata   enabled

  But it still does not work, it still complains about  one feature
  missing:

  classic_mirrored_queue_version

  unfortunately, this is required by rabbitmq 3.12 coming with Ubuntu
  24.04, but is not known and thus cannot be set by rabbitmq 3.9 coming
  with Ubuntu 22.04.

  Now the problem is: There is no Ubuntu coming with 3.9 or 3.10. Ubuntu
  jumps directly from rabbitmq 3.9 to rabbitmq 3.12, although there is
  no upgrade path from 3.9 to 3.12.

  Three nasty options:

  - loose your data and configuration and run from scratch
  - run an old version in either docker or LXD
  - try some upgrade path with docker/podman and non-ubuntu versions of rabbitmq

  Unfortunately,

  https://hub.docker.com/_/rabbitmq

  does not list versions older than 3.12, but they still seem to be
  available, so this might by some migration/workaround path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-release-notes/+bug/2074309/+subscriptions