[Bug 2074309] Re: upgrade 22.04 -> 24.04 won't start due to feature flags

[Mitchell Dzurick]

Great points Hadmut.

It is unfortunate how far behind we are in versions. We follow debian
which hasn't been updated in a bit. I was thinking about updating the
package there, but I need to fix the fires here first (and if I update
debian to the latest without handling this bug's issue, I will break
people's server).

I am definitely interested in updating rabbitmq-server to 3.13, however
we do not often update major versions because they are prone to breaking
things (hence, this bug report). If anything, I would consider making a
new binary `rabbitmq-server-3.13` similar to how say, gcc-13 does it.
That way users can upgrade to the new version but it's more of an
informed decision rather than things just breaking upon an `apt update
&& apt upgrade`.

Even if we stay on top of it, users might not `apt update && apt
upgrade` for an extended period of time, and then we would have to deal
with that.

You are on point, I was also thinking about the potential of snaps long
term.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/2074309

Title:
  upgrade 22.04 -> 24.04 won't start due to feature flags

Status in Release Notes for Ubuntu:
  New
Status in rabbitmq-server package in Ubuntu:
  Confirmed
Status in ubuntu-release-upgrader package in Ubuntu:
  New
Status in rabbitmq-server source package in Noble:
  Confirmed
Status in ubuntu-release-upgrader source package in Noble:
  In Progress
Status in rabbitmq-server source package in Oracular:
  Confirmed
Status in ubuntu-release-upgrader source package in Oracular:
  New

Bug description:
  [Impact]

  Upgrading systems with rabbitmq-server installed will leave the server
  in a bad state because upgrading directly from the version in Jammy to
  the version in Noble is not supported by upstream rabbitmq-server.

  [Test Plan]

  For now, we add an upgrade quirk to prevent upgrades from Jammy to
  Noble if rabbitmq-server is installed, and provide an brief
  explanation to the user.

  To test:

  1. Create a Jammy container

  $ lxc launch ubuntu:jammy jammy

  2. Install rabbitmq-server

  $ apt install -y rabbitmq-server

  3. Attempt the upgrade

  $ do-release-upgrade -d

  The upgrade should abort with a message explaining why.

  [Impact]

  This quirk uses a common pattern in ubuntu-release-upgrader for
  similar purposes. If the package name was typo'd, it would not work
  correctly.

  [Other information]

  We may eventually need another SRU to revert this change if another
  solution is found.

  [Original Description]

  Hi,

  I was just doing an upgrade from ubuntu 22.04 server to 24.04 server.

  After upgrading, the rabbitmq-server cannot be started anymore.

  Reason:

  See
  https://www.rabbitmq.com/blog/2022/07/20/required-feature-flags-in-rabbitmq-3.11

  Higher versions of rabbitmq require feature flags to be set *before*
  upgrade. In my rabbitmq installation these features had not been set,
  therefore the the rabbitmq server would not start. They ask you to
  downgerade to an earlier version and do the upgrade.

  I therefore used LXD to run a Ubuntu 22.04 machine with rabbitmq 3.9
  to set the missing feature flags and achieved:

  rabbitmqctl  enable_feature_flag all

  rabbitmqctl list_feature_flags
  Listing feature flags ...
  name    state
  implicit_default_bindings       enabled
  maintenance_mode_status enabled
  quorum_queue    enabled
  stream_queue    enabled
  user_limits     enabled
  virtual_host_metadata   enabled

  But it still does not work, it still complains about  one feature
  missing:

  classic_mirrored_queue_version

  unfortunately, this is required by rabbitmq 3.12 coming with Ubuntu
  24.04, but is not known and thus cannot be set by rabbitmq 3.9 coming
  with Ubuntu 22.04.

  Now the problem is: There is no Ubuntu coming with 3.9 or 3.10. Ubuntu
  jumps directly from rabbitmq 3.9 to rabbitmq 3.12, although there is
  no upgrade path from 3.9 to 3.12.

  Three nasty options:

  - loose your data and configuration and run from scratch
  - run an old version in either docker or LXD
  - try some upgrade path with docker/podman and non-ubuntu versions of rabbitmq

  Unfortunately,

  https://hub.docker.com/_/rabbitmq

  does not list versions older than 3.12, but they still seem to be
  available, so this might by some migration/workaround path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-release-notes/+bug/2074309/+subscriptions