[Bug 2107657] Re: upgrades to plucky incorrectly remove foreign packages from disabled sources

Thu Apr 24 17:14:58 UTC 2025

Hello Nick, or anyone else affected,

Accepted ubuntu-release-upgrader into plucky-proposed. The package will
build now and be available at
https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:25.04.15
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
plucky to verification-done-plucky. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-plucky. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: ubuntu-release-upgrader (Ubuntu Plucky)
       Status: In Progress => Fix Committed

** Tags added: verification-needed verification-needed-plucky

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/2107657

Title:
  upgrades to plucky incorrectly remove foreign packages from disabled
  sources

Status in ubuntu-release-upgrader package in Ubuntu:
  Triaged
Status in ubuntu-release-upgrader source package in Plucky:
  Fix Committed

Bug description:
  [Impact]

  Users may have third-party packages unnecessarily removed during their
  upgrade to plucky, because ubuntu-release-upgrader does not exclude
  such packages from the removal candidates, as it did previously.

  [Test Plan]

  Basically, install a package from a third-party PPA, and do the
  upgrade. Confirm that the package is considered foreign at the
  beginning of the upgrade, but that it is not later removed.

  1. Configure a PPA and install a package from it. Using a popular
  example that motivated this bug report:

  $ cat > /etc/apt/sources.list.d/code.sources << EOF
  Types: deb
  URIs: https://packages.microsoft.com/repos/code
  Suites: stable
  Components: main
  Signed-By: .
   -----BEGIN PGP PUBLIC KEY BLOCK-----
   Version: GnuPG v1.4.7 (GNU/Linux)
   .
   mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT
   LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV
   7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag
   OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j
   H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr
   M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs
   ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC
   AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH
   /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe
   MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy
   7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV
   KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ
   XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+
   NdCFTW7wY0Fb1fWJ+/KTsC4=
   =J6gs
   -----END PGP PUBLIC KEY BLOCK-----
  EOF
  $ apt update
  $ apt install code -y

  2. Run the upgrade. Note that due to the fact that plucky is now
  released, but upgrades are disabled, we cannot use the --proposed flag
  to test upgrades conveniently. Instead, we need to manually download
  the tarball from https://changelogs.ubuntu.com/meta-release-proposed,
  unpack it, and run the script.

  $ wget http://archive.ubuntu.com/ubuntu/dists/plucky-proposed/main/dist-upgrader-all/current/plucky.tar.gz
  $ tar xf plucky.tar.gz
  $ sudo -E ./plucky --frontend DistUpgradeViewText

  3. When the upgrade gets to the 'Remove obsolete packages?' stage,
  enter 'd' for details, and verify that 'code' is not suggested for
  removal.

  4. Check the upgrade log to ensure that 'code' was considered foreign
  at the beginning of the upgrade:

  $ grep "Foreign.*rewrite" /var/log/dist/upgrade/main.log

  [Where problems could occur]

  This patch fixes the list of foreign packages used to exclude from
  removal during the obsolete removal step. At this point, the package
  removals are only for clean up, and should not have any impact on the
  upgrade. Any theoretical issue would be seen in the package removal
  step.

  [Original Description]

  This is a bug introduced by commit ba84c16586 ("DistUpgrade: warn
  about foreign packages after rewriting sources"). This commit is
  intended to be non-functional with respect to package removals etc.,
  but inadvertently changed the list of packages which should be
  excluded from consideration for removal.

  The packages which are considered "foreign" will be different before
  and after rewriting the new sources. Later, packages which were deemed
  "foreign" at the beginning of the upgrade will be excluded from
  removal.

  The above patch tracks foreign packages before and after rewriting
  sources for the purposes of supplying the warning, but later uses the
  post-rewrite list when considering exclusions, whereas it should use
  the pre-rewrite list.

  This could be reproduced with any PPA, but to illustrate with a
  popular example (public key from
  https://packages.microsoft.com/keys/microsoft.asc):

  $ cat > /etc/apt/sources.list.d/code.sources << EOF
  Types: deb
  URIs: https://packages.microsoft.com/repos/code
  Suites: stable
  Components: main
  Signed-By: .
   -----BEGIN PGP PUBLIC KEY BLOCK-----
   Version: GnuPG v1.4.7 (GNU/Linux)
   .
   mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT
   LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV
   7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag
   OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j
   H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr
   M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs
   ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC
   AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH
   /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe
   MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy
   7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV
   KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ
   XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+
   NdCFTW7wY0Fb1fWJ+/KTsC4=
   =J6gs
   -----END PGP PUBLIC KEY BLOCK-----
  EOF
  $ apt update
  $ apt install code -y
  $ do-release-upgrade --devel

  [...]

  Eventually, looking at the logs, we can see the different in the pre-
  vs post-rewrite foreign packages:

  $ grep "Foreign.*rewriting" /var/log/dist-upgrade/main.log
  2025-04-18 15:38:36,057 DEBUG Foreign (before rewriting sources): code
  2025-04-18 15:38:36,057 DEBUG Foreign (after rewriting sources):

  Then, later, code will be shown as a remove candidate when the user is
  prompted about removals:

  [...]

  Remove obsolete packages?

  28 packages are going to be removed.

   Continue [yN]  Details [d]d

  Remove: code <------- Should not be here

  Remove (was auto installed) humanity-icon-theme libapt-pkg6.0t64
    libassuan0 libdrm-nouveau2 libdrm-radeon1 libdw1t64 libfwupd2
    libgl1-amber-dri libglapi-amber libgusb2 libicu74
    libmodule-scandeps-perl libnl-genl-3-200 libnsl2 libperl5.38t64
    libpython3.12-minimal libpython3.12-stdlib libpython3.12t64
    libsgutils2-1.46-2 libxcb-dri2-0 perl-modules-5.38 python3-colorama
    python3.12 python3.12-gdbm python3.12-minimal sosreport ubuntu-mono

   Continue [yN]  Details [d]

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/2107657/+subscriptions