[Bug 1775660] Re: ubuntu-release-upgrader should purge obsolete packages, not just remove them

[Brian Murray]

I performed an upgrade of Ubuntu 17.10 to Ubuntu 18.04 and ran "dpkg -l
| grep '^rc'" after purging packages and kind of captured the difference
in that with -proposed no packages were left in an rc state.

 $ cat artful-nopurge.txt 
rc  gnome-orca                                 3.28.0-3ubuntu1                             all          Scriptable screen reader - transitional package
rc  libunistring0:amd64                        0.9.3-5.2ubuntu1                            amd64        Unicode string library for C
rc  linux-image-4.13.0-16-generic              4.13.0-16.19                                amd64        Linux kernel image for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-4.13.0-21-generic              4.13.0-21.24                                amd64        Linux kernel image for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-4.13.0-41-generic              4.13.0-41.46                                amd64        Linux kernel image for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-4.13.0-43-generic              4.13.0-43.48                                amd64        Linux kernel image for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-extra-4.13.0-16-generic        4.13.0-16.19                                amd64        Linux kernel extra modules for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-extra-4.13.0-21-generic        4.13.0-21.24                                amd64        Linux kernel extra modules for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-extra-4.13.0-41-generic        4.13.0-41.46                                amd64        Linux kernel extra modules for version 4.13.0 on 64 bit x86 SMP
rc  linux-image-extra-4.13.0-43-generic        4.13.0-43.48                                amd64        Linux kernel extra modules for version 4.13.0 on 64 bit x86 SMP
rc  python                                     2.7.15~rc1-1                                amd64        interactive high-level object-oriented language (default version)
rc  python2.7-minimal                          2.7.15~rc1-1                                amd64        Minimal subset of the Python language (version 2.7)
rc  snapd-login-service                        1.23-0ubuntu1                               amd64        Daemon to allow non-root access to snapd

bdmurray at clean-artful-amd64:~$ grep "upgrader version" /var/log/dist-upgrade/main.log 
2018-07-11 14:09:34,978 INFO release-upgrader version '18.04.20' started
bdmurray at clean-artful-amd64:~$ dpkg -l | grep "^rc"
bdmurray at clean-artful-amd64:~$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04


** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1775660

Title:
  ubuntu-release-upgrader should purge obsolete packages, not just
  remove them

Status in ubuntu-release-upgrader package in Ubuntu:
  Fix Released
Status in ubuntu-release-upgrader source package in Bionic:
  Fix Committed

Bug description:
  [Justification]
  When an obsolete package leaves conffiles behind, these conffiles can sometimes interact poorly with the system and cause bugs.  Since the obsoleted package no longer exists in the targeted release, this can be difficult to resolve by applying fixes to a package.  Furthermore, removing-but-not-purging packages on upgrade means that a system which is upgraded will as a rule be inconsistent with a newly-installed system, which should always be considered a bug.

  Wherever possible, we should strive to make an upgraded system look
  exactly like a newly installed system, to minimize the combinatorics
  of configurations that must be supported.

  Examples of cases where this has been a problem in practice:

  LP: #1770965
  LP: #1749199

  [Test case]
  1. Do an upgrade from xenial with the release ubuntu-release-upgrader.  Accept the option to remove obsolete packages.
  2. Capture the list of packages that are in state 'rc' afterwards.
  3. Do an equivalent upgrade with the -proposed ubuntu-release-upgrader, including removal of obsolete packages.
  4. Confirm that the upgrade completes successfully.
  5. Capture the list of packages that are in state 'rc' afterwards.
  6. Confirm that there are fewer packages in state 'rc' in the second upgrade than the first.
  7. Repeat all of the above with an upgrade from artful.

  [Regression potential]
  As a consequence of this, we will also be running a lot more postrm scripts with the 'purge' option on users desktops, so any bugs in those maintainer scripts will now be apparent as part of a release upgrade where before they were latent.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1775660/+subscriptions