[Bug 1877769] Re: [SRU] Rewinding cache triggers obsolete adjustments consuming a lot of CPU

[Brian Murray]

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into focal-proposed. The package will build
now and be available at https://launchpad.net/ubuntu/+source/unattended-
upgrades/2.3ubuntu0.1 in a few hours, and then in the -proposed
repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
focal to verification-done-focal. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-focal. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: unattended-upgrades (Ubuntu Focal)
       Status: New => Fix Committed

** Tags added: verification-needed verification-needed-focal

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1877769

Title:
  [SRU] Rewinding cache triggers obsolete adjustments consuming a lot of
  CPU

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Focal:
  Fix Committed
Status in unattended-upgrades package in Debian:
  Fix Released

Bug description:
  [Impact]

   * Unattended-upgrades consumes excessive amount of CPU when rewinding
  cache trying to mark packages to upgrade or install even when they are
  already marked to be upgraded or installed. This can be triggered by
  packages held back.

  [Test Case]

   * Run test_rewind.py:
     -Fixed output:
  $ python3 test_rewind.py 
  INFO:root:Allowed origins are: o=Ubuntu,a=lucid-security
  INFO:root:Initial blacklist: ant-doc
  INFO:root:Initial whitelist (not strict): 
  DEBUG:root:Marking not allowed <apt_pkg.PackageFile object: filename:'/home/rbalint/projects/deb/unattended-upgrades/test/root.rewind/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_lucid-updates_main_binary-amd64_Packages'  a=lucid-updates,c=main,v=10.04,o=Ubuntu,l=Ubuntu arch='amd64' site='archive.ubuntu.com' IndexType='Debian Package Index' Size=894 ID:0> with -32768 pin
  DEBUG:root:Applying pinning: PkgFilePin(id=0, priority=-32768)
  DEBUG:root:Applying pin -32768 to package_file: <apt_pkg.PackageFile object: filename:'/home/rbalint/projects/deb/unattended-upgrades/test/root.rewind/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_lucid-updates_main_binary-amd64_Packages'  a=lucid-updates,c=main,v=10.04,o=Ubuntu,l=Ubuntu arch='amd64' site='archive.ubuntu.com' IndexType='Debian Package Index' Size=894 ID:0>
  DEBUG:root:Applying pinning: PkgPin(pkg='/^ant-doc/', priority=-32768)
  DEBUG:root:APT::VersionedKernelPackages is not set
  DEBUG:root:adjusting candidate version: test-package=2.0
  DEBUG:root:adjusting candidate version: test2-package=2.0
  DEBUG:root:falling back to adjusting test2-package's dependencies
  DEBUG:root:falling back to adjusting z-package's dependencies
  DEBUG:root:sanity check failed for: set() : no package is selected to be upgraded or installed
  .
  ----------------------------------------------------------------------
  Ran 1 test in 0.022s

  OK

    - Not fixed output:
   $ python3 test_rewind.py 
  INFO:root:Allowed origins are: o=Ubuntu,a=lucid-security
  INFO:root:Initial blacklist: ant-doc
  INFO:root:Initial whitelist (not strict): 
  DEBUG:root:Marking not allowed <apt_pkg.PackageFile object: filename:'/home/rbalint/projects/deb/unattended-upgrades/test/root.rewind/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_lucid-updates_main_binary-amd64_Packages'  a=lucid-updates,c=main,v=10.04,o=Ubuntu,l=Ubuntu arch='amd64' site='archive.ubuntu.com' IndexType='Debian Package Index' Size=894 ID:0> with -32768 pin
  DEBUG:root:Applying pinning: PkgFilePin(id=0, priority=-32768)
  DEBUG:root:Applying pin -32768 to package_file: <apt_pkg.PackageFile object: filename:'/home/rbalint/projects/deb/unattended-upgrades/test/root.rewind/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_lucid-updates_main_binary-amd64_Packages'  a=lucid-updates,c=main,v=10.04,o=Ubuntu,l=Ubuntu arch='amd64' site='archive.ubuntu.com' IndexType='Debian Package Index' Size=894 ID:0>
  DEBUG:root:Applying pinning: PkgPin(pkg='/^ant-doc/', priority=-32768)
  DEBUG:root:APT::VersionedKernelPackages is not set
  DEBUG:root:adjusting candidate version: test-package=2.0
  DEBUG:root:adjusting candidate version: test2-package=2.0
  DEBUG:root:falling back to adjusting test2-package's dependencies
  DEBUG:root:falling back to adjusting z-package's dependencies
  DEBUG:root:sanity check failed for: set() : no package is selected to be upgraded or installed
  DEBUG:root:falling back to adjusting test-package's dependencies
  DEBUG:root:falling back to adjusting test2-package's dependencies
  DEBUG:root:falling back to adjusting test3-package's dependencies
  DEBUG:root:falling back to adjusting test-package's dependencies
  DEBUG:root:falling back to adjusting test2-package's dependencies
  DEBUG:root:falling back to adjusting test3-package's dependencies
  .
  ----------------------------------------------------------------------
  Ran 1 test in 0.024s

  (Note the extra "DEBUG:root:falling back to adjusting" ... lines)

  [Regression Potential]

   * Minimal. The very small change fixes how call_checked() verifies that the package to be upgrades/installed is indeed marked to either being installed or upgraded.
  In the worst case, if call_checked() erroneously returns success for a package as a result of a regression, unattended-upgrades does not fall back to adjusting packages to make marking successful. Later unattended-upgrades finds out that the package could not be marked and should be treated as kept back.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1877769/+subscriptions