[Bug 1781176] Re: Blacklisted packages are included in the "upgradable origin", while they should not

Fri Feb 22 13:47:45 UTC 2019

Tested with 1.1ubuntu1.18.04.9 on Bionic:

...
root at bb-uu-lp-1781176-2:~# apt list --upgradable 
Listing... Done
e2fsprogs/bionic-updates 1.44.1-1ubuntu1.1 amd64 [upgradable from: 1.44.1-1]
ebtables/bionic-updates 2.0.10.4-3.5ubuntu2.18.04.3 amd64 [upgradable from: 2.0.10.4-3.5ubuntu2]
libext2fs2/bionic-updates 1.44.1-1ubuntu1.1 amd64 [upgradable from: 1.44.1-1]
wireshark-common/bionic-updates,bionic-security 2.6.6-1~ubuntu18.04.0 amd64 [upgradable from: 2.4.5-1]
...
root at bb-uu-lp-1781176-2:~# unattended-upgrade --verbose --debug
Initial blacklisted packages: ebtables
Initial whitelisted packages: 
Starting unattended upgrades script
...
fetch.run() result: 0
blacklist: ['ebtables']
whitelist: []
Packages that will be upgraded:=20
InstCount=3D0 DelCount=3D0 BrokenCount=3D0
Extracting content from /var/log/unattended-upgrades/unattended-upgrades-dp=
kg.log since 2019-02-22 13:20:19
Sending mail to root

>From root at bb-uu-lp-1781176-2.lxd Fri Feb 22 13:43:21 2019
Return-path: <root at bb-uu-lp-1781176-2.lxd>
Envelope-to: root at bb-uu-lp-1781176-2.lxd
Delivery-date: Fri, 22 Feb 2019 13:43:21 +0000
Received: from root by bb-uu-lp-1781176-2.lxd with local (Exim 4.90_1)
	(envelope-from <root at bb-uu-lp-1781176-2.lxd>)
	id 1gxB6n-0001bL-Kz
	for root at bb-uu-lp-1781176-2.lxd; Fri, 22 Feb 2019 13:43:21 +0000
Subject: unattended-upgrades result for bb-uu-lp-1781176-2: True
To: root at bb-uu-lp-1781176-2.lxd
Auto-Submitted: auto-generated
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: root <root at bb-uu-lp-1781176-2.lxd>
Message-Id: <E1gxB6n-0001bL-Kz at bb-uu-lp-1781176-2.lxd>
Date: Fri, 22 Feb 2019 13:43:21 +0000

Unattended upgrade returned: True

Packages that were upgraded:
 wireshark-common=20

Packages that were auto-removed:
 libwsutil8 libwiretap7 libwireshark10 libwscodecs1=20

...

root at bb-uu-lp-1781176-2:~# apt list --upgradable 
Listing... Done
e2fsprogs/bionic-updates 1.44.1-1ubuntu1.1 amd64 [upgradable from: 1.44.1-1]
ebtables/bionic-updates 2.0.10.4-3.5ubuntu2.18.04.3 amd64 [upgradable from: 2.0.10.4-3.5ubuntu2]
libext2fs2/bionic-updates 1.44.1-1ubuntu1.1 amd64 [upgradable from: 1.44.1-1]

** Tags removed: verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1781176

Title:
  Blacklisted packages are included in the "upgradable origin", while
  they should not

Status in unattended-upgrades:
  Fix Released
Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  In Progress
Status in unattended-upgrades source package in Bionic:
  Fix Committed

Bug description:
  [Impact]

   * Reports from u-u incorrectly list packages from non-upgradable
  origins as "Packages with upgradable origin but kept back"

   * Listing the packages incorrectly is a result of
  is_pkgname_in_blacklist() having a side effect and removing the side
  effect is part of fixing LP: #1396787 which fix is also being SRU-d.

   * The fix is removing the side effect of is_pkgname_in_blacklist()

  [Test Case]

   * There is a build-time test in test/test_blacklisted_wrong_origin.py
   * To reproduce the original problem set up a system where all security updates are installed but ebtables (from bionic-updates) is not updated:
  $ sudo unattended-upgrade  --verbose
  Initial blacklisted packages:
  Initial whitelisted packages:
  Starting unattended upgrades script
  Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
  No packages found that can be upgraded unattended and no pending auto-removals
  $ sudo apt upgrade
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  Calculating upgrade... Done
  The following packages will be upgraded:
    apt apt-utils ebtables initramfs-tools initramfs-tools-bin initramfs-tools-core libapt-inst2.0 libapt-pkg5.0
    liblxc-common liblxc1 libpython3-stdlib lxcfs lxd lxd-client netplan.io networkd-dispatcher nplan
    python-apt-common python3 python3-apt python3-minimal python3-update-manager snapd squashfs-tools
    unattended-upgrades update-manager-core update-notifier-common
  27 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
  Need to get 24.1 MB of archives.
  After this operation, 1454 kB of additional disk space will be used.
  Do you want to continue? [Y/n] n
  Abort.

  * blacklist ebtables, set up emails from u-u, then run u-u again:
  $ sudo echo 'Unattended-Upgrade::Package-Blacklist {"ebtables";};' > /etc/apt/apt.conf.d/51unattended-upgrades-blacklist-ebtables
  $ sudo echo 'Unattended-Upgrade::Mail "root";' > /etc/apt/apt.conf.d/51unattended-upgrades-mail
  $ sudo unattended-upgrade  --verbose
  Initial blacklisted packages: ebtables
  Initial whitelisted packages:
  Starting unattended upgrades script
  Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
  Packages that will be upgraded:

  * Observe ebtables listed as being kept back and having upgradable origin with buggy u-u:
  $ sudo cat /var/mail/mail
  ...
  Packages with upgradable origin but kept back:
   ebtables=20
  ...

  * Upgrade u-u to a fixed version and run it, observing ebtables to be
  not listed as having upgradable origin

  [Regression Potential]

   * Regressions may make packages incorrectly missing from u-u's
  report, but the autopkgtests also cover that to some extent.

  [Other Info]

   * Original report: https://github.com/mvo5/unattended-
  upgrades/issues/116

To manage notifications about this bug go to:
https://bugs.launchpad.net/unattended-upgrades/+bug/1781176/+subscriptions