[Bug 1789637] Re: Proper support for frontend lock

Thu Feb 7 13:07:29 UTC 2019

Tested 1.1ubuntu1.18.04.7~16.04.1:

Previous version:

root at x-uu-1789637-2:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
open("/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
open("/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 45
open("/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4

Previous version with python3-apt supporting frontend lock:

root at x-uu-1789637-2:~# apt install -qq python3-apt
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  python3-apt-dbg python-apt-doc
The following packages will be upgraded:
  python3-apt
1 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
Need to get 138 kB of archives.
After this operation, 6,144 B of additional disk space will be used.
(Reading database ... 25712 files and directories currently installed.)
Preparing to unpack .../python3-apt_1.1.0~beta1ubuntu0.16.04.3_amd64.deb ...
Unpacking python3-apt (1.1.0~beta1ubuntu0.16.04.3) over (1.1.0~beta1ubuntu0.16.04.2) ...
Setting up python3-apt (1.1.0~beta1ubuntu0.16.04.3) ...
root at x-uu-1789637-2:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
open("/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
open("/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 45
open("/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
open("/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 48
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5

Fixed version:

root at x-uu-1789637-2:~# apt install -qq unattended-upgrades
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  bsd-mailx default-mta | mail-transport-agent needrestart
The following packages will be upgraded:
  unattended-upgrades
1 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.
Need to get 40.2 kB of archives.
After this operation, 69.6 kB of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 25712 files and directories currently installed.)
Preparing to unpack .../unattended-upgrades_1.1ubuntu1.18.04.7~16.04.1_all.deb ...
Unpacking unattended-upgrades (1.1ubuntu1.18.04.7~16.04.1) over (0.90ubuntu0.10) ...
Processing triggers for systemd (229-4ubuntu21.15) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up unattended-upgrades (:) ...
Installing new version of config file /etc/kernel/postinst.d/unattended-upgrades ...
Installing new version of config file /etc/pm/sleep.d/10_unattended-upgrades-hibernate ...
root at x-uu-1789637-2:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
open("/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
open("/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
open("/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 50
open("/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1789637

Title:
  Proper support for frontend lock

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Released

Bug description:
  [Impact]

  Apt and dpkg implemented the Frontend Locking API and unattended-
  upgrades needs to adopt it to not leave the packaging system unlocked
  while passing control to python-apt and dpkg to perform package
  installations and removals. Leaving the packaging system unlocked
  caused many crashes of u-u when other tools took the lock and in the
  worse case let other package management tools operate on dpkg's
  database breaking systems.

  The change takes advantage of python-apt's new API and keeps the
  frontend lock during the run of u-u and unlocks only the inner locks
  for committing changes.

  [Test case]

  Run strace unattended-upgrades to upgrade several packages and check
  that lock-frontend is acquired at the beginning and not released until
  the end (not reacquired repeatedly).

  Unfixed u-u's output is like this:
  # strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 93
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  ...

  Fixed u-u's output is like that:
  # strace   unattended-upgrade --dry-run --verbose 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 8
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 57
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 69
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 70
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  ...

  [Regression potential]

  Incorrect lock API usage can make unattended-upgrades to crash, but it
  is unlikely to hang since the lock handling operations are non-
  blocking. Failing to reserve the locks can also cause problems by
  allowing other package management tools to install packages in
  parallel, but this problem existed in the past, too.

  [Additional Info]
  This is part of a wider series of changes for frontend locking

  - dpkg (bug 1796081)
  - apt (bug 1781169)
  - python-apt (bug 1795407)
  - packagekit (bug 1795614)
  - unattended-upgrades (bug 1789637)
  - aptdaemon (no bug filed yet)

  Further details about frontend locking can be found in
  https://lists.debian.org/debian-dpkg/2017/01/msg00044.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1789637/+subscriptions