[Bug 1789637] Re: Proper support for frontend lock

Wed Oct 17 16:53:09 UTC 2018

Tested with 1.1ubuntu1.18.04.6:

root at uu-frontend-lock:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 37
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
root at uu-frontend-lock:~# apt-get install una
unaccent             unace-nonfree        unagi                unagi-dev            unar                 
unace                unadf                unagi-dbg            unalz                unattended-upgrades  
root at uu-frontend-lock:~# apt-get install unattended-upgrades 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  bsd-mailx default-mta | mail-transport-agent needrestart
The following packages will be upgraded:
  unattended-upgrades
1 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
Need to get 37.7 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 unattended-upgrades all 1.1ubuntu1.18.04.6 [37.7 kB]
Fetched 37.7 kB in 0s (186 kB/s)               
Preconfiguring packages ...
(Reading database ... 28996 files and directories currently installed.)
Preparing to unpack .../unattended-upgrades_1.1ubuntu1.18.04.6_all.deb ...
Unpacking unattended-upgrades (1.1ubuntu1.18.04.6) over (1.1ubuntu1.18.04.5) ...
Processing triggers for ureadahead (0.100.0-20) ...
Processing triggers for systemd (237-3ubuntu10.3) ...
Setting up unattended-upgrades (1.1ubuntu1.18.04.6) ...
Processing triggers for man-db (2.8.3-2) ...
root at uu-frontend-lock:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 37
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
root at uu-frontend-lock:~# 

** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1789637

Title:
  Proper support for frontend lock

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Confirmed
Status in unattended-upgrades source package in Bionic:
  Fix Committed

Bug description:
  [Impact]

  Apt and dpkg implemented the Frontend Locking API and unattended-
  upgrades needs to adopt it to not leave the packaging system unlocked
  while passing control to python-apt and dpkg to perform package
  installations and removals. Leaving the packaging system unlocked
  caused many crashes of u-u when other tools took the lock and in the
  worse case let other package management tools operate on dpkg's
  database breaking systems.

  The change takes advantage of python-apt's new API and keeps the
  frontend lock during the run of u-u and unlocks only the inner locks
  for committing changes.

  [Test case]

  Run strace unattended-upgrades to upgrade several packages and check
  that lock-frontend is acquired at the beginning and not released until
  the end (not reacquired repeatedly).

  Unfixed u-u's output is like this:
  # strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 93
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  ...

  Fixed u-u's output is like that:
  # strace   unattended-upgrade --dry-run --verbose 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 8
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 57
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 69
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 70
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  ...

  [Regression potential]

  Incorrect lock API usage can make unattended-upgrades to crash, but it
  is unlikely to hang since the lock handling operations are non-
  blocking. Failing to reserve the locks can also cause problems by
  allowing other package management tools to install packages in
  parallel, but this problem existed in the past, too.

  [Additional Info]
  This is part of a wider series of changes for frontend locking

  - dpkg (bug 1796081)
  - apt (bug 1781169)
  - python-apt (bug 1795407)
  - packagekit (bug 1795614)
  - unattended-upgrades (bug 1789637)
  - aptdaemon (no bug filed yet)

  Further details about frontend locking can be found in
  https://lists.debian.org/debian-dpkg/2017/01/msg00044.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1789637/+subscriptions