[Bug 1789637] Re: Proper support for frontend lock

[Balint Reczey]

** Description changed:

- Frontend lock support for apt and dpkg is tracked in LP: #1781169, this
- bug tracks unattended-upgrades changes
+ [Impact]
+ 
+ Apt and dpkg implemented the Frontend Locking API and unattended-
+ upgrades needs to adopt it to not leave the packaging system unlocked
+ while passing control to python-apt and dpkg to perform package
+ installations and removals. Leaving the packaging system unlocked caused
+ many crashes of u-u when other tools took the lock and in the worse case
+ let other package management tools operate on dpkg's database breaking
+ systems.
+ 
+ The change takes advantage of python-apt's new API and keeps the
+ frontend lock during the run of u-u and unlocks only the inner locks for
+ committing changes.
+ 
+ [Test case]
+ 
+ Run strace unattended-upgrades to upgrade several packages and check
+ that lock-frontend is acquired at the beginning and not released until
+ the end (not reacquired repeatedly).
+ 
+ Unfixed u-u's output is like this:
+ # strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
+ openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
+ openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
+ openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
+ openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 93
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
+ openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
+ openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
+ ...
+ 
+ Fixed u-u's output is like that:
+ # strace   unattended-upgrade --dry-run --verbose 2>&1 | grep lock
+ openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
+ openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 8
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
+ openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 57
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
+ openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 69
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
+ openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 70
+ openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
+ ...
+ 
+ [Regression potential]
+ 
+ Incorrect lock API usage can make unattended-upgrades to crash, but it
+ is unlikely to hang since the lock handling operations are non-blocking.
+ Failing to reserve the locks can also cause problems by allowing other
+ package management tools to install packages in parallel, but this
+ problem existed in the past, too.
+ 
+ [Additional Info]
+ Frontend lock support for apt and dpkg is tracked in LP: #1781169, this bug tracks unattended-upgrades changes

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1789637

Title:
  Proper support for frontend lock

Status in unattended-upgrades package in Ubuntu:
  Confirmed

Bug description:
  [Impact]

  Apt and dpkg implemented the Frontend Locking API and unattended-
  upgrades needs to adopt it to not leave the packaging system unlocked
  while passing control to python-apt and dpkg to perform package
  installations and removals. Leaving the packaging system unlocked
  caused many crashes of u-u when other tools took the lock and in the
  worse case let other package management tools operate on dpkg's
  database breaking systems.

  The change takes advantage of python-apt's new API and keeps the
  frontend lock during the run of u-u and unlocks only the inner locks
  for committing changes.

  [Test case]

  Run strace unattended-upgrades to upgrade several packages and check
  that lock-frontend is acquired at the beginning and not released until
  the end (not reacquired repeatedly).

  Unfixed u-u's output is like this:
  # strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 93
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
  ...

  Fixed u-u's output is like that:
  # strace   unattended-upgrade --dry-run --verbose 2>&1 | grep lock
  openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
  openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 8
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 57
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 69
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 70
  openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
  ...

  [Regression potential]

  Incorrect lock API usage can make unattended-upgrades to crash, but it
  is unlikely to hang since the lock handling operations are non-
  blocking. Failing to reserve the locks can also cause problems by
  allowing other package management tools to install packages in
  parallel, but this problem existed in the past, too.

  [Additional Info]
  Frontend lock support for apt and dpkg is tracked in LP: #1781169, this bug tracks unattended-upgrades changes

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1789637/+subscriptions