[Bug 1602536] Re: /usr/bin/unattended-upgrade:apt.cache.LockFailedException:/usr/bin/unattended-upgrade at 1468:main:do_auto_remove:cache_commit:commit:_fetch_archives

[Balint Reczey]

** Description changed:

+ [Impact]
+ 
+  * Unattended-upgrades fails to autoremove packages after installing
+ updates or fails to install updates due to a parallel process acquiring
+ apt or dpkg lock while u-u is running.
+ 
+ [Test Case]
+ 
+  * Set up a system with packages (> 30) to be upgraded. In case of
+ Bionic -security has a low number of updates, thus set up the system to
+ install -updates to have more packages for u-u to upgrade:
+ 
+  echo 'Unattended-Upgrade::Allowed-Origins::
+ "${distro_id}:${distro_codename}-updates";' >
+ /etc/apt/apt.conf.d/51-updates
+ 
+  * Set up two shells to run commands in parallel
+  * In shell "A" run sudo apt update && sudo unattended-upgrade --dry-run --verbose --debug
+  * After u-u started run the following command in shell "B":
+  while sleep 1; do python3 -c 'import apt; import apt_pkg; print(apt_pkg.pkgsystem_lock())' ; done
+  * Observe the following exception repeated while running u-u and True being printed after u-u is finished:
+ ...
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ True
+ Traceback (most recent call last):
+   File "<string>", line 1, in <module>
+ apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
+ True
+ True
+ True
+ True
+ ...
+ 
+ [Regression Potential WIP]
+ 
+  * Unattended-upgrade may crash
+ 
+ [Original Bug Text]
+ 
  The Ubuntu Error Tracker has been receiving reports about a problem
  regarding unattended-upgrades.  This problem was most recently seen with
  version 0.90, the problem page at
  https://errors.ubuntu.com/problem/19f99745d7dce5aea118eb31bfffcbdcdf238c4f
  contains more details.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1602536

Title:
  /usr/bin/unattended-upgrade:apt.cache.LockFailedException:/usr/bin
  /unattended-
  upgrade at 1468:main:do_auto_remove:cache_commit:commit:_fetch_archives

Status in unattended-upgrades package in Ubuntu:
  Confirmed

Bug description:
  [Impact]

   * Unattended-upgrades fails to autoremove packages after installing
  updates or fails to install updates due to a parallel process
  acquiring apt or dpkg lock while u-u is running.

  [Test Case]

   * Set up a system with packages (> 30) to be upgraded. In case of
  Bionic -security has a low number of updates, thus set up the system
  to install -updates to have more packages for u-u to upgrade:

   echo 'Unattended-Upgrade::Allowed-Origins::
  "${distro_id}:${distro_codename}-updates";' >
  /etc/apt/apt.conf.d/51-updates

   * Set up two shells to run commands in parallel
   * In shell "A" run sudo apt update && sudo unattended-upgrade --dry-run --verbose --debug
   * After u-u started run the following command in shell "B":
   while sleep 1; do python3 -c 'import apt; import apt_pkg; print(apt_pkg.pkgsystem_lock())' ; done
   * Observe the following exception repeated while running u-u and True being printed after u-u is finished:
  ...
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  True
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  True
  True
  True
  True
  ...

  [Regression Potential WIP]

   * Unattended-upgrade may crash

  [Original Bug Text]

  The Ubuntu Error Tracker has been receiving reports about a problem
  regarding unattended-upgrades.  This problem was most recently seen
  with version 0.90, the problem page at
  https://errors.ubuntu.com/problem/19f99745d7dce5aea118eb31bfffcbdcdf238c4f
  contains more details.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1602536/+subscriptions