[Bug 2109495] Re: unattended-upgrades can (and regularly does) pause mid-update indefinitely with no error or feedback

[David Buckley]

I believe enabling the following ugly systemd unit should mitigate the
issue:

$ cat /etc/systemd/system/fix-apt-https.service
[Unit]
Description=Destroy broken apt fetches after resume
After=sleep.target

[Service]
Type=oneshot
ExecStart=-/usr/bin/killall https

[Install]
WantedBy=sleep.target

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/2109495

Title:
  unattended-upgrades can (and regularly does) pause mid-update
  indefinitely with no error or feedback

Status in apt package in Ubuntu:
  New

Bug description:
  unattended-upgrades can pause, probably forever, mid-upgrade. This
  prevents any further upgrades until the service is restarted.

  The direct cause is probably inside apt rather than the upgrade
  process itself, but the upgrade process has apparently no defence
  against apt hanging, either. An interactive user would see their
  command hang forever and would likely interrupt it, so the issue in
  apt is not a security problem (probably -- if it were missed, I guess
  it would hold the lock and prevent upgrades, too).

  I've observed this on both Debian and Ubuntu, on systems which suspend
  often, so I assume the issue is an infinite timeout somewhere in the
  networking code being hit because of a suspend operation occurring
  mid-update. It's hard to know how common it is, as it is not directly
  visible (see below).

  In this instance, the observed state was like this:

  $ ps aux|grep apt
  root      158770  0.0  0.0   2800  1536 ?        Ss   Mar24   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily update
  root      158812  0.0  0.0   2800  1584 ?        S    Mar24   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily lock_is_held update
  root      158857  0.0  0.0  22232  9876 ?        S    Mar24   2:00 apt-get -qq -y update
  _apt      158867  0.0  0.0  25440  7980 ?        S    Mar24   0:00 /usr/lib/apt/methods/http
  _apt      158868  0.0  0.0  25440  8072 ?        S    Mar24   0:00 /usr/lib/apt/methods/http
  _apt      158869  0.0  0.0  29468 12396 ?        S    Mar24   0:00 /usr/lib/apt/methods/https
  _apt      158870  0.0  0.0  29408 12424 ?        S    Mar24   0:00 /usr/lib/apt/methods/https
  _apt      159069  0.0  0.0  18280  5564 ?        S    Mar24   0:00 /usr/lib/apt/methods/gpgv
  _apt      159622  0.0  0.0  26792  6456 ?        S    Mar24   0:00 /usr/lib/apt/methods/store

  Note the date: I'd not had updates performed for over a month. I use
  the machine daily, but mostly suspend it overnight, so it's had
  multiple days of awake time since then, too. I recognise
  "apt/methods/http", so I think this is the common failure mode.

  There's no clues in the log files (they appear to most recently show a
  normal no-upgrades cycle on the 22nd).

  There are no errors reported to me as a user. Note that though I'm not
  on the default desktop environment, so it's possible the error
  reporting is broken due to me not starting a service which would have
  reported it, I do get popups telling me to restart applications when
  updates do run. The only feedback I have from the system is the
  suspicious absence of these requests to restart, and inability to run
  apt manually if I try. Error in this case:

  $ sudo apt update && sudo apt full-upgrade
  Reading package lists... Done
  E: Could not get lock /var/lib/apt/lists/lock. It is held by process 158857 (apt-get)
  N: Be aware that removing the lock file is not a solution and may break your system.
  E: Unable to lock directory /var/lib/apt/lists/

  Killing some part of the tree usually wakes it back up again. I
  usually find the issue because I want to change packages or force an
  upgrade pre-reboot; this also means "fixing" the upgrade task prevents
  what I want to do as it the immediately starts doing upgrades, so I
  usually just stop the service fully and start again either manually or
  after reboot.

  * Probably the service ought to set a robust, real time, timeout on the entire upgrade process, or at least on the networking part of it. Even a 24h timeout would be sufficient to mitigate most of the damage.
  * Having a timeout in the apt code would probably be useful, too.
  * Having some feedback from unattended-upgrades when it has failed to run due to lock contention many times in a row would be good, too.

  ProblemType: Bug
  DistroRelease: Ubuntu 24.04
  Package: unattended-upgrades 2.9.1+nmu4ubuntu1
  ProcVersionSignature: Ubuntu 6.8.0-52.53-generic 6.8.12
  Uname: Linux 6.8.0-52-generic x86_64
  NonfreeKernelModules: zfs
  ApportVersion: 2.28.1-0ubuntu3.5
  Architecture: amd64
  CasperMD5CheckResult: pass
  Date: Mon Apr 28 11:00:20 2025
  InstallationDate: Installed on 2025-01-17 (101 days ago)
  InstallationMedia: Ubuntu 24.04.1 LTS "Noble Numbat" - Release amd64 (20240827.1)
  PackageArchitecture: all
  RebootRequiredPkgs: Error: path contained symlinks.
  SourcePackage: unattended-upgrades
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/2109495/+subscriptions