[Bug 2017399] Re: `apt-get -s upgrade` calculates the upgrade differently depending on whether InRelease files are present in the cache

[Xiao Wan]

@juliank Thanks for confirming my guess regarding Phased-Update-
Percentage. However, I don't think I have gained any better
understanding of the role Release/InRelease files play in the upgrade
calculation (I assume by "Archive:" you meant "Suite:"?).

Doesn't the file naming scheme (managed by apt-get itself) already
provide enough information as to whether a package is a security update?
Why not just go through the `*security*Packages` files in the cache to
exclude those listed packages from phased update?

If Release/InRelease files are required to correctly calculate an
upgrade (if, say, `apt-get update` does not yet verify the files it
fetches), shouldn't `apt-get upgrade` display an error message and fail
conspicuously instead of silently producing an incorrect calculation?

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/2017399

Title:
  `apt-get -s upgrade` calculates the upgrade differently depending on
  whether InRelease files are present in the cache

Status in apt package in Ubuntu:
  Invalid

Bug description:
  # Background
  I have an in-house package manager that downloads packages from mirrors in parallel and maintains a versioned local repo. It has worked well since 2019. However, during a recent migration from 18.04 to 22.04 I encountered a strange bug: the install step keeps failing because the download step has missed the following packages: xserver-common xserver-xorg-core and xserver-xorg-legacy. It turns out that `apt-get -s`, which the download step uses to figure out the packages that need to be downloaded, produces different results depending on whether InRelease files are present in the cache (as specified in `Dir::State::Lists`). I suspect Phased-Update-Percentage is a related factor.

  # How to reproduce
  1. Launch a virtual machine with `xubuntu-22.04.2-desktop-amd64.iso` (sha256: c7072859113399bef0e680a08c987c360f41642d8089ebb8928689066a9c9759)

  2. Download `minimal.tar.xz` at https://github.com/xwcal/ubuntu-apt-
  bug/blob/main/minimal.tar.xz?raw=true (sha256:
  ff7c38c8db2f05813dd641da504182180bf81d72cca9306a0898684c6b839b65) and
  run `tar -xf minimal.tar.xz` under `/some/dir` to extract the index
  files. They were taken from a real scenario last week.

  3. Replace the content of /etc/apt/sources.list with the following three lines:
  ```
  deb file:/home/apt/repo jammy main universe multiverse restricted
  deb file:/home/apt/repo jammy-security main universe multiverse restricted
  deb file:/home/apt/repo jammy-updates main universe multiverse restricted
  ```

  4. run `apt-get -s -o Dir::State::Lists=/some/dir/lists2 upgrade` and observe the output -- you should hopefully get:
  ```
  The following packages have been kept back:
    linux-generic-hwe-22.04 linux-headers-generic-hwe-22.04 linux-image-generic-hwe-22.04
  ...
  123 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
  ```

  5. `rm /some/dir/*InRelease*` and then run `apt-get -s -o Dir::State::Lists=/some/dir/lists2 upgrade` again -- you should hopefully get:
  ```
  The following packages have been kept back:
    linux-generic-hwe-22.04 linux-headers-generic-hwe-22.04 linux-image-generic-hwe-22.04 xserver-common xserver-xorg-core xserver-xorg-legacy
  ...
  120 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
  ```

  # Desired behavior
  `apt-get -s` should produce consistent results regardless of whether InRelease files are present in the cache.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/2017399/+subscriptions