[Bug 2148338] Re: Mirror selection is broken for ports architecture (bad text substitutions)

[Olivier Gayot]

** Changed in: subiquity
       Status: New => Triaged

** Changed in: subiquity
   Importance: Undecided => Medium

** Also affects: curtin
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to curtin.
https://bugs.launchpad.net/bugs/2148338

Title:
  Mirror selection is broken for ports architecture (bad text
  substitutions)

Status in curtin:
  New
Status in subiquity:
  Triaged

Bug description:
  On ports architectures (i.e., s390x, ppc64el - and arm64 on some
  versions of Ubuntu) trying to use a country-mirror (or another mirror)
  in the TUI results in the setting being silently ignored:

  Mirror address: http://local-mirror.intranet

  Get:1 http://ports.ubuntu.com/ubuntu-ports questing InRelease [257 kB]
  Get:2 http://ports.ubuntu.com/ubuntu-ports questing-updates InRelease [136 kB]
  Get:3 http://ports.ubuntu.com/ubuntu-ports questing-backports InRelease [133 kB]
  Fetched 543 kB in 4s (140 kB/s)
  Reading package lists...

  ^ here on questing, I'm supposedly using local-mirror.intranet but
  uses ports.ubuntu.com

  Analysis
  --------

  Curtin's apt-config can deploy the configuration using one of two
  ways:

  * using a template (provided that a template ubuntu.sources file is provided in the ISO). This is quite robust - even though people have run into issues in the past.
  * using text substitutions to replace mirrors directly in URIs fields.

  In recent ISOs there is no template provided so we are using text
  substitutions in URI directives.

  The text substitutions are specified as a map where:
  * the key is the URI to search for in the sources.list.d/ubuntu.sources file
  * the value is what to replace that URI with

  And they are applied to all URIs fields in the ubuntu.sources file,
  where we have an exact match on the key

  E.g.

  substitutions = {
      "http://archive.ubuntu.com/ubuntu/": "http://fr.archive.ubuntu.com/ubuntu",
      "http://ports.ubuntu.com/ubuntu-ports": "http://local-mirror.intranet",
  }

  The keys (i.e., http://archive.ubuntu.com/ubuntu/ and
  http://ports.ubuntu.com/ubuntu-ports) are hard-coded in curtin. And
  they need to strictly match what's in the ISO, otherwise the text
  substitutions cannot succeed.

  And this is what happens for ports architectures. ISOs (resolute here)
  are built with:

  Types: deb
  URIs: http://ports.ubuntu.com/ubuntu-ports/
  Suites: resolute resolute-updates resolute-backports
  Components: main universe restricted multiverse
  Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

  The URIs field *almost* matches but the trailing "/" makes it fail:

  * http://ports.ubuntu.com/ubuntu-ports     <- configured in curtin
  * http://ports.ubuntu.com/ubuntu-ports/    <- in ubuntu.sources

  A simple fix would involve adding the "/" in curtin, or matching with
  an optional "/".

  However, we need to be careful, ports architectures use
  ports.ubuntu.com for both the security pocket and the other pockets:

  ```
  Types: deb
  URIs: http://ports.ubuntu.com/ubuntu-ports/
  Suites: resolute resolute-updates resolute-backports
  Components: main universe restricted multiverse
  Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

  ## Ubuntu security updates. Aside from URIs and Suites,
  ## this should mirror your choices in the previous section.
  Types: deb
  URIs: http://ports.ubuntu.com/ubuntu-ports/
  Suites: resolute-security
  Components: main universe restricted multiverse
  Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
  ```

  This is a key difference with other architectures that use
  archive.ubuntu.com / security.ubuntu.com

  and naively search and replacing ports.ubuntu.com by something else
  would apply to both stanzas.

  We probably want something like:
  ```
  Types: deb
  URIs: http://local-mirror.intranet/ubuntu-ports/
  Suites: resolute resolute-updates resolute-backports
  Components: main universe restricted multiverse
  Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

  ## Ubuntu security updates. Aside from URIs and Suites,
  ## this should mirror your choices in the previous section.
  Types: deb
  URIs: http://ports.ubuntu.com/ubuntu-ports/
  Suites: resolute-security
  Components: main universe restricted multiverse
  Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/2148338/+subscriptions