[Bug 1926990] Re: update-notifier should not alert users about esm-apps in a ESM machine

[Lucas Albuquerque Medeiros de Moura]

I can confirm that the xenial, bionic, focal and hirsute packages are
working as expected.

To perform the verification I have used the following script:

--------------------------------------
#!/bin/sh
set -x

series=xenial
name=$series-dev

function generate_motd_message() {
    message=$1
    echo "-----------------------------------------------------------"
    echo $message
    lxc exec $name -- /usr/lib/update-notifier/update-motd-updates-available --force
    lxc exec $name -- update-motd
    echo "-----------------------------------------------------------"
}

function turn_distro_into_esm_mode() {
    # guarantee that xenial distro is on ESM mode
    lxc exec $name -- sed -i 's/is_esm_supported and is_not_currently_supported/True #comment/' /usr/lib/update-notifier/apt-check
}

function turn_distro_into_non_esm_mode() {
    # guarantee that xenial distro is on ESM mode
    lxc exec $name -- sed -i 's/True #comment/False/' /usr/lib/update-notifier/apt-check
}

function setup_update_notifier() {
    lxc exec $name -- sh -c "cat <<EOF >/etc/apt/sources.list.d/ubuntu-$series-proposed.list
    deb http://archive.ubuntu.com/ubuntu/ $series-proposed restricted main multiverse universe"
    lxc exec $name -- apt update
    lxc exec $name -- sh -c "apt install update-notifier -yq > /dev/null"
}

function install_ua_from_branch() {
    lxc exec $name --cwd /var/tmp/uac -- apt-get update
    lxc exec $name --cwd /var/tmp/uac -- apt-get install make
    lxc exec $name -- git clone https://github.com/canonical/ubuntu-advantage-client.git /var/tmp/uac
    lxc exec $name --cwd /var/tmp/uac -- sh -c "make deps > /dev/null"
    lxc exec $name --cwd /var/tmp/uac -- sh -c "DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -us -uc > /dev/null"
    lxc exec $name -- apt-get remove ubuntu-advantage-tools --assume-yes
    lxc exec $name --cwd /var/tmp/uac -- dpkg -i /var/tmp/ubuntu-advantage-tools_27.0_amd64.deb
    lxc exec $name -- ua version
}

function install_ua() {
    lxc exec $name -- add-apt-repository ppa:ua-client/daily -y
    lxc exec $name -- sudo apt-get update
    lxc exec $name -- sudo apt-get install ubuntu-advantage-tools -y
    lxc exec $name -- ua version
    lxc exec $name -- sudo apt-get update
}

function ua_disable_esm_apps() {
    lxc exec $name -- sudo ua disable esm-apps
}

function install_all_upgrades() {
    lxc exec $name -- sudo sh -c "apt update > /dev/null"
    lxc exec $name -- sudo sh -c "apt upgrade -y > /dev/null"
}

function ua_attach() {
    lxc exec $name -- sed -i 's/contracts.can/contracts.staging.can/' /etc/ubuntu-advantage/uaclient.conf
    lxc exec $name -- ua attach $UACLIENT_BEHAVE_CONTRACT_TOKEN_STAGING
}

function ua_detach() {
    lxc exec $name -- ua detach --assume-yes
}

function generate_ua_motd_messages() {
    lxc exec $name -- python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py
}

function install_update_motd() {
    lxc exec $name -- apt install update-motd -yq
}

function turn_esm_apps_into_non_beta() {
    lxc exec $name -- sh -c "echo 'features:\n allow_beta: true' >> /etc/ubuntu-advantage/uaclient.conf"
}

function update_contract_effectiveto() {
    operation=$1
    num_days=$2
    replace_date=$(date -d "$date $operation$num_days days" +"%Y-%m-%dT00:00:00Z")
    echo $replace_date

    lxc exec $name -- sed -i "s/\"effectiveTo\": \"[^\"]*\"/\"effectiveTo\": \"$replace_date\"/g" /var/lib/ubuntu-advantage/private/machine-token.json
}

lxc delete --force $name
lxc launch ubuntu-daily:$series $name
sleep 10

setup_update_notifier
install_update_motd
generate_motd_message "$series: ua not installed"
install_ua
turn_esm_apps_into_non_beta
generate_ua_motd_messages
turn_distro_into_esm_mode
generate_motd_message "$series esm: ua not attached"
ua_attach
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached"
ua_disable_esm_apps
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached, apps disabled"
turn_distro_into_non_esm_mode
ua_detach
generate_ua_motd_messages
generate_motd_message "$series non-esm: ua not attached"
ua_attach
generate_ua_motd_messages
generate_motd_message "$series non-esm: ua attached"
update_contract_effectiveto - 30
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached - contract expired"
update_contract_effectiveto - 5
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached - contract grace period"
update_contract_effectiveto + 5
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached - contract expire soon"
install_all_upgrades
generate_ua_motd_messages
generate_motd_message "$series esm: ua attached - all upgrades installed"

set +x
-----------------------------

I verified the messages and they are consistent with the modifications
delivered by the proposed package


** Tags removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-hirsute verification-needed-xenial
** Tags added: verification-done verification-done-bionic verification-done-focal verification-done-hirsute verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-notifier in Ubuntu.
https://bugs.launchpad.net/bugs/1926990

Title:
  update-notifier should not alert users about esm-apps in a ESM machine

Status in update-notifier package in Ubuntu:
  Fix Released
Status in update-notifier source package in Xenial:
  Fix Committed
Status in update-notifier source package in Bionic:
  Fix Committed
Status in update-notifier source package in Focal:
  Fix Committed
Status in update-notifier source package in Hirsute:
  Fix Committed

Bug description:
  [Impact]
  When users are running an ESM distro and looking at the MOTD they may be surprised by an alert generated by update-notifier that both esm-infra and esm-apps could be enabled in the machine to receive future updates. However, esm-apps will not bring much to an ESM distro. This means that users will be receiving alerts about esm-apps unnecessarily.

  [Test case]

  To reproduce the issue:

  1. Launch a xenial container
  2. Create the file /etc/ubuntu-advantage/uaclient.conf with the following content:

  # Ubuntu-Advantage client config file.
  contract_url: 'https://contracts.canonical.com'
  security_url: 'https://ubuntu.com/security'
  data_dir: /var/lib/ubuntu-advantage
  log_level: debug
  log_file: /var/log/ubuntu-advantage.log
  features:
    allow_beta: true

  3. Install ubuntu-advantage-client from this PPA:
  https://code.launchpad.net/~ua-client/+archive/ubuntu/daily

  PS: Make sure to not allow the installation to override the config
  file you have just created.

  4. Update the version of update-notifier to the one in proposed
  5. Run `/usr/lib/update-notifier/apt-check --human-readable`
  6. Verify that both esm-infra and esm-apps alerts appear at the end of the message

  To verify that the error is fixed:

  1. Run the past scenario until step 3
  2. Install the new update-notifier from this ppa:
  https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa/
  3. Run `/usr/lib/update-notifier/apt-check --human-readable`
  4. Verify that only esm-infra is now showing an alert message at the end

  [Where problems could occur]

  We are changing some aspects of the package that could generate
  problems:

  1) We are changing the condition for when to output esm-apps alerts. If that condition is not right, we may miss displaying the messages to users that need them.
  2) We are adding a new dependency to the package, lsb-release. But since the package was already relying on that dependency, we are just fixing a packaging mistake here.
  3) Instead of relying on `lsb_release` for distro-info, we are now getting that info directly from `/etc/os-release`. Since is an attempt to not rely on subprocess python calls on that script. However, since we are adding more python code, logic errors can not happen because of that new chunk of code.
  4) Since we are touching translatable message, translation errors can happen because of the modifications we are providing here.
  5) We are now adding a ESM Apps headers for the case where users now have
  esm-apps source files in the system. If our logic is incorrect, we could be displaying that info on situations that are not needed.

  However, we believe that all of the risks mentioned above are
  manageable and should not block the changes we are bringing to the
  package.

  [Discussion]
  When running /usr/lib/update-notifier/apt-check on a ESM machine, we could alert the user about the existence of both esm-infra and esm-apps if the system has unauthenticated source files for both services. Therefore, we would display a message like this in the system:

  --------------------------
  Enable UA Apps: ESM to receive additional future security updates.
  See https://ubuntu.com/security/esm or run: sudo ua status

  Enable UA Infra: ESM to receive additional future security updates.
  See https://ubuntu.com/security/esm or run: sudo ua status
  ----------------------------------

  This is not ideal, since ESM machines will not gain much from esm-apps.
  Also, this messaging may hide the importance of using esm-infra on ESM
  machines, since the users will not know beforehand which service to
  prioritize.

  Furthermore, we are also bringing back the message:

  0 updates can be applied immediately

  When the system has no packages to install. This is because users
  found that omission confusing, since they thought that maybe something
  went wrong with motd if display no messages when the system is up to
  date.

  Additionally, we are adding an esm-apps header to the `apt-check`
  message if the system is a LTS one with esm-apps source files on it.
  The header should behave exactly like the one we have for esm-infra:

   UA Apps: Extended Security Maintenance (ESM) is (enabled|disabled).

  We doing this to be consistent with the message we have on ESM distros

  Finally, we are no longer relying on `lsb_release` on `apt-check` to
  get distro info, like name and version. We are parsing `/etc/os-
  release` for that now. The rationale for this change is some errors we
  have saw regarding the use of `subprocess.check_output`. Since we can
  parse the os-release file directly, we believe it is safer to do so
  than to rely on shell out for getting that info.

  == Changelog ==

    * debian/control: add lsb-release to the update-notifier-common depends   as it is used by apt-cdrom-check. (LP: #1927996)
    * data/apt_check.py
      - Only show esm-apps alerts on lts and non-esm distros (LP: #1926990)
      - Update esm-infra alert for distros on ESM mode 
      - Show message with number of upgradable packages even if that 
        number is zero (LP: #1926819)
      - Get distro name and version directly from /etc/os-release
      - Show esm-apps status header when running on LTS distro that 
        has not yet entered Extended Security Maintenance

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1926990/+subscriptions