[Bug 1906436] Re: Resolve autopkgtest failures Pep8/pyflakes on Xenial and Bionic

Wed Dec 2 04:36:26 UTC 2020

** Summary changed:

- Resolve autopkgtest failures Pep8 on Xenial and Bionic
+ Resolve autopkgtest failures Pep8/pyflakes on Xenial and Bionic

** Description changed:

- [Impact]
+ Cherry-picking upstream changes for ESM Product renaming and messaging
+ 1901627 triggered autopkgtest failures on for Xenial(pep8) and
+ Bionic(pep8 & pyflakes) during SRU review.

-  The product names and product URLs of Ubuntu Advantage Extended
- Security Maintenance offerings have been rebranded prior to an official
- launch of the product ESM product in Ubuntu Xenial, Bionic and Focal.
- The backport of these commits introduced pep8 linter errors in unittests
- that broke autopkgtests for xenial version 3.168.11 and bionic version
- 3.192.1.8.
- 
- The failed autopkgtests pointed to a need for an explicit build-depends
- on pep8 package because cloud-images:
- 
- https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
- /autopkgtest-xenial/xenial/amd64/u/update-
- notifier/20201016_043912_51752@/log.gz
- 
- 
- In order to publish to xenial and bionic, introduce pep8 biuld-depends and fix pep8 lints
- 
- [Test Case]
- 
-  * Install unreleased ubuntu-advantage-tools from a PPA
-  * Run: ua enable <yourToken> to activate UA apt repositories
-  * Check MOTD messaging related to UA Infra: ESM prior to upgrade to -proposed update-notifier via /usr/lib/update-notifier/apt-check --human-readable
- 
-  * Upgrade update-notifier to -proposed
-  * Re-check MOTD messaging related to UA Infra: ESM to see expected messages
- 
- #!/bin/bash
- #
- # SRU Verification update-notifier + ubuntu=advantage-tools
- # Test procedure:
- # - launch container Trusty, Xenial or Bionic
- # - Install ubuntu-advantage-tools from https://launchpad.net/~ua-client/+archive/ubuntu/proposed which supports esm on trusty, xenial, bionic, and focal
- # - Attach container to UA subscription (which activates the ESM APT repos
- # - run apt_check --human-readable to assert ESM pkg counts ARE NOT reported
- # - Upgrade update-notifier to -proposed
- # - re-run apt_check --human-readable to assert ESM pkg counts ARE reported
- 
- set -e
- UA_TOKEN=$1
- if [ -z "$1" ]; then
-  echo "Usage: $0 <contractTOKEN>"
-  exit 1
- fi
- # sources:
- # ua.proposed:
- # source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
- # keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7
- 
- cat > test-un.yaml <<EOF
- #cloud-config
- ssh_import_id: [chad.smith]
- package_update: true
- package_upgrade: true
- apt:
-   sources:
-       ua.proposed:
-           source: deb http://ppa.launchpad.net/ua-client/staging/ubuntu \$RELEASE main
-           keyid: 6E34E7116C0BC933
- EOF
- 
- cat > setup_proposed.sh <<EOF
- #/bin/bash
- mirror=http://archive.ubuntu.com/ubuntu
- echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
- apt-get update -q
- apt-get install -qy update-notifier-common
- EOF
- 
- wait_for_boot() {
-   local vm=$1 release=$2
-   echo "--- Wait for cloud-init to finish"
-   sleep 5
-     lxc exec ${vm} -- cloud-init status --wait --long
- }
- 
- for release in xenial bionic focal groovy; do
-   echo "--- BEGIN $release update-notifier testing"
-   vm=test-sru-$release
-   echo "--- Launch cloud-init with ppa:ua-client/proposed enabled"
-   lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-un.yaml)"
-   wait_for_boot ${vm} ${release}
-   echo "--- Attach Ubuntu-Advantage, enabling services"
-   lxc exec ${vm} -- ua attach ${UA_TOKEN}
-   echo "--- Install a downgraded hello package which ESM-focal delivers"
-   lxc exec ${vm} -- apt-get install hello=2.10-2ubuntu2
-   echo "--- Expect 0 upgradable packages for MOTD from apt_check before upgrade"
-   lxc exec ${vm} -- /usr/lib/update-notifier/apt-check --human-readable
-   lxc exec ${vm} -- /usr/lib/update-notifier/apt-check --human-readable | grep '0 of these updates are security updates' && echo "SUCCESS: found 0 ESM security updates pre-upgrade" || echo "FAILURE: did not find expected 0 ESM security updates"
-   echo "--- Upgrade update-notifier from -proposed"
-   lxc file push setup_proposed.sh ${vm}/
-   lxc exec ${vm} -- bash /setup_proposed.sh | grep update-notifier
-   echo "--- Expect non-zero upgradable packages for MOTD from apt_check AFTER upgrade"
-   lxc exec ${vm} -- /usr/lib/update-notifier/apt-check --human-readable | grep '1 of these updates is a security update' && echo "SUCCESS: found 1 ESM security updates pre-upgrade" || echo "FAILURE: did not find expected 1 ESM security updates"
- done
- 
- [Where problems could occur]
- 
-  *  Low risk:
-     pep8 lint fixes only here in data/apt_check.py, data/hooks.py and test_package-data-downloader.py. Only need to confirm that current apt-hook logic remains functional and that motd messaging is present without tracebacks using  /usr/lib/update-notifier/apt-check --human-readable
- 
- 
- [Other Info]
- 
- [Original Description]
- 
- Cherry-picking upstream changes for ESM Product renaming and messaging
- 1901627 triggered autopkgtest failures on Xenial and Bionic during SRU
- review. Also changes in the base cloud-images used for testing dropped
+ Also changes in the Xenial base cloud-images used for testing dropped
  pep8 package from the build environment and unittests rely on calling
- pep8 on Cenial and Bionic. minimally a build-depends on pep8 needs to be
- added and pep8 test failures addresssed.
+ pep8 on Xenial and Bionic. minimally a build-depends on pep8 needs to be
+ added and pep8 test failures addresssed for Xenial. Bionic also needs
+ minor pyflakes fixes from master.

  This does not directly affect Focal or later because update-notifier
  moved to pycodestyle validation in tests instead of using pep8.

  Initial pep8 build-dependency error on Xenial/Bionic autopkgtests:

  Fetched 4 B in 0s (0 B/s)
  ....Get:1 http://localhost:17195/canary-file.txt [4 B]
  Fetched 4 B in 0s (0 B/s)
  .E
  ======================================================================
  ERROR: test_pep8_clean (test_pep8.TestPep8Clean)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/tmp/autopkgtest.RQGkx9/build.3cC/src/tests/test_pep8.py", line 32, in test_pep8_clean
      ["pep8", "--ignore={0}".format(IGNORE_PEP8), py_file])
    File "/usr/lib/python3.5/subprocess.py", line 557, in call
      with Popen(*popenargs, **kwargs) as p:
    File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
      restore_signals, start_new_session)
    File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
      raise child_exception_type(errno_num, err_msg)
  FileNotFoundError: [Errno 2] No such file or directory: 'pep8'

  Once resolving the build-depends: example pep8 test failures: seen
  running nosetests3 on update-notifier on Xenial amd Bionic

  ./tmp/autopkgtest.wN63go/build.IFg/src/tests/../data/apt_check.py:157:21:
  W503 line break before binary operator ....

  autopackage test failure logs showing missing pep8 dependency:

  xenial: update-notifier/3.168.11
     - autopkg logs: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/u/update-notifier/20201016_043912_51752@/log.gz

  bionic: update-notifier/3.192.1.8
      - autopkg failures: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/u/update-notifier/20201016_045656_8c126@/log.gz

  focal: update-notifier/3.192.30.1
    - rejected this upload in Focal to reduce SRU thrashing because a followup will be provided for Focal anyway to sync with Xenial and Bionic.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-notifier in Ubuntu.
https://bugs.launchpad.net/bugs/1906436

Title:
  Resolve autopkgtest failures Pep8/pyflakes on Xenial and Bionic

Status in update-notifier package in Ubuntu:
  New

Bug description:
  Cherry-picking upstream changes for ESM Product renaming and messaging
  1901627 triggered autopkgtest failures on for Xenial(pep8) and
  Bionic(pep8 & pyflakes) during SRU review.

  Also changes in the Xenial base cloud-images used for testing dropped
  pep8 package from the build environment and unittests rely on calling
  pep8 on Xenial and Bionic. minimally a build-depends on pep8 needs to
  be added and pep8 test failures addresssed for Xenial. Bionic also
  needs minor pyflakes fixes from master.

  This does not directly affect Focal or later because update-notifier
  moved to pycodestyle validation in tests instead of using pep8.

  Initial pep8 build-dependency error on Xenial/Bionic autopkgtests:

  Fetched 4 B in 0s (0 B/s)
  ....Get:1 http://localhost:17195/canary-file.txt [4 B]
  Fetched 4 B in 0s (0 B/s)
  .E
  ======================================================================
  ERROR: test_pep8_clean (test_pep8.TestPep8Clean)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/tmp/autopkgtest.RQGkx9/build.3cC/src/tests/test_pep8.py", line 32, in test_pep8_clean
      ["pep8", "--ignore={0}".format(IGNORE_PEP8), py_file])
    File "/usr/lib/python3.5/subprocess.py", line 557, in call
      with Popen(*popenargs, **kwargs) as p:
    File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
      restore_signals, start_new_session)
    File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
      raise child_exception_type(errno_num, err_msg)
  FileNotFoundError: [Errno 2] No such file or directory: 'pep8'

  Once resolving the build-depends: example pep8 test failures: seen
  running nosetests3 on update-notifier on Xenial amd Bionic

  ./tmp/autopkgtest.wN63go/build.IFg/src/tests/../data/apt_check.py:157:21:
  W503 line break before binary operator ....

  autopackage test failure logs showing missing pep8 dependency:

  xenial: update-notifier/3.168.11
     - autopkg logs: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/u/update-notifier/20201016_043912_51752@/log.gz

  bionic: update-notifier/3.192.1.8
      - autopkg failures: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/u/update-notifier/20201016_045656_8c126@/log.gz

  focal: update-notifier/3.192.30.1
    - rejected this upload in Focal to reduce SRU thrashing because a followup will be provided for Focal anyway to sync with Xenial and Bionic.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1906436/+subscriptions