[Bug 2109339] [NEW] Test suite failures: DSA1024 keys / xenial

[Julian Andres Klode]

Public bug reported:

[Impact]
Autopkgtests in oracular onward are broken, and the APT 2.8.3 noble SRU also breaks the tests by revoking 1024-bit DSA keys fully; this in turn results in an inability to determine regressions.

There is no direct user-facing impact, however the SRU cannot simply be
staged in proposed, as for testing purposes, the version in testing is
triggered by e.g. new APT changes, so it needs to be released.

test_update_origin.py has a whole bunch of issues preventing it from
passing:

- It uses an included keyring, a copy of only the 2012 archive signing key, which used DSA. With the APT 2.8.3 SRU to noble, and APT in the oracular release pocket, that test is now failing
- The test verifies xenial which also will EOL at some point and then would fail again, so it needs to be updated to test noble.

[Test plan]
autopkgtests should succeed after upload. For noble, they need to be tried against apt 2.8.3 from proposed.

[Where problems could occur]
We only modify the test, so tests could continue to fail, or inadvertently pass.

[Other information]
We simply change the test case to test noble instead of xenial. It may be worthwhile exploring how to test the current release we are testing on instead such that we don't run into broken tests when noble disappears.

However, that requires determining the release correctly which may have
odd effects on downstream distros wishing to test update-manager; and is
only a concern for 26.04 onward given that noble is staying around the
longest of all SRU candidates.

** Affects: update-manager (Ubuntu)
     Importance: Undecided
     Assignee: Julian Andres Klode (juliank)
         Status: In Progress

** Affects: update-manager (Ubuntu Focal)
     Importance: Undecided
         Status: New

** Affects: update-manager (Ubuntu Jammy)
     Importance: Undecided
         Status: New

** Affects: update-manager (Ubuntu Noble)
     Importance: Undecided
         Status: New

** Affects: update-manager (Ubuntu Oracular)
     Importance: Undecided
         Status: New

** Affects: update-manager (Ubuntu Plucky)
     Importance: Undecided
         Status: New

** Changed in: update-manager (Ubuntu)
       Status: New => In Progress

** Changed in: update-manager (Ubuntu)
     Assignee: (unassigned) => Julian Andres Klode (juliank)

** Summary changed:

- Test suite failures: DSA1024 keys / xenial / stderr confusion in plucky
+ Test suite failures: DSA1024 keys / xenial

** Description changed:

  [Impact]
  Autopkgtests in oracular onward are broken, and the APT 2.8.3 noble SRU also breaks the tests by revoking 1024-bit DSA keys fully; this in turn results in an inability to determine regressions.
  
  There is no direct user-facing impact, however the SRU cannot simply be
  staged in proposed, as for testing purposes, the version in testing is
  triggered by e.g. new APT changes, so it needs to be released.
  
  test_update_origin.py has a whole bunch of issues preventing it from
  passing:
  
  - It uses an included keyring, a copy of only the 2012 archive signing key, which used DSA. With the APT 2.8.3 SRU to noble, and APT in the oracular release pocket, that test is now failing
  - The test verifies xenial which also will EOL at some point and then would fail again, so it needs to be updated to test noble.
- 
- Additionally in plucky, the tests will also fail due to a change in
- xvfb-run that now causes stderr to remain on stderr, and hence a
- Testsuite-Restriction: allow-stderr needs to be added.
  
  [Test plan]
  autopkgtests should succeed after upload. For noble, they need to be tried against apt 2.8.3 from proposed.
  
  [Where problems could occur]
  We only modify the test, so tests could continue to fail, or inadvertently pass.
  
  [Other information]
  We simply change the test case to test noble instead of xenial. It may be worthwhile exploring how to test the current release we are testing on instead such that we don't run into broken tests when noble disappears.
  
  However, that requires determining the release correctly which may have
  odd effects on downstream distros wishing to test update-manager; and is
  only a concern for 26.04 onward given that noble is staying around the
  longest of all SRU candidates.

** Also affects: update-manager (Ubuntu Plucky)
   Importance: Undecided
       Status: New

** Also affects: update-manager (Ubuntu Focal)
   Importance: Undecided
       Status: New

** Also affects: update-manager (Ubuntu Oracular)
   Importance: Undecided
       Status: New

** Also affects: update-manager (Ubuntu Noble)
   Importance: Undecided
       Status: New

** Also affects: update-manager (Ubuntu Jammy)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-manager in Ubuntu.
https://bugs.launchpad.net/bugs/2109339

Title:
  Test suite failures: DSA1024 keys / xenial

Status in update-manager package in Ubuntu:
  In Progress
Status in update-manager source package in Focal:
  New
Status in update-manager source package in Jammy:
  New
Status in update-manager source package in Noble:
  New
Status in update-manager source package in Oracular:
  New
Status in update-manager source package in Plucky:
  New

Bug description:
  [Impact]
  Autopkgtests in oracular onward are broken, and the APT 2.8.3 noble SRU also breaks the tests by revoking 1024-bit DSA keys fully; this in turn results in an inability to determine regressions.

  There is no direct user-facing impact, however the SRU cannot simply
  be staged in proposed, as for testing purposes, the version in testing
  is triggered by e.g. new APT changes, so it needs to be released.

  test_update_origin.py has a whole bunch of issues preventing it from
  passing:

  - It uses an included keyring, a copy of only the 2012 archive signing key, which used DSA. With the APT 2.8.3 SRU to noble, and APT in the oracular release pocket, that test is now failing
  - The test verifies xenial which also will EOL at some point and then would fail again, so it needs to be updated to test noble.

  [Test plan]
  autopkgtests should succeed after upload. For noble, they need to be tried against apt 2.8.3 from proposed.

  [Where problems could occur]
  We only modify the test, so tests could continue to fail, or inadvertently pass.

  [Other information]
  We simply change the test case to test noble instead of xenial. It may be worthwhile exploring how to test the current release we are testing on instead such that we don't run into broken tests when noble disappears.

  However, that requires determining the release correctly which may
  have odd effects on downstream distros wishing to test update-manager;
  and is only a concern for 26.04 onward given that noble is staying
  around the longest of all SRU candidates.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/2109339/+subscriptions