[Bug 1892358] Re: autopkgtest success rate dropped inhibiting proposed migration
Launchpad Bug Tracker
1892358 at bugs.launchpad.net
Mon Jan 18 09:34:34 UTC 2021
This bug was fixed in the package systemd - 246.6-1ubuntu1.1
---------------
systemd (246.6-1ubuntu1.1) groovy; urgency=medium
[ Dan Streetman ]
* d/t/boot-smoke: update test to avoid false negatives
(LP: #1892358)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=205c30ca53b0e421db28bb56afaf5f88650ce592
* d/t/boot-and-services: remove unneeded test lines
(LP: #1892358)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=71853082af4e668996db574915c5a156f9897fd3
* d/t/systemd-fsckd: rewrite test to try to fix false negatives
(LP: #1892358)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=6ae6be039ec582410769d2d6d131e12bdcd19a68
* d/p/lp1905044-test-use-cap_last_cap-for-max-supported-cap-number-n.patch:
test: use cap_last_cap() instead of capability_list_length()
(LP: #1905044)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=84a4832f5f7d4f939c1c78c6be4c3f9e05cd7f59
* d/p/lp1907306/0001-sd-dhcp-client-don-t-log-timeouts-if-already-expired.patch,
d/p/lp1907306/0002-sd-dhcp-client-track-dhcp4-t1-t2-expire-times.patch,
d/p/lp1907306/0003-sd-dhcp-client-add-RFC2131-retransmission-details.patch,
d/p/lp1907306/0004-sd-dhcp-client-simplify-dhcp4-t1-t2-parsing.patch,
d/p/lp1907306/0005-sd-dhcp-client-correct-dhcpv4-renew-rebind-retransmi.patch,
d/p/lp1907306/0006-sd-dhcp-client-correct-retransmission-timeout-to-mat.patch,
d/p/lp1907306/0007-test-network-increase-wait_online-timeout-to-handle-.patch,
d/p/lp1907306/0008-sd-dhcp-client-fix-renew-rebind-timeout-calculation-.patch:
Send correct number of dhcpv4 renew and rebind requests
(LP: #1907306)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0a96dc16ac00e90cd3904e6d490d676b9bb98f1f
* d/p/lp1902960-udev-re-assign-ID_NET_DRIVER-ID_NET_LINK_FILE-ID_NET.patch:
Run net_setup_link on 'change' uevents (LP: #1902960)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7183e2ef4758ce47b152dec735e7d213d6003e37
* d/t/root-unittests:
Remove any corrupt journal files (LP: #1881947)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3d0ea66f0db4a204759fa0005f6f27579ee4195a
[ Balint Reczey ]
* d/t/systemd-fsckd: Plymouth-start stays active in 20.10 and later
(LP: #1908067)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=e3ddd09301c8bdaa59b4fe54d7906f609552370d
-- Dan Streetman <ddstreet at canonical.com> Wed, 06 Jan 2021 15:40:39
-0500
** Changed in: systemd (Ubuntu Groovy)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1892358
Title:
autopkgtest success rate dropped inhibiting proposed migration
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Fix Released
Status in systemd source package in Focal:
Fix Released
Status in systemd source package in Groovy:
Fix Released
Bug description:
[impact]
autopkgtests are failing/flaky and prevent other packages from
migrating to -updates
[test case]
check autopkgtest history
[regression potential]
in regard to the changed test cases, any regression would likely
result in either an incorrectly passed test, or an incorrectly failed
test.
[scope]
for systemd, this is needed for x, b, and f.
tests in g appear to be mostly stable, but I've opened MR (linked from
this bug) to update the tests there as well.
i don't plan to update x, as it's reaching ESM in ~6 months, and
backporting the test fixes is more work than just a simple code copy,
since there are additional differences/changes needed in the older
version of systemd (and python3). the failing/flaky tests in x have
been like that forever, and people have just retried them; we can keep
retrying them until x moves into ESM next year.
[original description]
Hi,
we had such cases in the past like bug 1817721 for bionic and maybe bug 1892130 is about the same as well. There were more but I didn't want to search for all of them - what I checked is that there are no open ones clearly pointing out the recent further drop in already flaky subtests.
In particular the tests "tests-in-lxd" and "systemd-fsckd" were known
to be flaky before, but got even worse.
Here stats of the last 40 runs, it might be a coincidences that this
is after 246-2ubuntu1 landed. Could as well be any other change
groovy
amd64
tests-in-lxd (F 42% S 0% B 10% => P 45%/) FFFFBFFFFFFFB....FF.B.....F.....F...FBF
build-login (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
unit-config (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
networkd-testpy (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
boot-and-services (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
boot-smoke (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
logind (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
storage (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
upstream (F 35% S 0% B 10% => P 52%/) ..FFB.FFF.FFB....FF.B.....F.F..F....FBF
udev (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
systemd-fsckd (F 37% S 0% B 10% => P 50%/) FFFFBFFFFFFFB.FF...FB.....F..........B.
root-unittests (F 0% S 0% B 10% => P 87%/) ....B.......B.......B................B.
ppc64el
tests-in-lxd (F 25% S 0% B 0% => P 75%/) FFFF....FF............FFF.....F.........
systemd-fsckd (F 35% S 0% B 0% => P 65%/) FFFFFFF...FF........F....FF.F..F........
root-unittests (F 2% S 0% B 0% => P 97%/) ..............................F.........
s390x
tests-in-lxd (F 52% S 0% B 0% => P 47%/) FFFFFFF.FFFFFFF.FF.........FFFF...F.....
timedated (F 2% S 0% B 0% => P 97%/) ...........F............................
upstream (F 17% S 0% B 0% => P 82%/) .....F......F.F.............FFF...F.....
systemd-fsckd (F 32% S 0% B 0% => P 67%/) FFFFFFF..FF..F.................FF..F....
root-unittests (F 10% S 0% B 0% => P 90%/) ............................FFF...F.....
arm64
tests-in-lxd (F 40% S 0% B 2% => P 57%/) FFFFF.B...FFF.FF..F..F.........FFF.F....
logind (F 2% S 0% B 2% => P 95%/) ......B...................F.............
upstream (F 22% S 0% B 2% => P 75%/) ...F.FB.....F.F.............F..FFF.F....
root-unittests (F 12% S 0% B 2% => P 85%/) ......B.F...........F.F........F...F....
(I'm sure LP will make this unreadable, but is is nice in monospace)
Whatever the root cause is - the success rate of these has reduced so
much that the (even formerly questionable) practice of retry-until-
success won't work anymore.
I have run the two tests in a local VM and systemd-fsckd works there
while tests-in-lxd seems to trip over the old flaky fellow being
"boot-and-services".
We had the discussion in the past, but I think I need to again bring
up the suggestion to skip "tests-in-lxd" and "systemd-fsckd" until
they are on reasonable success rates.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1892358/+subscriptions
More information about the foundations-bugs
mailing list