[Bug 1829829] Re: Ubuntu CI has been flaky for a week
Dan Streetman
dan.streetman at canonical.com
Fri Jun 28 15:08:45 UTC 2019
> right now the upstream test timeouts on i386
ah, ok - so I did look into this, or at least some failures really
similar to this, a while back, for bug 1831468. As far as I could tell,
the problem in those cases was that the test was starting a service and
waiting for notification of its completion; but for some reason, dbus
seemed to be crashing/dying/exiting, and the notification of the
completed service never actually reached the test suite, so it just
waited forever until timeout. I do see in the logs for the test you
referenced:
[0;38;5;245mUnable to add match type='signal',sender='org.freedesktop.DBus',path='/org/freedesktop/DBus',interface='org.freedesktop.DBus',member='NameOwnerChanged',arg0=':1.7', failing connection: Method call timed out[0m
[0;38;5;245mBus bus-api-system: changing state RUNNING → CLOSING[0m
[0;38;5;245mUnable to add match type='signal',sender='org.freedesktop.DBus',path='/org/freedesktop/DBus',interface='org.freedesktop.DBus',member='NameOwnerChanged',arg0=':1.7', failing connection: Connection terminated[0m
[0;38;5;245mBus bus-api-system: changing state CLOSING → CLOSED[0m
[0;38;5;245msystemd-rfkill.socket: Changed dead -> listening[0m
[0;38;5;245msystemd-rfkill.socket: Job 364 systemd-rfkill.socket/start finished, result=done[0m
[[0;32m OK [0m] Listening on [0;1;39mLoad/Save RF …itch Status /dev/rfkill Watch[0m.
[0;38;5;245mBus bus-api-system: changing state UNSET → OPENING[0m
[0;38;5;245mBus bus-api-system: changing state OPENING → AUTHENTICATING[0m
that's quite similar to what i saw before. I never had a chance to get
farther than that, but I can try to look more into it soon. Let me know
if that sounds familiar or if you have any debugging suggestions.
> Plus, in https://github.com/systemd/systemd/pull/12861#issuecomment-506556062 the
> test is completely broken on Ubuntu CI and I have no idea why.
ugh, yeah TEST-15 and TEST-22 aren't giving any clues at all in the log
to why they failed.
> As far as I can tell, it has been abandoned since then. If you're ready to keep
> the list up to date by visiting PRs where Ubuntu CI fails I'm of course all for it.
Let's add TEST-15 and TEST-22 for now, and I'll see if I can both keep
up with other flaky tests, and also try to figure out why they're being
flaky in ubuntu-ci.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1829829
Title:
Ubuntu CI has been flaky for a week
Status in systemd package in Ubuntu:
Confirmed
Bug description:
It was originally reported in https://github.com/systemd/systemd/pull/12583#issuecomment-492949206 5 days ago. To judge from the logs VMs can't be rebooted there:
```
Ubuntu 18.04.2 LTS autopkgtest ttyS0
autopkgtest login:
---------------------------------------------------
------- nova show 91e76a78-d05c-412a-b383-55a26010ae69 (adt-bionic-amd64-systemd-upstream-20190516-051604) ------
+--------------------------------------+------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | euler |
| OS-EXT-SRV-ATTR:hypervisor_hostname | euler.lcy01.scalingstack |
| OS-EXT-SRV-ATTR:instance_name | instance-003d216a |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2019-05-16T07:00:42.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2019-05-16T07:00:33Z |
| flavor | autopkgtest (f878e70e-9991-46e0-ba02-8ea159a71656) |
| hostId | 1722c5f2face86c3fc9f338ae96835924721512372342f664e6941bd |
| id | 91e76a78-d05c-412a-b383-55a26010ae69 |
| image | adt/ubuntu-bionic-amd64-server-20190516.img (d00bf12c-467e-433f-a4f5-15720f13bff1) |
| key_name | testbed-juju-prod-ues-proposed-migration-machine-11 |
| metadata | {} |
| name | adt-bionic-amd64-systemd-upstream-20190516-051604 |
| net_ues_proposed_migration network | 10.42.40.13 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | autopkgtest at lcy01-27.secgroup |
| status | ACTIVE |
| tenant_id | afaef86b96dd4828a1ed5ee395ea1421 |
| updated | 2019-05-16T07:00:42Z |
| user_id | 8524250971084851b3792a68fbc398dd |
+--------------------------------------+------------------------------------------------------------------------------------+
---------------------------------------------------
<VirtSubproc>: failure: Timed out on waiting for ssh connection
autopkgtest [07:07:45]: ERROR: testbed failure: cannot send to testbed: [Errno 32] Broken pipe
```
Though judging by https://github.com/systemd/systemd/pull/12626, it
appears that sometimes the tests pass.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1829829/+subscriptions
More information about the foundations-bugs
mailing list