[Bug 2100765] Re: Consistent DEP8/s390x failure: boot notice unexpectedly in journalctl logs

Fri Mar 14 09:24:39 UTC 2025

Hello Andreas, or anyone else affected,

Accepted rsyslog into oracular-proposed. The package will build now and
be available at
https://launchpad.net/ubuntu/+source/rsyslog/8.2406.0-1ubuntu2.1 in a
few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
oracular to verification-done-oracular. If it does not fix the bug for
you, please add a comment stating that, and change the tag to
verification-failed-oracular. In either case, without details of your
testing we will not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: rsyslog (Ubuntu Oracular)
       Status: New => Fix Committed

** Tags added: verification-needed verification-needed-oracular

** Changed in: rsyslog (Ubuntu Noble)
       Status: New => Fix Committed

** Tags added: verification-needed-noble

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/2100765

Title:
  Consistent DEP8/s390x failure: boot notice unexpectedly in journalctl
  logs

Status in rsyslog package in Ubuntu:
  Fix Released
Status in rsyslog source package in Noble:
  Fix Committed
Status in rsyslog source package in Oracular:
  Fix Committed

Bug description:
  [ Impact ]

  There is a consistent autopkgtest failure in s390x only, where the
  logcheck test fails because there is an unexpected boot notice in the
  output[1]:

  136s ** Running logcheck
  136s This email is sent by logcheck. If you no longer wish to receive
  136s such mail, you can either uninstall the logcheck package or modify
  136s its configuration file (/etc/logcheck/logcheck.conf).
  136s
  136s System Events
  136s =-=-=-=-=-=-=
  136s -- Boot f50e9bf0cfe04fdbb0e7815e3d03ce39 --
  136s
  136s * FAIL: unmatched lines - logcheck rules may need updating
  136s This email is sent by logcheck. If you no longer wish to receive
  136s such mail, you can either uninstall the logcheck package or modify
  136s its configuration file (/etc/logcheck/logcheck.conf).
  136s
  136s System Events
  136s =-=-=-=-=-=-=
  136s -- Boot f50e9bf0cfe04fdbb0e7815e3d03ce39 --
  136s
  137s autopkgtest [19:22:39]: test logcheck: -----------------------]
  137s logcheck             FAIL stderr: * FAIL: unmatched lines - logcheck rules may need updating

  At some point I thought it was related to the new apparmor rules that
  now allow reading some systemd files, but the test last worked, and
  started failing, with the same rsyslog version, so nothing changed in
  rsyslog itself. A migration-reference/0 would also fail, and then this
  failure would not block migration/release. But that also means the
  other, valuable, working, and important, tests would not be looked at.

  On its own, this does not warrant an SRU, but it's being bundled
  together with other fixes.

  [ Test Plan ]

  Since this is an autopkgtest fix, the test plan is that all
  autopkgtests must pass on all architectures, with no bad-test hint or
  migration-reference/0 help.

  [ Where problems could occur ]

   The fix is restricting the log lines that logcheck will analyze.
  Specifically, it's restricting them to the current system boot. If
  there were errors logged before the boot, logcheck will not see them.
  Note, however, that logcheck is already being fed just rsyslog
  messages from a particular timestamp that is AFTER the testbed
  rebooted (notice the journalctl filtering), so it's already a subset
  of all messages.

  [ Other Info ]

  This was only seen on s390x, and the root cause was not found.

  [ Original Description ]

  There is a consistent autopkgtest failure in s390x only, where the
  logcheck test fails because there is an unexpected boot notice in the
  output[1]:

  136s ** Running logcheck
  136s This email is sent by logcheck. If you no longer wish to receive
  136s such mail, you can either uninstall the logcheck package or modify
  136s its configuration file (/etc/logcheck/logcheck.conf).
  136s
  136s System Events
  136s =-=-=-=-=-=-=
  136s -- Boot f50e9bf0cfe04fdbb0e7815e3d03ce39 --
  136s
  136s * FAIL: unmatched lines - logcheck rules may need updating
  136s This email is sent by logcheck. If you no longer wish to receive
  136s such mail, you can either uninstall the logcheck package or modify
  136s its configuration file (/etc/logcheck/logcheck.conf).
  136s
  136s System Events
  136s =-=-=-=-=-=-=
  136s -- Boot f50e9bf0cfe04fdbb0e7815e3d03ce39 --
  136s
  137s autopkgtest [19:22:39]: test logcheck: -----------------------]
  137s logcheck             FAIL stderr: * FAIL: unmatched lines - logcheck rules may need updating

  At some point I thought it was related to the new apparmor rules that
  now allow reading some systemd files, but the test last worked, and
  started failing, with the same rsyslog version, so nothing changed in
  rsyslog itself.

  Last pass:
  8.2412.0-2ubuntu1	postgresql-common/274		2025-02-22 22:37:20 UTC	0h 28m 43s	-	pass	da643947-4f61-4567-97ff-930a77206d06	log   artifacts

  First failure after that:
  8.2412.0-2ubuntu1	systemd/257.3-1ubuntu2 glibc/2.41-1ubuntu1 php-twig/3.20.0-2 samba/2:4.21.4+dfsg-1ubuntu1 symfony/6.4.18+dfsg-1ubuntu1 tinysparql/3.8.2-3		2025-02-24 19:30:36 UTC	0h 10m 13s	-	fail	74381588-1934-40f9-b06a-831634f68bdb	log   artifacts

  It also fails with other triggers, simpler ones:
  8.2412.0-2ubuntu1	krb5/1.21.3-4ubuntu2		2025-02-26 03:56:06 UTC	0h 12m 30s	-	fail	90d5a082-37fe-4859-8044-4973c5d68202	log   artifacts

  And a migration-reference/0 run from a few days before also failed, with the same error:
  8.2412.0-2ubuntu1	migration-reference/0		2025-02-20 01:14:47 UTC	0h 10m 33s	-	fail	d7415807-95c5-4bc6-8fb2-0a4c3dc1b3f6	log   artifacts

  1. https://autopkgtest.ubuntu.com/results/autopkgtest-
  plucky/plucky/s390x/r/rsyslog/20250224_193036_105c8@/log.gz

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/2100765/+subscriptions