[Bug 1935617] Re: systemd autopkgtest broken on ppc64el with qemu 6.0

Christian Ehrhardt  1935617 at bugs.launchpad.net
Mon Jul 12 15:48:46 UTC 2021 

I've collected the instructions that are around the failures and it is suspicious for
 lbzx / lvx / ld
All failures are around those:

Impish

ubuntu at i-systemdtest:/var/crash$ for i in */CoreDump; do p=$(dirname $i | tr -d '.0123456789' | tr '_' '/' ); gdb $p $i -q -ex "x/-10i \$pc" -ex "x/3i \$pc" -ex "q"; done
Reading symbols from /usr/bin/chgrp...
Reading symbols from /usr/lib/debug/.build-id/27/193823a3e38fa68e68ecfcb8bd98b2cb41d617.debug...
[New LWP 855]
Core was generated by `/bin/chgrp adm /var/log/dmesg '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000e0cb28f5190 in quotearg_buffer_restyled (buffer=buffer at entry=0xe0cb2920130 <slot0> "'", buffersize=buffersize at entry=256, 
    arg=arg at entry=0x606513d00c0e0000 <error: Cannot access memory at address 0x606513d00c0e0000>, argsize=argsize at entry=18446744073709551615, quoting_style=shell_always_quoting_style, 
    flags=flags at entry=1, quote_these_too=quote_these_too at entry=0x7fffd194aca0, left_quote=<optimized out>, right_quote=<optimized out>) at lib/quotearg.c:400
400	lib/quotearg.c: No such file or directory.
   0xe0cb28f5168 <quotearg_buffer_restyled+1256>:	cmpdi   r28,0
   0xe0cb28f516c <quotearg_buffer_restyled+1260>:	cmpdi   cr3,r22,-1
   0xe0cb28f5170 <quotearg_buffer_restyled+1264>:	addi    r9,r31,1
   0xe0cb28f5174 <quotearg_buffer_restyled+1268>:	addi    r29,r29,1
   0xe0cb28f5178 <quotearg_buffer_restyled+1272>:	cmpld   cr7,r9,r16
   0xe0cb28f517c <quotearg_buffer_restyled+1276>:	addi    r31,r9,1
   0xe0cb28f5180 <quotearg_buffer_restyled+1280>:	blt     cr7,0xe0cb28f51b0 <quotearg_buffer_restyled+1328>
   0xe0cb28f5184 <quotearg_buffer_restyled+1284>:	bne     0xe0cb28f4eb0 <quotearg_buffer_restyled+560>
   0xe0cb28f5188 <quotearg_buffer_restyled+1288>:	li      r24,0
   0xe0cb28f518c <quotearg_buffer_restyled+1292>:	bne     cr3,0xe0cb28f4eb4 <quotearg_buffer_restyled+564>
=> 0xe0cb28f5190 <quotearg_buffer_restyled+1296>:	lbzx    r30,r23,r29
   0xe0cb28f5194 <quotearg_buffer_restyled+1300>:	cntlzw  r30,r30
   0xe0cb28f5198 <quotearg_buffer_restyled+1304>:	rlwinm  r30,r30,27,5,31
Reading symbols from /usr/bin/chmod...
Reading symbols from /usr/lib/debug/.build-id/4d/106a49cb4c9f6f8f4ab6db8ce5930efdec021e.debug...

warning: Can't open file /usr/lib/powerpc64le-linux-gnu/libc-2.31.so
during file-backed mapping note processing

warning: Can't open file /usr/lib/powerpc64le-linux-gnu/ld-2.31.so during file-backed mapping note processing
[New LWP 2300]
Core was generated by `chmod 0711 /var/snap/lxd/common/lxd '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000147684c5050 in ?? ()
No line number information available for address 0x147684c504f
=> 0x147684c5050:	lbzx    r30,r24,r29
   0x147684c5054:	cntlzw  r30,r30
   0x147684c5058:	rlwinm  r30,r30,27,5,31
Reading symbols from /usr/bin/find...
Reading symbols from /usr/lib/debug/.build-id/3c/c37af50368782eb631f8c451e0830803c5e669.debug...
[New LWP 3876]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
Core was generated by `find /var/lib/update-notifier/updates-available -newermt now-7 days '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __strlen_power9 () at ../sysdeps/powerpc/powerpc64/le/power9/strlen.S:44
44	../sysdeps/powerpc/powerpc64/le/power9/strlen.S: No such file or directory.
No line number information available for address 0x7b70090875bf
   0x7b70090875c0 <__strlen_power9>:	vspltisb v18,0
   0x7b70090875c4 <__strlen_power9+4>:	vspltisb v19,-1
   0x7b70090875c8 <__strlen_power9+8>:	neg     r5,r3
   0x7b70090875cc <__strlen_power9+12>:	clrldi  r9,r5,60
=> 0x7b70090875d0 <__strlen_power9+16>:	lvx     v0,0,r3
   0x7b70090875d4 <__strlen_power9+20>:	lvsr    v1,0,r3
   0x7b70090875d8 <__strlen_power9+24>:	vperm   v0,v19,v0,v1
Reading symbols from /usr/bin/rm...
Reading symbols from /usr/lib/debug/.build-id/57/6e94c66b0b81cf99c708ab49e3db8e9f32c036.debug...
[New LWP 3942]
Core was generated by `rm -f /var/lib/update-notifier/tmp.2y4Fd9GVp5 '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000798b9f361d0 in quotearg_buffer_restyled (buffer=buffer at entry=0x798b9f60130 <slot0> "'", buffersize=buffersize at entry=256, 
    arg=arg at entry=0x800369e198070000 <error: Cannot access memory at address 0x800369e198070000>, argsize=argsize at entry=18446744073709551615, quoting_style=shell_always_quoting_style, 
    flags=flags at entry=1, quote_these_too=quote_these_too at entry=0x7fffcf48d7c0, left_quote=<optimized out>, right_quote=<optimized out>) at lib/quotearg.c:400
400	lib/quotearg.c: No such file or directory.
   0x798b9f361a8 <quotearg_buffer_restyled+1256>:	cmpdi   r28,0
   0x798b9f361ac <quotearg_buffer_restyled+1260>:	cmpdi   cr3,r22,-1
   0x798b9f361b0 <quotearg_buffer_restyled+1264>:	addi    r9,r31,1
   0x798b9f361b4 <quotearg_buffer_restyled+1268>:	addi    r29,r29,1
   0x798b9f361b8 <quotearg_buffer_restyled+1272>:	cmpld   cr7,r9,r16
   0x798b9f361bc <quotearg_buffer_restyled+1276>:	addi    r31,r9,1
   0x798b9f361c0 <quotearg_buffer_restyled+1280>:	blt     cr7,0x798b9f361f0 <quotearg_buffer_restyled+1328>
   0x798b9f361c4 <quotearg_buffer_restyled+1284>:	bne     0x798b9f35ef0 <quotearg_buffer_restyled+560>
   0x798b9f361c8 <quotearg_buffer_restyled+1288>:	li      r24,0
   0x798b9f361cc <quotearg_buffer_restyled+1292>:	bne     cr3,0x798b9f35ef4 <quotearg_buffer_restyled+564>
=> 0x798b9f361d0 <quotearg_buffer_restyled+1296>:	lbzx    r30,r23,r29
   0x798b9f361d4 <quotearg_buffer_restyled+1300>:	cntlzw  r30,r30
   0x798b9f361d8 <quotearg_buffer_restyled+1304>:	rlwinm  r30,r30,27,5,31
Reading symbols from /usr/bin/rm...
Reading symbols from /usr/lib/debug/.build-id/57/6e94c66b0b81cf99c708ab49e3db8e9f32c036.debug...
[New LWP 4024]
Core was generated by `rm /foo '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000007d8753761d0 in quotearg_buffer_restyled (buffer=buffer at entry=0x7d8753a0130 <slot0> "'", buffersize=buffersize at entry=256, 
    arg=arg at entry=0x80410981d8070000 <error: Cannot access memory at address 0x80410981d8070000>, argsize=argsize at entry=18446744073709551615, quoting_style=shell_always_quoting_style, 
    flags=flags at entry=1, quote_these_too=quote_these_too at entry=0x7fffef260f50, left_quote=<optimized out>, right_quote=<optimized out>) at lib/quotearg.c:400
400	lib/quotearg.c: No such file or directory.
   0x7d8753761a8 <quotearg_buffer_restyled+1256>:	cmpdi   r28,0
   0x7d8753761ac <quotearg_buffer_restyled+1260>:	cmpdi   cr3,r22,-1
   0x7d8753761b0 <quotearg_buffer_restyled+1264>:	addi    r9,r31,1
   0x7d8753761b4 <quotearg_buffer_restyled+1268>:	addi    r29,r29,1
   0x7d8753761b8 <quotearg_buffer_restyled+1272>:	cmpld   cr7,r9,r16
   0x7d8753761bc <quotearg_buffer_restyled+1276>:	addi    r31,r9,1
   0x7d8753761c0 <quotearg_buffer_restyled+1280>:	blt     cr7,0x7d8753761f0 <quotearg_buffer_restyled+1328>
   0x7d8753761c4 <quotearg_buffer_restyled+1284>:	bne     0x7d875375ef0 <quotearg_buffer_restyled+560>
   0x7d8753761c8 <quotearg_buffer_restyled+1288>:	li      r24,0
   0x7d8753761cc <quotearg_buffer_restyled+1292>:	bne     cr3,0x7d875375ef4 <quotearg_buffer_restyled+564>
=> 0x7d8753761d0 <quotearg_buffer_restyled+1296>:	lbzx    r30,r23,r29
   0x7d8753761d4 <quotearg_buffer_restyled+1300>:	cntlzw  r30,r30
   0x7d8753761d8 <quotearg_buffer_restyled+1304>:	rlwinm  r30,r30,27,5,31
Reading symbols from /usr/bin/rm...
Reading symbols from /usr/lib/debug/.build-id/57/6e94c66b0b81cf99c708ab49e3db8e9f32c036.debug...
[New LWP 906]
Core was generated by `rm -rf /tmp/pollinate.tzqlorZOCM3X '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  last_component (name=0x804170e3450d0000 <error: Cannot access memory at address 0x804170e3450d0000>) at lib/basename-lgpl.c:36
36	lib/basename-lgpl.c: No such file or directory.
No line number information available for address 0xd45ade2549f
=> 0xd45ade254a0 <last_component>:	lbz     r9,0(r3)
   0xd45ade254a4 <last_component+4>:	cmplwi  r9,47
   0xd45ade254a8 <last_component+8>:	bne     0xd45ade254bc <last_component+28>


Focal

$ for i in */CoreDump; do p=$(dirname $i | tr -d '.0123456789' | tr '_' '/' ); gdb $p $i -q -ex "x/-10i \$pc" -ex "x/3i \$pc" -ex "q"; done
Reading symbols from /usr/bin/find...
Reading symbols from /usr/lib/debug/.build-id/67/c0d4df7aff2f7c3113a270e0839de23b1942ff.debug...
[New LWP 997]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
Core was generated by `find /var/lib/update-notifier/updates-available -newermt now-7 days '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __strlen_power8 () at ../sysdeps/powerpc/powerpc64/power8/strlen.S:36
36	../sysdeps/powerpc/powerpc64/power8/strlen.S: No such file or directory.
No line number information available for address 0x7e0f566e1cdf
   0x7e0f566e1ce0 <__strlen_power8>:	dcbt    0,r3
   0x7e0f566e1ce4 <__strlen_power8+4>:	rldicr  r4,r3,0,60
   0x7e0f566e1ce8 <__strlen_power8+8>:	rlwinm  r6,r3,3,26,28
   0x7e0f566e1cec <__strlen_power8+12>:	li      r0,0
   0x7e0f566e1cf0 <__strlen_power8+16>:	li      r5,-1
=> 0x7e0f566e1cf4 <__strlen_power8+20>:	ld      r12,0(r4)
   0x7e0f566e1cf8 <__strlen_power8+24>:	sld     r5,r5,r6
   0x7e0f566e1cfc <__strlen_power8+28>:	orc     r9,r12,r5
Reading symbols from /usr/bin/rm...
Reading symbols from /usr/lib/debug/.build-id/52/ef9ac8e7ffd3c14fe97d9efcd463c65318d6d7.debug...
[New LWP 1107]
Core was generated by `rm -f /var/lib/update-notifier/tmp.7RITiEiwnq '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000727ed5063b0 in quotearg_buffer_restyled (buffer=buffer at entry=0x727ed530130 <slot0> "'", buffersize=buffersize at entry=256, 
    arg=arg at entry=0x800314f727070000 <error: Cannot access memory at address 0x800314f727070000>, argsize=argsize at entry=18446744073709551615, quoting_style=shell_always_quoting_style, 
    flags=flags at entry=1, quote_these_too=quote_these_too at entry=0x7fffdbb42770, left_quote=0x0, right_quote=0x0) at lib/quotearg.c:400
400	lib/quotearg.c: No such file or directory.
   0x727ed506388 <quotearg_buffer_restyled+1160>:	addi    r9,r31,1
   0x727ed50638c <quotearg_buffer_restyled+1164>:	addi    r29,r29,1
   0x727ed506390 <quotearg_buffer_restyled+1168>:	addi    r31,r31,2
   0x727ed506394 <quotearg_buffer_restyled+1172>:	cmpld   cr7,r9,r27
   0x727ed506398 <quotearg_buffer_restyled+1176>:	blt     cr7,0x727ed506440 <quotearg_buffer_restyled+1344>
   0x727ed50639c <quotearg_buffer_restyled+1180>:	bne     0x727ed506448 <quotearg_buffer_restyled+1352>
   0x727ed5063a0 <quotearg_buffer_restyled+1184>:	cmpdi   r22,-1
   0x727ed5063a4 <quotearg_buffer_restyled+1188>:	li      r9,0
   0x727ed5063a8 <quotearg_buffer_restyled+1192>:	std     r9,112(r1)
   0x727ed5063ac <quotearg_buffer_restyled+1196>:	bne     0x727ed5060c4 <quotearg_buffer_restyled+452>
=> 0x727ed5063b0 <quotearg_buffer_restyled+1200>:	lbzx    r30,r24,r29
   0x727ed5063b4 <quotearg_buffer_restyled+1204>:	cntlzw  r30,r30
   0x727ed5063b8 <quotearg_buffer_restyled+1208>:	rlwinm  r30,r30,27,5,31
Reading symbols from /usr/bin/rm...
Reading symbols from /usr/lib/debug/.build-id/52/ef9ac8e7ffd3c14fe97d9efcd463c65318d6d7.debug...
[New LWP 838]
Core was generated by `rm -rf /tmp/pollinate.8UMuSzjLatRW '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  last_component (name=0x8041f076230e0000 <error: Cannot access memory at address 0x8041f076230e0000>) at lib/basename-lgpl.c:36
36	lib/basename-lgpl.c: No such file or directory.
No line number information available for address 0xe234bc956df
=> 0xe234bc956e0 <last_component>:	lbz     r9,0(r3)
   0xe234bc956e4 <last_component+4>:	cmplwi  r9,47
   0xe234bc956e8 <last_component+8>:	bne     0xe234bc956fc <last_component+28>

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1935617

Title:
  systemd autopkgtest broken on ppc64el with qemu 6.0

Status in qemu package in Ubuntu:
  New
Status in systemd package in Ubuntu:
  New

Bug description:
  I'm not sure yet if this is flaky or a real issue, but I'm filing it
  to avoid multiple people analyzing the same.

  The Qemu 6.0 upload https://launchpad.net/ubuntu/+source/qemu/1:6.0+dfsg-1~ubuntu2 triggers a test failure like
  https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/ppc64el/s/systemd/20210708_223311_e3bbb@/log.gz

  I have tested the new qemu on ppc64 and it worked fine for device emulation and migration cases.
  But this is suspicious.
  Of the last tests exactly and only those with the new qemu failed.

  
  impish
    ppc64el
      tests-in-lxd                   (F  5% f  0% S  0% B  0% => P 95%/) F.........F.............................
      systemd-fsckd                  (F  0% f  0% S 100% B  0% => P  0%/) SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
      upstream-1                     (F 15% f  0% S  0% B  0% => P 85%/) F..FFFF....F............................
      upstream-2                     (F 12% f  0% S  0% B  0% => P 87%/) F..FFFF.................................

  
  For an insight in flakyness/reproducibility I've retriggered the missing qemu and the non-qemu cases a few times. If those reproduce all-bad vs all-good again this would further indicate a real issue.

  Unfortunately the ppc maas seems down right now and canonistack also
  isn't too nice this week - overall that inhibits the testing a bit :-/

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1935617/+subscriptions

More information about the foundations-bugs mailing list