[Bug 1987387] [NEW] [UBUNTU 20.04] zgetdump can not handle multivolume dumps

Launchpad Bug Tracker 1987387 at bugs.launchpad.net
Sun Aug 28 20:22:12 UTC 2022 

You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

SRU Justification:
==================

[Impact]

 * The zgetdump tool (as part of the current s390-tools version in focal)
   is not able to handle multi-volume dumps (DASD disk) dumps.

 * While this is rarely needed, it is extremely annoying if one is
   in usually urgent need to use it and it does not work.

 * On s390x systems multi-volume (DASD disk) dumps are pretty common,
   due to usually costly and therefore limited disk resources,
   DASD disks are usually relatively small.

[Fix]

 * d55b787 d55b787d05eb9bd70f93c36cf859b66b2ad02038 "zgetdump: Fix
device node determination via sysfs"

[Test Plan]

 * Have an IBM zSystems LPAR with Ubuntu 20.04 / focal (latest).

 * Have two (or more) additional DASD disks reserved for storing dumps,
   assigned to the system and enabled:
   sudo chzdev -e 0002, 0003
   (Assuming 0001 has Ubuntu 20.04 installed - all 3390 disks).
   Get the block device names using 'lsdasd' - assuming here dasdb, dasdc

 * If needed low-level format these disks:
   sudo dasdfmt -y -b 4096 /dev/dasdb
   sudo dasdfmt -y -b 4096 /dev/dasdc

 * Create one single partition per disk:
   fdisk -a /dev/dasdb
   fdisk -a /dev/dasdc

 * Create an mvdump.conf file that points to the above disks
   sudo vi /etc/mvdump.conf
   ...
   cat /etc/mvdump.conf
   /dev/dasdb1
   /dev/dasdc1

 * Re-write the zipl boot record like this:
   sudo zipl -n -M /etc/mvdump.conf

 * Now ipl the system from the first dump DASD: 0002
   and initiate the DASD dump by:
   1. Stop all CPUs.
   2. Store status on the IPL CPU.
   3. IPL the dump tool on the IPL CPU.

 * Wait until the dump is completed and re-ipl the Ubuntu 20.04 again
(0001).

 * Without this fix one will see a message like this:
   zgetdump -i /dev/dasdb
   zgetdump: Could not open "/sys/bus/ccw/devices/0.0.0002/dasdb/dev" (No such file or directory)

 * With the fix one will see a message like this:
   zgetdump -i /dev/dasdb
   General dump info:
     Dump format........: s390mv_ext
     Version............: 1
     Dump created.......: 
     ...
     Dump device info:
       Volume 0: 0.0.0002 (online/active)
       Volume 1: 0.0.0003 (online/valid)

 * For more in-depth details see the
   'Linux on System z. Using the Dump Tools.' documentation:
   https://www.ibm.com/docs/en/linux-on-systems?topic=dump-hmc-se-example

[Where problems could occur]

 * A new function got introduced to 'check whether a device with a given
   busid is online'.
   Issues could occur in case this function is broken and 
   checks for a wrong busid, has a wrong path
   or handled the status wrongly.

 * The kind of 'little' refactoring of that patch may lead to
   further unexpected issues (that can largely identified by a test build).

 * The additional use of libutil functions may cause issues
   in case of an outdated libutil that does not offer all needed functions.
   (Testable with a test build.)

 * Erroneous code may even break single volume dumps

[Other Info]
 
 * This code is known to work with hirsute and newer Ubuntu releases,
   esp. jammy (respectively their s390-tools versions).

 * The upstream code can be cleanly cherry-picked,
   hence is applied as-is.

 * An updated s390-tools version 2.12.0-0ubuntu3.7 with a
   patched zgetdump tool was build and made available via this PPA:
   https://launchpad.net/~fheimes/+archive/ubuntu/lp1987387
   and was already successfully tested!
__________

== Comment: #0 - Thorsten Diehl <thorsten.diehl at de.ibm.com> - 2022-08-16 12:40:46 ==
I installed Ubuntu 20.04.4 LTS on IBM z14, enabled two DASDs, created one partition on each DASD and created an mvdump.conf file like this:
/dev/dasdc1
/dev/dasdd1

I wrote the boot record by zipl -n -M mvdump.conf and IPLed the system from dasdc devno.
The dump completed succesfully.
Then I tried to get this dump via zgetdump on the restarted Ubuntu 20.04.4 system (zgetdump -v reports version 2.12.0-build-20220506), I got the following error:
root at m8330032:~# zgetdump -i /dev/dasdc
zgetdump: Could not open "/sys/bus/ccw/devices/0.0.9405/dasdc/dev" (No such file or directory)
root at m8330032:~# zgetdump -i /dev/dasdc1
zgetdump: Could not open "/sys/bus/ccw/devices/0.0.9405/dasdc/dev" (No such file or directory)
root at m8330032:~# zgetdump -i /dev/dasdd
zgetdump: Could not open "/sys/bus/ccw/devices/0.0.9405/dasdc/dev" (No such file or directory)
root at m8330032:~# zgetdump -i /dev/dasdd1
zgetdump: No valid dump found on "/dev/dasdd1"
root at m8330032:~#

However, If I'm doing the same zgetdump on another system (e.g. with newer s390-tools version), I get the expected result
m83lp32:~ # zgetdump -i /dev/dasdc
General dump info:
  Dump format........: s390mv_ext
  Version............: 1
  Dump created.......: Tue, 16 Aug 2022 18:31:57 +0200
  Dump ended.........: Tue, 16 Aug 2022 18:32:02 +0200
  Dump CPU ID........: ff1fa1e739068000
  UTS node name......: m8330032.lnxne.boe
  UTS kernel release.: 5.4.0-124-generic
  UTS kernel version.: #140-Ubuntu SMP Thu Aug 4 02:23:07 UTC 2022
  Build arch.........: s390x (64 bit)
  System arch........: s390x (64 bit)
  CPU count (online).: 16
  CPU count (real)...: 16
  Dump memory range..: 4096 MB
  Real memory range..: 4096 MB
  Dump file size.....: 849 MB

Memory map:
  0000000000000000 - 00000000ffffffff (4096 MB)

Dump device info:
  Volume 0: 0.0.9405 (online/active)
  Volume 1: 0.0.9406 (online/valid)
m83lp32:~ #

The error is easily reproducible.
Please update zgetdump to a newer version to solve this RAS problem.

With Jammy (22.04.1; s390-tools version 2.20.0-build-20220623) this
problem does not occur.

== Comment: #3 - Jan Hoeppner <Jan.Hoeppner at de.ibm.com> - 2022-08-22 01:43:45 ==
There were several issues fixed in s390-tools v2.15.1 in regards to multivolume dumps: https://github.com/ibm-s390-linux/s390-tools/releases/tag/v2.15.1

Especially the following upstream commit for zgetdump:
https://github.com/ibm-s390-linux/s390-tools/commit/d55b787d05eb9bd70f93c36cf859b66b2ad02038

** Affects: ubuntu-z-systems
     Importance: Medium
     Assignee: Skipper Bug Screeners (skipper-screen-team)
         Status: In Progress

** Affects: s390-tools (Ubuntu)
     Importance: Medium
         Status: In Progress


** Tags: architecture-s39064 bugnameltc-199411 patch severity-medium targetmilestone-inin2004
-- 
[UBUNTU 20.04] zgetdump can not handle multivolume dumps
https://bugs.launchpad.net/bugs/1987387
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.

More information about the Ubuntu-sponsors mailing list