[Bug 1540407] Re: multipathd drops paths of a temporarily lost device
Launchpad Bug Tracker
1540407 at bugs.launchpad.net
Sun Mar 27 03:35:42 UTC 2016
This bug was fixed in the package multipath-tools -
0.5.0+git1.656f8865-5ubuntu1
---------------
multipath-tools (0.5.0+git1.656f8865-5ubuntu1) xenial; urgency=medium
* Merge from Debian. (LP: #1551952) (LP: #1540407)
Remaining changes:
- debian/control:
- Bump debhelper dependency to install udev rules to
/lib/udev/rules.d
- Bump udev dependencies as well
- debian/control:
- multipath-tools: add sg3-utils-dev Depends
- multipath-udeb: add sg3-udeb Depends
- debian/initramfs/hooks: use 95 not 60 for multipath rules priority
- debian/rules: Move udev rules to priority 95, because rules that load modules should be >90.
- debian/multipath-tools.preinst: modprobe dm-multipath.
This will make sure that multipathd will be able to start.
- debian/initramfs/local-top: wait for udev to settle before running
'multipath' in order to avoid race condition on device-mapper calls.
- debian/initramfs/local-top: remove '--timeout 10' which causes my
test system to not boot roughly 3 out of 4 times.
- Split kpartx initramfs bits into kpartx-boot for dmraid (LP: #941874)
- debian/initramfs/kpartx.hook
- debian/kpartx-boot.postinst
- debian/kpartx-boot.postrm
- debian/kpartx.install
- debian/control: Add kpartx-boot package for dmraid
- debian/rules: Install kpartx initramfs hook
- debian/patches/1000--set-umask-in-multipathd.patch: Set umask in multipathd.
- debian/patches/handle_spaces_in_rev_attr.patch: support IBM IPR devices
and others which may have only spaces for the rev attribute.
- debian/patches/path_selector.patch: switch the default path selector
back to round-robin while service-time isn't available to the installer
multipath-modules.
- debian/patches/0015-shared-lock-for-udev.patch: (LP: #1431650)
- debian/initramfs/hooks: also copy wwids file on the installed system to
ensure all paths come up on boot. (LP: #1479929)
- Disable -fexceptions on multipath-udeb (LP: #1489379): the flag causes
libchecktur.so to link with libgcc_s.so.1 (even with -static-libgcc),
which is not available in the installer environment.
- debian/patches/disable-fexceptions-udeb.patch: conditionally disable
-fexceptions with CFLAGS_DISABLE_FEXCEPTIONS.
- debian/rules: set CFLAGS_DISABLE_FEXCEPTIONS to build multipath-udeb.
- debian/patches/handle_spaces_in_rev_attr.patch: update patch to apply the
change to the right line (LP: #1492425)
- debian/initramfs/local-premount: wait for udev to settle before the call
to resolve_device() in local_mount_root(), so the by-uuid/ symlinks have
a chance to be updated by the multipath udev rules (LP: #1503286).
- debian/multipath-tools.postinst: handle upgrades from < 0.5.0 by migrating
from the old device names with device numbers to using letters for devices.
- debian/patches/mpath_name_migration.patch: ship a multipath_migrate binary
to make translation from pre-0.5.0 device naming to the new scheme.
- debian/initramfs/hooks: install multipathd and required directories.
- debian/initramfs/local-premount: reload all maps to make sure they're
indeed loaded and ready before we end premount.
- debian/initramfs/local-top: run multipathd rather than a one-off call to
multipath so that new paths can be correctly added as detected while we're
still in the initramfs.
- debian/initramfs/local-bottom: remember to stop multipathd.
- debian/rules: install debian/initramfs/local-bottom
- debian/patches/0053-readonly-bindings_multipathd_prod.patch: support -B to
allow multipathd to handle cases where the bindings file is read-only.
(LP: #1526984)
- debian/patches/git-kpartx-support-spaces-in-dev-names-b407050a.patch: deal
with spaces in device names in kpartx too (LP: #1432062)
- debian/tests/kpartx-file-loopback: add an autopkgtest to catch future cases
where uploads might break kpartx's loopback file handling.
- debian/patches/kpartx_more_loopback_fixes.patch: fix loopback mounted
files some more: since we stat() the loopback device node, we can't rely
on S_ISREG() tests to handle this case, and should look at the device
itself instead. (LP: #1543430)
- Remove partition device nodes of individual paths (for LVM on multipath)
(LP: #1540401)
- debian/multipath-tools.dm-mpath-lvm.udev: udev rule for that.
- debian/initramfs/hooks: copy the udev rule and partx to the initramfs.
- debian/initramfs/init-top: load the dm-multipath module for 'multipath -c'.
- debian/rules: install the udev rule (priority 56: after 55-scsi-sg3_id)
- debian/rules: don't ship 95-multipath.rules udev rules anymore; they are
not necessary with multipath-tools listening for udev events directly.
* Drop changes:
- debian/patches/series: Drop obsolete patches already included upstream
- debian/patches/ignore_devices_nodename_fail.patch
- debian/patches/skip_usb_devices.patch
- debian/patches/0014-kpartx-long-path.patch
- debian/patches/0052-readonly-bindings_multipathd.patch
- debian/patches/add_find-multipaths.patch
- debian/patches/dm-multipath-backlist-nvme-5c412e47.patch
-- Ryan Harper <ryan.harper at canonical.com> Wed, 09 Mar 2016 10:21:26
-0600
** Changed in: multipath-tools (Ubuntu)
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1540407
Title:
multipathd drops paths of a temporarily lost device
Status in multipath-tools package in Ubuntu:
Fix Released
Bug description:
== Comment: #0 - Thorsten Diehl <thorsten.diehl at de.ibm.com> - 2016-02-01 08:57:28 ==
# uname -a
Linux s83lp31 4.4.0-1-generic #15-Ubuntu SMP Thu Jan 21 22:19:04 UTC 2016 s390x s390x s390x GNU/Linux
# dpkg -s multipath-tools|grep ^Version:
Version: 0.5.0-7ubuntu9
# cat /etc/multipath.conf
defaults {
default_features "1 queue_if_no_path"
user_friendly_names yes
path_grouping_policy multibus
dev_loss_tmo 2147483647
fast_io_fail_tmo 5
}
blacklist {
devnode '*'
}
blacklist_exceptions {
devnode "^sd[a-z]+"
}
---------------------------------------
On a z Systems LPAR with a single LUN, 2 zfcp devices, 2 storage ports, and the following multipath topology:
mpatha (36005076304ffc3e80000000000003050) dm-0 IBM,2107900
size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 0:0:0:1079001136 sda 8:0 active ready running
|- 0:0:1:1079001136 sdb 8:16 active ready running
|- 1:0:0:1079001136 sdc 8:32 active ready running
`- 1:0:1:1079001136 sdd 8:48 active ready running
I observed the following:
When I deconfigure one of the two zfcp devices (e.g. via chchp -c 0, or directly on the HMC), the multipathd removes the two paths via these devices from the pathgroup after 10 seconds. When the zfcp devices comes back, it runs through zfcp error recovery and is being set up properly, and also the mid layer objects are looking fine. However, the multipathd does not add them to the path group again.
Expected behaviour: multipathd does not remove the paths from topology
list, but holds them as "failed faulty offline" until dev_loss_tmo
timout is reached (which is infinite here).
I discussed this already with zfcp development, and it looks most
likely as a problem with multipathd, rather than zfcp or mid-layer.
Easy to reproduce: you need two zfcp devices, one LUN, and preferably
two ports on the storage server (WWPNs). Configure LUN via 2 zfcp
devices * 2 WWPNs = 4 paths.
This can be also reproduced on a z/VM guest. Instead of configuing the
CHPID off, just detach one zfcp device and re-attach it after 30....60
seconds. Same problem.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1540407/+subscriptions
More information about the foundations-bugs
mailing list