[Bug 1020436] Re: Cannot read superblock after FC multipath failover

[Steve Fisher]

It looks like all the /dev/mapper/mpathX targets, and the remaining 'sd'
unique paths (with one fabric disabled) were all readable after the path
shutdown, but the underlying LVs were somehow still deactivated when the
paths disappeared.

root@<hostname>-03:/srv/mysql# dmesg
<snip>
[1039812.298161] I/O error in filesystem ("dm-4") meta-data dev dm-4 block 0x1f414af0       ("xlog_iodone") error 5 buf count 3072
[1039812.311419] xfs_force_shutdown(dm-4,0x2) called from line 1043 of file /build/buildd/linux-2.6.32/fs/xfs/xfs_log.c.  Return address = 0xffffffffa0162516
[1039812.311433] Filesystem "dm-4": Log I/O Error Detected.  Shutting down filesystem: dm-4
[1039812.326038] Please umount the filesystem, and rectify the problem(s)

root@<hostname>:/srv/mysql# ls -l
ls: cannot open directory .: Input/output error

root@<hostname>:/srv/mysql#  lvs -o lv_attr
  /dev/test2/lvol0: read failed after 0 of 2048 at 0: Input/output error
  /dev/test/lvol0: read failed after 0 of 4096 at 0: Input/output error
  /dev/mvsan/san: read failed after 0 of 2048 at 0: Input/output error
  Attr  
  -wi-ao
  -wi-ao
  -wi-ao
  -wi-ao
  -wi-ao
root@<hostname>:/srv/mysql# dd if=/dev/mapper/mpath
mpath0  mpath1  mpath2  
root@<hostname>:/srv/mysql# dd if=/dev/mapper/mpath0 of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 0.0191405 s, 267 kB/s
root@<hostname>:/srv/mysql# dd if=/dev/mapper/mpath1 of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 0.00693051 s, 739 kB/s
root@<hostname>:/srv/mysql# dd if=/dev/mapper/mpath2 of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 0.00559024 s, 916 kB/s

-----

root@<hostname>:/srv/mysql# ls -l /dev/sd*
brw-rw---- 1 root disk 8,  0 2012-07-11 14:24 /dev/sda
brw-rw---- 1 root disk 8,  1 2012-07-11 14:24 /dev/sda1
brw-rw---- 1 root disk 8, 16 2012-07-11 14:24 /dev/sdb
brw-rw---- 1 root disk 8, 32 2012-07-11 14:24 /dev/sdc
brw-rw---- 1 root disk 8, 48 2012-07-11 14:24 /dev/sdd
brw-rw---- 1 root disk 8, 64 2012-07-11 14:24 /dev/sde
brw-rw---- 1 root disk 8, 80 2012-07-11 14:24 /dev/sdf
brw-rw---- 1 root disk 8, 96 2012-07-11 14:24 /dev/sdg
root@<hostname>:/srv/mysql# dd if=/dev/sdb of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 5.0139e-05 s, 102 MB/s
root@<hostname>:/srv/mysql# dd if=/dev/sdc of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 4.0726e-05 s, 126 MB/s
root@<hostname>:/srv/mysql# dd if=/dev/sdd of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 4.0935e-05 s, 125 MB/s
root@<hostname>:/srv/mysql# dd if=/dev/sde of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 4.2647e-05 s, 120 MB/s
root@<hostname>:/srv/mysql# dd if=/dev/sdf of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 4.1718e-05 s, 123 MB/s
root@<hostname>:/srv/mysql# dd if=/dev/sdg of=/dev/null count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 4.3425e-05 s, 118 MB/s

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1020436

Title:
  Cannot read superblock after FC multipath failover

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+subscriptions