[Bug 1687004] Re: multipath crash generating core dump

[Łukasz Zemczak]

Hello Rafael, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build
now and be available at https://launchpad.net/ubuntu/+source/multipath-
tools/0.4.9-3ubuntu7.16 in a few hours, and then in the -proposed
repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: multipath-tools (Ubuntu Trusty)
       Status: In Progress => Fix Committed

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1687004

Title:
  multipath crash generating core dump

Status in multipath-tools package in Ubuntu:
  Fix Released
Status in multipath-tools source package in Trusty:
  Fix Committed

Bug description:
  [Impact]

   * mutexes are being allocated from memory with no checks and, if allocation fails, it causes these kind of segfaults later in the code execution.
   expect multipath daemon to crash and not run any checkers on path groups.
   * not checking path groups, in an event of failure, the mpath won't change path prios.
   * openstack relies on flushing device maps frequently when using iscsi.

  [Test Case]

   * i'm fixing this based on a dump analysis and not on reproduction.
   * if you disallow memory overcommit - facilitating memory exhaustion - you would be able to reproduce that by stressing multipathd with paths being flushed, but that is theory only.

  [Regression Potential]

  * the patch is changing the locking mechanism for log thread, based on
  upstream commit.

  * major change is to use the mutexes from stack instead of allocating
  from heap.

  * multipath log thread could not work as designed.

  * tested by reported and reported to be good.

  * What releases are affected ?

   The following releases already got the fix
   - Xenial/Yakkety/Zesty/Artful

   Note that Debian also has the fix.
   Meaning that ONLY Trusty is affected by this bug.

  * This SRU contained fixes for 2 LP bugs:
  ​https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1687004
  https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1695789

  [Other Info]

  It was brought to my attention that:

  multipath-tools: 0.4.9-3ubuntu7.15

  Faced a crash and generated a dump.

  ## multipath (trusty) crashed and its dump shows:

  (gdb) bt full
  #0  __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
          __PRETTY_FUNCTION__ = "__pthread_mutex_lock"
          type = 0
  #1  0x00007f48700b606e in flush_logqueue () at log_pthread.c:39
          empty = 0
  #2  0x00007f48700b611b in log_thread (et=0x0) at log_pthread.c:57
  No locals.
  #3  0x00007f4870964184 in start_thread (arg=0x7f4870d8b700) at pthread_create.c:312
          __res = <optimized out>
          pd = 0x7f4870d8b700
          now = <optimized out>
          unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139949107623680, -3200163692152804016, 0, 0, 139949107624384, 139949107623680, 3244383534590274896,
                  3244383107817352528}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
          not_first_call = <optimized out>
          pagesize_m1 = <optimized out>
          sp = <optimized out>
          freesize = <optimized out>
          __PRETTY_FUNCTION__ = "start_thread"
  #4  0x00007f486fdb537d in __ecvt_r (value=9.532824124368238e-130, ndigit=0, decpt=0x0, sign=0x0, buf=0x7f4870d8b9c0 "\220R\267pH\177", len=139949107623680)
      at efgcvt_r.c:218
          d = 0
          f = 3.2378592100206092e-319
          exponent = 1893250816
  #5  0x0000000000000000 in ?? ()
  No symbol table info available.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1687004/+subscriptions