[Bug 1616213] Re: Core dump on multipathd shutdown - trusty 14.04.4

[Dragan S.]

Thread 5 is on the way out:

(gdb) thread 5
[Switching to thread 5 (Thread 0x7f21ca11f840 (LWP 41582))]
#0  0x00007f21c9111dfd in nanosleep () at ../sysdeps/unix/syscall-template.S:81
81	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) up
#1  0x00007f21c9111c94 in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:137
137	../sysdeps/unix/sysv/linux/sleep.c: No such file or directory.
(gdb) 
#2  0x00000000004084e6 in child (param=0x0) at main.c:1539
1539			sleep (1); /* This is weak. */
(gdb) list
1534		free_polls();
1535	
1536		unlock(vecs->lock);
1537		/* Now all the waitevent threads will start rushing in. */
1538		while (vecs->lock.depth > 0) {
1539			sleep (1); /* This is weak. */
1540			condlog(3,"Have %d wait event checkers threads to de-alloc, waiting..\n", vecs->lock.depth);
1541		}
1542		pthread_mutex_destroy(vecs->lock.mutex);
1543		FREE(vecs->lock.mutex);

The code in thread 5 before the sleep(1) is:
(gdb) list -
1524	
1525		pthread_cancel(check_thr);
1526		pthread_cancel(uevent_thr);
1527		pthread_cancel(uxlsnr_thr);
1528		pthread_cancel(uevq_thr);
1529	
1530		free_keys(keys);
1531		keys = NULL;
1532		free_handlers(handlers);
1533		handlers = NULL;
(gdb) list -
1514		/*
1515		 * exit path
1516		 */
1517		block_signal(SIGHUP, NULL);
1518		lock(vecs->lock);
1519		if (conf->queue_without_daemon == QUE_NO_DAEMON_OFF)
1520			vector_foreach_slot(vecs->mpvec, mpp, i)
1521				dm_queue_if_no_path(mpp->alias, 0);
1522		remove_maps_and_stop_waiters(vecs);
1523		free_pathvec(vecs->pathvec, FREE_PATHS);
(gdb) 

So as you can see at line 1523, thread 5 freed vecs->pathvec. Which thread 1 is accessing:
1150	vector_foreach_slot (vecs->pathvec, pp, i) {

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1616213

Title:
  Core dump on multipathd shutdown - trusty 14.04.4

Status in multipath-tools package in Ubuntu:
  In Progress

Bug description:
  On ubuntu trusty 14.04.4 in multipath-tools version 0.4.9-3ubuntu7.14
  there is bug in multipathd on shutdown.

  The code will access pathvec pointer which is a valid address:

  Reading symbols from /sbin/multipathd...Reading symbols from /usr/lib/debug//sbin/multipathd...done.
  done.
  [New LWP 41631]
  [New LWP 41584]
  [New LWP 41633]
  [New LWP 41632]
  [New LWP 41582]
  [New LWP 41583]
  [Thread debugging using libthread_db enabled]
  Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
  Core was generated by `/sbin/multipathd'.
  Program terminated with signal SIGSEGV, Segmentation fault.
  #0  0x00000000004075db in checkerloop (ap=0x1b81040) at main.c:1150

  1150				vector_foreach_slot (vecs->pathvec, pp, i) {
  (gdb) list
  1145			pthread_cleanup_push(cleanup_lock, &vecs->lock);
  1146			lock(vecs->lock);
  1147			condlog(4, "tick");
  1148	
  1149			if (vecs->pathvec) {
  1150				vector_foreach_slot (vecs->pathvec, pp, i) {
  1151					check_path(vecs, pp);
  1152				}
  1153			}
  1154			if (vecs->mpvec) {

  Pathvec is a valid pointer:
  (gdb) p vecs->pathvec
  $1 = (vector) 0x1b81280

  But the contents of the structure are just garbage:

  (gdb) p *vecs->pathvec
  $2 = {allocated = 1651076143, slot = 0x756e696c2d34365f}
  (gdb)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1616213/+subscriptions