[Bug 2089565] Re: Issue upgrading CEPH on ubuntu 24.04 LTS

[stephane maubian]

adding ceph-mds  is having the same behavior / crash


Nov 25 14:29:16  ceph-mds[1315223]:  ceph version 19.2.0 (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable)
Nov 25 14:29:16  ceph-mds[1315223]:  1: /lib/x86_64-linux-gnu/libc.so.6(+0x45320) [0x79eec6645320]
Nov 25 14:29:16  ceph-mds[1315223]:  2: pthread_kill()
Nov 25 14:29:16  ceph-mds[1315223]:  3: gsignal()
Nov 25 14:29:16  ceph-mds[1315223]:  4: abort()
Nov 25 14:29:16  ceph-mds[1315223]:  5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5) [0x79eec6aa5ff5]
Nov 25 14:29:16  ceph-mds[1315223]:  6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da) [0x79eec6abb0da]
Nov 25 14:29:16  ceph-mds[1315223]:  7: (std::unexpected()+0) [0x79eec6aa5a55]
Nov 25 14:29:16  ceph-mds[1315223]:  8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391) [0x79eec6abb391]
Nov 25 14:29:16  ceph-mds[1315223]:  9: (ceph::buffer::v15_2_0::list::iterator_impl<true>::copy(unsigned int, std::__cxx11::basic_st>
Nov 25 14:29:16  ceph-mds[1315223]:  10: (MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0xca1) [0x79eec76c3ab1]
Nov 25 14:29:16  ceph-mds[1315223]:  11: (MDSDaemon::handle_mds_map(boost::intrusive_ptr<MMDSMap const> const&)+0x2c6) [0x5d6582d14b>
Nov 25 14:29:16  ceph-mds[1315223]:  12: (MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)+0x2c9) [0x5d658>
Nov 25 14:29:16  ceph-mds[1315223]:  13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0xf9) [0x5d6582d18919]
Nov 25 14:29:16  ceph-mds[1315223]:  14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr<Message> const&)+0x118) [0x79eec734a78>
Nov 25 14:29:16  ceph-mds[1315223]:  15: (DispatchQueue::entry()+0x6c9) [0x79eec7348929]
Nov 25 14:29:16  ceph-mds[1315223]:  16: (DispatchQueue::DispatchThread::entry()+0x11) [0x79eec7418f61]
Nov 25 14:29:16  ceph-mds[1315223]:  17: /lib/x86_64-linux-gnu/libc.so.6(+0x9ca94) [0x79eec669ca94]
Nov 25 14:29:16  ceph-mds[1315223]:  18: /lib/x86_64-linux-gnu/libc.so.6(+0x129c3c) [0x79eec6729c3c]
Nov 25 14:29:16  ceph-mds[1315223]:  NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Nov 25 14:29:17  systemd[1]: ceph-mds at .service: Main process exited, code=dumped, status=6/ABRT
Nov 25 14:29:17  systemd[1]: ceph-mds at .service: Failed with result 'core-dump'.
Nov 25 14:29:17  systemd[1]: ceph-mds at .service: Scheduled restart job, restart counter is at 3.
Nov 25 14:29:17  systemd[1]: ceph-mds at .service: Start request repeated too quickly.
Nov 25 14:29:17  systemd[1]: ceph-mds at .service: Failed with result 'core-dump'.
Nov 25 14:29:17  systemd[1]: Failed to start ceph-mds at .service - Ceph metadata server daemon.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2089565

Title:
  Issue upgrading  CEPH  on ubuntu 24.04 LTS

Status in ceph package in Ubuntu:
  New

Bug description:
  This issue is a continuation of
  https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2065515

  
  On Ubuntu 24.04 lts we did upgrade Ceph to  19.2.0-0ubuntu0.24.04.1

  Previous release is : 19.2.0~git20240301.4c76c50-0ubuntu6

  whenever  upgrading (tested on 2 different clusters)  the ceph-mon
  ends up crashing repeatedly with the below stack error

  ```
   ceph version 19.2.0 (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable)
   1: /lib/x86_64-linux-gnu/libc.so.6(+0x45320) [0x788409245320]
   2: pthread_kill()
   3: gsignal()
   4: abort()
   5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5) [0x7884096a5ff5]
   6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da) [0x7884096bb0da]
   7: (std::unexpected()+0) [0x7884096a5a55]
   8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391) [0x7884096bb391]
   9: (ceph::buffer::v15_2_0::list::iterator_impl<true>::copy(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)+0x193) [0x78840a293593]
   10: (MDSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0xca1) [0x78840a4c3ab1]
   11: (Filesystem::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x1c3) [0x78840a4e4303]
   12: (FSMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)+0x280) [0x78840a4e6ef0]
   13: (MDSMonitor::update_from_paxos(bool*)+0x291) [0x631ac5dea801]
   14: (Monitor::refresh_from_paxos(bool*)+0x124) [0x631ac5b7a164]
   15: (Monitor::preinit()+0x98e) [0x631ac5bb2fbe]
   16: main()
   17: /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca) [0x78840922a1ca]
   18: __libc_start_main()
   19: _start()
   NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

  ```

  
  mitigation:
  a rollback to the previous release 19.2.0~git20240301.4c76c50-0ubuntu6 is still possible to restore service

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2089565/+subscriptions