[Bug 2000630] Re: "crash" module is always on but not properly configured

[Launchpad Bug Tracker]

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: ceph (Ubuntu)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2000630

Title:
  "crash" module is always on but not properly configured

Status in Ceph Monitor Charm:
  Fix Released
Status in Ceph Monitor Charm quincy.2 series:
  Fix Released
Status in Ceph OSD Charm:
  Fix Released
Status in Ceph OSD Charm quincy.2 series:
  Fix Released
Status in ceph package in Ubuntu:
  Confirmed

Bug description:
  cloud:focal-yoga (quincy)

  $ juju ssh ceph-mon/leader -- sudo ceph version
  ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)

  
  How to reproduce:

  1. make sure "crash" module is on (it's a part of "always on" modules)

      https://docs.ceph.com/en/quincy/mgr/crash/

      $ juju ssh ceph-mon/leader -- sudo ceph mgr module ls | grep crash
      crash              on (always on)

  2. intentionally crash ceph-osd process (in this example I used
  SIGSEGV)

      $ juju ssh ceph-osd/leader -- sudo pkill --signal SIGSEGV ceph-osd

  3. make sure a normal crash file is generated for apport *and* a set
  of files for ceph crash module.

      # ll -h /var/crash/
      total 121M
      drwxrwxrwt  2 root root 4.0K Dec 28 10:42 ./
      drwxr-xr-x 13 root root 4.0K Dec 12 21:41 ../
      -rw-r-----  1 ceph ceph 121M Dec 28 10:42 _usr_bin_ceph-osd.64045.crash

  
      # ll -h /var/lib/ceph/crash/*
      '/var/lib/ceph/crash/2022-12-28T10:42:04.661282Z_51be6c87-4a42-4fbb-afe5-264d94cd6c79':
      total 1.6M
      drwx------ 2 ceph ceph 4.0K Dec 28 10:42 ./
      drwxr-xr-x 4 ceph ceph 4.0K Dec 28 10:42 ../
      -r--r--r-- 1 ceph ceph    0 Dec 28 10:42 done
      -rw-r--r-- 1 ceph ceph 1.6M Dec 28 10:42 log
      -rw------- 1 ceph ceph  926 Dec 28 10:42 meta
      
      /var/lib/ceph/crash/posted:
      total 8.0K
      drwxr-xr-x 2 root root 4.0K Sep 13 17:47 ./
      drwxr-xr-x 4 ceph ceph 4.0K Dec 28 10:42 ../

  4. check syslog for post failures to MON units.

  Dec 28 10:51:18 famous-skunk ceph-crash[10667]: WARNING:ceph-
  crash:post
  /var/lib/ceph/crash/2022-12-28T10:42:04.661282Z_51be6c87-4a42-4fbb-
  afe5-264d94cd6c79 as client.crash.famous-skunk failed: (None,
  b'2022-12-28T10:51:18.368+0000 7f427dc2f700 -1 auth: unable to find a
  keyring on /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
  (2) No such file or directory\n2022-12-28T10:51:18.368+0000
  7f427dc2f700 -1 AuthRegistry(0x7f427805f4f0) no keyring found at
  /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
  disabling cephx\n2022-12-28T10:51:18.376+0000 7f427c9cd700 -1 auth:
  unable to find a keyring on /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
  (2) No such file or directory\n2022-12-28T10:51:18.376+0000
  7f427c9cd700 -1 AuthRegistry(0x7f4278065748) no keyring found at
  /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
  disabling cephx\n2022-12-28T10:51:18.376+0000 7f427c9cd700 -1 auth:
  unable to find a keyring on /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
  (2) No such file or directory\n2022-12-28T10:51:18.376+0000
  7f427c9cd700 -1 AuthRegistry(0x7f427c9cc000) no keyring found at
  /etc/ceph/ceph.client.crash.famous-
  skunk.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
  disabling cephx\n[errno 2] RADOS object not found (error connecting to
  the cluster)\n')

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-ceph-mon/+bug/2000630/+subscriptions