[Bug 2093880] Re: libcephfs: flush the caps release in filesystem sync

[OhmyPosh]

To effectively flush the caps release in the filesystem sync, you'll
need to ensure the following steps:

Understand the Context: In CephFS, "caps" refer to capabilities granted
to clients on specific files or directories. Flushing cap releases
ensures clients relinquish permissions properly, preventing data
inconsistency.

Sync Process: When flushing caps during a filesystem sync, you’re
ensuring all outstanding operations on a file are written and released.
This can involve sending a flush message to clients to signal
completion.

Flush Implementation: Modify the code in libcephfs where caps are
managed, ensuring that before releasing the capability, all related
operations are synchronized.

Testing: After implementing the changes, thoroughly test the filesystem
to ensure the caps are flushed correctly and no data is lost during
sync.

This process is crucial for maintaining file system consistency in
distributed environments.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2093880

Title:
  libcephfs: flush the caps release in filesystem sync

Status in ceph package in Ubuntu:
  New

Bug description:
  Hi,

  The bug is as mentioned in the Ceph upstream: https://tracker.ceph.com/issues/67221
  and also Linux upstream: https://github.com/torvalds/linux/commit/ccda9910d8490f4fb067131598e4b2e986faa5a0

  Under some situations, the libcephfs forgets the acquired capabilities
  and thus gets evicted by the Ceph MDS.  I would like to see if a
  backport will be available for Ubuntu 22.04 LTS (Jammy) Ceph Quincy
  client (17.2.x).

  Logs on the client:
    Dec 19 04:32:45 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state OPEN)
    Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 session reset
    Dec 19 04:32:46 *** kernel: ceph: mds0 closed our session
    Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect start
    Dec 19 04:32:46 *** kernel: ceph: mds0 reconnect denied
    Dec 19 04:32:46 *** kernel: libceph: mds0 (1)***:6801 socket closed (con state V1_CONNECT_MSG)
    Dec 19 04:32:47 *** kernel: ceph: mds0 rejected session

  Logs on the server: 
    Dec 19 04:30:17 *** ceph-mds[1408372]: log_channel(cluster) log [WRN] : client.911386 isn't responding to mclientcaps(revoke), ino 0x1004e6bede5 pending pAsLsXsFs issued pAsLsXsFs, sent 240.313055 seconds ago
    Dec 19 04:32:44 *** ceph-mds[1408372]: log_channel(cluster) log [INF] : Evicting (and blocklisting) client session 911386 (v1:***:0/362122962)

  The Ubuntu client version:
    Description:	Ubuntu 22.04.5 LTS
    Release:	22.04

  Package: 
    libcephfs2/jammy-updates,jammy-security,now 17.2.7-0ubuntu0.22.04.2 amd64 [installed,automatic]

  What expected to happen:
  * There shall be no 'client isn't responding to ... pending pAsLsXsFs ...' messages, and no eviction.

  What happened instead:
  * The error appeared and the client is evicted.  

  Thanks,
  Kinson

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2093880/+subscriptions