[Bug 2105431] Re: Issue with NFS 4.0 mounted to Kubernetes

[Chen Wang]

** Description changed:

  We stored some public files of the service dependencies on NFS, and then mount the NFS path to Kubernetes for all pods to utilize for unified management. Some issues encountered while we use NFS are as follows:
- 1. Suppose we mount the path `/root/nfs/bigdata-client/conf/hadoop` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/bigdata-client/conf/hadoop_bak`, and create a new `/root/nfs/bigdata-client/conf/hadoop` directory.
- 2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/bigdata-client/conf/hadoop` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
+ 1. Suppose we mount the path `/root/nfs/hadoop` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/hadoop_bak`, and create a new `/root/nfs/hadoop` directory.
+ 2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/hadoop` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
  3. Switch to NFSv3 mounting, perform the same operation as above, and the pod will automatically restart normally.
  The error message is displayed as follows:
  ```
  Warning Failed 4s kubelet Error: failed to generate container "00aa47e731507adfb0473d5440cce5e5dacf66ddee4adb1d8eced29543b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/hadoop-conf": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/hadoop-conf: stale NFS file handle
  ```

** Description changed:

  We stored some public files of the service dependencies on NFS, and then mount the NFS path to Kubernetes for all pods to utilize for unified management. Some issues encountered while we use NFS are as follows:
- 1. Suppose we mount the path `/root/nfs/hadoop` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/hadoop_bak`, and create a new `/root/nfs/hadoop` directory.
- 2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/hadoop` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
+ 1. Suppose we mount the path `/root/nfs/common/conf/conf_dir` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/common/conf/conf_dir_bak`, and create a new `/root/nfs/common/conf/conf_dir` directory.
+ 2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/common/conf/conf_dir` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
  3. Switch to NFSv3 mounting, perform the same operation as above, and the pod will automatically restart normally.
  The error message is displayed as follows:
  ```
  Warning Failed 4s kubelet Error: failed to generate container "00aa47e731507adfb0473d5440cce5e5dacf66ddee4adb1d8eced29543b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/hadoop-conf": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/hadoop-conf: stale NFS file handle
  ```

** Description changed:

  We stored some public files of the service dependencies on NFS, and then mount the NFS path to Kubernetes for all pods to utilize for unified management. Some issues encountered while we use NFS are as follows:
  1. Suppose we mount the path `/root/nfs/common/conf/conf_dir` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/common/conf/conf_dir_bak`, and create a new `/root/nfs/common/conf/conf_dir` directory.
  2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/common/conf/conf_dir` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
  3. Switch to NFSv3 mounting, perform the same operation as above, and the pod will automatically restart normally.
  The error message is displayed as follows:
  ```
- Warning Failed 4s kubelet Error: failed to generate container "00aa47e731507adfb0473d5440cce5e5dacf66ddee4adb1d8eced29543b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/hadoop-conf": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/hadoop-conf: stale NFS file handle
+ Warning Failed 4s kubelet Error: failed to generate container "00aa47e731507adfb0473d5440cce5e5dacf66ddee4adb1d8eced29543b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/conf_dir": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/conf_dir: stale NFS file handle
  ```

** Description changed:

  We stored some public files of the service dependencies on NFS, and then mount the NFS path to Kubernetes for all pods to utilize for unified management. Some issues encountered while we use NFS are as follows:
  1. Suppose we mount the path `/root/nfs/common/conf/conf_dir` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/common/conf/conf_dir_bak`, and create a new `/root/nfs/common/conf/conf_dir` directory.
  2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/common/conf/conf_dir` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
  3. Switch to NFSv3 mounting, perform the same operation as above, and the pod will automatically restart normally.
  The error message is displayed as follows:
  ```
- Warning Failed 4s kubelet Error: failed to generate container "00aa47e731507adfb0473d5440cce5e5dacf66ddee4adb1d8eced29543b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/conf_dir": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/conf_dir: stale NFS file handle
+ Warning Failed 4s kubelet Error: failed to generate container "00aad7e731507adfba473d5440cce5e5dac798ae4adb1d8eced295c3b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/conf_dir": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/conf_dir: stale NFS file handle
  ```

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to glibc in Ubuntu.
https://bugs.launchpad.net/bugs/2105431

Title:
  Issue with NFS 4.0 mounted to Kubernetes

Status in glibc package in Ubuntu:
  New

Bug description:
  We stored some public files of the service dependencies on NFS, and then mount the NFS path to Kubernetes for all pods to utilize for unified management. Some issues encountered while we use NFS are as follows:
  1. Suppose we mount the path `/root/nfs/common/conf/conf_dir` to Kubernetes, update the configuration to change the hadoop directory to `/root/nfs/common/conf/conf_dir_bak`, and create a new `/root/nfs/common/conf/conf_dir` directory.
  2. Under NFSv4, if the pod on Kubernetes restarts automatically, the `/root/nfs/common/conf/conf_dir` path cannot be found, resulting in the failure of the pod to restart automatically. We can use the `kubectl delete pod xxx` method to restart normally.
  3. Switch to NFSv3 mounting, perform the same operation as above, and the pod will automatically restart normally.
  The error message is displayed as follows:
  ```
  Warning Failed 4s kubelet Error: failed to generate container "00aad7e731507adfba473d5440cce5e5dac798ae4adb1d8eced295c3b49db9" spec: failed to generate spec: failed to start "/var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernetes.io~nfs/conf_dir": stat /var/lib/kubelet/pods/b4e3af40-3713-4b77-a4c5-5334550860d7/volumes/kubernettes.io~nfs/conf_dir: stale NFS file handle
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/2105431/+subscriptions