[Bug 2028304] [NEW] continuous handle_nullreq: failed; please move to gssproxy

[Charles Hedrick]

Public bug reported:

[don't look at the system config. The actual server can't contact your
bug system, so this is being submitted from a desktop]

Ubuntu 22.04.1. NFS server using kerberized NFS.

Occasionally individual users lose access to their files, typically at
login. They get permission failed. The server shows continuous messages:

rpc.svcgssd[5672]: WARNING: handle_nullreq: failed reading request.

Looking at the code, this seems to happen when the GSS token is too
large for the fixed kernel buffer. The limit is documented as 2K, except
that the protocol to svcgssd is in text, so it's 4K there. Using strace,
I can see that when the system is working, some of the tokens are very
nearly 4K.

This problem should be fixed by using gssproxy instead of svcgssd. I'd
suggest that you move to gssproxy as the default. I think we'll see
increasing evidence of this problem as sites move to newer versions of
Kerberos.

The problem seems to have started when we upgraded our IPA servers. The
newest version of Kerberos includes PACs in the Kerberos tickets. That
increases the size of tickets. We're guessing that this was enough to
push the system into this failure mode intermittently.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: nfs-common 1:2.6.1-1ubuntu1.2
ProcVersionSignature: Ubuntu 5.19.0-35.36~22.04.1-generic 5.19.17
Uname: Linux 5.19.0-35-generic x86_64
.etc.request-key.d.id_resolver.conf: create	id_resolver	*	*	/usr/sbin/nfsidmap -t 600 %k %d
ApportVersion: 2.20.11-0ubuntu82.4
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Thu Jul 20 14:56:18 2023
InstallationDate: Installed on 2022-11-07 (254 days ago)
InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
NFSMounts:
 
NFSv4Mounts: /staff/users temp.lcsr.rutgers.edu:/beta/users nfs4   rw,nosuid,nodev,relatime,vers=4.2,rsize=65536,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5,clientaddr=128.6.26.16,local_lock=none,addr=172.16.90.121
SourcePackage: nfs-utils
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.default.apport: [modified]
modified.conffile..etc.default.nfs-common: [modified]
mtime.conffile..etc.default.apport: 2022-11-23T13:25:53.902233
mtime.conffile..etc.default.nfs-common: 2022-11-23T12:56:19.514493

** Affects: nfs-utils (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug jammy wayland-session

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/2028304

Title:
  continuous handle_nullreq: failed; please move to gssproxy

Status in nfs-utils package in Ubuntu:
  New

Bug description:
  [don't look at the system config. The actual server can't contact your
  bug system, so this is being submitted from a desktop]

  Ubuntu 22.04.1. NFS server using kerberized NFS.

  Occasionally individual users lose access to their files, typically at
  login. They get permission failed. The server shows continuous
  messages:

  rpc.svcgssd[5672]: WARNING: handle_nullreq: failed reading request.

  Looking at the code, this seems to happen when the GSS token is too
  large for the fixed kernel buffer. The limit is documented as 2K,
  except that the protocol to svcgssd is in text, so it's 4K there.
  Using strace, I can see that when the system is working, some of the
  tokens are very nearly 4K.

  This problem should be fixed by using gssproxy instead of svcgssd. I'd
  suggest that you move to gssproxy as the default. I think we'll see
  increasing evidence of this problem as sites move to newer versions of
  Kerberos.

  The problem seems to have started when we upgraded our IPA servers.
  The newest version of Kerberos includes PACs in the Kerberos tickets.
  That increases the size of tickets. We're guessing that this was
  enough to push the system into this failure mode intermittently.

  ProblemType: Bug
  DistroRelease: Ubuntu 22.04
  Package: nfs-common 1:2.6.1-1ubuntu1.2
  ProcVersionSignature: Ubuntu 5.19.0-35.36~22.04.1-generic 5.19.17
  Uname: Linux 5.19.0-35-generic x86_64
  .etc.request-key.d.id_resolver.conf: create	id_resolver	*	*	/usr/sbin/nfsidmap -t 600 %k %d
  ApportVersion: 2.20.11-0ubuntu82.4
  Architecture: amd64
  CasperMD5CheckResult: pass
  CurrentDesktop: ubuntu:GNOME
  Date: Thu Jul 20 14:56:18 2023
  InstallationDate: Installed on 2022-11-07 (254 days ago)
  InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
  NFSMounts:
   
  NFSv4Mounts: /staff/users temp.lcsr.rutgers.edu:/beta/users nfs4   rw,nosuid,nodev,relatime,vers=4.2,rsize=65536,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5,clientaddr=128.6.26.16,local_lock=none,addr=172.16.90.121
  SourcePackage: nfs-utils
  UpgradeStatus: No upgrade log present (probably fresh install)
  modified.conffile..etc.default.apport: [modified]
  modified.conffile..etc.default.nfs-common: [modified]
  mtime.conffile..etc.default.apport: 2022-11-23T13:25:53.902233
  mtime.conffile..etc.default.nfs-common: 2022-11-23T12:56:19.514493

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/2028304/+subscriptions