Krb5p NFSv4 stops after 24hours

igor101

Cadet
Joined
May 24, 2023
Messages
1
I've set up a TrueNAS-13.0-U4 CORE (for testing) and I use CentOS 7 clients integrated with AD using sssd. The TrueNAS CORE storage is also integrated with the AD all green checks) and have encrypted and "ticketed" krb5p NFS4 mounts. All clients mount (nfsvers=4.1, sec=krpb5p), Kerberos tickets and encryption work, permissions all good, AD users and groups all good, too. So, it's been set up OK and it also works fine.

Until it stops after about 24 hours as it refuses mount requests. A restart of NFS service from the GUI makes it work again, for another 24 hours. Interestingly, a restart of the NFS daemons from the CLI works only sometimes. I have "Require Kerberos for NFSv4" enabled.

So, on a CentOS 7.9 client:
mount command produces this:
mount.nfs: access denied by server while mounting storage.domain.com:/home
dmesg shows this:
[ 8.142033] RPC: server storage.domain.com requires stronger authentication.

now the interesting part from /var/log/messages
May 24 14:51:04 box-p04 kernel: RPC: server storage.domain.com requires stronger authentication. May 24 14:51:20 box-p04 systemd: Got automount request for /home, triggered by 1422 (ERAAgent) May 24 14:51:20 box-p04 systemd: Mounting /home... May 24 14:51:20 box-p04 mount: mount.nfs: access denied by server while mounting storage.domain.com:/home May 24 14:51:20 box-p04 kernel: RPC: server storage.domain.com requires stronger authentication. May 24 14:51:20 box-p04 systemd: home.mount mount process exited, code=exited status=32 May 24 14:51:20 box-p04 systemd: Failed to mount /home. May 24 14:51:20 box-p04 systemd: Unit home.mount entered failed state. May 24 14:51:20 box-p04 systemd: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 1422 (ERAAgent)

A restart of the NFS service on the TrueNAS storage, and suddently no issues with the authenticaion and all working nicely again.
Interestingly, to restore the operation the NFS service needs to be restarted from within the web GUI. Restarting nfsd & nfsuserd (in that order), would not affect/remove the cause of this issue. Something else getting restarted through the GUI along the NFS?

On the TrueNAS side I've nothing in the logs. I'm not sure which logs should contain something on this on FreeBSD. I checked the usual suspects in /var/log/, sort them by time modified and checked any logs changed near my time testing the issue. There's nothing there about NFS or mounts, no errors.

Where to look for NFS logs, mount attempts logs, kerberos logs etc. relevant on TrueNAS CORE 13? Enable debug?

Thanks!
 
Top