NFS4 broken after upgrade to TrueNAS 12

[xenu]

I am wondering if anyone has kerberized nfs4 working with 12.0-U5+ or if this is some issue with my setup? Or is this feature not commonly used "in the wild"?

 

[cscutcher]

I completely forgot about this issue (was a busy 2021!) and stupidly upgraded by TrueNAS box to TrueNAS-12.0-U7 a couple of weeks ago. This issue doesn't seem to have got better, for me at least.

I'm going to try and stick with it and collect more debug this time around! Although perhaps with a bit more intensity than when I tried to debug back in August (since it was so frustrating I gave up!)

 

[cscutcher]

I've had a similar issue happen to me too when upgrading from U5.0 to U5.1. I could mount but not access files.
The solution was to move entries in "maproot" to "mapall".

Thanks for the tip. Unfortunately I don't think that was it for me, at least the affected shares don't use either mapall or maproot.

As an aside though, and I'm sure your already aware but just in case you hadn't considered it; your suggested change would give anyone connected to that share full root permissions right? I'm assuming you knew that and it's a situation where it doesn't particularly matter, but just wanted to mention it in-case you hadn't realized.

 

[cscutcher]

I am wondering if anyone has kerberized nfs4 working with 12.0-U5+ or if this is some issue with my setup? Or is this feature not commonly used "in the wild"?

I too wonder this. I gotta say that kerberized nfs4 has been a pain for me since I first stuck my toe in the water years ago. When I have had it working, it always feels held together with ducttape and I daren't look at it too hard lest it falls apart.

I keep trying to convince myself that it's just my unfamiliarity or wonky homelab setup, and that the sysadmin greybeards just have the dark arts of making it work locked up in their heads as tribal knowledge, but I increasingly wonder it's just an awkward set of tools to work with. I keep meaning to try out a similar setup with pure linux-linux kerberized nfs4 to see if it's easier (and since I have more experience with linux than bsd) although even then it doesn't exactly feel like the mainstream either.

On the other hand, if people aren't using kerberized nfs4 then what are they using? nfs <4 or with sec=sys seem easier/more reliable but lack integrity or encryption which seems a bit limited in the new world of zero-trust security, especially when the encryption performance overhead is relatively minor. Samba seems popular, and I have too little experience with it to say with any certainty, but from the little I've used it, it never really feels Linux native, especially when trying to get centralized auth and identity setup. sshfs also seems popular, and while I see the appeal for quick and easy access to remote files, it doesn't seem like a practical replacement for NFS. Searching for alternatives, everything else either seems aimed at a much larger scale (glustefs and the like) or too obscure (webdav I guess is an example).

I think this will be my last attempt to get it working solidly again on TrueNAS. If it doesn't slay the beast, I think I'll try if I can get smb working nicely, or I guess maybe if linux-linux nfs4 is less painful that might be an option. Although how that will help me with TrueNAS...I don't know if TrueNAS Scale is any less painful being Linux based.

 

[morganL]

@cscutcher 12.0-U8 is coming out today.... I don't see any relevant NFSv4-Kerberos fixes.

These are relatively complex and lesser used technologies.. mostly used for larger scale enterprise deployments. Often there are integration issues that have to be resolved at deployment time. There are not many home labs using it.

If you can document your set-up and see that TrueNAS is not behaving as expected, then please report a bug.

TrueNAS SCALE has a different (linux-based) NFS implementation..... it may behave differently . If it works, it would be an indication of an issue with FreeBSD NFSv4.

 

[TrueChris]

Issue on Jira has been closed with comment "Unfortunately, we do not have time or resources to track this down."
That's disappointing but understandable. Like morganL wrote, either you've an enterprise installation with Kerberos than you likely use SCALE or you've a homelab with CORE and likely don't use Kerberos.
Still I'd like to have it working. After I wasn't able to get Kerberos working on TrueNAS CORE I tried it on FreeBSD and got it working pretty straightforward with release 12.2 (base of TN 12) and release 13.0.

 

[TrueChris]

Just tested TrueNAS 13.0-BETA1 and what can I say, kerberized NFS works again. It didn't for me with U5, U7 and U8.
That's great news but has a bad taste nevertheless. Because if noone knows what broke with U5, it can happen again.

 

[jgreco]

Just tested TrueNAS 13.0-BETA1 and what can I say, kerberized NFS works again. It didn't for me with U5, U7 and U8.
That's great news but has a bad taste nevertheless. Because if noone knows what broke with U5, it can happen again.

That's why you always need a lab to test and a plan to roll back if there's a fire.

 

[cscutcher]

If you can document your set-up and see that TrueNAS is not behaving as expected, then please report a bug.

I was monitoring NAS-112125 since I'm pretty certain that's the same issue, but tbh I wasn't sure what extra information I could provide that others hadn't already included, and as you say these are relative complex and not often used on a small scale, so there's not a wealth of information on how to even narrow down the kind of information that's useful.

As TrueChris says;

That's disappointing but understandable. Like morganL wrote, either you've an enterprise installation with Kerberos than you likely use SCALE or you've a homelab with CORE and likely don't use Kerberos.

I too think this is a bit of a shame. Feels like there's a gap for home users for reliable centralised user management IPA and some NFS like thing with defaults for authentication that's encrypted by default. homelabbers like me are definitely outliers, but I think there's still a still a problem there for less techie users too (although I'll admit whether the average user, even the more tech savvy see its a problem is another thing). Obviously TrueNAS and open source projects don't owe us free solutions to these problems, but it feels like more and more the answer is to rely more on the dominance of big cloud providers, and closed source walled gardens which I find a bit scary.

 

[anodos]

Feels like there's a gap for home users for reliable centralised user management IPA and some NFS like thing with defaults for authentication that's encrypted by default.

I'm in the process of writing CI for NFSv4 + krb5 for SCALE. This will be in Active Directory (not FreeIPA) context. Just recently added NFSv4 ACL support to SCALE and CI for that (over NFSv4 protocol). There's a certain amount of load-balancing that I'm doing between the different TrueNAS releases.

Once that's in place, I'll transition back to TN 13.0 to add CI there and fix any issues that I come across vis-a-vis kerberos and NFSv4 in an AD environment. FreeIPA is a rather lower priority for now than Active Directory.