truenas scale 22.02.0.1 nfs service failed to start

[LiX47]

As in the title, just updated truenas scale and now the nfs service refuses to start, can't use my share at all and don't see any useful debugging info

 

[LiX47]

Looking into it a bit more, it may be related to this https://access.redhat.com/solutions/5545331 which seems to be coming from a few datasets with spaces in their name

Code:

service nfs-server status
● nfs-server.service - NFS server and services
     Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Thu 2022-04-07 17:55:48 EDT; 9min ago
    Process: 410530 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=1/FAILURE)
    Process: 410532 ExecStopPost=/usr/sbin/exportfs -au (code=exited, status=0/SUCCESS)
    Process: 410533 ExecStopPost=/usr/sbin/exportfs -f (code=exited, status=0/SUCCESS)

Apr 07 17:55:48 <host> exportfs[410530]: exportfs: Failed to resolve Drive
Apr 07 17:55:48 <host> exportfs[410530]: exportfs: Failed to resolve Drive
Apr 07 17:55:48 <host> exportfs[410530]: exportfs: duplicated export entries:
Apr 07 17:55:48 <host> exportfs[410530]: exportfs:         *:/mnt/data/backup/Seagate
Apr 07 17:55:48 <host> exportfs[410530]: exportfs:         *:/mnt/data/backup/Seagate
Apr 07 17:55:48 <host> exportfs[410530]: exportfs: Failed to stat /mnt/data/backup/Seagate: No such file or directory
Apr 07 17:55:48 <host> exportfs[410530]: exportfs: Failed to stat /mnt/data/backup/My: No such file or directory
Apr 07 17:55:48 <host> systemd[1]: nfs-server.service: Control process exited, code=exited, status=1/FAILURE
Apr 07 17:55:48 <host> systemd[1]: nfs-server.service: Failed with result 'exit-code'.
Apr 07 17:55:48 <host> systemd[1]: Stopped NFS server and services.

the datasets in question being the ones that start with "Seagate" or "My"

 

[Nick2253]

For this very reason, I don't recommend using spaces in either pool or dataset names. Connect words with hyphens if needed.

Now, I'd agree that this is a "bug", but fixing problems caused by spaces in the name are likely going to be low on the priority list, so you're probably best served by renaming (https://docs.oracle.com/cd/E19253-01/819-5461/gamnn/index.html).

 

[LiX47]

For this very reason, I don't recommend using spaces in either pool or dataset names. Connect words with hyphens if needed.

Now, I'd agree that this is a "bug", but fixing problems caused by spaces in the name are likely going to be low on the priority list, so you're probably best served by renaming (https://docs.oracle.com/cd/E19253-01/819-5461/gamnn/index.html).

It wasn't an issue prior to updating so I'd call this a regression. I figured that was the case and I've started working on that, however in this case these datasets mirror the folders/drives/etc that they're backing up, the choices for a workaround in my case are either moving terabytes of data, or breaking that naming symmetry and making automation more difficult :/

 

[morganL]

It wasn't an issue prior to updating so I'd call this a regression. I figured that was the case and I've started working on that, however in this case these datasets mirror the folders/drives/etc that they're backing up, the choices for a workaround in my case are either moving terabytes of data, or breaking that naming symmetry and making automation more difficult :/

What version did you update from RCx or the .0 release?

 

[LiX47]

What version did you update from RCx or the .0 release?

It looks like I probably updated from TrueNAS-SCALE-22.02-RC.1-2, basing off of another machine I have that I generally keep in sync (but held off because I saw this break and wasn't sure what was going on), but I can't say with 100% certainty

 

[LiX47]

I had to manually correct the exports file for zfs, and in doing so I noticed that the paths with spaces were unquoted. It seems that whatever was generating that file should be quoting its path names as per https://linux.die.net/man/5/exports

Code:

Blank lines are ignored. A pound sign ("#") introduces a comment to the end of the line. Entries may be continued across newlines using a backslash. If an export name contains spaces it should be quoted using double quotes. You can also specify spaces or other unusual character in the export name using a backslash followed by the character code as three octal digits. 

 

[morganL]

It looks like I probably updated from TrueNAS-SCALE-22.02-RC.1-2, basing off of another machine I have that I generally keep in sync (but held off because I saw this break and wasn't sure what was going on), but I can't say with 100% certainty

There was a major change between RC and release.. the NFS server changed from Ganesha to kernel for stability and performance reasons.

SCALE 22.02.0 has some NFS issues which will be mostly addressed in SCALE 22.02.1. The currently nightly has all the fixes.

 

[LiX47]

There was a major change between RC and release.. the NFS server changed from Ganesha to kernel for stability and performance reasons.

SCALE 22.02.0 has some NFS issues which will be mostly addressed in SCALE 22.02.1. The currently nightly has all the fixes.

well, it's certianly not a small issue, I'm just now noticing that my installed applications section was showing as empty, even though images were listed under "manage docker images", and my plex server was still running...rebooting the nas fixed it so i suspect it may have been related to me correcting the exports as well

 

[morganL]

well, it's certianly not a small issue, I'm just now noticing that my installed applications section was showing as empty, even though images were listed under "manage docker images", and my plex server was still running...rebooting the nas fixed it so i suspect it may have been related to me correcting the exports as well

There are some issues with updates where certain tasks aren't completed in time on more complex setups .. seems to be fixed by a reboot. The 1st update (22.02.1) addresses at least some of them. Data is safe, but there is more work to be done.

 

[john60]

I also have nfs will not start:

root@truenas[~]# service nfs-server status
● nfs-server.service - NFS server and services
Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2023-02-24 12:43:39 CST; 1 day 18h ago
CPU: 10ms

Feb 24 12:43:39 truenas systemd[1]: Starting NFS server and services...
Feb 24 12:43:39 truenas exportfs[6233]: exportfs: can't open /etc/exports for reading
Feb 24 12:43:39 truenas systemd[1]: nfs-server.service: Control process exited, code=exited, status=1/FAILURE
Feb 24 12:43:39 truenas systemd[1]: nfs-server.service: Failed with result 'exit-code'.
Feb 24 12:43:39 truenas systemd[1]: Stopped NFS server and services.

My dataset does not have a space in it's name but contains files that do. Am I having a different issue? I can't rename the contained files since they are provided to me by others and i need to preserve their names.