"Stale file handle" when list snapshots (.zfs)

[marcelo.roxa]

Hi,

After upgrading version 12 to 13 (included 13U1), unable to list snapshot contents at NFS clients (NFv3 and NFSv4).

[root@machine01 _mroxa]# mount -o vers=4 10.30.0.2:/mnt/storage/TESTE /mnt
[root@machine01 _mroxa]# df -h /mnt

Filesystem Size Used Avail Use% Mounted on
10.30.0.2:/mnt/storage/TESTE 10G 1,5M 10G 1% /mnt

[root@machine01 _mroxa]# ls /mnt/
teste

[root@machine01 _mroxa]# ls /mnt/.zfs/snapshot/
manual-2022-07-20_10-29/ manual-2022-07-20_10-49/

[root@machine01 _mroxa]# ls /mnt/.zfs/snapshot/manual-2022-07-20_10-49/
ls: cannot access /mnt/.zfs/snapshot/manual-2022-07-20_10-49/: Stale file handle

[root@machine01 _mroxa]# mount | grep "10.30.0.2"

10.30.0.2:/mnt/storage/TESTE on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.30.19.103,local_lock=none,addr=10.30.16.190)

someone with the same problem?

 

[anodos]

It's probably happening because the snapshot has different fsid (different filesystem). Try mounting through v4 pseudoroot instead of mounting the dataset directly.

 

[marcelo.roxa]

The problem remains with pseudoroot.

With version 12 it doesn't happen, only with versions 13 and above.
I tried using CentOS, SUSE and Debian as a client and the same problem happens.
After downgrading to version 12 everything is resolved.
I also tried a fresh install on version 13, a new pool, a new dataset and a new snapshot and the problem persists.

locally (On Truenas) I can list the snapshots normally.

 

[anodos]

The problem remains with pseudoroot.

With version 12 it doesn't happen, only with versions 13 and above.
I tried using CentOS, SUSE and Debian as a client and the same problem happens.
After downgrading to version 12 everything is resolved.
I also tried a fresh install on version 13, a new pool, a new dataset and a new snapshot and the problem persists.

locally (On Truenas) I can list the snapshots normally.

Sorry. I somehow glossed over this and read SCALE in there somewhere. Please file a bug ticket. There might be an issue with how NFS file handle is being generated for additional datasets. Probably warrants investigation.

 

[GuillaumeB]

Hi,

We have the same problem (TrueNAS-13.0-U2). Is there a bug ticket ? Or any solution ?

 

[marcelo.roxa]

Hi,

Bug ticket: https://ixsystems.atlassian.net/browse/NAS-117270

 

[anodos]

Hi,

We have the same problem (TrueNAS-13.0-U2). Is there a bug ticket ? Or any solution ?

Ticket is still open.

 

[GuillaumeB]

Ticket is still open.

Is it possible to have a link for this ticket please ?

 

[nomad-fr]

Same trouble with FreeBSD 13.1 ... no trouble with previous version 13.0-p7 .
see :Reddit : FreeBSD 13.1 : ZFS NFS : .zfs/snapshot : Stale file handle

 

[GuillaumeB]

This bug causes another problem: deleting a snapshot in which a simple "ls" via NFS has been attempted will completely block and leave the nfsd daemon KO. The storage then becomes unstable or even totally unusable...

(Sorry for my second post, I didn't see the link. Unfortunately, even with an account I can't see the ticket.)

 

[nomad-fr]

On FreeBSD it seems to be the same : for me deleting a snapshot in which a simple "ls" via NFS has been attempted will completely block and leave the zfs destroy process in an unkillable state IO (trouble).

 

[nomad-fr]

I added a Bug repport see #266236

 

[nomad-fr]

I just add a procedure to reproduce in the bug report.

 

[randyg503]

NAS-117270

I receive a 'You don't have access to this issue' message when attempting to visit the above referenced bug ticket. Thoughts?

 

[blanchet]

I have the same issue on TrueNAS-13.0u2

• I cannot see the content of snapshots through NFSv3 (I have not tried NFSv4)

This bug is really annoying because it prevents users to restore their files in self-service mode.

 

[nomad-fr]

still no news from FreeBSD ...

 

[grahamperrin]

still no news from FreeBSD ...

Maybe an upstream issue. Please see <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266236#c7>.

Bug ticket: https://ixsystems.atlassian.net/browse/NAS-117270

Someone at Atlassian should liaise with Mozilla, to make login with Google less of a crash course. I wasted fifteen minutes …

Ticket is still open.

… why is the issue not accessible?

 

[mpfusion]

Is there any update? The bug report is inaccessible and the issue is a major problem if you need users to restore their own files.

 

[randyg503]

Is there any update? The bug report is inaccessible and the issue is a major problem if you need users to restore their own files.

FreeBSD 13+ : NFS access of snapshot returns stale file handle; server zfs commands hang · Issue #13974 · openzfs/zfs

System information Type Version/Name Distribution Name FreeBSD Distribution Version 13.1-RELEASE-p2 Kernel Version 13.1-RELEASE-p2 Architecture amd64 OpenZFS Version 2.1.4 Attempting to access to a...

github.com

 

[mpfusion]

So they reported upstream, but no fix as of now. Bummer. I'd really like to have this fixed in 13.0-U3, which seems very unlikely.