NFS client: bad sequence-id error

[snowshi]

Hi, we are facing an issue with nfs (based on truenas) server or client.
Server: TrueNAS-SCALE-22.12.2
Client: rhel6 with nfs-utils version nfs-utils-1.2.3-78.el6_10.2.x86_64


Client terminal:
Cannot write to /nfs-mounted-dir/, errno =5[Input/Output error] *WARN* Cannot write to /nfs-mounted-dir/, failed to lock the file[Input/output error]

at this time, the /var/log/messages will have


Oct 24 19:36:31 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff880211e76420! Oct 24 19:37:14 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff880b2fa8aa20! Oct 24 19:37:19 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff883414082220! Oct 24 19:41:16 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff8821148d8e20! Oct 24 19:47:47 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff8811bafdbe20! Oct 24 19:52:28 tcnode8 kernel: NFS: v4 server returned a bad sequence-id error on an unconfirmed sequence ffff881f71ae1420!


help someone can explain, thanks.