NFS complete lockup

[qwertymodo]

I am trying to share my user folders over NFS to a VM server running ProxMox via a 10Gb direct connection (no switch), which ProxMox is then re-sharing to its VM's via virtio-9p. This way, I can set the NFS server to only allow access to a single IP address, and the VM's don't need to know about the existence of that 10Gb connection (which is being used exclusively for this sharing setup, all other network traffic on each box is going through normal 1Gb ethernet NICs). For the most part, this works. However, after a day or two, the entire NFS connection locks up completely. Even trying to ls on the ProxMox machine directly just freezes. So far, I have not seen the connection recover without a physical power cycle of both machines. I understand the performance ramifications of NFS-on-ZFS, but when it's working, the performance has been reasonable for my needs. CPU and memory usage on both machines is pretty low, 1-2 cores in use on FreeNAS, ~20-30% RAM usage, ProxMox is ~3% CPU and ~25% RAM. Any ideas?

Here's a connection diagram if it helps at all:


FreeNAS 9.10.2-U3:
AsRock C2750D4I 8-core CPU
32GB RAM, ECC
Boot: 2x8GB Kingspec SATA DOM
Data: 6x2TB WD Red, RAIDZ2
2x1Gbe On-board Ethernet
Mellanox ConnectX-2 10Gbe

ProxMox VE 4.4-13:
Dell R710
2xXeon L5638 (6 cores each, 24 threads total)
64GB RAM, ECC
PERC H700 RAID Controller
2x1TB NL-SAS, RAID1
4x1Gbe On-board Ethernet
Mellanox ConnectX-2 10Gbe

FreeNAS NFS configuration:

ProxMox NFS configuration (/etc/fstab):

Code:

172.24.42.2:/mnt/vdev0/shared /mnt/freenas/shared nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/shared/media /mnt/freenas/shared/media nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/home /mnt/freenas/home nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/home/user1/files /mnt/freenas/home/user1 nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/home/user2/files /mnt/freenas/home/user2 nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/home/user3/files /mnt/freenas/home/user3 nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0
172.24.42.2:/mnt/vdev0/www /mnt/freenas/www nfs nfsvers=3,auto,bg,_netdev,rsize=8192,wsize=8192,intr,sync 0 0

 

[dlavigne]

Anything related in /var/log/messages on either side of the connection?

 

[qwertymodo]

FreeNAS (server-side) shows nothing. ProxMox (client-side) has a ton of

nfs: server 172.24.42.2 not responding, still trying
nfs: server 172.24.42.2 OK

 

[qwertymodo]

Also, I should mention that I do have the MTU set to 9000 on both sides.

 

[dlavigne]

Were you able to figure this out?

 

[qwertymodo]

Nope, no change. School's pretty busy so I haven't had much time to really dig into it, but the setup is essentially useless in this state.