TrueNAS 12.0 Features

[morganL]

Persistent L2ARC is in the OpenZFS software, but hasn't been tested and verified yet.

 

[JoeAtWork]

Interesting that the microbusting is such a big deal, I have seen over the years that Dell/EQL would show you how to get a cisco cat 37xx to work but suggested just buying dedicated dell switches and enable the iscsi option. That must be the ticket they just were too chicken sh!t to say the csico swiches has too small of buffers.

I knew that a newer version of VMware fixed some storage issues. The only way in the docs to get VAII to work is with device extents, since I was using these as NFS/iSCSI/FC I tried to use a file extent I think and that was most likely my mistake. Good info and maybe why I have always had very good success with NFS.

VMware 6.x supported NFS 4.1 and with FreeNAS 11.x we finally got NFS 4.1. In my tests with a virtual FreeNAS running under VMware I was able to use just the standard vSwitch and set VLAN101 to use vmNIC1 as active and vmNIC2 as standby and VLAN102 to use vmNIC2 as active and vmNIC1 as standby. This allowed VMware to force the traffic on an interface and FreeNAS to happly think it had just 1 NIC trunked or two nics access and the load balancing by vmware was even when using the ip addresses in 2 subnets.

Serve the home has an interesting forum post on brocade 10/40gb switches, for my home lab I might have to look into those.

 

[Yorick]

Probably overkill for a home lab, and: Catalyst is designed for campus, Nexus for data center. Nexus 3100 has 12 to 48MB buffer, depending on model, and Nexus 3600 8GB (or 16GB in the bigger version). PFC, Priority Flow Control, is supported, which means these will interact with other DCB components. See also https://www.cisco.com/c/dam/en/us/s...ta-center-bridging/at_a_glance_c45-460907.pdf for an overview of the technologies making up DCB.

 

[kspare]

Am I missing something? NFS 4.1 is a thing on freenas??

 

[morganL]

NFSv4.0 is the nominal support on FreeNAS
NFSv4.1 nominally included pNFS (parallel NFS = scale-out) which FreeNAS does not support
NFSv4.1 clients should be backward compatible with NFSv4 storage.. but not via pNFS
So.... FreeNAS supports NFSv4.1 clients, but not pNFS
TrueNAS SCALE has a plan to support NFSv4.1 with pNFS (scale-out).

 

[kspare]

Correct me if i'm wrong, but vmware esxi only supports nfs 3 and 4.1 and will not work with 4.0? So as far as vmware is concerned you are still using 3?

 

[kspare]

NFSv4.0 is the nominal support on FreeNAS
NFSv4.1 nominally included pNFS (parallel NFS = scale-out) which FreeNAS does not support
NFSv4.1 clients should be backward compatible with NFSv4 storage.. but not via pNFS
So.... FreeNAS supports NFSv4.1 clients, but not pNFS
TrueNAS SCALE has a plan to support NFSv4.1 with pNFS (scale-out).

Freenas supports 4.1 clients that will operate with backward compatibility.....huge caveat there. It does not *fully* or *properly* support 4.1 clients. I just want to make sure I got that right.

 

[Ericloewe]

Is there such a thing as a 4.1-only client?

 

[kspare]

Is there such a thing as a 4.1-only client?

esxi? It does not support 4. only 3. or 4.1 unless i'm wrong? which could be true!

 

[morganL]

The better way of saying it is

FreeNAS (or TrueNAS CORE) does not support the pNFS part of NFS v4.1

If a client or application expects pNFS, it would not be satisfied. TrueNAS SCALE will be the appropriate solution.

 

[morganL]

ESXi uses an NFSv4.1 client, but can support single node NFS storage without pNFS.

 

[kspare]

ESXi uses an NFSv4.1 client, but can support single node NFS storage without pNFS.

Agreed, but not on NFSv4.0 only NFS3.0 am I correct?

 

[morganL]

There have been VMware issues with ESXI NFS v4.1 client
I don't know whether they have ben resolved. https://www.ixsystems.com/community/threads/nfs-4-1-datastore-mounted-as-read-only.69173/

 

[Ericloewe]

I'm far from an NFS expert, but I'd be very surprised if the design goal for anything would be "break standard NFS 4.0 clients". It doesn't make sense.

 

[JoeAtWork]

There have been VMware issues with ESXI NFS v4.1 client
I don't know whether they have ben resolved. https://www.ixsystems.com/community/threads/nfs-4-1-datastore-mounted-as-read-only.69173/

This was fixed some time ago, you can now make nfs 4.1 mounts with multiple ip addresses and VMware will round robin them to get all the bandwidth of the individual links. The trick is to use VMware to carve up the networking.

 

[JoeAtWork]

I'm far from an NFS expert, but I'd be very surprised if the design goal for anything would be "break standard NFS 4.0 clients". It doesn't make sense.

There was a time FreeBSD lacked one feature VMware was looking for and it would make the mount RO. It was a year or two for it to get fixed.

 

[kspare]

This was fixed some time ago, you can now make nfs 4.1 mounts with multiple ip addresses and VMware will round robin them to get all the bandwidth of the individual links. The trick is to use VMware to carve up the networking.

My point is that you can't do this with Freenas because it doesn't support nfs 4.1, to use freenas, nfs and esxi, you have to use nfs 3

 

[RegularJoe]

My point is that you can't do this with Freenas because it doesn't support nfs 4.1, to use freenas, nfs and esxi, you have to use nfs 3

NFS4 feature used with VMware 6+ is session trunking.

Nope I have it now running at a few sites. When you link to vCenter it even coordinates the snapshots with the guests. If you have 4 VLANs and 4 subnets to put your NFS on you only need to tell ESXi that it should use NFS 4 and those 4 ip addresses, you get 4x the speed of a single connection. It is very simple and you can expose a snapshot, copy the folder to your production NFS mount point on the FreeNAS cli and be able to boot that vm that you are investigating. The thing I don't like is the copy command in FreBSD will not support sparse files and someone at iXsystems does not want to bundle coreutils with FreeNAS or allow mortals an wasy way to add coreutils to FreeNAS. I also feel since FreeNAS is a NAS it should have fdupes and not be forced to make a jail just to get fdupes.

 

[kspare]

Very interesting. I tried it out on our dev data store. I'm not sure what went wrong but when veeam starting making snapshots of the vm's we starting to have alot of issues....so definitly some R&D to do there on my part.

How would you get 4x the speed on a single connection?

 

[RegularJoe]

4x with 4 NICs in 4 subnets and in vmware you feed it the 4 ip addresses of the FN filer when you config a NFS 4 datastore. That is assuming your on the same switch or your cross connected switches have a big pipe/backplane. In VMware you have to tell it that for vlan1 use vnic0 as active and the others as standby and repeat the logic for all threee other NICs.

VEEAM is terrible, they do dirty snapshots. If you do the vcenter integration with FreeNAS even the NFS snapshots are application aware as they chat with the vmware tools.