L2Arc - NFS Vs iSCSI

[kspare]

I'm doing some experimenting with iscsi and nfs, we have 2 Intel 750 PCI ssd's in our box.

Sync is set to always.

We have one drive set for zil and one for l2arc.

I'm wondering if L2Arc is more effective with NFS
or with iSCSI

It gets a little blurry when I read that nfs would appear to be the better option because the vmdk files would be sitting right on the file system vs iscsi that would be sitting within the iscsi extent.

Has anyone spent some time on this?

Are there any suggestions on some tests I could run?

 

[mav@]

L2ARC, same as ARC, works on level of blocks, and blocks are completely the same, whether they belong to file or zvol. If you are going to do some comparisons, just remember that default dataset block is 128K, while default zvol block is 16K+, so you should compare comparable things.

 

[HoneyBadger]

default zvol block is 16K+

Did this change recently? I thought that zvols were still defaulting to 8K.

 

[mav@]

ZFS internal default for zvol is indeed still 8K. But since some time (9.3, or some 9.2.1.x) FreeNAS tries to tune zvol block size to specifics of the pool. For example, if you try to create zvol on top of 10-disk RAIDZ2, FreeNAS will propose block size of like 64K, since lower value will be too space-inefficient in that case.

Generally things changed dramatically since 4K disks appeared, and FreeNAS switched to using their layout as default. For 512-byte disks this problem of space-efficiently was almost non-existent.