Performance NFS vs iSCSI with Veeam B&R

Kosta

Contributor
Joined
May 9, 2013
Messages
106
Hello,
I've been searching a bit, both here and on the web, and the best I have been able to find that NFS has lower performance that iSCSI.
In my configuration is not like that. Here iSCSI is faster than NFS. Let me explain:

I have both Backup-Server and TrueNAS virtualized in my ESXi server. The server itself doesn't matter much in my opinion, but if it's of any use:
Supermicro X11 with one Silver 4110, 128GB RAM, NVME as ESXi-Datastore.
Additionally I have 8 physical 2TB drives, which are connected via HBA to the TrueNAS. HBA is an LSI 9211-8i and is being directly assigned to TrueNAS. This is working flawlessly.
I must also say that this is mostly about the performance of VBR with TrueNAS. The backups that go on iSCSI are way slower than NFS.

Now, I first tried NFS.
I simply created a new vSwitch, connected both VMs to it, gave each machine an IP manually and off we go. Around 300MB/s. I'd say that's OK. I didn't do any performance measurement via SMB or so with CrystalDisk, so I have no comparison.
And then I decided to try iSCSI, in the end, that is a recommended solution for Backups and Veeam. Block-Storage.
And apparently, doesn't matter what I do, I get half the speed. About 100-150MB/s.
I tried following iSCSI setups:
1) Over VMXNET3, same as NFS (basically both have an IP, and Windows iSCSI connects to Truenas)
2) Connecting ESXi to TrueNAS and VM via RDM (ESXi has a VMkernel Adapter and PortGroup on the same vSwitch, PG is assigned to the TrueNAS VM, VMK and TrueNAS have IPs from the same subnet, ESXi connects the iSCSI, then I use RDM to directly assign iSCSI to the VM, in windows create local ReFS/64 disk and locally save backups)
3) iSCSI ESXi->TrueNAS, VMDK for Backup-Server (basically same as 2nd way, with the exception that I create a Datastore on TrueNAS-iSCSI, then create VMDK on the datastore and assign it to the VM).

Additionally, I tried in 1st configuration to set MTU to 9000 in TrueNAS and vSwitch. Absolutely no change.

Any ideas what the problem might be?

Thanks.
 
Last edited:

Kosta

Contributor
Joined
May 9, 2013
Messages
106
Update:
I just did some performance tests with CrystalMark. And it is as I was afraid. Write performance via iSCSI is suffering extremely.

iSCSI:
2022-06-08 10_34_40-Window.png

SMB (I just don't want to configure NFS under windows server if not necessary, but one can see the difference):
2022-06-08 10_44_08-Window.png

This shows to me that it's not Veeam that's the problem too. This is either in TrueNAS or ESXi or the configuration of both.
I am also asking myself why is the write performance THIS much lower than read performance. Is that normal? I could implement SSD caching...
 
Top