Backup settings TrueNAS 12RC1 / ESXi 6.7 / Veeam 10

[Seimari]

Hello,

I bought backup server (check signature) with 4x Seagate Exos X14 12TB SAS drives and fast formatted them to 4096 bytes sector size and updated latest firmware.
I know with 4 disks (2 vdevs) write speed is really slow, i'm going to by 4 more drives.

I am clueless about optimal record and block settings
Current settings:
TrueNas Pool record size: 1 MiB
16TB zvol with lz4 compression on, dedup off.
iscsi logical block size: 512
2x 10G iscsi mpio with round robin iops 1 setting in vSphere.
vSphere VMFS 6.82: 512 format , thick privision, block size 1MB (cannot change), space reclamation granularity 1MB (cannot change)

Everything works but even when Veeam does incremental backup and only transfers 20GB, it's quite slow (30-112MB/s).

Max Veeam read speed is 643MB/s and Processing rate 304MB/s.
Veeam is virtual machine (8 vcores, 12GB Ram) and i attached this storage as D drive, so it looks like local drive.
Backup Proxy max concurrent tasks: 8
Backup Repository maximum concurrent tasks to 4
Use per-VM backup files on.
Backup job storage optimization: Local target

Other possibility is to replicate from shared storage to backup server.

I apologize that i have multiple questions but I wanted to show big picture.

 

[joeschmuck]

Everything works but even when Veeam does incremental backup and only transfers 20GB, it's quite slow (30-112MB/s).
Is there possibility that SLOG is full or something?

I'm not the expert here but you should do some basic benchmark testing on the backup server, find out how fast the pool reads/writes and if you can, also do it from within the Veeam VM. Once you have done benchmark testing then you can make an informed decision where the problem exists. It could be that the incremental backups are a lot of small files and that slows your overall transfer rate.

As for the SLOG/ZIL, you have quite a bit of RAM (256GB) so using a ZIL would doubtfully speed things up. Here is a link to read.

Also, the fact that you top out at 112MB/s seems to indicate that you are using a 1Gb connection speed, what I get on my home wired network. Maybe you have something mis-configured in the VM or hardware?

Take a step back and look it all over again.

Good Luck

 

[Seimari]

I ran CrystalDiskMark 7 from Veeam VM and targeted D: (TrueNas backup server).

 

[Seimari]

 

[joeschmuck]

I wish I had those numbers

 

[jenksdrummer]

You may want a larger test file - if your test file is sitting in ARC you are reading your cache rates which will give great numbers but they aren't realistic. Needs to be at least the size of the RAM on your storage box...

To you OP, we moved off FreeNAS for backend LUN storage for Veeam Storage VM...bought a RAID Controller for the former FreeNAS box, and turned it up as Windows Server 2019 system and use the proxy and storage roles on it. Screams. Seriously; getting over 1GByte/sec backups and we're using reverse incremental backups. Also, as an aside, check you're using ReFS 64K on your storage LUN...

Never could get that level of performance out of FreeNAS save for blowing out the data and starting fresh; then after a few weeks it would just tank and we'd get maybe 25-100Mb/sec per iSCSI channel speeds. FreeNAS seemed fine, but the LUN within the VM reported 100% utilization. Seemed OK with AppAssure before that, but, that was a different process for backups; IE, incrementals then it would periodically roll them forward and discard data, vs Veeam Reverse Incrementals, where it's ingesting new data and pooping out the changed blocks from the 'full' backup.