TrueNAS Scale - iSCSI Pauses on Write

jbar

Cadet
Joined
Jul 3, 2022
Messages
2
New to TrueNas and looking at storing some network backups and performance doesn't seem right with constant pausing during transfers.

First Try
Server 10gb to 10GB switch with TrueNas connected via 1gb motherboard connection (waiting on a new network card)
2 vDevs RAIDz2 using 900GB 10k SAS drives using lz4 compression.
Write performance is like 100mb for a few seconds then 0mb as reported by windows.
zpool iostat 1 was reporting around 49mb/s on the write side.
Memory seem to creep up while the dashboard showed over half still free
Setup using an iSCSI share.
Usually on another server I can get around 100-200mb/sec with a raid 1 or raid 10 with these types of drives.

Second Try
Switched to a new network card 10gb->10gb switch->10gb on TrueNas
Same issue as before, a bit higher peeks reported by Windows around 300mb then paused to 0mb for a bit.
Used prior pool and vDevs.

Third Try
Wiped the pool, new pool. 4 mirrors using 8 drives lz4 compression
Same pause when transferring a large file. Higher zpool iostat 1 write speed of around 60mb that peaks a bit but also started to show pauses.


Other bits for testing, on this 10gb switch I've tested these cards, cables to full 10gb wire rate between ports since it hardware offloads vlan->vlan access. I searched around and found tuneables but that was for the old version of TrueNas not Scale and I can't find a lot of performance tuning options for Scale. The goal would be a consitent 100mb write speed with no pauses. I have 12 drive bays and 1 open PCIE x8 slot if I need to add more hardware to it. I figured I would ask as I don't want to keep spending too much time if there is an easy fix for this since I'm like a weekend into learning truenas.

  • Motherboard make and model: Supermicro X8DTT-HEF+
  • CPU make and model: Dual Xeon E5645 Hex (6) Core 2.4GHz
  • RAM quantity: 32gb
  • Hard drives: 10x 900gb 2.5" 10K SAS
  • Hard disk controllers Broadcom 2008 SAS2 Controller
  • Network cards Intel onboard gigabit and Intel x520 dual SFP+ cards
 

jbar

Cadet
Joined
Jul 3, 2022
Messages
2
After changing some settings I found a few things.

1. Windows seems to issue Sync Writes with iSCSI. Everywhere on this forum I read that iSCSI was not sync writes.
2. Turning off sync writes netted a 250mb/s transfer of a 15gb file, which makes me think #1 is true.
3. I added a SLOG to a new Pool not the existing pool.
4. a 900GB 10k sas drive as a SLOG with sync writes was not better than a 4 way mirror of 900gb sas drives.
5. Tried out jumbo frames and something wasn't 100% right. 250mb/s is fine but I want to see if I can make out the network before i start optimizing drives.


Right now this is a backup device but I would like to have some mixed workloads and thinking of a PCIe SSD card for the slog to turn back on. Might have to order parts for that as I only have a samsung pm983.
 
Top