Slog for 12 ssds?

[TheMaxSti]

Hi All.

Long term lurker here..

So iv been playing with Truenas for years. I've always shied away from making it home prod as I'm not great with Linux. But when the pandemic hit I decided to grab this opportunity and break this cycle..

Anyway, I currently use (For my sins) a Synology 1817+ (10Gb card, 6x12tb fronted with 2x1,9tb Intel SSDs (Cache)) the performance is not what you would call blow your socks off... 300Mb tops.

So I decided to repurpose one of my spare esxi nodes ( ASROCK D1541D4U with 64Gb RAM, LSI 9400-16i connected to 12x 960Gb SSDs in a 4vdev x 3 Disk rz1 Config (For testing)) I can now pin the network to 10gb easily and this is making the VMS feel much more snappy, especially the SQL Servers (DBA 9-5 job). esxi boxes connected via multipath nfs 4.1 to truenas.

Iv maxed out the RAM (64gb, technically I could do 128GB if I could find the DIMMs without having to sell body parts.) but was wondering if an optane m.2 in the nvme slots as a SLOG would help at all with so many SSD behind this pool. From research it seems the optane drives top out at 280mb but as i understand that this is only for random writes? I don't think using the nvme slots as an L2ARC would be useful.

Anyone got any useful info on this.. Most seem to be using a SLOG in front of HDDs and i can see the use for this just don't know if it will be anyuse with SSDs.
Also not 100% sure on the optain drive will work in this board.

Anyway Cheers
Rob

 

[ChrisRJ]

Please check out "ZIL and SLOG" from the recommended readings in my signature.

 

[TheMaxSti]

Thanks Chris, I seem to remember seeing this before in my research..

Unfortunately I'm still at a loss.. The article does not mention optaine drives or adding a SLOG to a pool of SSD drives. Your article does mention another drive i've been looking into the DC P3700 but again is this needed when iv got a pool of SSDs anyway. Most of the articles I've read its all about adding a SLOG to a pool of HDDs.

I understand that to get the best performance out of a pool with NFS you need a SLOG or for the fool hardy to disable sync writes. Just wanted to see if anyone had any thoughts or personal experience of this scenario and save me buying some DC P3700 or STEC ZeusRAM as a trial.

Cheers

 

[Etorix]

So iv been playing with Truenas for years. I've always shied away from making it home prod as I'm not great with Linux.

TrueNAS CORE is FreeBSD, not Linux. Anyway, in normal use you're not supposed to see what's running under the GUI so it shouldn't really matter.

So I decided to repurpose one of my spare esxi nodes ( ASROCK D1541D4U with 64Gb RAM, LSI 9400-16i connected to 12x 960Gb SSDs in a 4vdev x 3 Disk rz1 Config (For testing)) I can now pin the network to 10gb easily and this is making the VMS feel much more snappy, especially the SQL Servers (DBA 9-5 job). esxi boxes connected via multipath nfs 4.1 to truenas.

Iv maxed out the RAM (64gb, technically I could do 128GB if I could find the DIMMs without having to sell body parts.) but was wondering if an optane m.2 in the nvme slots as a SLOG would help at all with so many SSD behind this pool.

NFS, VMs and databases should be doing sync writes, possibly a lot of them, so you may benefit from a SLOG.
Two caveats: The M.2 slots on D1541D4U are x1 (they are really intended for boot), so your SLOG would be doing better in a PCIe slot. Consumer Optane M.2 are severely limited, and even the top 64 GB models may not fare so well performance and endurance-wise under heavy load; 900p/905p or Optane DC would be better—except for body parts maybe…

I understand that to get the best performance out of a pool with NFS you need a SLOG or for the fool hardy to disable sync writes. Just wanted to see if anyone had any thoughts or personal experience of this scenario and save me buying some DC P3700 or STEC ZeusRAM as a trial.

Don't bother with ZeusRAM at this point. Good modern SLOG drives are Optane and Radian RMS.

 

[TheMaxSti]

@Etorix thanks makes sense.Just wanted to be a bit more confident if the brown stuff was to hit the fan on what commands could help and how to use them.

Ok, check hear you on the Optane drives.. and 905p is too far out for me / and I don't have enough lanes to make it worthwhile.. Good spot as id not fully twigged on this.

The DC P3700 I think should work fine in the LSI 9400-16i as it's a tri-mode adapter so NVME should be ok as well. Just may need another one as I'm tempted to add another 12 drives to the mix so giving me 24 drives for more space, I think I'm bottlenecked with the CPU, not that I need more speed just the random io would be nice to have.
Both PCIe slots on the board are 8x so splitting the NVME p3700 across 2 cards Maybe better than all on one card. I Will look out for some cheap small m.2 drives for boot then giving me back to sata / sas ports.
Cheers