Sanity Check

[dbesade]

Hello,

Long time follower of the community. I have not posted often. I was hoping to get a sanity check on my configuration, as well as any feedback. I don't have any performance issues at the moment, but I'm always looking to Optimize. This is my current home setup.

Base Storage Head
-Dell PowerEdge T420
-TrueNAS Core 12.0 u4
-2 x Intel E5-2430L-V2 Processors
-80GB/RAM
-Onboard SATA (for 6 Internal Drives)
-LSI 9207-8i (8 Internal Hot-Swap Bays)
-LSI 9207-8e (External 24 Slot Shelf... MD1220)
-Broadcom BCM57810S Dual Port 10G CNA

TrueNAS Configuration
-Pool A (Backups via NFS)
-2 x 1TB 7200RPM SATA Drives combined into a mirrored vDEV
-1 x 40G Intel 320 SSD as SLOG
-Backups runs very well, easily hitting 150MB/sec via NFS using Veeam

-Pool B (Nearline Storage)
-8 x 2TB 7200RPM Self Encrypting SAS2 Drives split into 4 x Mirrored vDEV
-1 x 200G Intel DC S3700 SSD as SLOG
-Primarily Serves as Datastore in vSphere using iSCSI for Video Storage and some VMs.
-Sync-Always enabled

-Pool C (10K SAS Storage)
-13 x 600G 10KRPM 2.5" SAS Drives split into 6 x Mirrored vDEV with 1 Spare
-1 x 200G Intel DC S3700 SSD as Slog
-Primarily Serves as Datastore for VM Storage via iSCSI
-Sync-Standard Enabled

-Pool D (SAS SSD Storage)
-5 200G 2.5" SAS SSD (High Endurance w/Power Protection) split into RAID-Z1 with 1 spare.
-No SLOG
-Primarily Serves are Datastore for VM Storage via iSCSI

Thoughts

• The SLOG in Pool C is wasted.
  • My ARC Summary doesn't necessarily show a benefit for an L2-Arc in my use case. One alternative I thought about was pulling the SLOG from Pool A and B, putting the 2 x Intel S3700s together and doing a Mirrored Meta-Data for Pool-B to help offset the slow drives. Backups via NFS will be fine without it I reckon.
• Pool D
  • I do realize I'm chewing up Endurance on the SSD's. I needed lowish seek time and more space than mirrored vDev's would give me. Medium-term plan is to get 4 more of the same SSD's and do mirrored vDEVs for performance.

​

 

[dbesade]

Guessing everything is all good?

 

[Spearfoot]

Hello,

Long time follower of the community. I have not posted often. I was hoping to get a sanity check on my configuration, as well as any feedback. I don't have any performance issues at the moment, but I'm always looking to Optimize. This is my current home setup.

Base Storage Head
-Dell PowerEdge T420
-TrueNAS Core 12.0 u4
-2 x Intel E5-2430L-V2 Processors
-80GB/RAM
-Onboard SATA (for 6 Internal Drives)
-LSI 9207-8i (8 Internal Hot-Swap Bays)
-LSI 9207-8e (External 24 Slot Shelf... MD1220)
-Broadcom BCM57810S Dual Port 10G CNA

TrueNAS Configuration
-Pool A (Backups via NFS)
-2 x 1TB 7200RPM SATA Drives combined into a mirrored vDEV
-1 x 40G Intel 320 SSD as SLOG
-Backups runs very well, easily hitting 150MB/sec via NFS using Veeam

-Pool B (Nearline Storage)
-8 x 2TB 7200RPM Self Encrypting SAS2 Drives split into 4 x Mirrored vDEV
-1 x 200G Intel DC S3700 SSD as SLOG
-Primarily Serves as Datastore in vSphere using iSCSI for Video Storage and some VMs.
-Sync-Always enabled

-Pool C (10K SAS Storage)
-13 x 600G 10KRPM 2.5" SAS Drives split into 6 x Mirrored vDEV with 1 Spare
-1 x 200G Intel DC S3700 SSD as Slog
-Primarily Serves as Datastore for VM Storage via iSCSI
-Sync-Standard Enabled

-Pool D (SAS SSD Storage)
-5 200G 2.5" SAS SSD (High Endurance w/Power Protection) split into RAID-Z1 with 1 spare.
-No SLOG
-Primarily Serves are Datastore for VM Storage via iSCSI

Thoughts

• The SLOG in Pool C is wasted.
  • My ARC Summary doesn't necessarily show a benefit for an L2-Arc in my use case. One alternative I thought about was pulling the SLOG from Pool A and B, putting the 2 x Intel S3700s together and doing a Mirrored Meta-Data for Pool-B to help offset the slow drives. Backups via NFS will be fine without it I reckon.
• Pool D
  • I do realize I'm chewing up Endurance on the SSD's. I needed lowish seek time and more space than mirrored vDev's would give me. Medium-term plan is to get 4 more of the same SSD's and do mirrored vDEVs for performance.

My suggestions:

• Install more RAM -- this is always a good idea!
• Pull the Intel 320 SLOG from pool A -- it's not a good choice for a SLOG device and you don't need one anyway for the pool's intended purpose.
• Enable synchronous writes on pool C -- no use having a SLOG device if you're not forcing synchronous writes.
• Replace the DC S3700 SSD SLOG devices with Intel Optane 900-series AIC-type SSDs, assuming you have the spare PCIe slots and can afford them; they're pricey and hard to find.
• Your plan to use mirrored SSDs for pool D is a good idea -- you'll get more IOPS that way.

If you haven't already, read @jgreco's SLOG/ZIL thread:

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com

I once used Intel DC S3700 SSDs as SLOG devices for my NFS-based ESXi datastores... but not anymore. I don't run any mission-critical virtual machines.

 

[ChrisRJ]

The CPUs have a relatively low frequency. So you might get some improvement going for a higher clock speed, even with 2x 4 cores. There was a very good post here (from @jgreco ?) a while ago, that goes into why the lower TDP argument isn't really relevant for a NAS usage.

 

[jgreco]

Some people argue for lower TDP CPU's on the basis that they "use less power".

However, the CPU cores between a low TDP CPU and a full blast CPU primarily differ in the clock regulation, and so if they end up idling at the same amount of power consumption, you get an unexpected result: the low TDP CPU may have to run for a much longer time at its full clock speed to match the same amount of work that the normal CPU might have gotten done in half the time. The normal CPU guns the engine and burns somewhat more power for a much shorter period of time before returning to idle, quite possibly saving power in the process.

Now of course this is dependent on a bunch of factors; a 12 core CPU inherently uses more power at idle than a 4 core CPU, for example.

But the other thing is that protocols like SMB benefit from the higher clock speeds; you get better single-client performance from higher clock speed CPU's.

 

[dbesade]

This is great! Thank you to all who have replied. Here are some replies, in-line.

• Pull the Intel 320 SLOG from pool A -- it's not a good choice for a SLOG device and you don't need one anyway for the pool's intended purpose.
  • Believe is or not, there is write latency reduction with the 320 as a SLOG.
  • Though you are correct, for the Pools intended purpose its probably unneeded.
• Enable synchronous writes on pool C -- no use having a SLOG device if you're not forcing synchronous writes.
  • I actually really don't want to do this as my writes will suffer.
  • I would entertain doing a Mirror of S3700s, but not sure thats going to help the relative slow write speed (in comparison to 12 spindles).
• Replace the DC S3700 SSD SLOG devices with Intel Optane 900-series AIC-type SSDs, assuming you have the spare PCIe slots and can afford them; they're pricey and hard to find.
  • For $650 - 800 its a bit for a single pool
  • What about the Optane P4800s
• Your plan to use mirrored SSDs for pool D is a good idea -- you'll get more IOPS that way.
  • Yeah need to add more Drives.
• The CPUs have a relatively low frequency. So you might get some improvement going for a higher clock speed, even with 2x 4 cores.
  • Unfortunately 2.5Ghz is the fastest speed on the E5-2400 V2 Series CPUs.
  • I'm investigating moving to a PowerEdge 720XD with E5-2600 Series V2 Processors which would have some Higher Speed, Lower Core Processors.
• But the other thing is that protocols like SMB benefit from the higher clock speeds; you get better single-client performance from higher clock speed CPU's.
  • I'm doing NFS and iSCSI and thats it.