Requesting Feedback/Suggestions on New build + configuration

[haregh28]

Hello,

I'm a long time lurker - Created an account to ask for help with my first build!
I plan to create a VM for Free NAS with details as follows :

Physical Server :

• Supermicro 836 chassis with 920SQ PSU
• SAS2 backplane
• Supermicro X10SDV-7TP4F moherboard ( Xeon D-1537 CPU )
• 64GB ECC DDR4 RAM
• Server is connected to a UPS

TrueNAS VM :

• CPU : 4
• RAM : 16GB
• Mirrored boot drives ( as VMDKs on different SSDs - The SSDs are formatted as VMFS )
• Disks will be passed through to the VM and I'm thinking of the following pools :
  • Pool 1 - 4 x 4tb wd red plus + 4 x 400GB Intel S3700 in RAIDZ2 => for file storage ( photos / media etc )
  • Pool 2 - 2 x 960GB Synology SATA SSD in RAID mirror => for VM Storage
• VM will be connected to vSwitch connected to one 10G port which is connected to a 10G switch.

Questions :

• Is this a reasonable build / configuration ?
  • What changes should I make for better results ?
• In Pool1 : I'm thinking of using the 4 x S3700 as metadata vdev for storing small files
  • I've observed that small files take a long time to read/write from magnetic disks and my understanding is that situating the metadata vdev on a faster storage will help - Am I having the right idea ?
• Will my use case benefit from an SLOG device ?
  • From what I read, an SLOG will not be useful for the VM storage pool since writes to this pool need to be ack'ed by the SSDs in Pool 2 and isn't(?) needed for the Pool 1 since it is for file storage.
• If an L2ARC would be useful, what size is recommended ( If a higher RAM is needed for TrueNAS VM - I can increase it ) ?
• Can I use a 2x250GB Samsung 980 Pro partitioned as an SLOG of 20GB AND L2ARC device with the remaining ~200GB ?

Thanks!

 

[Etorix]

Why run TrueNAS as a VM rather than bare metal?
You do not allocate enough RAM to even think about L2ARC.
SLOG is for when sync writes cannot be avoided (databases, NFS shares), which is apparently not your use case. A 980 Pro would not be a good SLOG anyway.

 

[haregh28]

Thanks @Etroix for your feedback!

Why run TrueNAS as a VM rather than bare metal?

I'm going to be using this for my home and it's going to be one user or maximum 2 users with very less actual usage.
Hence I would like to not have to dedicate the physical server for a NAS only.

You do not allocate enough RAM to even think about L2ARC

How much RAM should I allocate before considering L2ARC ? Can you point me to any document / guides about the considerations to take into account for this choice ?

SLOG is for when sync writes cannot be avoided (databases, NFS shares), which is apparently not your use case.

Both pools will likely be NFS shares with pool 2 being exported to ESXi for VM storage - My understanding is that I will have to enable sync writes for pool 2 atleast ( pool 2 - vm storage might be iSCSI - I'm unsure what to pick - maybe another question for the forum, but regardless will need sync writes enabled ).

A 980 Pro would not be a good SLOG anyway.

What are some good SLOG options ? NAS will not be heavily used. I imagine < 100GB of data written to Pool 2 on days when it is used ( say 10% of the days ). Pool 1 will not have a SLOG configured. I imagine I will not need a high endurance device considering my specific usage.

 

[Etorix]

Depending on what you want to do, it might be easier to run TrueNAS on bare metal and run other services as jails/VMs. That would at least take out some points of failure for ZFS. If not, read this carefully:

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.truenas.com

The general rules of thumb for L2ARC are that 64 GB RAM is the minimum before considering L2ARC (because managing a Level 2 ARC takes up RAM and evicts the Level 1 ARC) and to keep L2ARC no larger than 5*RAM.

A good SLOG has low latency, high endurance and, most importantly, power-loss protection. Mirroring it is not critical for home/lab use, but PLP and endurance are. Current choices would be a data center class NVMe SSD (Intel DC Pxxxx), an Optane drive or a Radian RMS-200/300 card.

 

[haregh28]

@Etorix Thanks for the response and apologies for my delay..

Depending on what you want to do, it might be easier to run TrueNAS on bare metal and run other services as jails/VMs. That would at least take out some points of failure for ZFS.

I'm comfortable with running on VMs - I plan to passthrough the SAS controller to the VM - no RDM business.
The NAS is for a home environment - not too critical and I will have backups as well.
I have a few other ESXi servers and do not want more than 1 virtualization technology - Besides I'm no FreeeBSD expert and and am also very comfortable with VMware ESXi.

The general rules of thumb for L2ARC are that 64 GB RAM is the minimum before considering L2ARC (because managing a Level 2 ARC takes up RAM and evicts the Level 1 ARC) and to keep L2ARC no larger than 5*RAM.

A good SLOG has low latency, high endurance and, most importantly, power-loss protection. Mirroring it is not critical for home/lab use, but PLP and endurance are. Current choices would be a data center class NVMe SSD (Intel DC Pxxxx), an Optane drive or a Radian RMS-200/300 card.

I'm wondering if I should start without an L2ARC and then add if needed ?
If that's do-able - what can I monitor to decide if I need either of these - Can you please point me to the relevant docs for this ?

Ah yes - PLP.. My server would be connected to a UPS. can you enlighten me under what conditions an SLOG device without PLP would cause data loss in this scenario ( I'm sure there are - Asking so I can see if they are worth the risk ) ?
Same here as for L2ARC - Thinking of just starting without an SLOG and then adding when needed.. Are there metrics I can monitor for SLOG as well ?