Pool Setup & Recommendations for Light-Duty Home NAS

[jordiorlando]

Hello! I have been running TrueNAS (first Core, then Scale) on my old desktop for several years now. I recently took advantage of some sales to purchase new hardware for the purpose of building a second NAS. My intention is to relocate the old NAS to a family member's house while keeping the new one in mine, so as to have offsite replication for both my data as well as my families. My main question now is on how to set up my new pool(s) to best match my use case and available hardware.

New NAS2 hardware (PCPartPicker)​

• OS: TrueNAS SCALE 22.02.2.1
• Chassis: Jonsbo N1
• CPU: AMD Ryzen 7 1700X
• RAM: 64GB DDR4-3200 (non-ECC, unfortunately)
• Storage adapter: 4x onboard SATA plus 4x SATA via LSI 9211-4i SAS HBA
• Storage:
  • Boot drive: Samsung 850 Evo 250GB SATA SSD
  • Samsung 960 Evo 250GB M.2 NVME SSD
  • 2x Crucial MX100 256GB SATA SSDs
  • 5x Seagate IronWolf 12TB HDDs (ST12000VN0008)

Intended workload​

• General purpose SMB shares for Windows and Mac users, mostly just personal files and photos
• Plex server (I know I lack support for hardware transcoding, so far it hasn't been too much of an issue)
• Traefik and maybe Pi-Hole for local DNS, etc
• Maybe a Minecraft server for veeery occasional use

Data protection​

• Regular scrubs, snapshots, and SMART tests
• Replication of important datasets to old NAS1 offsite
• Regular cloud syncs of important datasets to B2

My question is mainly, how should I set up my pool or pools on the new system? For reference, my old system (PCPartPicker) has 4x 4TB drives in RAID-Z1 as the main documents+media+plex pool, as well as the 2x Crucial SSDs mirrored as a data pool just for my ix-applications dataset. Given the hardware I have available, I have been considering a couple of options:

1. Single pool for everything, but using all 5 HDDs:
   1. 5x 12TB HDDs in RAID-Z2
   2. 250GB NVME SSD as L2ARC (do I need this? would it hurt more than help?)
2. Single pool for everything, but only using 4 HDDs:
   1. 4x 12TB HDDs in stripe of 2 mirrored Vdevs
   2. 250GB NVME SSD as L2ARC
   3. 2x 256GB SATA SSDs mirrored as special Metadata + small files Vdev
3. Similar to NAS1, separate pools for data and ix-applications:
   1. 4x 12TB HDDs in RAID-Z1 (?)
   2. 250GB NVME SSD as L2ARC
   3. Separate pool containing 2x 256GB SATA SSDs mirrored for ix-applications
4. Something else??

(Note: my chassis has 5x 3.5" bays and only 1x 2.5" bay, so I would sacrifice one of the 3.5" bays in order to mount the 2x 256GB SATA SSDs)

It seems to me like Option 1, perhaps minus the L2ARC, is probably the most conservative and simple one, and might be more than enough for my use case. However, Option 2 tickles my performance bone and seems like fun (until it goes horribly wrong and eats my data / burns my house down). Option 3 is basically what I already have in NAS1, which seems to work fine but maybe could be improved.

I'm aware that my setup is probably far from optimal and that I've most likely made some dumb decisions w.r.t. hardware purchases or software configuration. I appreciate any advice given, even if that advice is that I'm overthinking things! Thank you!

 

[Heracles]

how should I set up my pool or pools on the new system?

First thing will be to set a single pool. The only reason to have multiple pools is when you need different pool-level options for two cases. Also, with only 5 disk, there would not be much options : one mirror and one Raid-Z1 pool. The Raid-Z1 pool would not be protected properly.

That means the only reasonable option is for a single Raid-Z2 pool. (your option 1)

Option no2 would not use all your drives and would have a lower protection. If it would gain from 2 vdev instead of 1, the difference will not matter much for your use case. Actually, should you have any IO intensive need, 2 vdev would not be enough for that anyway.

Option 3 is bad. 12TB drives are way too big for Raid-Z1.

As for the L2ARC, you have significant RAM already for what you describe as a light usage. You sure can go for it but it is not of a major importance.

Good that you plan for your backups in the process. Better to do it now instead of after you lost everything like so many try to do...

 

[jordiorlando]

First thing will be to set a single pool. The only reason to have multiple pools is when you need different pool-level options for two cases. Also, with only 5 disk, there would not be much options : one mirror and one Raid-Z1 pool. The Raid-Z1 pool would not be protected properly.

That means the only reasonable option is for a single Raid-Z2 pool. (your option 1)

Option no2 would not use all your drives and would have a lower protection. If it would gain from 2 vdev instead of 1, the difference will not matter much for your use case. Actually, should you have any IO intensive need, 2 vdev would not be enough for that anyway.

Option 3 is bad. 12TB drives are way too big for Raid-Z1.

As for the L2ARC, you have significant RAM already for what you describe as a light usage. You sure can go for it but it is not of a major importance.

Good that you plan for your backups in the process. Better to do it now instead of after you lost everything like so many try to do...

Thanks for the feedback!

I‘m tempted to agree that Option 2 is overkill, so I should just go with my initial plan of all 5 drives in Raid-Z2. Maybe I’ll try with and without L2ARC and see if I can notice any difference.

As for RAM, I had initially gone with 32GB but got scared into purchasing a 64GB kit due to the 1GB/TB of raw storage rule-of-thumb, plus the extra space required to index the L2ARC. Do you think 32 is sufficient?

And yeah, I don’t skimp on backups. I’ve had enough friends and family lose precious photos. It’s an awful experience.

 

[Arwen]

While their used to be a suggestion for 1GByte of memory per 1TByte of disk, it's just that, a suggestion. Today, it is not really relevant.

One so called "rule" for a L2ARC is to max out memory first, before adding a L2ARC. Any entry in L2ARC requires memory for the pointers / table entries for the L2ARC entries.

Unless you know for sure, which sound like you don't, skipping the L2ARC and going with less memory, (16GB or 32GB), can be a good option. Then you check the statistics and see how long it takes ZFS to fill the ARC, (in memory). Then, if appropriate add memory and later potentially a L2ARC drive.

Remember, in some use cases, like random file access, the in-memory ARC does not help much. It's repetitive file retrieval that builds entries in ARC.

 

[Davvo]

I would go ECC. Don't use L2ARC.