SOLVED Picking Storage Drives

[spiceygas]

I'm looking for input on my first FreeNAS build. I've got experience with traditional RAID, but this is my first time on ZFS.

Usage

• Location: Home
• Primary Use: Media storage (i.e. Data at rest. More reads than writes)
  • 1-5 users
• Secondary use: 2-5 VMs
  • 1 user
• Goals: Maximize storage, reasonable rebuild time in event of drive failure.

System

• 2x Xeon 2680v3
• 128 gigs RAM
• 10 gb copper LAN
  • Clients are limited to 1 gb
  • Internet bandwidth is limited to 300 mb
• Case that can handle 36 hotswap drives
• Separate UPS

Proposed storage

• OS Install
  • SATA DOM
• ZFS Pool 1 (For bulk storage)
  • VDev 1 - RAIDZ2
    • 5x 14 TB Drives
  • VDev 2 - RAIDZ2
    • 5x 14 TB Drives
• ZFS Pool 2 (For VMs)
  • VDev 1 - MIRROR
    • 2x SATA SSDs (Probably 1 TB each)

Rationale

1. In case of drive failure and rebuild, each VDev in Pool1 is isolated, reducing the time needed to re-silver.
2. Two drive redundancy in data is sufficient for my home media usage. Each VDev is 42TB (14 x 3) unformatted storage.
3. Adding another VDev to Pool 1 for future growth will be easy.
4. No dedicated SLOG. I anticipate it to be unnecessary given my light workload, but can add it on later.
5. 128 GB RAM seems sufficient given light workload, but the mobo has plenty of room to expand if necessary.

Thoughts? Am I making any poor assumptions?

Questions

1. Is more drives per VDev appropriate? What's the general tolerance towards number of drives in RAIDZ2, and balancing resilver time against desire to minimize space "wasted" (invested?) on redundancy?
2. Do folks agree that an SLOG is likely unnecessary in this build?

 

[Samuel Tai]

This is a reasonable layout.

 

[sretalla]

Is more drives per VDev appropriate?

6 is probably mathematically better for RAIDZ2, but in real-world terms not much of a difference other than a slightly better data to parity ratio.

What's the general tolerance towards number of drives in RAIDZ2, and balancing resilver time against desire to minimize space "wasted" (invested?) on redundancy?

Thinking is that 12 is as high as it makes sense to go and 6 or 8 are some kind of sweet spot.

Do folks agree that an SLOG is likely unnecessary in this build?

Yes, seems so.

 

[spiceygas]

Very helpful. Thank you both.

I am now wondering if it would be better to put each VDev into separate pools. In the worst case of 3 drive failures in a single VDev, it would then only take down that one pool. Of course, it would result in two separate pools of storage, but as long as I don't mind dealing with that as a user, it seems "safer." No?

6 is probably mathematically better for RAIDZ2, but in real-world terms not much of a difference other than a slightly better data to parity ratio.

Cool. I might increase the number of drives per VDev, then. The big wildcard in my mind is resilvering time when a drive fails. I've done some googling but cannot seem to find reliable stats on rebuild times.

 

[Constantin]

I lean towards multiple VDEVs in a pool vs. multiple pools. Granted, the latter has a measurable benefit re: fault tolerance over a 2-VDEV layout but in a home setting where you can swap in a drive quickly for a resilver, that seems somewhat unnecessary. The key is to have pre-qualified cold spare or two drive handy for emergency use.

I do wonder if your use case will result in a lot of power draw for those CPUs and how much you'll actually stress them with 2-5 VMs. But I have seen some very energy efficient systems here using standard Xeon processors, that make my D-1537 look greedy.

 

[spiceygas]

I lean towards multiple VDEVs in a pool vs. multiple pools. Granted, the latter has a measurable benefit re: fault tolerance over a 2-VDEV layout but in a home setting where you can swap in a drive quickly for a resilver, that seems somewhat unnecessary. The key is to have pre-qualified cold spare or two drive handy for emergency use.

I see your point and will mull it over.

Thank you for the insights and for humoring what probably seems like simple questions.

 

[Constantin]

Not a simple question at all considering how long you have to live with the results. 14TB drives are not cheap and presumably you have some sort of a backup set also. For me, the added benefit of speed via the 2-VDEV pool outweighs the risk vs. a 2-pool, 1-VDEV each approach.

Looking ahead, it will likely be easier, less costly to take advantage of SSDs to speed up your pool performance with TrueNAS 12.x via special VDEVs with a single pool rather than multiple pools. This may or may not be interesting in your use case.

Like others, I would consider going for a 6-drive Z2 VDEV config though to reduce the parity-penalty.