18TB hard drives choice WD NAS vs Seagate Enterprise vs WD Enterprise

[Diff]

Thinking about hard drives for new TrueNAS build. This is second TrueNAS server for my HomeLab, main focus is backup, so speed is not a priority, but reliability is. Thinking to start with 3-way Mirror VDEV, and extend later.

Seems like recommendation in general to use WD Red Pro NAS drives, but with many other choices questions come up and I do not feel I have enough knowledge to answer, so would appreciate any opinions.

Choices
- 18TB WD Red Pro NAS drive is $599 (SATA 6GB), CMR ( ), MTBF 1M hours
- 18TB Seagate Exos X18 Enterprise drive is $393 (SATA 6GB) or $472 (SAS 12Gb), CMR ( ), MTBF 2.5M hours

Seems like Seagate less expensive and also more durable, at least from spec sheet.
Also in WD family, there are more Enterprise class drives, like:

- 18TB WD Gold Enterprise drive, $583 (SATA 6GB), spec states EAMR (??? ), MTBF 2.5M hours
- 18TB Ultrastar DC HC550 Enterprise drive, $649 (SATA 6GB) or $709 (SAS 12Gb), CMR ( ), MTBF 2.5M hours

Q1: What is EAMR? Is that acceptable for NAS use cases? Is WD Gold a reasonable drive to consider?
Q2: Seems like Seagate is a winner here.. Am I missing something?
Q3: Any other advice / considerations?

 

[Etorix]

EAMR stands for Energy-Assisted Magnetic Recording. Nothing wrong as long as it is not Shingled (SMR) on top of it.
Ultrastar DC or WD Gold (should be rebranded Ultrastar DC) are enterprise drives and perfectly worthy of consideration.
SAS buys you nothing over SATA. I suppose you do not have a SAS HBA and do not use a SAS backplane anyway.

Any of the listed drives should be fine. Exos X18 is the clear winner on price.

3-way mirrors are reliable, but quite an expensive choice for backup (33% space efficiency). What is the cost per available TB if you go raidz2 with 4 or more drives of lower capacity? (50% space efficiency and above)

 

[Diff]

Thank you @Etorix !

I suppose you do not have a SAS HBA and do not use a SAS backplane anyway.

Sorry, did not clarify that part. I use for this build Dell PowerEdge R7515, with PERC H730P controller, which supports HBA mode. So it has 12 x 3.5 bays SAS/SATA.

3-way mirrors are reliable, but quite an expensive choice for backup (33% space efficiency). What is the cost per available TB if you go raidz2 with 4 or more drives of lower capacity? (50% space efficiency and above)

My thinking was:
1. Want to start with 3 drives, but Z2 requires 4 at least.
2. With Z2 (4 drives) will have ~50% vs 33%, but in case of failure my understanding would be bigger impact on performance, comparing to Mirror will be almost no impact (only on READs I assume).
3. Next time I expand, I can even do 2-way mirror (2 drives) or same 3-way(mirrors), still a little bit cheaper comparing to minimum 4 drives next time.
4. Seems like most of recommendations (if I understand correctly) for Z2 to use it in 6-10 drives VDEVs.

Does this make sense or I am still missing some understanding of ZFS?

 

[Etorix]

Resilvering raidz# takes more resources than resilvering mirrors. I don't know if there is an impact on using the pool while resilvering but is this a concern for a backup NAS?
Mixing 2-way and 3-way mirrors is possible but not a good idea. The pool is only as resilient as its weakest vdev.
6-10 is a sweet spot for raidz2, but it works from 4 drives.

Your understanding is correct. I'm only questioning the cost per usable TB, given that you want storage but that mirrors are rather a performance choice.
With 12 bays you may target either 4-wide Z2, possibly up to 3 vdevs, or 6-wide, up to 2 vdevs. (If and when raidz expansion becomes possible, there might be a sidesway path to move from 4-wide to 6-wide.)
18 TB is about $400. A 3-way mirror costs $1200 for 18 TB raw space, at $67/TB (usable TB).
Let's say 12 TB drives are $300. A 4-wide Z2 costs $1200 for 24 TB raw space at $50/TB. That's better value, even as the individual drives cost more per TB!
Let's say 10 TB are $275, worse cost per TB than the above 12 TB but approximately matching the prices I see in my local market. A 6-wide Z2 costs $1650 for 40 TB raw space at $41/TB. Higher absolute cost, but even better value due to the better space efficiency.
You may want to check your options with 8-16 TB drives and 4-/6-wide Z2 before committing to 3-way mirrors.

 

[Redcoat]

Please take a look at this thread that addresses the PercH730p and it’s use with TrueNAS https://www.truenas.com/community/threads/zpool-atop-hardware-raid5.94037/#post-650799

 

[Diff]

Resilvering raidz# takes more resources than resilvering mirrors. I don't know if there is an impact on using the pool while resilvering but is this a concern for a backup NAS?

Good point.

Mixing 2-way and 3-way mirrors is possible but not a good idea. The pool is only as resilient as its weakest vdev.

Make sense, noted!

With 12 bays you may target either 4-wide Z2, possibly up to 3 vdevs, or 6-wide, up to 2 vdevs. (If and when raidz expansion becomes possible, there might be a sidesway path to move from 4-wide to 6-wide.)
18 TB is about $400. A 3-way mirror costs $1200 for 18 TB raw space, at $67/TB (usable TB).
Let's say 12 TB drives are $300. A 4-wide Z2 costs $1200 for 24 TB raw space at $50/TB. That's better value, even as the individual drives cost more per TB!
Let's say 10 TB are $275, worse cost per TB than the above 12 TB but approximately matching the prices I see in my local market. A 6-wide Z2 costs $1650 for 40 TB raw space at $41/TB. Higher absolute cost, but even better value due to the better space efficiency.
You may want to check your options with 8-16 TB drives and 4-/6-wide Z2 before committing to 3-way mirrors.

I did not think about looking on it this way and play with different drive sizes and cost per $/TB. Really cool. Thank you for this idea!

 

[Diff]

Please take a look at this thread that addresses the PercH730p and it’s use with TrueNAS https://www.truenas.com/community/threads/zpool-atop-hardware-raid5.94037/#post-650799

With Dell PowerEdge R7515 chassis, all wired up to PERC H730P, can probably buy an HBA330 to replace it, my question is how to actually check if H730P, switched to HBA mode is different from HBA330 itself. That part I do not understand yet, probably need to research more on this.

Actually posted a PERC in HBA mode question on separate thead

RAID controllers HBA mode

It feels like there is a lot of different, sometimes confusing information on different RAID controllers suitable for FreeNAS/TrueNAS and ZFS in general. Following official documentation - https://www.ixsystems.com/blog/hardware-guide/ it states that RAID controllers switched to HBA mode are...

www.truenas.com

 

[phier]

Hi,
what exactly is "4-wide Z2" ? And whats the point of 3 mirror? Isnt 2 drives - ie standard mirror enough?

thanks

 

[NugentS]

"4-wide Z2" is another way of saying a vdev consisting of 4 disks in RAIDZ2 - so 50% capacity, 50% parity
3 Way mirror is the same as a normal mirror expcept an aditional disk in case of failure and 33% capacity. If you have a pool of say 4 mirrors (8 disks total), you can definately lose 1 disk, but the second disk in the same vdev kills the pool. a 3 way mirror and you can lose 2 disks in a single vdev

 

[firesyde424]

With Dell PowerEdge R7515 chassis, all wired up to PERC H730P, can probably buy an HBA330 to replace it, my question is how to actually check if H730P, switched to HBA mode is different from HBA330 itself. That part I do not understand yet, probably need to research more on this.

Actually posted a PERC in HBA mode question on separate thead

RAID controllers HBA mode

It feels like there is a lot of different, sometimes confusing information on different RAID controllers suitable for FreeNAS/TrueNAS and ZFS in general. Following official documentation - https://www.ixsystems.com/blog/hardware-guide/ it states that RAID controllers switched to HBA mode are...

www.truenas.com

In my experience, it's usually not worth messing around with this. The HBA330 and HBA330+ are relatively cheap and require no additional configuration to work with TrueNAS.