Go wide or go deep?

[chuggs]

I am building a new NAS system. It will live at my father's house, a considerable way away from me. The primary purpose is twofold: to provide him on-site backup so he can stop juggling external USB drives, and to provide me a place to back up my NAS offsite. He will have no need for transcoding or running VMs. The system needs to be as quiet as possible and as small as possible so it does not get in his way.

I am debating two different options. One uses 8 4TB disks, the other 6 6TB disks. Both would use a single raidZ2 pool. I found enclosures I like for each. The prices are roughly comparable, so I am hoping someone here can help me decide one way or the other.

Option 1:

Processor/Motherboard: SuperMicro X10SDV-2C-TLN2F ($195.00)
Chassis: U-NAS NSC-810A ($219.99)
Memory: Crucial - 16GB (1 x 16GB) DDR4-2400 Memory ($186.99)
OS Storage: Team - L5 LITE 120GB 2.5" Solid State Drive ($34.89)
Storage: Seagate - BarraCuda 4TB 3.5" 5400RPM Internal Hard Drive ($84.99) x 8
Case Fan: Noctua - NF-A12x25 PWM 60.1 CFM 120mm Fan ($29.90) x 2
HBA: LSI SAS 9211-8i 8-port 6Gb/s PCI-E Controller Card ($44.88)
Cables: Athena Power GPU Server 6G Data Cable for RM-3U8G1043 0.5M Mini SAS ($16.98) x 2
Power Supply: SeaSonic SS-350M1U 350W 80 PLUS ($64.28)
Total: $1519.71

One advantage with this approach is that if a disk needed to be replaced, it would be much easier for my Dad to do in this case than it would be on the Node 304. In addition, I wouldn't need to mess with USB drives for the OS. The USB drives in my current build have given me fairly consistent trouble over the years.

Option 2:

Processor/Motherboard: SuperMicro X10SDV-2C-TLN2F ($195.00)
Chassis: Fractal Design - Node 304 Mini ITX Tower Case ($84.99)
Memory: Kingston - 16GB (1 x 16GB) Registered DDR4-2133 Memory
OS Storage: PNY Attache USB 2.0 Flash Drive, 32GB/ BLACK (P-FD32GATT03-GE) ($7.99) x 2
Storage: Seagate - Barracuda 6TB 3.5" 5400RPM Internal Hard Drive ($167.66) x 6
Power Supply: SeaSonic - FOCUS Gold 450W 80+ Gold Certified Semi-Modular ATX Power Supply ($53.99)
Total: $1355.92

I really like the Node 304, but I don't think my Dad would be enthusiastic about switching a disk in it. Also, no HBA means I am stuck with USB for the OS.

 

[Dice]

Hello!

Looks like you've done a fair amount of work and thought. Great.

I've a few cents for you:
4TB's are not the way to go in this case, as you can afford to run 'deep' and get a larger drive.
I've a few reasons. 4TB's are on the brink on passing over the 'sweet spot' to 6TB drives (or in some cases, 8TBs even). Wider vdevs should preferably be avoided where possible (at least <7) due to the negative impact on resilvering, heat and possibility to get a slow drive in the mix causing a slow experience.

Generally, it is cool to look at and feel the idea of tremendous amounts of data packed in a little Node box... However, once maintenance is needed, it is really not all that fun anymore.
I (among many on the forums) suggest avoiding mITX cases.
A far better contender (and more commonly used these days) are the larger Fractal Design cases.

I'd really avoid anything Seagate other than Ironwolf, and even those due to their SMART attribute mayhem (you'll need their proprietary software to properly interpret their values - which is not supported on FreeNAS). The preferred drives are WD RED, since they both run cool, reliably and has a functioning SMART output.

I'd avoid using a USB for Os-drive.

 

[Inxsible]

Generally, it is cool to look at and feel the idea of tremendous amounts of data packed in a little Node box... However, once maintenance is needed, it is really not all that fun any more.

Agreed. Ask me how I know !

I (among many on the forums) suggest avoiding mITX cases.

Agreed again. ATX and m-ATX give you room for expansion later which m-ITX don't.

I'd really avoid anything Seagate other than Ironwolf,

I disagree here. Many users have used desktop seagate drives. I use Seagate Constellations without issues. I also use 2x 6TB Seagate Ironwolfs. I will say though that the Ironwolfs run super hot. In Winter they ran at 38C and now in the summer they sometimes touch 48C-49C. I freaked out when I saw those temps. I use stock fans on the Node 304 running them at high in summer and medium in winter.

I really like the Node 304, but I don't think my Dad would be enthusiastic about switching a disk in it.

Since your dad is going to be the one to do the maintenance -- then don't use Node 304. Cabling is difficult in the Node 304 to say the least.

Also, no HBA means I am stuck with USB for the OS.

I think you can use a HBA as long as it is half height/half length. Agreed that it will be difficult to find. You'd just have to get 2x breakout cable to connect all the drives. 1 port handling 4 drives and another port handling the remaining 2. If you use a full height, then you can only have 4 drives total

I'd avoid using a USB for Os-drive.

I wouldn't. I use an USB drive on my Node 304 obviously and it's been the same one since I installed FreeNAS the first time about 3.5+ years ago. No issues whatsoever. So if you have spare SATA ports and space in the case, then use SSD. But using an USB is not so bad.

 

[Inxsible]

Chassis: U-NAS NSC-810A ($219.99)

Frankly that's a lot to spend on a case. Rackmount chassis from Supermicro which allow hot-swap are available for less if you go the used route. I know you mentioned you wanted small cases, but even those can be had for cheaper used.

Are you willing to look at used components or do you have to have new and pay more for it ?

 

[Chris Moore]

Chassis: U-NAS NSC-810A ($219.99)

I like this, because drives fail and this (as you said) will make it easier to fix.
The Node case is not easy to service.

Storage: Seagate - BarraCuda 4TB 3.5" 5400RPM Internal Hard Drive ($84.99) x 8

I use the Seagate drives, almost exclusively, because the WD drives give incomplete and misleading SMART status information.
The specific ones I have are the even older model Seagate Desktop drives, like this:
https://www.newegg.com/Product/Prod...1190&cm_re=ST4000DM000-_-22-178-338-_-Product
I bought a 20 pack of them last year. and have 6 in my primary NAS 12 in my backup NAS and 2 cold spares on the shelf.
They have worked fine for me and I really don't know what problem there is supposed to be with the SMART output.
This is what I get by email from my reporting script:

Code:

+------+------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
|Device|Serial			|Temp|Power|Start|Spin |ReAlloc|Current|Offline |Seek  |Total	 |High  |Command|Last|
|	  |Number			|	|On   |Stop |Retry|Sectors|Pending|Uncorrec|Errors|Seeks	 |Fly   |Timeout|Test|
|	  |				  |	|Hours|Count|Count|	   |Sectors|Sectors |	  |		  |Writes|Count  |Age |
+------+------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
 -snip-
|da12  |Z307xxxx		  | 35 | 6704|   51|	0|	  0|	  0|	   0|	 0|  22717947|	 0|	  0|   0|
|da13  |Z307xxxx		  | 36 | 9283|   94|	0|	  0|	  0|	   0|	 0|  24895265|	 0|	  0|   0|
|da14  |Z307xxxx		  | 36 | 8923|   53|	0|	  0|	  0|	   0|	 0|  24608832|	 0|	  0|   0|
|da15  |Z305xxxx		  | 37 | 6592|   22|	0|	  0|	  0|	   0|	 0|  28286562|	 0|	  0|   0|
|da16  |Z307xxxx		  | 38 |10204|   86|	0|	  0|	  0|	   0|	 0|  26939081|	 0|	  0|   0|
|da17  |Z305xxxx		  | 37 | 6577|   20|	0|	  0|	  0|	   0|	 0|  27825059|	 0|	  0|   0|
 -snip-
+------+------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+

I have 36 drives in this system, so only showing the relevant ones.

For my purposes, the next time I buy drives, it will likely be the 8TB model, still Seagate. Since I just bought these 4TB ones last year and it will be a little while before I expand the array again.

For what you are doing, this doesn't sound bad.

 

[chuggs]

Wider vdevs should preferably be avoided where possible (at least <7) due to the negative impact on resilvering, heat and possibility to get a slow drive in the mix causing a slow experience.

Could you expand on this a bit? My (limited) understanding was that resilvering was actually more of an issue for smaller vdevs with large disks.

Generally, it is cool to look at and feel the idea of tremendous amounts of data packed in a little Node box... However, once maintenance is needed, it is really not all that fun any more.
I (among many on the forums) suggest avoiding mITX cases.
A far better contender (and more commonly used these days) are the larger Fractal Design cases.

I hear you (and others). My personal build is in a larger mATX Fractal Design case. But space is a major consideration for this build. It has to tuck in to my dad's limited office space. I have built two SFF desktop machines before, so I am not too concerned about it the size.

 

[chuggs]

I wouldn't. I use an USB drive on my Node 304 obviously and it's been the same one since I installed FreeNAS the first time about 3.5+ years ago. No issues whatsoever. So if you have spare SATA ports and space in the case, then use SSD. But using an USB is not so bad.

Sadly, I have not been so lucky. My current Freenas system (which is of similar age as yours) started with two SanDisk Cruzers. Both have failed, as have one of the replacements. Even if they haven't disappeared from their vdev entirely, I will sporadically have issues where the machine fails to boot correctly due to failures talking to the drives. If I have to use USB drives I will, but I would prefer to avoid them if possible.

 

[chuggs]

Frankly that's a lot to spend on a case. Rackmount chassis from Supermicro which allow hot-swap are available for less if you go the used route. I know you mentioned you wanted small cases, but even those can be had for cheaper used.

Are you willing to look at used components or do you have to have new and pay more for it ?

No argument about the cost. That said, it is a niche product made by a smaller company, so it isn't completely surprising. As mentioned above though, size is a consideration, and rack mount is totally out of the question in the space I have to work with.

I am willing to look at used components. In fact, the X10SDV-2C-TLN2F and LSI SAS 9211-8i will both be purchased second hand.

 

[MrToddsFriends]

In fact, the X10SDV-2C-TLN2F and LSI SAS 9211-8i will both be purchased second hand.

I would expect heat/airflow problems with these components in a cramped case like the U-NAS NSC-810A. Be prepared to have a close look on this while performing burnin tests.

 

[IQless]

This is what I get by email from my reporting script:

Would this script be available for the public? :P

 

[Inxsible]

Would this script be available for the public? :p

https://forums.freenas.org/index.ph...d-identification-and-backup-the-config.27365/

 

[chuggs]

I would expect heat/airflow problems with these components in a cramped case like the U-NAS NSC-810A. Be prepared to have a close look on this while performing burnin tests.

That concerns me as well. I am hoping that by using the 2 core variant of the Xeon-D that I avoid some heat issues there, but I have no experience at all with HBA cards. Do they typically have sensors that are accessible by the OS for monitoring purposes?

 

[Bidule0hm]

I use the Seagate drives, almost exclusively, because the WD drives give incomplete and misleading SMART status information.
[...]
They have worked fine for me and I really don't know what problem there is supposed to be with the SMART output.

I agree, WD drives aren't bad on the SMART infos but they don't have as much info as the Seagate drives. And you don't see problems with my script because I interpret the weird raw values with the formulas found here http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html

NB: after a few years of comparing the WD and the Seagate drives I'd say I prefer the WD ones because they run just a bit cooler and quieter but honestly both are great drives brands, I'm fine with either, I'm just being really picky here.

 

[Chris Moore]

I prefer the WD ones because they run just a bit cooler and quieter

I found it the other way around, but I was using a mix of the Wd Re 2TB (Enterprise Storage) drives and the HGST Ultrastar 2TB drives that all ran at 7200RPM. All of those were hot for my taste, when I compared them to the temp of a batch of Seagate drives in the same kind of chassis... The Red Pro drives we have in the data center at work get pretty warm also.

There are options and it really doesn't make that much difference what drive you go with. They either fail or they don't. The thing that I didn't like about the WD drives may not even be applicable to the drives they make now because it has been five years now that I have been using exclusively Seagate drives at home and I have to take what I get at work. Last time I asked for drives, I got a totally different model than what I asked for. They bought the WD Gold Datacenter drives as spares for a server that came with WD Red Pro drives. I wanted to match what was already in it, but they got the others because they cost less at the time.

 

[MrToddsFriends]

That concerns me as well. I am hoping that by using the 2 core variant of the Xeon-D that I avoid some heat issues there, but I have no experience at all with HBA cards. Do they typically have sensors that are accessible by the OS for monitoring purposes?

HBAs typically don't have such sensors, AFAICT. See also
https://forums.freenas.org/index.php?threads/cooling-for-a-dell-perc-h310.61941/

Most Denverton (Atom C3000) boards have 8 or 12 SATA connectors, eliminating the need for a separate HBA in your case.
https://www.supermicro.nl/products/motherboard/ATOM/

 

[Chris Moore]

HBAs typically don't have such sensors

No, they don't. Even running the Avago MegaRAID Storage Manager on a Windows server with a LSI 9271-81 Hardware RAID controller, it doesn't tell you a thing about the temperature of the controller, but you can see the temperature of each of the drives.
These controller cards rely upon airflow to keep the chip cool. It is usually a bad thing to put them in small chassis with poor airflow.

 

[Chris Moore]

Would this script be available for the public? :P

There is a link to the scripts under the useful links button in my signature.
Credit where it is due, I didn't write it, @Bidule0hm wrote it and the version I linked to is a modified/updated version.

Sent from my SAMSUNG-SGH-I537 using Tapatalk