BUILD 1RU Build Advice - Restart Thread, Have Existing 1RU Chassis now

[adamgoldberg]

I'm preparing to build a server for backups, CFS file sharing for one user, and Plex for as many as 3 simultaneous streams.

I have a rack in the basement which serves as a networking closet -- this is where everything terminates, and sound isn't a big problem -- but I do want to continue moving away from tower/minitower on a shelf in the rack to regular rack-mounted devices. I have a Dell R210-II for my Exchange server, for example.

Here's my current thoughts:

MB: X10SLH ($208ish) is a minor upgrade from the X10SLL (~$170), but I could go either way
CPU: E3-1230V3 3.1ghz (~$250). Seems like the graphics stuff in the E3 v4s wouldn't be useful (unless Plex transcoding benefits from this?)

Case: 813MTQ-350 (~$275). Is 350watts enough? (Or, is 29amp +12v enough?) There's enough fans in here to keep things cool, and I don't think things are too loud probably. Is the low profile passive heat-sink ok (SNK-P0046, ~$25) or is it worth getting an active sink (SNK-P0049A4, ~$35)? I think both will fit in the 1RU case. If there were an identical case with non-hot-swap drive bays, I'd be interested -- is there such a thing (nb, I'm already sold on supermicro cases)?

HD: $/TB seems to favor the 3TB WD Red, but the 4TB aren't too much more (less than $1/TB more), which would yield a few more TB of space. Qty: 4.

Boot disk: an internally-mounted thing. Either a USB drive or a SATA DOM, but the DOMs cost a few bucks more. Thoughts?

RAM: 2 @ 8GB (CT4486296 8GB DDR3-1600 ECC UDIMM). No need for 32 GB, I think.

As I've got this priced out, it comes to about $1325.

 

[adamgoldberg]

Struggling through the motherboard line, and it seems like the X10SL7 might be a better choice. It's about the same price as the X10SLH, but instead of the 6 @ 6Gbps (X10SLH) it has the 2@6, 4@3Gbps of the X10SLL plus 6 @ 6Gpbs SAS/Sata via the built-in LSI 2308. That seems like a better idea (even though there's not much way to use all 12 (!) ports in the 1RU case.

 

[joeschmuck]

Boot disk: an internally-mounted thing. Either a USB drive or a SATA DOM, but the DOMs cost a few bucks more. Thoughts?

I wouldn't use a DOM, instead purchase a budget SSD, any size will do, whatever is cheapest and the sales are good right now. But a USB Flash drive is fine too if you want to save a few bucks, however I prefer a SSD if you have the SATA port available.

16GB RAM is probably fine however since FreeNAS 10 will be coming out in the near future, and if you feel you may do more with your device, 32GB would be a safer bet however it's your money and you could add two more 8GB sticks later if you find out you need more.

The MB should be fine, you are only looking for four hard drives and there are six SATA ports so that works.

Before you purchase anything, ensure you have an idea of how much storage space you will desire over the next 3 years and then double it. This will be the safe value and then you can figure out how many hard drives you really need and their size. And you need to choose the redundancy level, RAIDZ1 or RAIDZ2. We here will push you towards RAIDZ2. This means that with four 6TB drives you would have about 10.5TB of storage. With 4TB drives that would be about 7.1TB of storage. So storage capacity is a must to know ahead of time, it's not cheap to upgrade at a later date and adds risk.

 

[danb35]

What benefit are the extra 8 ports of the X10SL7 if you're in a 1U chassis that only holds 4 drives?

 

[joeschmuck]

8 extra ports just gives you the flexibility to expand into a different chassis later, if you desire that to add more storage.

 

[adamgoldberg]

8 extra ports just gives you the flexibility to expand into a different chassis later, if you desire that to add more storage.

This is what I was thinking about... but the X10SL7 is the same price as the board I was looking at, and the 833T-653B is a 2u 650W, 8 drive hot-swap case and adds about $125 to the cost. So, I'd be half-populated with drives (4 x 3TB, 4 empty), and have enough ports on the board and enough space in the case to double the number of drives... Or, alternately, 3U supermicro 8-drive cases seem rather available on ebay...

My current WHSv1 server has 2.5TB of drives, and it's about full (7% free). 4x4TB at RaidZ2 is just more than 7TB of storage, between 2-3x what I currently have. That might not give enough breathing room...

Ugh. I thought I was zeroing in on what I needed. Maybe not.

 

[tvsjr]

Buy something 2U at least... 8-12 bays. You'll end up needing them sooner or later :)

 

[jgreco]

MB: X10SLH ($208ish) is a minor upgrade from the X10SLL (~$170), but I could go either way
CPU: E3-1230V3 3.1ghz (~$250). Seems like the graphics stuff in the E3 v4s wouldn't be useful (unless Plex transcoding benefits from this?)

Look at the E3-1231 V3. Small bump up, same price.

Case: 813MTQ-350 (~$275). Is 350watts enough? (Or, is 29amp +12v enough?) There's enough fans in here to keep things cool, and I don't think things are too loud probably.

It's a good chassis except that it only holds four drives.

Is the low profile passive heat-sink ok (SNK-P0046, ~$25) or is it worth getting an active sink (SNK-P0049A4, ~$35)? I think both will fit in the 1RU case.

Okay, so, let's see.

One option has a heat sink with a LOT of surface area and redundant fans cooling that surface area.

The other option has a tiny heat sink with a single squirrel cage fan. When that squirrel cage fan eventually fails, it becomes a big hot blob on top of the CPU.

I dunno, which seems better...

If there were an identical case with non-hot-swap drive bays, I'd be interested -- is there such a thing (nb, I'm already sold on supermicro cases)?

Not really, the SC812L is fixed three bay but I don't think there's a four bay, and the prices aren't remarkably lower.

HD: $/TB seems to favor the 3TB WD Red, but the 4TB aren't too much more (less than $1/TB more), which would yield a few more TB of space. Qty: 4.

$/TB typically favors the larger drives. Be sure that you're aware that the cost per TB for your NAS is an integral part of the cost per TB of storage.

So you get some 3TB Red's and let's say your NAS platform costs $800. Four 3TB red is $400, total platform cost is $1200 to get 6TB of space, or $200/TB. 5TB Red is $200/ea, or $800, total platform cost is $1600 to get 10TB of space, or $160/TB. 6TB Red is $250/ea, or $1000, total platform cost is $1800 to get 12TB of space, or $150/TB. Just an example of how to do that math.

Because the nasty truth is that once you fill those four bays, your two options are to pull the existing drives and upgrade them, which basically costs even MORE, or to add a second NAS.

Boot disk: an internally-mounted thing. Either a USB drive or a SATA DOM, but the DOMs cost a few bucks more. Thoughts?

The USB stuff is prone to rapid failure. The SSD-stuck-inside solution may work but can be a bit hacky. The SATA DOM solution works great but is ridiculously expensive and eats a SATA port. No truly happy solutions, just choices.

Also as someone else noted, now is the time to contemplate whether perhaps it would be smarter to seek out a 12 bay 2U chassis.

 

[Fuganater]

IMHO buying a 1U chassis for a NAS is a huge waste of money.

 

[depasseg]

IMHO buying a 1U chassis for a NAS is a huge waste of money.

Unless you have JBOD expansion chassis in mind.

 

[adamgoldberg]

IMHO buying a 1U chassis for a NAS is a huge waste of money.

Yea, ok, message received.

So, above specs except a 2ru or 3ru supermicro case, 8 or more drive slots. How big of a power supply is necessary? 350 or so still enough?

 

[jgreco]

https://forums.freenas.org/index.php?threads/proper-power-supply-sizing-guidance.38811/

 

[adamgoldberg]

Ah very helpful.

 

[tvsjr]

Unless you have JBOD expansion chassis in mind. :)

Most people aren't as nuts as you... (pay no attention to the two external ports hanging off my HBA, just waiting for an expansion chassis or two :D)

 

[depasseg]

I think @pclausen takes the cake on using expansion JBOD's.

 

[Fuganater]

Unless you have JBOD expansion chassis in mind. :)

Not really. 1U chassis are SO expensive for what you are getting. You can get a 2U or 3U for either the same price or like $50 more and it will be far more useful in the future.

I think @pclausen takes the cake on using expansion JBOD's.

Absolutely YES!

 

[jgreco]

Not really. 1U chassis are SO expensive for what you are getting. You can get a 2U or 3U for either the same price or like $50 more and it will be far more useful in the future.

Depends. The capacity of our 2U SC813A-R740B + X10SRW hypervisor boxes, for example, includes space for 16 drives and five x8 PCIe, four of which are full height slots. The 1U Supermicro 1018R-WC0R includes space for 10 drives, two x16 full height, and one x8 technically-half-height. That's an amazing amount of crap to pile into a 1U!

The 1018R-WC0R is almost exactly the same price, so if you look at it on a per-drive-bay or per-expansion-card basis, yes, you're correct, it's more expensive, but if you're looking at it from a per-host perspective, it's a wash. And in the data center, where rack costs of ~$2K/month translate to $45 per rack unit, the savings of going with a 1U over a 2U is $2700 over a reasonable lifetime (5 years service). Per machine.

Right now, large FreeNAS systems for inexpensive mass storage still favor 3.5" HDD's and that'll probably be the case for the indefinite future, but at this point, smaller filers are quite possible. You can do a 12TB-usable RAIDZ3-with-hot-spare built out of 10 x 2TB drives with a 1018R-WC0R. In 1U. Or a 4TB-usable mirror setup built out of 10 x 960GB SSD's (SanDisk Ultra II recently seen for $199!) for lightning fast high end NAS.

Of course, all of this assumes 2.5" drives. For 3.5", the typical 4 drive limit is definitely hurt-y. But I do have an ASRock 1U12LW around here somewhere ;-)

 

[adamgoldberg]

Ok, 1RU seems to be a poor choice. See near-future thread on a 3u system :)

 

[adamgoldberg]

Ok, I'm going to try this again/restart this thread.

Above, I wrote

I'm preparing to build a server for backups, CFS file sharing for one user, and Plex for as many as 3 simultaneous streams.

This is still true, but some additional information: I'm replacing a Windows Home Server (v1) that has 3TB of storage (about 2TB in use, which includes all the workstation backups), which is used for workstation backups, and a file server for a Plex server (the server is running on a Mac, but using the WHS share as the storage, which is suboptimal). I also have a Win2008r2 server that runs Exchange, and is also a file server for other storage. Currently the Win2008r2 server backs up to a locally-connected drive using Windows Server Backup.

I have several workstations to backup -- a total of about 3TB of raw storage on the workstations.

I had a need for a 1RU server for other reasons, and was able to pick up a 813MTQ-350CB with an X9 motherboard (forget which one) and an e3-1230 w/16G RAM and four drives for a good price ($200). Such a good price, I bought two (one for the original use, and one to set up a test FreeNAS system until I got my sh--t together).

It turns out one of the systems came with 4x2TB drives, so I installed FreeNAS and got a CIFS share working and Plex, and got a trial of Acronis Backup. I set it up with all four drives in a Raidz2 configuration, so it has about 4TiB available storage using the 2TB drives.

Above, when we first had this conversation, you guys cautioned me about the cost-per-TB of storage of a 1RU system. I still think that makes sense (that is, you guys are still correct generally), but given that my needs are to replace 3TB of storage capacity on the WHS, plus a bit more, and I already have a 1RU chassis/mobo/memory, I'm thinking I can maybe just proceed with the existing hardware (plus new drives).

I was looking for a Supermicro 2U (12-bay) or 3U (16-bay) system (following the earlier guidance), but what's available seems to be about $500-600 for a system like that with a X8 motherboard. There's some guidance about heat, power & speed about these older mobos, that it's probably better to go with an X9 (or X10)... so to get equivalent CPU performance in a 826 or 836 chassis, I'd be looking at several hundred dollars over and above the 2U or 3U chassis cost.

So, given all that, it seems to me that I might be just fine (at least for now) with the existing 1RU hardware plus 4 new, large WD Red drives (or Red Pro). If I got 4x4TiB drives, I'd have about 6TB available storage. (Bidule0hm's calculator gives 5.728TiB after subtracting 20% minimum free space) (11.4TiB using 6TB drives, but there's some concern about long-term reliability of those drives, right?).

With the deal I got on the 1RU system, $200 plus 4 new drives @ $160 ea (NewEgg WD Red 4TB) yields a system with a total cost of about $147 per TiB and 5.7 TiB capacity -- sufficient capacity for at least the near term.

Am I still being a moron? Or do you think the numbers work out this-a-way?

Adam

PS: This is one of the more helpful forums I've used in recent memory. Lots of knowledgable, opinionated (in a good way), active participants with advice to give. Thanks!

 

[rogerh]

I'm not particularly knowledgeable, but I think at an early stage you need to see how noisy those machines have to be to keep cool in your environment, and whether the noise is acceptable. Apart from that, I think you have a good bargain.