BUILD 20TB Rackmount Build, am I headed in the right direction?

Status
Not open for further replies.

VWITEA

Cadet
Joined
Aug 5, 2014
Messages
3

D G

Explorer
Joined
May 16, 2014
Messages
60
Welcome to the forum!

You pretty much can't go wrong with Supermicro and a Xeon :) Not only are Supermicro motherboards the gold standard on here, but Supermicro chassis hold a special place in certain member's hearts. Just be sure to read the guides that are available, especially cyberjock's presentation he put together. A handful of the pros on here have spent a lot of time putting together not only hardware recommendations, but just general best practice for configuration and use of FreeNAS. Humbly follow their advice and you'll be set :)

As for the RAM, there's been a lot of buzz about Kingston RAM. They used to be the go-to RAM for FreeNAS builders, but apparently they've been caught doing some shifty things with quality of their SSD's and apparently their RAM is off the suggested list. People seem to be looking for Samsung or Hynix ECC lately.

That said, I did my build back in May and bought Kingston RAM (it seems like that's about the time the concerns started coming up, and I didn't find out about it until later), and it seems to be doing just fine. I wouldn't risk it though. Kingston is the ECC RAM that's easy to find, but look a bit harder to find a set that's going to be more reliably good.

Also, one important consideration is to get an UPS. With all that data being constantly backed-up, it seems like it would be extremely undesirable to lose it all from a power outage. That won't always happen, but it can. It just seems silly to spend all that money to build the server but not add the extra layer of protection you get from having an UPS.

You also know that having 21TB of disk space in RAID-Z2 won't give you 20TB of storage, right? Just checking :)
 

titan_rw

Guru
Joined
Sep 1, 2012
Messages
586
I don't think that MB / CPU combo supports 16 gig dimms, even if you're only running 32 total.

The max is 32, and I think that has to be achieved via 4x 8gb dimms.

I could be wrong though.
 

Mr_N

Patron
Joined
Aug 31, 2013
Messages
289
I'd go with a M1115 controller if you can find one cheap, any 1015's around are getting a bit old these days.

Also 7 drives isn't a recommended number for RaidZ2 volumes, if you need 20tb formatted space i'd consider the following:

1 vdev of 10x3TB drives or 2 vdev's of 6x3TB drives will give you ~ 22.3TB
 

Dennis K.

Explorer
Joined
Feb 17, 2014
Messages
70
  • The linked mainboard only supports unbuffered ram, but your linked modules is registered ram. They are not compatible.
  • You might want to consider the newer X10 series, like the X10SL7-F. It already has a LSI 2308 chip on board which gives you a total of 14 SATA (8 SAS) ports (which still needs to be flashed to IT mode)
  • You might want to reread http://doc.freenas.org/index.php/Hardware_Recommendations for determing your pool configuration. Especially on how many drives you need.
 

Dennis K.

Explorer
Joined
Feb 17, 2014
Messages
70
  • There are reports in this forum regarding an incompatibility between Supermicro X10 and Kingston ram, as DG already mentioned. Filling only 2 slots may run fine, but there are definitely problems when using all 4 slots. Just search for the topic. Or you just use ram listed as recommended by Supermicro, Although they only lists some Hynix modules for that particular model. I just wont use Kingston and buy Hynix or Samsung modules.
  • Yes, a 10-drive raidz2 vdev will be optimized for data safety. If you need more safety you can consider a raidz3 with 11 drives. If you need more performance you can go with two raidz2 vdevs with 6 drives each. The performance topic is also nicely discussed here. It's up to you what you want/need.
  • I don't have any recommendations for Chassis/PSU
Edit: There's also a huge topic on that Mainboard: http://forums.freenas.org/index.php?threads/supermicro-x10sl7-f.14105/ Maybe you can find some RAM recommendations in there. I think Crucial RAM was mentioned as working.
 

titan_rw

Guru
Joined
Sep 1, 2012
Messages
586
1 Vdev of 10 drives (For better redundancy, less performance?)

Depends on your needs.

For the best redundancy, a single mirror vdev consisting of a 10 way mirror would be best. Capacity of 1 drive, however any 9 of them can fail. Also reads would be balanced between all 10.

That's the extreme anyway. There's always a trade off between redundancy, capacity, speed and expense.

For critical stuff (not a home movie archive), I'd be a bit leery of a 10 drive z2. I'd rather go 11 drive z3 as Dennis mentioned, or since you have a 12 drive chassis, 2 vdev's of 6 drives in z2. Total of 4 drive redundancy, but only if they're the right 4. It'll be a little bit faster as there's 2 vdevs instead of 1 huge one.

I have both. One of my nas's is 11 drives in z3 (3tb drives). The other is 12 drives in double 6 drive z2 (still 3tb drives). Both have the same capacity (8 drive). In the first, any 3 drives can fail. In the second, it's only ANY 2, but possibly 3 or 4 depending on which drives fail. And random iops should theoreticaly be a bit better on the second example (2 vdev's instead of 1).
 

VWITEA

Cadet
Joined
Aug 5, 2014
Messages
3
Great! You guys have been really helpful. I was worried about being flamed/ignored for my nooby post. I do my best to search, but some things can't be searched if you don't have the background knowledge.

I'll definitely take the advice on the ram.

As far as the chassis goes. I would like to find a less expensive 12x 3.5" drive rackmounted solution that will work with this board. I only chose the Supermicro because I assume it will work well with their motherboard. I am not even positive that they are compatible though.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,455
I have that chassis (in black), and it's working very well for me so far. Way more power than is necessary for your build, but that won't hurt anything. I was able to find it on eBay for around $300US shipped, so you'll want to look around.
 

Dennis K.

Explorer
Joined
Feb 17, 2014
Messages
70
I'm on the same page as titan regarding the pool configuration. I would go either with a 11 drive raidz3 (24TB usable), or with a striped two 6 drive raidz2 (also 24TB usable) setup.
 

jamiejunk

Contributor
Joined
Jan 13, 2013
Messages
134
If you buy a "super server" from supermicro (basically a logic board and chassis together) you will get 24 hour phone support from them. If you buy them separately you will not. Just an FYI.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Ok round 2

Chassis: Supermicro 12 bay
Motherboard: X10SL7-F-O
Processor: E3-1230V3
RAM: 4x Kingston 8GB 1600Mhz ECC
HDD: 10x WD Red 3TB 3.5" 1 Vdev of 10 drives (For better redundancy, less performance?)

I'm open to a different chassis/psu, but this was the best I could find with the knowledge I have.

Avoid Kingston like the plague for X10 motherboards.

Check the Tested Memory List for recommendations and note that the 1.35V DIMMs recommended by Crucial for that board are the exact same DIMMs listed as "Micron".
 
Status
Not open for further replies.
Top