Home Server Questions - lots of questions....

Status
Not open for further replies.

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Ok, I am getting closer to purchasing all the hardware for the new FreeNAS setup, but I am trying to figure everything out and have a plan going farther into the future so I can properly plan the build now.

So, a little background...

I have been running a openmediavault nas for years now, it has done its job, and I have had no issues with it, but going into the future the greater "power" freenas and ZFS provide would be much appreciated. Problem is, ZFS is a huge difference coming from a much more standard computer mentality, lots of terms I have not fully digested and then the statistics of proper vdev and pool setup to support the redundancy I want, fun stuff for sure.

My current questions are based around harddrive setup and the chassis required to make it all work in an economical way.

I currently have 11 3 TB drives I will want to use, I already own them, so they are "locked in" to this build. I will want to add more storage space to this immediately, and possibly more in the future and my plan to do this is to buy another 2-3 3 TB drives now to total 13 or 14, and run these in a RAIDZ3 configuration.

First, is this a sufficiently redundant way to do this? Should I instead use two 7 vdev arrays with RAID Z2? (I hope that is the proper use of vdev, I think it goes vdev<Pool?)

But then, going into the future, I will need to expand to a larger amount of storage. So, would it be smarter to just buy say 5 3 TB drives today for a total of 16 3 TB's, set up a single vdev with RAID Z3 for a usable pool size of of 39TB? Or 2 vdev's with Z2 for a total of 36 TB usable? Or, just chill with my 13 or 14 drive pool now (in whatever vdev/Z* I decide) and in a year add in 5 4TB drives in RAID Z2 for an additional 12 TB of usable pool space?

I am just not sure what the best way to go about this is, and how to properly plan for it today. Obviously I will want to plan on utilizing the hardware I buy today properly, so I would want to buy a chassis either with a SAS backplane or a barebone chassis and a HBA card that will support the total amount of drives I plan to have in the future.

Another question that I suppose should be asked in this same thought process, how much RAM is required for such a setup? I know I know, 1 GB per 1 TB, but does that scale 100%? If I buy 2 16 gig kits and fill out my motherboard with 32 gigs of RAM today, and end up with ~39TB usable space in the future, will I have kneecapped myself? Just get 2x16GB kits today so I can add another 16 gigs in the future if needed <--- probably the best idea huh. Well, this is a moot point, I knew, but forgot to actually acnowledge, this mobo can only support 32 gigs of RAM. I guess the real question here..... is this enough? Do I need to move up a class to get more RAM slots? This won't be a heavy hitting server, it just needs to stream multimedia to a few destinations at once, nothing too crazy. If it will max out gigabit LAN, I will be happy!

And on the topic of chassis, any suggestions? At the 16+ drive option, I think I am fully into a server rack type of case, I don't see much way around that which is fine. I just need to figure out the way to best spend my money. I don't mind buying a "nice" 4U chassis with a backplane which eliminates the need for an HBA card since I plan on using a X10SL7-F motherboard which I can use reverse breakout cables with to get 2 mini SAS connectors out of. So whatever chassis I use would need to be setup so I can use 2 SAS connectors and plug in all of my potential drives, or, have a plan to get an HBA card if required. I am just not sure how many drives I can use per SAS connector, maybe that depends how the backplane is setup? I am 100% new to backplans and SAS in general, so I am trying to learn up on it all.

Also, if it matters, I am planning to run a XEON CPU in this guy, so CPU power shouldn't be a concern, at least for this thread.

Thanks for the advice guys. I am really trying to set this all up correctly from the get go, and have proper future expansion ability without spending an exuberant amount.
 
Last edited:

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
This thread should clear any question you have about what is a vdev, what is a pool, etc... ;)

Please note you can't add drives to an existing vdev, however you can replace drives by bigger drives or add a new vdev to pool.

Do not go over 11 drives per vdev with RAID-Z3 and 8 with RAID-Z2.

32 GB will be fine if it's for home use but if you plan to rise the capacity of your pool in the future you might want a MB and CPU who can go to 64 GB.

Regarding the question about the drives per SAS connector and co you might want to read about SAS expanders, @jgreco wrote a very good thread about SAS here: https://forums.freenas.org/index.ph...-sas-sy-a-primer-on-basic-sas-and-sata.26145/ ;)
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
This thread should clear any question you have about what is a vdev, what is a pool, etc... ;)

Please note you can't add drives to an existing vdev, however you can replace drives by bigger drives or add a new vdev to pool.

Do not go over 11 drives per vdev with RAID-Z3 and 8 with RAID-Z2.

32 GB will be fine if it's for home use but if you plan to rise the capacity of your pool in the future you might want a MB and CPU who can go to 64 GB.

Regarding the question about the drives per SAS connector and co you might want to read about SAS expanders, @jgreco wrote a very good thread about SAS here: https://forums.freenas.org/index.ph...-sas-sy-a-primer-on-basic-sas-and-sata.26145/ ;)


Will read up on those, but as far as vdev additions, I know you change a vdev once it's deployed, but can you not add more vdev's to a pool? I thought this was possible, but maybe I misunderstood something.


Sent from my jailbroke iPhone using Tapatalk
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Yes. BiduleOhm told you that you could "add a new vdev to pool"

O. I read that wrong. My bad, thanks.


Sent from my jailbroke iPhone using Tapatalk
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
I would suggest going with a 24 bay chassis (eventually) then adding a vdevs in multiples of 6 or 8 raidz2 depending on if you prefer 25% parity loss or 33%.

Go with a skylake/X11 SuperMicro board for 8 sata ports (first vdev is free ;) ). 64GB ram capacity and a good amount of PCI slots.

Start with 16 or 32GB of ram

A cheap sas card (m1015/m1115) will then add another 8 disks.

Another sas card or an expander will then add another 8 or more disks.

Each minisas connector supports 4 drives. But an expander can multiply that.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
I would suggest going with a 24 bay chassis (eventually) then adding a vdevs in multiples of 6 or 8 raidz2 depending on if you prefer 25% parity loss or 33%.

Go with a skylake/X11 SuperMicro board for 8 sata ports (first vdev is free ;) ). 64GB ram capacity and a good amount of PCI slots.

Start with 16 or 32GB of ram

A cheap sas card (m1015/m1115) will then add another 8 disks.

Another sas card or an expander will then add another 8 or more disks.

Each minisas connector supports 4 drives. But an expander can multiply that.

Actually, I ordered one of these! Its used, but I got it for 420 bucks. And I plan to add another 10x3TB Z3 VDEV on top of my current setup, which will be expanded to 10x3TB Z3 as well.

SYSTEM SUPERMICRO 4U 846E16-R1200B X8DTE-F WITH SAS2 BACKPLANE
MEMORY 32GB DDR3 MEMORY (4X 8GB + 8X 2GB)
PROCESSOR DUAL INTEL XEON L5520 8M CACHE 2.26GHZ
BOOT DRIVE 2 X 90GB SATA 2.5'' SSD
STORAGE DRIVES 24 X TRAYS (NO STORAGE INCLUDED)
HBA 6GBPS SAS HBA (WITH LSI 9211-8I FIRMWARE)
POWER SUPPLIES DUAL PSU INCLUDED (SEE IMAGE)
NIC 2X MELLANOX 10GBE NETWORK INTERFACE CARD
RAILS RAILS INCLUDED
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Ok, so after creating a helpful little cheat sheet, as of today's prices, 4TB's would be best. But I plan to test out the used hardware I got for a while which will put me near Black Friday when I will actually be ready to throw drives in and test them.... So I figure I should just wait and see if there are any epic sales on drives.

443e20f525c064f0890b1552392b9789.png



Sent from my jailbroke iPhone using Tapatalk
 
Last edited:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,994
Might I ask what your system is being built for?
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Might I ask what your system is being built for?

Backups, plex, general file server type stuff (but for only a few clients, won't be hit by more than 2-3 clients at once), and then just holding the media that plex will be streaming.


Sent from my jailbroke iPhone using Tapatalk
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
cheat sheet
I think it makes more sense to decide how you want to lay out your pool, for medium term at least, and then pick a size of drive that delivers a suitable amount of storage within your budget. I don't think comparing drive today's drive prices, which are a short-term parameter, should drive pool layout decisions, which should not be short-term.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
I think it makes more sense to decide how you want to lay out your pool, for medium term at least, and then pick a size of drive that delivers a suitable amount of storage within your budget. I don't think comparing drive today's drive prices, which are a short-term parameter, should drive pool layout decisions, which should not be short-term.

I am confused at your advice. How else would you decide on how to sat up your pool of not by price, size, and redundancy delivered?

I am planning on at least 25% redundancy regardless, most are right at the 30% "safe zone" area. Then it just comes down to total desired pool size, and then how to attain that size in the manner that makes the most sense. In order to determine what "makes sense" is cost per usable GB and power consumption.

I guess my question is, how else do you determine pool layout? If I have a good redundancy percentage, isn't the next step figure out what pool layout will be in my wallets best interest?

Also, I currently have 11 3TB drives. I have 2 waiting to be used, and 9 already deployed in my previous openmediavault Nas that I will be repurposing to this new server. So I might as well fully utilize those 3TB's I have, weather I use all 11 in a single vdev + a new 4TB vdev, or break them up and purchase more 3TB's to create two vdevs of appropriate long term size.


Sent from my jailbroke iPhone using Tapatalk
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
How else would you decide on how to sat up your pool
Factors include:
  • desired redundancy (which you noted)
  • desired performance characteristics (which leads to vdev layout and number of vdevs)
  • expectations for data growth over time and desired upgrade path (replacing drives vs adding vdevs)
I can't deny budget as a factor, but your cheat sheet runs from 6 drives in RAIDZ2 to 10 drives in RAIDZ3, which offer quite difference performance characteristics and upgrade paths.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Factors include:
  • desired redundancy (which you noted)
  • desired performance characteristics (which leads to vdev layout and number of vdevs)
  • expectations for data growth over time and desired upgrade path (replacing drives vs adding vdevs)
I can't deny budget as a factor, but your cheat sheet runs from 6 drives in RAIDZ2 to 10 drives in RAIDZ3, which offer quite difference performance characteristics and upgrade paths.

True. Upgrade paths are important, speed in this case is much less important. All I need to do is saturate gigabit, which shouldn't be an issue here.

But, future upgrade paths is an important factor, so I will give that a thought based on having a 24 bay chassis, *and knowing I don't want to run more than 80% full on a ZFS file system*.


Sent from my jailbroke iPhone using Tapatalk
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
Two 6-disk RAIDZ2 vdevs would be the obvious choice.

Except that won't give me enough space if I use my current 3TB drives, and if I don't use my current 3TB's I am not utilizing over a thousand bucks worth of drives I already own.


Sent from my jailbroke iPhone using Tapatalk
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
c3bef0f67e84c243133c91e12f373b1f.png



Sent from my jailbroke iPhone using Tapatalk
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
6 drive vdevs or 8 drive vdevs in raid z2 would be the way to go. Now take your current drivers and figure out how the squeeze them into that and buy the couple extra if you need them.

I choose 8 disks in z2 for my 24bay build. I currently have 2vdevs.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
525
6 drive vdevs or 8 drive vdevs in raid z2 would be the way to go. Now take your current drivers and figure out how the squeeze them into that and buy the couple extra if you need them.

I choose 8 disks in z2 for my 24bay build. I currently have 2vdevs.

Isn't 8xZ2 to low of a redundancy? 25% I guess is OK

And actually, apparently...... I'm wrong anyways. I missed a drive. Have to redo my thought process lol. I currently have 9 drives deployed and 3 more just waiting to go into my Nas, but I have kept them in their boxes waiting until I figure out my new server.

Need to rethink this.


Sent from my jailbroke iPhone using Tapatalk
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
Isn't 8xZ2 to low of a redundancy? 25% I guess is OK

And actually, apparently...... I'm wrong anyways. I missed a drive. Have to redo my thought process lol. I currently have 9 drives deployed and 3 more just waiting to go into my Nas, but I have kept them in their boxes waiting until I figure out my new server if you go above 10.

Need to rethink this.


Sent from my jailbroke iPhone using Tapatalk
8 disks in a vdev is fine, even 10 is OK. What starts to happen is rebuild times get really slow with more drives and that can get scary.
 
Status
Not open for further replies.
Top