BUILD My 1st FreeNAS build; noob from Windows

[FATeknollogee]

marbus90, I like your thinking....guess what, I own 2x 9207-8e, so I'm all set :)

 

[FATeknollogee]

Never heard of OVH till you mentioned it, I'll definitely read more on their website

 

[FATeknollogee]

If you use striped mirrors, you can add 2 more disks at a time, equaling a pool growth by 1x disk size. The JBOD seems a good idea to not worry about expansion and layout - just split your mirrors across the JBOD and the server chassis. 10 HDDs, 3x S840, 2x Z16IZF2D-400UCM, 1x Samsung SSD per chassis.

When I first read this (I had my Windows "hat" on) I was thinking, what the hell is this guy talking about.......this makes no sense
Then I remembered with ZFS, the striped mirror sets can be a mix of different drives as long as we keep the same style together.
10 HDD chassis 1 + 10 HDD chassis 2, 3x S840 chassis 1 + 3x S840 chassis 2 etc..

Is this the best way to take advantage of the SSDs?

 

[marbus90]

One pool across the S840 mirrorsets and another pool across the HDD mirrorsets would be the best way. In case you'll be using VMs, add the ZeusIOPS as SLOG/ZIL to the HDD pool. With utilizing the JBOD and the server chassis with 2x SFF-8087/8088 cables per chassis you can bump your total bandwidth available. With that config you can look into some 10Gbe interfaces, with Gbe it'll be extremely bored. ;)

 

[FATeknollogee]

One pool across the S840 mirrorsets and another pool across the HDD mirrorsets would be the best way

If my understanding is correct:
The S840 pool will contain 3x striped mirror sets (6 drives)
The HDD pool will contain 10x striped mirror sets (20 drives)
The Z16IZF2D pool will contain 2x striped mirror sets (4 drives)

In case you'll be using VMs, add the ZeusIOPS as SLOG/ZIL to the HDD pool. With utilizing the JBOD and the server chassis with 2x SFF-8087/8088 cables per chassis you can bump your total bandwidth available.

Definitely plan on having some VMs

With that config you can look into some 10Gbe interfaces, with Gbe it'll be extremely bored. ;)

Does FreeBSD support Mellanox 56 gig NICs?

 

[marbus90]

Your understanding is correct with the slight adjustment that the Z16IZF2D's would be dedicated to SLOGs for the HDD pool. Or two of them for the HDD pool and the other two for the SSD pool, but nevertheless the SLOGs will also be mirrored.

FreeBSD itself yes, FreeNAS not yet: https://bugs.pcbsd.org/issues/2014

 

[FATeknollogee]

For offsite backups I'd look into OVH's FS-30T. thats 5x6TB Enterprise drives in an ECC capable system, so create an raidz on there (considering it's not your only point of the backup of the backup that's okay), maybe order their vRack gateway as well for a secure connection between your OVH servers and your home. I'd order one in Canada and one in France for replication of your home backup box.

What kind of experience have you had dealing with OVH? How is their service/support etc?

 

[jgreco]

CPU: Intel Xenon E5-1620v2 or *E5-2620*

Consider an X10 board and the E5-1650v3. Winner.

 

[FATeknollogee]

Will def check out the SuperMicro X10 line

 

[jgreco]

X10 plus that 1650 is about $800. RAM is very pricey but it performs really nicely!

 

[FATeknollogee]

Those DDR4 sticks sure haven't hit mainstream pricing!

You say perform nicely, but how does that compare to the prev generation (X9)?

 

[marbus90]

Doesn't really compare. Of course if you would buy everything new, you should consider X10 only - but with that collection of X9 hardware, I'd repurpose that first.

 

[FATeknollogee]

Doesn't really compare. Of course if you would buy everything new, you should consider X10 only - but with that collection of X9 hardware, I'd repurpose that first.

Repurposing the X9 is definitely the plan. Need to get my feet wet w ZFS

 

[FATeknollogee]

Your understanding is correct with the slight adjustment that the Z16IZF2D's would be dedicated to SLOGs for the HDD pool. Or two of them for the HDD pool and the other two for the SSD pool, but nevertheless the SLOGs will also be mirrored.

How does the storage all look when it's done? Is this one large sized "LUN" that can be carved up? (I know we don't speak LUNs in ZFS)

 

[Ericloewe]

How does the storage all look when it's done? Is this one large sized "LUN" that can be carved up? (I know we don't speak LUNs in ZFS)

Every pool can be arbitrarily divided into essentially as many datasets or zvols as you wish. Each is a somewhat indepedent filesystem, so you can have different rules for different data (permissions type, snapshots, replication...).

 

[marbus90]

It would be served with the following biggest sizes, smaller subsets of which can be configured via datasets:
1x zPool with 20x1TB striped mirrors = 10TB after RAID (HDDs)
1x zPool with 6x200GB striped mirrors = 600GB after RAID (s840)
1x zPool with 4x200GB striped mirrors = 400GB after RAID (ZeusIOPS) (if used as SLOG devices, they wouldn't show up)

For example your Windows Storage-VM would get a 30GB vHD on the s840 RAID for boot and a 5TB vHD on the HDD RAID for storage.

 

[FATeknollogee]

It would be served with the following biggest sizes, smaller subsets of which can be configured via datasets:
1x zPool with 20x1TB striped mirrors = 10TB after RAID (HDDs)
1x zPool with 6x200GB striped mirrors = 600GB after RAID (s840)
1x zPool with 4x200GB striped mirrors = 400GB after RAID (ZeusIOPS) (if used as SLOG devices, they wouldn't show up)

Thanks for the detailed example.
To increase these pools, I will always need a minimum of 2 drives:
The 1st pool will increase in size by 1TB for every 2 x 1TB drives
The 2nd pool will increase in size by 200GB for every 2 x S840 SSD
The 3rd pool is SLOG, so no worries there.

 

[marbus90]

Exactly.

On another note, if you want to upgrade your workstation: E5-2637 v3 (Quadcore) or E6-2643 v3 (Sixcore) would be the best high-end CPU choices for that, since they have the highest singlethreaded speeds. 3.6GHz if 2 Cores per CPU are in use (4 total), 3.5GHz if all cores of all CPUs are in use.