BUILD Enterprise Grade Build

Status
Not open for further replies.

vikingboy

Explorer
Joined
Aug 3, 2014
Messages
71
Add some Chelsio T420-CR 10GbE cards from eBay.

curios why not a t520-cr?
Is there no perf advantage of the t5 chipset over the t4? Seems unlikely spec given small $ delta in contrast to complete spec price.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
The T420-CR goes for $110-$120 often on eBay. The T520-CR goes for more like $400.

If you feel like picking up a pair of each and benchmarking them, be my guest. :smile:

For most typical FreeNAS users, my guess is that the T420-CR is sufficient.

And if anyone wants to ship me a T580-CR, I do have a 40Gbps port free... heheh
 

Mlovelace

Guru
Joined
Aug 19, 2014
Messages
1,111
I am the Director of Information Systems for a medium sized non-profit. I am looking to build or purchase a "SANs" system. For my whitebox solution, I was looking into using FreeNAS. I have a current FreeNAS running as a file server for my department for the last few years. It has been solid and stable.

Here is the whitebox solution I have been looking into.

Case: SUPERMICRO CSE-216E16-R1200LPB Black 2U Rackmount Server Case
MB: SUPERMICRO MBD-X10DRI-T4+-O Enhanced Extended ATX Xeon Server Motherboard Dual LGA 2011 Intel C612
Processor: Intel Xeon E5-2660 v2 Ivy Bridge-EP 2.2 GHz LGA 2011 95W BX80635E52660V2 Server Processor
Controller: areca ARC-1883ix24-8SA PCI-Express 3.0 x8 SAS RAID Adapter
Memory: Kingston 32GB ECC DDR4 2133 (PC4 17000) Server Memory LRDIMM QR x4 w/TS Model KVR21L15Q4/32
Cache Drives: Intel Fultondale 3 DC P3600 AIC 400GB PCI-Express 3.0 MLC Solid State Drive - OEM
Storage Drives: Kingston SSDNow KC300 SKC300S37A/120G 2.5" 120GB SATA III Enterprise Solid State Drive with Adapter

The setup would feature Quad 10gbe, Dual Processor, PCIe SSD for 400 GB of caching (in/out), 24 SSD as storage drives (RAID-Z3) and memory maxed to 768GB.

Priced at about $17,000 USD.

My goal here is maximum performance for multiple SQL databases including our new ERP system. Secondary use use of Hyper-V for MS Terminal Server or Individual VMs. Most of this would be done via iSCSI over 10gbe.

Some other SAN/NAS units/companies I am looking into are PureStorage, Tegile, and Nimble. My secondary option for the whitebox would to use MS Server 2012 R2 Datacenter, which I am currently using for all my VM servers.

Q1: Tiered Storage - Does FreeNAS set a priority of RAM>PCIe SSD > SSD Storage in the configuration. I know that SSD caching is possible.
Q2: FUTURE QUESTIONS
Hmm $17000, have you looked at the dell MD3220i. I'd be reluctant to use a filer that didn't have dual controllers as my primary SAN. Nimble storage has a CS-215 that might come in around that price point. You might be able to get a demo system from them too.
 

MtK

Patron
Joined
Jun 22, 2013
Messages
471
Here's an alternative design that I suspect would be pretty awesome:

SuperChassis 216A-R900LPB - $800
Supermicro X10SRL-F - $220
Intel E5-1650 v3 - $550
Samsung M393A4K40BB0-CPB 32GB x 8 units - $1800
LSI 9211-8i x 2 units - $400
Mainboard ports for remaining 8 or 10 slots if you get the two-bay rear drive tray
Intel DC S3500 480GB x 6 units - $2400
Intel DC P3700 400GB x 2 units - $1800
Add some Chelsio T420-CR 10GbE cards from eBay.

$8500. With LOT of room for additional SSD growth.
In this case, how would you set up these drive?
3*2-way mirror vdevs + 2 mirror L2ARC?

How about the rest of the WD REDs you mentioned...?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
The S3500's as three two-way mirrors, yes, but the P3700 for mirror SLOG. There's no point in L2ARC when the pool is insanely fast, and no need to mirror L2ARC in any case (I think maybe you meant SLOG anyways). That would give you about 1TB of insanely fast SSD storage.

You could deploy the WD Reds in whatever fashion made sense. If it was going to be used for live VM, mirrors still makes the most sense, 6 1TB drives gives you a 3TB pool and about 1TB-to-1.5TB-usable of modest speed HDD VM storage, or in RAIDZ2 a 4TB pool with about 3TB of usable space for backups or something like that.

The really cool thing about this system is that it'd have a lot of space remaining for future expansion, and the system would only get faster as you expanded it.
 
  • Like
Reactions: MtK

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
also mod note: I evicted some of the posts discussing Microsoft licensing to the off-topic forum.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
No. SATA drives work fine in SAS bays. See the SAS sticky. You do not get multipathing but that's not so important for a single enclosure.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
@jgreco thanks!
you would need an interposer to connect those SATA drives in the SAS bays, right?
I'm asking because of this thread: https://forums.freenas.org/index.php?threads/24-bays-2-pools.41251/

Oh, I see, if you're using the SBB, obviously you CAN use the interposers but I seem to recall that you can also use an actual SAS drive in the SBB (maybe removing the interposer board? I forget how that works) and if so you could probably also remove the interposer and just stick a SATA drive in and see if that works. :smile: The purpose of the interposer is to give you the SAS dual communications channels to the drive.
 

MtK

Patron
Joined
Jun 22, 2013
Messages
471
Oh, I see, if you're using the SBB, obviously you CAN use the interposers but I seem to recall that you can also use an actual SAS drive in the SBB (maybe removing the interposer board? I forget how that works) and if so you could probably also remove the interposer and just stick a SATA drive in and see if that works. :) The purpose of the interposer is to give you the SAS dual communications channels to the drive.
sorry, you are right, my confusion.
the interposer is only needed for the dual port so both "sides" of the SBB see the drives.
do you have any suggestion for that other thread?
 

MtK

Patron
Joined
Jun 22, 2013
Messages
471
The S3500's as three two-way mirrors, yes, but the P3700 for mirror SLOG. There's no point in L2ARC when the pool is insanely fast, and no need to mirror L2ARC in any case (I think maybe you meant SLOG anyways). That would give you about 1TB of insanely fast SSD storage.

You could deploy the WD Reds in whatever fashion made sense. If it was going to be used for live VM, mirrors still makes the most sense, 6 1TB drives gives you a 3TB pool and about 1TB-to-1.5TB-usable of modest speed HDD VM storage, or in RAIDZ2 a 4TB pool with about 3TB of usable space for backups or something like that.

The really cool thing about this system is that it'd have a lot of space remaining for future expansion, and the system would only get faster as you expanded it.
isn't 400Gb (x2) a bit too much for SLOG?
 

MtK

Patron
Joined
Jun 22, 2013
Messages
471
Status
Not open for further replies.
Top