Outrageous Number of Hard Drives

[rgm]

I realize I am not giving much information but any words of wisdom would be much appreciated!

Will it be possible to have 3 4U NAS chassis, each with 36 4TB had drives, operate in unison as a massive storage unit using FreeNAS?

 

[Mlovelace]

Yep, you just daisy chain the two 36 bay jbod units off the first NAS. Freenas won't have any trouble with that number of disks provided enough resources.

 

[rgm]

Yep, you just daisy chain the two 36 bay jbod units off the first NAS. Freenas won't have any trouble with that number of disks provided enough resources.

Mlovelace,

Thank you for the speedy reply, much appreciated!

 

[DJ9]

This isn't going to go well. What's your system specs for all of these hard drives?

 

[joeschmuck]

If you got enough money to purchase 36 4TB drives then you have enough money to purchase the proper hardware to run it. Don't skimp on the hardware. I'm curious too to see your system specs.

 

[rogerh]

He's proposing 108 4TB drives, actually!

 

[joeschmuck]

He's proposing 108 4TB drives, actually!

Oh Snap! (yes, I'm old)

 

[Mlovelace]

You can do it. He'll need dual E5's and 512gig of ram. I'd go with dual E5-2697v2 so you've got plenty of threads and still fast for single core.

 

[mjws00]

We had someone here a month or so ago with ~116 drives on like 16 or 32Gb. Workloads are the name of the game. I don't know why you wouldn't just put another motherboard in those jbods when you're spending that kind of dough. But to each their own.

 

[Mlovelace]

We had someone here a month or so ago with ~116 drives on like 16 or 32Gb. Workloads are the name of the game. I don't know why you wouldn't just put another motherboard in those jbods when you're spending that kind of dough. But to each their own.

It's much more cost effective to scale out instead of scaling up, but people do crazy stuff all the time.

 

[L]

I know a group that has 252 drives running on 64GB ram. I wrote a blog about them..http://kateleyco.com/?p=815

they are running raw freebsd though not freenas

 

[rgm]

Seems I have sparked quite the discussion haha.

Well this is a work in progress so any input is more then welcome!!!!

As of right now this is the proposed hardware:

• Case: Supermicro 4U 36 Bay NAS case
• Motherboard: supermicro Super X9DRD-7LN4F-JBOD or Asus
• Memory: 100 plus GB
• CPU: Intel E5 2600 (open to suggestions)
• Hard Drives: WD 4TB hard drives or Ultrastar He6 6TB hard drives
• Compact flash: SanDisk to deploy FreeNAS
• Power supply: doesn't matter
• Cooling: doesn't matter

Architecture: One master server with two slave servers (possibly Daisy chain setup as recommended by mlovelace). The overall goal of creating a NAS that will appear as one large storage repository on a network.

 

[marbus90]

there aren't "slave servers", there are just dumb JBOD chassis. also supermicro offers 45drive JBODs in 4U, so if you want to bump that drive count even further...

Supermicro offers such a server as barebone including the JBOD Expansion ports:
http://www.supermicro.nl/products/system/4U/6047/SSG-6047R-E1R36L.cfm
Then you just cascade those JBOD chassis via SFF-8088 cables:
http://www.supermicro.nl/products/chassis/4U/847/SC847E16-R1K28JBOD.cfm

CPU-wise I would look into a E5-2630 v2 with tenendies towards E5-2650 v2. They both have a nice TurboBoost in case you need some singlethreaded oomph.

 

[rgm]

there aren't "slave servers", there are just dumb JBOD chassis. also supermicro offers 45drive JBODs in 4U, so if you want to bump that drive count even further...

Supermicro offers such a server as barebone including the JBOD Expansion ports:
http://www.supermicro.nl/products/system/4U/6047/SSG-6047R-E1R36L.cfm
Then you just cascade those JBOD chassis via SFF-8088 cables:
http://www.supermicro.nl/products/chassis/4U/847/SC847E16-R1K28JBOD.cfm

CPU-wise I would look into a E5-2630 v2 with tenendies towards E5-2650 v2. They both have a nice TurboBoost in case you need some singlethreaded oomph.

marbus,

Thank you for the advice, it is much appreciated I will look into both links asap.

 

[anodos]

marbus,

Thank you for the advice, it is much appreciated I will look into both links asap.

You could also contact iXsystems and see what they can provide you. My experience as a customer has been pretty good.

 

[rgm]

You could also contact iXsystems and see what they can provide you. My experience as a customer has been pretty good.

I will keep that in mind, thanks.