60TB+ New Build Need Hardware check

[Chris Moore]

I have a about 10 wired on 1GB and 4-5 on Wireless. The switch is a Cisco 200 series Smart Switch that has worked well.

I have a similar switch at home and a separate 10GB switch. You could pickup a fairly inexpensive 10GB switch on eBay if you do some looking. You just have to be careful to not get something that isn't what you thought it was.

 

[danb35]

A switch like this would probably do great for you: https://www.neweggbusiness.com/product/product.aspx?item=9b-33-150-165

The Dell X1052 is quite a bit less expensive, and has 48 GbE ports plus 4 SFP+.

 

[Stux]

My Primary NAS is a similar build. One of its most important tasks is as a Time Machine target for a bunch of macs.

Currently I'm working on a 10gbe upgrade for it, as it means being able to service up to 10 clients simultaneously at full gbit speed each.

Time machine writes are temporarily cached in ram before being flushed to disk. No need for ssd cache.

Would recommend either 6-way or 8-way RaidZ2 vdevs in a 24 bay backup device.

 

[Chris Moore]

The Dell X1052 is quite a bit less expensive, and has 48 GbE ports plus 4 SFP+.

That looks like a good option. I just went with the Cisco because that is the brand he already had.

 

[iHeartMacs]

With regard to the chassis, I would save some cash and get a used one. You can always toss the old system board and put a new one in it.
Something like this: http://www.ebay.com/itm/Supermicro-...ore-Barebones-24-Bay-SAS2-846EL1/292252408126

Just for the sake of future expansion, I would go in increments of 6,

That one sold but I see this one. What do you think?
https://www.ebay.com/itm/Supermicro...094725?hash=item36214f89c5:g:38gAAOSwgIpZiM1Y

 

[danb35]

That one sold but I see this one. What do you think?
https://www.ebay.com/itm/Supermicro...094725?hash=item36214f89c5:g:38gAAOSwgIpZiM1Y

Looks like a good choice but for the RAID card--but it'd be easy enough to replace that with an HBA, if it doesn't support IT mode on its own.

 

[Chris Moore]

That one sold but I see this one. What do you think?
https://www.ebay.com/itm/Supermicro...094725?hash=item36214f89c5:g:38gAAOSwgIpZiM1Y

That unit has a hardware RAID controller that would need to come out and be replaced by a plain SAS HBA. You might want to go with this model instead, and it is less expensive:
https://www.ebay.com/itm/Supermicro...ore-128gb-24-Bay-JBOD/372154285756?rmvSB=true

 

[danb35]

That unit has a hardware RAID controller that would need to come out and be replaced by a plain SAS HBA.

Of course, the RAID controller could be resold--it's what I did when I bought my server.

 

[jgreco]

The Dell X1052 is quite a bit less expensive, and has 48 GbE ports plus 4 SFP+.

In general, the Dell switches are a good deal with less cost and usually less drama. One of our 7048P's here did lose its PoE controller, though, and as it wasn't covered by a Dell warranty ended up being a costly bit of unhappiness. The 5524/5524P/5548/5548P are also good options, and a bit cheaper than the X1052. They have 2x SFP+ and also 2x dedicated stacking ports that are connected via HDMI, which isn't necessarily as good as the 4x SFP+ of the X1052, but I want to say the X1052 is a web-managed switch which means that CLI may not be available, if that matters to you.

One of the side effects of the slowdown in the growth of common networking speeds is that we now get crazy nice equipment being sold at a modest price.

Of course, the RAID controller could be resold--it's what I did when I bought my server.

Yeah, right?

 

[Chris Moore]

I have an Adaptec RAID controller that I have been trying to get sold for a year.
It must not be a desired model...

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[jgreco]

I have an Adaptec RAID controller that I have been trying to get sold for a year.
It must not be a desired model...

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Adaptec has basically been a has-been starting after the 29160 (what a slam!).

Most of the Supermicro gear out there is from whitebox vendors who integrate Supermicro parts, which in recent years have been OEM versions of things like LSI RAID controllers. These work very well under Windows, VMware, Linux, FreeBSD, etc., unlike that Adaptec RAID controller which is known to be somewhat problematic.

Now, if you happen to find one of the higher end RAID controllers, which is what an integrator is going to put into a 24- or 36-bay chassis, and it happens to come fully loaded with the supercap ("battery backup") option, this becomes a very attractive thing especially for VMware hosts. VMware has some of its own special snowflake write sync issues, and one of the least expensive ways to get high performance storage is to use direct-attached disk or direct-attached SSD. But you have to use a BBU RAID controller to get good write speeds.

As it happens, LSI has released a lot of 12Gbps stuff now, and a lot of older 6Gbps stuff is falling out of the lease/hardware-renewal cycles. Since a BBU RAID controller is fairly specialized gear, the prices have been trending downwards, and just recently I acquired some of the Supermicro 2208's with supercap for $125/ea (?). But these are still DAMN GOOD at what they do! Especially for HDD, where the average HDD is still 6Gbps and so are the inexpensive SATA SSD's. So it is now perfectly feasible for a small business, home user, etc., to pick up a gently used RAID controller that is *nearly* as good as the $750-for-12Gbps units new today.

The LSI BBU RAID aren't any good for FreeNAS, because FreeNAS needs access to the raw disks, but for many uses, you can get a really high performance 6Gbps RAID controller for a hella-great price and it'll make things go fast. For the right things, like an ESXi host, the difference is like night and day.

Adaptec made great products back in the day, but based on how infrequently I see their products, and how frequently I see LSI or OEM LSI, LSI won the RAID wars. And even the LSI top-of-the-line cards don't go for that much $$$ used, so I guess I am not shocked you're having problems getting rid of it. I hope this helps explain what might be some contributing factors to the problem.

Anyways, the part that's missing here is that I think I was probably the one who bought danb35's RAID controller, because when you buy from eBay, you never really know if it's going to arrive wrapped in newsprint instead of a static bag.

 

[Ericloewe]

because when you buy from eBay, you never really know if it's going to arrive wrapped in newsprint instead of a static bag.

Hey... Deng Xiaoping died!

 

[droeders]

Adaptec has basically been a has-been starting after the 29160 (what a slam!).

Wow - haven't thought of my last SCSI card for quite some time. Had a 10K Seagate Cheetah HDD to go with it - something like 150GB.

Ahhh the memories...

 

[ethereal]

static bag.

anti static :)

 

[Ericloewe]

anti static :)

Static shield, really. Or ESD shield.

 

[iHeartMacs]

Looks like a good choice but for the RAID card--but it'd be easy enough to replace that with an HBA, if it doesn't support IT mode on its own.

I just printed out the hardware guide and I'm now aware of HBA and that hardware RAIDS are a no no. OH and wow thx for the fast reply.

 

[Stux]

Static shield, really. Or ESD shield.

And did you know that the "anti-static" bags and foams are colour coded? Pink is dissipative, and Black/Grey is conductive. Ie one will dissipate static to atmosphere to prevent build up, but won't protect contents from a static discharge, the other will conduct static discharges around/away from the contents.

 

[jgreco]

anti static :)

No, really, static. With a little static electricity generator and everything. For when you want to make SURE it's dead.

 

[Chris Moore]

anti static :)

I have a couple cans of anti static spray.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[iHeartMacs]

My Primary NAS is a similar build. One of its most important tasks is as a Time Machine target for a bunch of macs.

Currently I'm working on a 10gbe upgrade for it, as it means being able to service up to 10 clients simultaneously at full gbit speed each.

Time machine writes are temporarily cached in ram before being flushed to disk. No need for ssd cache.

Would recommend either 6-way or 8-way RaidZ2 vdevs in a 24 bay backup device.

Hi, I just read this and didn't realize you had a similar build with similar requirements (Time Machine).

That unit has a hardware RAID controller that would need to come out and be replaced by a plain SAS HBA. You might want to go with this model instead, and it is less expensive:
https://www.ebay.com/itm/Supermicro...ore-128gb-24-Bay-JBOD/372154285756?rmvSB=true

Great. Much cheaper too. Ok so here is what this one has. I'd be grateful for any "gotchas" about the parts listed that you might now about. I really want to "....do it right!"

-X9DRI-F Motherboard (Supports V2 CPUs / 16 Memory Slots)
Not the X10SL7-F but could it be good for future media serving possibility?

-2x E5-2650 v2 2.6ghz 8-Core 8.0 GT/s / 20mb Smart Cache CPUs
Guide says not much benefit of 2 CPU's unless this is just to get more RAM capacity

-16x 8gb PC3-10600R Server Memory
Following the rule of thumb this seems about right for upgrade of storage in the future.

-24x 3.5" Trays (2 screws each)

-SAS2-846EL1 Backplane
Yay SAS2 is this a good one? I didn't see it in the new build guide.

-LSI 9210-8i JBOD Controller
it looks like this one must be cross-flashed? I'll look that up.

-2x 1200w PSU
Seems like more than enough power.

I know these are extremely loud. I've read about changing out the fans or putting a fan controller on the stock fans. I'll read more into that. I originally wanted to keep this in the office that has AC but the noise would be unbearable. I'm trying to make a judgement call on what would be better. I can have quite fans that might compromise integrity of the machine (if I don't research the hell out of it first) in a 24/7 AC's office or put it in a well ventilated server closet that might reach 110 degrees.

Any suggestions on Cabinets with wheels for this beast are more than welcome.