New Norco 4224 Build

[kskreider]

NAS Intent: JBOD
OS: FreeNAS version TBD
Chassis: Norco RPC-4224 - PURCHASED
Motherboard: Supermicro X10SRi-F-O - PURCHASED
CPU: Intel Xeon E5-1620 v4 - PURCHASED
Cooler: Noctua NH-U9DX i4 - PURCHASED
PSU: Corsair RMX 1000W CP-9020094-NA - PURCHASED
RAM: 4 x 16 GB Kingston ECC RDIMM DDR4-2400
Boot: 2 x 16GB Sandisk Cruiser Fit 3.0 - PURCHASED
Vdev1: 8 x 8TB WD80EFAX NAS-HDD (RaidZ2) - PURCHASED
Vdev2: TBD
HBA1: SUPERMICRO AOC-USAS2-L8I (cross-flashed to LSI 9211-8i IT) - PURCHASED
HBA2: SUPERMICRO AOC-USAS2-L8I (cross-flashed to LSI 9211-8i IT) - PURCHASED
Backplane Fans: 3 x 120mm Cooler Master A12025-12CB-3BN-F1 - PURCHASED
Exhaust Fans: 2 x 80mm (unknown)
Cables: 4 x 50cm LSI SAS SFF-8087, 2 x 19" Molex 79576-3002 SFF-8087 Reverse Breakout - PURCHASED

Props to user @Stux and his build thread HERE

Questions:

1. Is it better to connect backplanes using the two HBA cards or the reverse breakout cables straight to the motherboard? Initially I am only running with 8 x 8tb Reds. My intent is to have 3VDEVs of 8 drives each in the pool but with the initial cost of the build it might be a while.
2. Can/should I add more RAM when I add more HDDs? I was thinking about starting off with 32GB and upgrading to 64GB later.

Insight is more than helpful since I am such a noob here.

Thanks,

Keith

 

[Redcoat]

Props to @Stux I believe ..

 

[Chris Moore]

You probably should have asked for advice before buying all that. I would have suggested getting a Supermicro chassis with a SAS expander backplane. One of the reasons @Stux used the Norco case is because he is in Australia. Being in California, your availability of used data center hardware is much different and it would have been a better solution. Less wiring mess. SAS os better than SATA.
Sure, you can upgrade the memory later but I would suggest that you don't fill all your RAM slots because you might want even more as you get your storage built. You might want to use 32 GB sticks.

 

[kskreider]

You probably should have asked for advice before buying all that. I would have suggested getting a Supermicro chassis with a SAS expander backplane. One of the reasons @Stux used the Norco case is because he is in Australia. Being in California, your availability of used data center hardware is much different and it would have been a better solution. Less wiring mess. SAS os better than SATA.
Sure, you can upgrade the memory later but I would suggest that you don't fill all your RAM slots because you might want even more as you get your storage built. You might want to use 32 GB sticks.

Well the chassis, cards and cables all came used. It was a good price but an impulse but so I am not opposed to changing any of it.

In fact my first post at STH was and ISO post for an 836. I am a bit nervous about the older Norco Backplane reliability.

 

[Chris Moore]

Well the chassis, cards and cables all came used. It was a good price but an impulse but so I am not opposed to changing any of it.

In fact my first post at STH was and ISO post for an 836. I am a bit nervous about the older Norco Backplane reliability.

The Norco boards are not much more than a wiring harness. Unless you wear the connectors out from putting drives in and out, I don't really see how it could fail. No need to change it if it is working.

It really looks like the only thing to do is buy RAM and hard drives and you already have 8 drives...
Theoretically, you should have a GB of RAM for every TB of drive space, but it kind of depends on your intended use.
Will you be running Plex or other jails / plugins?
Besides pure storage what are your intentions?

 

[Chris Moore]

Is it better to connect backplanes using the two HBA cards or the reverse breakout cables straight to the motherboard? Initially I am only running with 8 x 8tb Reds. My intent is to have 3VDEVs of 8 drives each in the pool but with the initial cost of the build it might be a while.

The SAS HBA cards use about 10 watts each, so you could save a tiny amount of power if you pull them out until you need them, but it might not be worth it. Your call. When you need them, you can put them back in. FreeNAS will re-enumerate the drives but it won't hurt your storage because the drives are mapped by their 'gptid' which looks like this:

Code:

pool: Irene																													   
state: ONLINE																													 
  scan: scrub repaired 0 in 3h23m with 0 errors on Mon Sep 25 09:23:48 2017														 
config:																															 
																																	
		NAME											STATE	 READ WRITE CKSUM												 
		Irene										   ONLINE	   0	 0	 0												 
		  raidz2-0									  ONLINE	   0	 0	 0												 
			gptid/959cd1b0-6694-11e6-867e-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/6c70d9a6-6cb0-11e6-b454-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/97462e3c-6694-11e6-867e-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/79506ccf-963c-11e7-8ac8-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/98e163cd-6694-11e6-867e-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/99c7cd32-6694-11e6-867e-002590af8b19  ONLINE	   0	 0	 0												 
		  raidz2-1									  ONLINE	   0	 0	 0												 
			gptid/16d1736e-8b2f-11e7-92b6-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/cd7a14e5-8c44-11e7-92b6-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/1fe96c3f-8dec-11e7-a5a5-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/6458c3c0-8c77-11e7-92b6-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/aca060b0-8d0e-11e7-a5a5-002590af8b19  ONLINE	   0	 0	 0												 
			gptid/01c07df6-8ae8-11e7-9652-002590af8b19  ONLINE	   0	 0	 0												 
																																	
errors: No known data errors

 

[Inxsible]

One of the reasons @Stux used the Norco case is because he is in Australia. Being in California, your availability of used data center hardware is much different

I always wonder why that is. Any business worth it's salt would have some server grade hardware which would need replacing once the warranty runs out or it comes to End Of Life. And I am sure Australia has many businesses. What do they do with those? Do they just have much stricter recycling policies?

 

[Chris Moore]

I always wonder why that is. Any business worth it's salt would have some server grade hardware which would need replacing once the warranty runs out or it comes to End Of Life. And I am sure Australia has many businesses. What do they do with those? Do they just have much stricter recycling policies?

Used hardware is in short supply in Britain also, so I guess it is about the recycling but it is something that someone living there would need to answer.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Stux]

it doesn't make much of a difference if you use the HBA or the Sata ports. You should check every bay to make sure they're all functional.

I'm in the process of switching to a single HBA and 36 port expander in the same chassis, because I want to pass-through the HBA in ESXi. But two HBAs and the built in ports is super performing and just fine for bare Metal

 

[Inxsible]

Used hardware is in short supply in Britain also, so I guess it is about the recycling but it is something that someone living there would need to answer.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Being curious, I asked my President, here at work, what the company policy was regarding End of Life hardware that the company uses. He said they scrub the data clean and send it over to Africa where the company has some partnerships with certain NGOs. The NGO's then re-distribute the hardware to local schools, charitable organizations etc. I don't know if they are sold at a nominal fee or given out for free. I work in the US, but my company is Irish which might explain the existence of the policy. Perhaps UK/AUS/EU have certain mandates regarding this.

 

[kskreider]

The final bits came in over the weekend so here is the update on my build:

NAS Intent: JBOD
OS: FreeNAS version 11 Stable
Chassis: Norco RPC-4224
Motherboard: Supermicro X10SRi-F-O
CPU: Intel Xeon E5-1620 v4
Cooler: Noctua NH-U9DX i4
PSU: Corsair RMX 1000W CP-9020094-NA
RAM: 2 x 32GB Crucial CT32G4RFD424A
Boot: 2 x 16GB Sandisk Cruiser Fit 3.0
Vdev1: 8 x 8TB WD80EFAX NAS-HDD (RaidZ2)
Backplane Fans: 3 x 120mm Cooler Master A12025-12CB-3BN-F1
Exhaust Fans: 2 x 80mm NOCTUA NF-A8 PWM
Cables: 2 x 19" Molex 79576-3002 SFF-8087 Reverse Breakout

It is all set up and doing burn-in tests now. I have Memtest currently running a 5-7 day test, then I will start a week of SMART testing, and a 24 hour CPU stress test.