BUILD Now What ... time to revisit my hardware(?)

[Ruff.Hi]

My build is in my sig ... but the high level details are ...

NAS Configuration
Motherboard ..... ASRock C2550D4I
Memory .............. Crucial 32GB (4 x 8GB) DDR3-1600 Memory
Storage .............. 8 x Western Digital Red 4TB 3.5" 5900RPM Internal Hard Drive
Hotswap Bays .. 2 x Kingwin KF-4001-BK 3.5″ Internal hot swap rack bay
HDD Cooling ..... 2 x Gentle Typhoon 2150rpm 120mm fans (swapped out the fans from the Kingwins)
Case .................... CaseLabs Nova X2M

I haven't visited this site for a while ... and when I did return, I found this thread (How long your ASROCK c2550d4i or c2750d4i lasted). Why was I returning? My shell script had stopped returning my motherboard temperature. Why? Because the IPMI wasn't being found by the script. Why? Because my BMC was dead. Why? The watchdog had run itself crazy chasing its tail. Hmmm.

I now have the watchdog tied up in its kennel (disabled in BIOS) and I have an RMA in play for the board.

That fixes the (short term) problem. It also gives me a window to review my hardware for my 2nd NAS. I would like to keep / reuse my hardware as much as possible. I don't see much of a problem with most of the hardware (Hard drive cages, HDDs, case, fans) ... but, (obviously) the board has to go. And that will probably take the RAM - it seems most boards have moved on to DDR4 memory.

I have time (cross fingers) so I'll read up on the hardware comments and post back once that is complete.

 

[melloa]

While you are reading, check: https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/. You will find it useful if buying new equipment.

 

[Ruff.Hi]

While you are reading, check: https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/. You will find it useful if buying new equipment.

Noted. I also found the following ...

• FreeNAS Quick Hardware Guide
• Supermicro Motherboard Part Number Guide
• Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

supermicro stuff

• change ipmi password

 

[Ruff.Hi]

Further ... in case I need to move away from the micro ATX motherboard form factor ...

Cases with 6+ 5.25" bays from pcPartPicker.

The Zalman MS800 is dirt cheap and interesting.
The Cooler Master HAF 932 is another option.
Really like the Cooler Master Storm Trooper with its multiple orientation option for the 5.25" bays.

Are their any build logs around using any of the three (3) cases above?

 

[ChriZ]

Perhaps going for a x10 series supermicro could allow the reuse of your Ram (they aren't SO-Dimms, are they?)
Just make sure that you check their compatibility on crucial website.

 

[Ruff.Hi]

A bit of research over lunch ...

Zalman MS800 case from Newegg for $40 (after rebate) ... tempting
Zalman MS800 case from Amazon ... $60

Motherboard ... Supermicro Motherboard Micro ATX DDR4 LGA 1151 X11SSL-CF-O
CPU ... Intel Xeon Processor 3.4 4 BX80662E31230V5

I'll have to check the motherboard for suggested / approved RAM.

Re the motherboard ... am I right in thinking that this (the two black thingos to the bottom right) ...

Show : pic

... is the LSI SAS3008 HBAs? And that I will need something like this ...

Show : pic

... to connect to my HDD cages.

 

[Ruff.Hi]

Perhaps going for a x10 series supermicro could allow the reuse of your Ram (they aren't SO-Dimms, are they?)
Just make sure that you check their compatibility on crucial website.

Ohh - that is useful.

The memory is Crucial 16GB (2 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3L 1600 (PC3L 12800) Server Memory Model CT2KIT102472BD160B ... so if I understand what SODUMMs are ... the answer is NO.

Edit: Link to supermicro Xeon® Board - X10SL7-F that has more than enough HDD options (2x SATA (6Gbps), 4x SATA (3Gbps), 8x SAS2 (6Gbps) via LSI 2308). I could add another 4xCage and still have 2 left over for boot SSDs.

BTW ... I am using thumb drives to boot from ... and boot is sooooo slow. As is system updates. Would they (boot, updates) be faster if my OS was on SSDs?

 

[Ruff.Hi]

I knocked this together using PCPartPicker ...

PCPartPicker part list / Price breakdown by merchant

CPU: Intel Xeon E3-1231 V3 3.4GHz Quad-Core Processor ($239.99 @ SuperBiiz)
CPU Cooler: CRYORIG M9i 48.4 CFM CPU Cooler (Purchased For $0.00)
Motherboard: Supermicro X10SL7-F Micro ATX LGA1150 Motherboard ($241.98 @ Newegg)
Memory: Crucial 32GB (4 x 8GB) DDR3-1600 Memory (Purchased For $0.00)
Storage: 8 x WD Red 4TB 3.5" HDDs (Purchased For $0.00)
Case: Zalman MS800 ATX Mid Tower Case ($65.98 @ Newegg)
Power Supply: Silverstone Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (Purchased For $0.00)
Total: $547.95

... reusing stuff from my current NAS. I have no idea about the CPU ... just one of the many that worked with that motherboard (more research to do). I also don't know if that cooler will be enough - I just happen to have one downstairs.

 

[Ruff.Hi]

And here is a version not re-using my RAM ...

CPU: Intel Xeon E3-1230 V5 3.4GHz Quad-Core OEM/Tray Processor ($239.99 @ SuperBiiz)
CPU Cooler: CRYORIG M9i 48.4 CFM CPU Cooler (Purchased For $0.00)
Motherboard: Supermicro X11SSL-CF Micro ATX LGA1151 Motherboard - no price at PCParts ... but it looks like $275 from a quick google.
Memory: 4 x Crucial 8GB (1 x 8GB) DDR4-2133 Memory (4 x $67.98 @ Jet)
Storage: 8 x Western Digital Red 4TB 3.5" 5400RPM Internal Hard Drive (Purchased For $0.00)
Case: Zalman MS800 ATX Mid Tower Case ($65.98 @ Newegg)
Power Supply: Silverstone Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (Purchased For $0.00)
Total: $852.89

 

[Ruff.Hi]

So ... options ... #1 and #2 will cost the same (excluding the ram in #2) ... I'm thinking option #1 ... use old ram, add case, mobo and cpu ... and look for an OEM tray option as I am ditching the cooler anyway.

First step ... buy new case. Wait on the mobo and cpu. I'll check my couch for change.

 

[Ruff.Hi]

CPU: Intel Xeon E3-1231 V3 3.4GHz Quad-Core Processor
Motherboard: Supermicro X10SL7-F Micro ATX LGA1150 Motherboard
Total: $475

I picked up the above from NewEgg using one of their combo options. It saved me $20 on buying the items separately. It is due to be delivered tomorrow.

Then it is strip down the old box, rebuild in the new box. I'll be keeping my two thumbdrives (mirrored o/s) from my old build ... can I just plug them into the back of the motherboard, set them to the #1 boot option and turn the new server on?

What else?

• update / review the bios
• update / review the IP addresses
• should I use different IP addresses for the new box?
• do I need to remap my network drives / does using different IP addresses change the answer to this question?
  

 

[Ruff.Hi]

I'll be keeping my two thumbdrives (mirrored o/s) from my old build ... can I just plug them into the back of the motherboard, set them to the #1 boot option and turn the new server on?

What else?

• update / review the bios
• update / review the IP addresses
• should I use different IP addresses for the new box?
• do I need to remap my network drives / does using different IP addresses change the answer to this question?

To answer my own questions ...
1) yes ... but remember to actually plug in the keyboard

2) still working on this ... it mapped the GUI to two new IP addresses (.11 and .13) ... not the static address I was using (.3)
WARNING: Feb. 2, 2017, 7:08 p.m. - The WebGUI Address could not bind to 192.168.1.3; using wildcard
(haven't dug into this one yet)

3) it didn't seem to matter ... the GUI got to new dynamic address ... I will swap that back to a static address

4) nope ... plug them all in and everything came back as good as gold.

So ... why haven't I looked at the static IP address thing? ... because I am getting this ...
WARNING: Feb. 2, 2017, 7:08 p.m. - Firmware version 15 does not match driver version 21 for /dev/mps0. Please flash controller to P21 IT firmware.

I google searched to some success ... and then searched here on this forum ... golden!

https://forums.freenas.org/index.ph...e-on-a-supermicro-x10sl7-f-motherboard.38884/

The software at 'ftp://ftp.supermicro.com/Driver/SAS/LSI/2308/Firmware/IT/' (I am not sure how to turn this into a clickable link) is now up to PH.20.00.07.00-IT (instead of the .04 mentioned in the above thread). I'll flash this over the week end.

In other news ... and this is weird ... my computer boots up when I turn the PSU on. I don't need to press the power button. When I shut it down via the GUI, it reboots. What is that about?

Edit on the reboot issue: I just went downstairs and pulled the power / reset wires from the motherboard ... then told the NAS to shut down. When I did this last night (without pulling the wires), it rebooted. This time, it shut down and stayed off. Do I have something wrong with the power / reset wires ... the motherboard ... or the power switch?

 

[CraigD]

The Zalman MS800 is dirt cheap and interesting.

I use this case it is good, don't get the PLUS version

Bend up some of the 5.25" bay tabs get 3 of the drive converters, fans, even dust filters in my signature and 16 3.5" HDDs fit in it no problems

I also have two SSDs next to the PSU secured with velcro

Have Fun
EDIT: Found some photos, cables and SSDs got managed after the photos were taken

 

[CraigD]

In other news ... and this is weird ... my computer boots up when I turn the PSU on. I don't need to press the power button. When I shut it down via the GUI, it reboots. What is that about?

BIOS setting in the X10SL7-F can be set to after power loss return to previous state

 

[Stux]

And here is a version not re-using my RAM ...

CPU: Intel Xeon E3-1230 V5 3.4GHz Quad-Core OEM/Tray Processor ($239.99 @ SuperBiiz)
CPU Cooler: CRYORIG M9i 48.4 CFM CPU Cooler (Purchased For $0.00)
Motherboard: Supermicro X11SSL-CF Micro ATX LGA1151 Motherboard - no price at PCParts ... but it looks like $275 from a quick google.
Memory: 4 x Crucial 8GB (1 x 8GB) DDR4-2133 Memory (4 x $67.98 @ Jet)
Storage: 8 x Western Digital Red 4TB 3.5" 5400RPM Internal Hard Drive (Purchased For $0.00)
Case: Zalman MS800 ATX Mid Tower Case ($65.98 @ Newegg)
Power Supply: Silverstone Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (Purchased For $0.00)
Total: $852.89

For future reference, when someone digs this up ;), its always recommended to go for 2x16GB in skylake rather than 4x8GB, as this leaves you with room for another 32GB as an upgrade down the road (say after FreeNAS 10 comes out with Docker support). The 32 vs 64GB ram limits is one of the fundamental differences between the Haswell and Skylake platforms.

 

[Ruff.Hi]

I use this case it is good, don't get the PLUS version.

Have Fun

Thanks for replying. I am having fun with the case. I have 2 x Kingwin 4HDD hot swap cages ... they have groves in the side and just slide right in ... no need to bend anything. I have 10" sata cables ... and they are too long(!) ... ordered a set of 6" cables. Will add some pictures down the track. I removed the front panel USB 2 & 3, headphone / sound card to cut down on the wires coming from the front.

Edit: Newegg has it out of stock. I paid $60 and should get a $20 gift card in about 4 months for it. I wonder how many others did that.

BIOS setting in the X10SL7-F can be set to after power loss return to previous state

I did read that somewhere ... if my BIOS has that setting set ... then why didn't it restart after I had pulled the power / reset wires? I'll have a look in the BIOS and see what the setting is.

Found this thread ... https://forums.freenas.org/index.php?threads/shutdown-doesnt.27752/

Edit:
BIOS setting set to STAY OFF on power restore. Power switch plugged in - shut down command results in reboot
BIOS setting set to STAY OFF on power restore. Power switch NOT plugged in - shut down command results in shutdown
BIOS setting set to STAY OFF on power restore. Power switch plugged in. UPS not involved - shut down command results in shutdown

 

[Ruff.Hi]

Update1: Flashed my LSI to P20 following the instructions above. Easy as pie after I didn't fat finger the last 9 digits of the LSI serial number. So ... that warning is gone.
Update2: The incorrect IP address for the GUI has been fixed. I added an interface entry for the GUI as well as using the 1-14 menu to set the GUI to 192.168.1.3.

I think I am go to finish my build, wiring and close the case up.

The power down command that results in a reboot seems to be gone. I have the UPS thumb drive plugged in but the PC is plugged into the wall. I will revert it at some stage.

 

[Ruff.Hi]

Update2: The incorrect IP address for the GUI has been fixed. I added an interface entry for the GUI as well as using the 1-14 menu to set the GUI to 192.168.1.3.

... and totally screwed up my network connection ... killing any change of a system update or sending emails. I deleted the interface entry and rebooted. Warning is back but I read somewhere that saving the config using '0.0.0.0' as your IP will solve that. Maybe my config had 192.168.1.3 and was trying to find that ... and couldn't ... hence the warning message. By saving (ie over-writing) the config, it will stop looking for .3. Maybe.

Seems I will have to get use to using .11 instead of .3 for my GUI.

With the new motherboard, the IPMI info is back ... and this version has more motherboard temps. Here is the top part of my daily temp email ...

Code:

System Temp      31.000  C
Peripheral Temp  34.000  C
PCH Temp         37.000  C
VRM Temp         32.000  C
DIMMA1 Temp      23.000  C
DIMMA2 Temp      25.000  C
DIMMB1 Temp      25.000  C
DIMMB2 Temp      25.000  C

cpu.0.temperature: 36 C
cpu.1.temperature: 35 C
cpu.2.temperature: 33 C
cpu.3.temperature: 32 C

 

[Ruff.Hi]

The power down command that results in a reboot seems to be gone. I have the UPS thumb drive plugged in but the PC is plugged into the wall. I will revert it at some stage.

Plugging the PC back into the UPS ... and the shutdown --> reboot issue is back. I have just issued another shutdown command ... we'll see if that one works.

Edit: it worked. NAS shut down. I can live with having to issue back to back shutdown commands.

 

[Ruff.Hi]

One of my mirrored boot thumb drives is failing (chksum=6). I am trying to swap to a SSD ... is there any secret sauce to this?

I am ...

• creating a USB boot drive with freenas installer on it
• booting to that USB
• installing freenas to SSD
• changing boot order to put SSD first
• off to the races?

Should I disconnect my HDDs during the initial boot up?