BUILD Hardware selection - 1st FreeNAS build

[DataKeeper]

Sent you both a message and thanks.

 

[pclausen]

Bag of screws is on its way! :)

 

[DataKeeper]

damn! Pal Error on the IT flash

Used Rufus to create a DOS boot drive.
Downloaded M1015_X9SCM_BOOT_P16.zip and unzipped it to DOS drive.
Popped the flash drive into the server and rebooted.
At the prompt I entered the following

Code:

megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0

Worked fine and rebooted. Then went to enter the following

Code:

sas2flsh -o -f 2118it.bin -b mptsas2.rom
sas2flsh -o -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card)

Only to have it fail on the first cmd with the following

ERROR: Failed to initialize PAL. Exiting Program.

 

[pclausen]

You will want these on your USB stick if you're doing this from a UEFI motherboard (which you would be if using the X10):

 

[DataKeeper]

Yeah the file I linked to above had everything. Noticed your other post and booted into the UEFI shell, switched to the flash drive, ran the two commands and it worked fine! :D

Now.. I just need some screws for my hard drives :p

 

[DataKeeper]

While I've been plugged into a UPS that was all I had done. Wanted to get them talking. SNMP is a ways off since the damn AP9617 management card is used and must be reset. The ONLY way to do this is via serial connection and the only thing I currently have with a serial port in the freenas server.

Question: Any type of serial hyperterminal in freenas to allow me to connect to the APC? If not I'll have to temporarily install something on the X8 1U system sitting on top of the USP.

So I couldn't find any others in my big box of 10+ year old cables so stole the APC 940-0127B cable off my desktop, went downstairs and plugged it into the FreeNAS server and the APC SMART-UPS SUA2200RM2U. Back upstairs I noticed it connected via the footer messages on ugen1.4. Configured the UPS Service and it started right away.

Running "upsc -l" displayed the UPS name (SMART-UPS_SUA2200RM2U) and "upsc SMART-UPS_SUA2200RM2U" displays all the status data. Without drives it is showing 6600 seconds runtime or 110 minutes.

Pulled the plug for about 5 minutes then plugged it back in. Updates in messages log and at the shell as well as email alerts had been sent to my email.

Now to order another 940-0127B cable sometime soon to get back on my desktop and Back-UPS XS 1500. :D

 

[DataKeeper]

So while I wait for the drive screws to arrive (BIG thanks guys!) I got impatient and created labels containing the serial number for each drive and affixed one to each of the caddy levers. I then opened 12 new WD RED 4TB drives and simply holding them in the caddies, slid them each in until they locked in place. Opened a SecureCRT window then a dozen tabs and kicked off my badblocks test on all 12 drives at the same time. (I so need to relearn screen!)

All are currently at 35% at 2:35:00 elapsed time with zero errors so far. Drive temps holding at between 24c through 28c.

@Bidule0hm .. Wanted to say thanks! I have your SMART Report and UPS Report scripts running from cron with a few slight changes. Likely go back and look at a few other scripts you have there.


Regarding vdevs. I'm thinking of the following.. A single vdev with 4-drives and two vdevs with 10 drives - all raidz2 and 4TB.

The smaller 4-drive vdev would give me around 7TB for my photo collection and important personal documents specifically. This is all backup archived data and would be rarely accessed. This data is currently on a 2nd drive in my desktop, on an external drive (in a 1 hour fire/water safe inside my larger gun safe) and yet another external drive at a friends place. Yes.. I value my photos! The remaining 20 bays split into 2 10-drive vdevs for Media and everything else. I have 14 now so could set up the 4 drive and 10 drive vdevs most anytime. I'm considering just ordering another 14 to be done with it and have it fully populated. I'll decide in the next day or so.

I have 10s of thousands of photos. Most are my own. More by my 4 year old son over the last 1.5 years. I also have nearly 1000 family photos going back to the late 1800s which I've spent a lot of time preserving digitally.

Question: Am I correct in that a smaller 4 drive vdev will be safer then a larger 8 or 10 vdev in that if a drive should fail the resliver process will be quicker and hence less likely that a 2nd drive could fail?

 

[marbus90]

Resilver is usually limited by mass of data and write speed of the target device.

 

[rogerh]

Presumably you mean to use the 4 drive vdev as a separate pool on its own, rather than a member of a larger, multi-vdev pool? Otherwise you can't use it separately, and the data is vulnerable to damage to the other vdevs. Just a terminology question, but vital to avoid talking at cross purposes.

 

[DataKeeper]

Yes, call the 4-drive pool gallery. It would consist of only the 4 drives in raidz2.

Let's call the larger 10-drive pool garage. I'll likely extend the 2nd 10-drive vdev to this one.

 

[pclausen]

DataKeeper,

I know you flashed your 1015 controller to P16, but did you also flash your SAS2 backplane?

I have done all 3 of mine, and they were on ancient versions. Here's a ftp site with the binaries:

ftp://211.214.88.34/bios/BP_FW/

And here's how to perform the actual upgrade:

Firmware Upgrade through running Linux system
i)For expander behind SAS HBA or On-board SAS Controller
1. Download and extract all files to a local drive.
2. Boot the target system to Linux.
3. Navigate to directory with the Linux utility.
4. Find SAS address of expander(s) in system.
a. Run "xflash -i get avail" command
b. The format of SAS Address is shown as below
(xxxxxxxx:yyyyyyyy)
c. Record returned SAS Address as "xxxxxxxxyyyyyyyy"
5. Update expander fw.
a. Run "xflash -i <SAS Address> down fw sas2xfwBasic.fw 0 -y"
6. Power cycle system once expander has completed firmware update.

 

[DataKeeper]

Ya know I didn't even realize the backplane could be flashed. Thanks.

 

[Darren Myers]

Ya know I didn't even realize the backplane could be flashed. Thanks.

I need to do this too...also screws on the way!

 

[DataKeeper]

@Darren Myers .. Thanks! Received some from @pclausen today as well. Thanks to you both :)

Badblocks testing at 42:29:00 and no errors yet...

I received the replacement 48-port switch last week and it works fine except for one of the three side fans was whining and of course they were all loud. Ordered 3 of these Noctua 40x10mm fans to replace them and its nice and quiet now.

 

[DataKeeper]

Not much new really. Badblocks ran for about 70 hours and completed fine on 12 4TB drives. These have also had a conveyance test run on them. I received 6 more drives for this system today so will need to start testing those tonight. Installed ESXi6 for the firsttime on the 4-bay 1U to a SSD drive and damn those things are fast! Yes its my first SSD drive. This is just playing around with learning ESXi and I've installed Debian and Red Hat along with pfSense, Windows 7 and Windows 10. Just playing around with it and gotta say its pretty neat. This is installed on the X8SIE-F board with X3440 CPU and 8GB RAM.

Along with the 6 new drives, I received a sweet little 1U CSE-510T-200B chassis for a dedicated pfSense install. Got the XS708E 8-port 10GbE switch today and gave it an IP but haven't had time to do anything else with it yet. Got a 3rd X540-T1 10GbE network card so I'll have 1 in the FreeNAS server, 1 in the ESXi server and another in my Desktop for now. I'll admit the cost of the switch and 3 cards was hard but its done and that is that.

I'm still looking for a rack so everything remain here on my worktable until I locate one. The FreeNAS system with 16 4TB drives is pulling 198W at idle, CPU is 34C, RAM is 29C with System temp at 21C. The APC UPS with the 2 systems (18x4TB, 2x6TB NAS drives, +1 SSD & 2 DOMS) and both switches shows 3360 seconds (56 minutes) runtime. I've tested both power supplies in the 846 by pulling each power cord out and each worked as expected.

For now here is a pic of my assemble and test desk with everything laid out.

(Edit.. ohh and last night one of my 8 year old Dell 24" displays fizzled and died with a puff of smoke so I'm looking for 2 new 24" LED displays now as I figure I should replace the one that is still working as well)

 

[pclausen]

Hey, you are missing the serial number label on the 18th WD Red :)

Are you still running the fans in Heavy I/O mode?

My January 2006 Dell 2405 is still going strong. :)

What's that showing on the vertical tablet? It almost looks like IPMI sensor readings via the web interface, but if so, how did you get it to display more than the first 10 rows at once?

 

[DataKeeper]

I ran out of label tape :(

Yes I am. I'll change the fan mode later to see what works better.

I have/had 2 2407s. The one that fried was the one that still had a great screen. The remaining screen is still good but it's not as bright as it used to be. Still useable but I'm going to replace them both soon. Wife laughed yesterday saying one screen simply wasn't going to work for me and she's correct, as usual. I can't remember having only a single screen and I hated it today. I figure within a week I'll have two new screens on the desk.

That is correct. It's the IPMI screen and when viewed with the iPad it displays the entire screen. :D Now if only Java was available on the iPad!

 

[Ericloewe]

I ran out of label tape :(

Yes I am. I'll change the fan mode later to see what works better.

I have/had 2 2407s. The one that fried was the one that still had a great screen. The remaining screen is still good but it's not as bright as it used to be. Still useable but I'm going to replace them both soon. Wife laughed yesterday saying one screen simply wasn't going to work for me and she's correct, as usual. I can't remember having only a single screen and I hated it today. I figure within a week I'll have two new screens on the desk.

That is correct. It's the IPMI screen and when viewed with the iPad it displays the entire screen. :D Now if only Java was available on the iPad!

Try salvaging the good panel from the dead monitor and use it in the working monitor to replace the aged one. That way you get three screens. You can never have too many screens.

 

[pclausen]

My main "home office" work screen is a Seiko 39" 4k "TV". It will only do 30Hz @ 4k, but for ssh sessions, web browsing, running IPMI screens, etc, it's perfect. I think I paid a little over $300 from tigerdirect for mine. Once sharpness is turned all the way down and a few other settings tweaked, it is just as good as my 24" dell and my 30" dual-link DVI at work. The cost per pixel is hard to beat. :)

 

[marbus90]

Yep. below 2 monitors I can't work. had 3 for a brief period of time - that was glorious... 5120x1200px in screen estate :)

I'm going to order a few U2515H soon. 25" 2560x1440, DP output, LED, IPS and ~300EUR each. Waiting for the bonus check. :P