Build advice, what steps next...

[Grewterd]

Hi, my name is Scott, and I’m a statistic.

I’m doing my best not to be a statistic again so here I am testifying to my situation. The last time I lost data was when I went from FreeNAS .7 to what would become NAS4Free (knock on wood). Now I’m coming back home as it seems the support for NAS4Free just isn’t there any more, the documentation is spotty at best and as it looks like I’m going to have to take off and nuke the site from orbit, so I may as well rethink the OS while I’m at it.

I have “some” things right with my system, and some things need to be fixed, it’s a work in progress. This system was originally built from spare hardware about 7 years ago. It’s gone through 3 cases, 3 motherboards, 3 sets of RAM, 4 or 5 CPU upgrades (I keep passing my CPU’s down to it), 2 or 3 PSUs, the addition of a second vdev, replacement of the original vdev (that was not too bad considering I had two failed drives in a RAIDZ1 vdev, backups are wonderful!), and now the second array is due to be upgraded (one drive has pending bad blocks and the drives are out of warranty). I’m not sure if anything actually remains from the first build now!

I’ve done something to my file system’s permissions while trying to setup ACLs from Windows and things have just gone crazy. I’m seeing the system stop responding completely to commands via SSH or the direct console which forces me to reboot…a lot. I’ve not been able to get any assistance with it so, I’m about to back up everything and start over, else I’ll be using the thing for target practice soon.

Given what I’ve learned in the last few weeks reading the documentation, hardware recommendations, best practices, worst practices, posts, etc. (which has been just outstanding in it's content) I have a list of objectives that I’d like to accomplish and here is where I’d appreciate some input on where to concentrate.

Here is my current setup:

• ASUS M5A97 R2.0 AM3+ AMD 970 SATA 6Gb/s USB 3.0 ATX AMD Motherboard (yes, I know)
• AMD Phenom quad core CPU
• TWO (aka 32GB total) x Crucial 16GB Kit (8GBx2) DDR3/DDR3L-1600MT/s (PC3-12800) DR x8 ECC UDIMM Server Memory CT2KIT102472BD160B/CT2CP102472BD160B (woo hoo, I got something right!)
• DELL PERC H200 cross flashed with LSI IT firmware (I think p19 but I do know that I just flashed my backup HBA to p20 after reading that FreeNAS is using p20 currently.)
• PSU is bad, 500W. I think the thing came from BestBuy (please don’t hurt me).
• vdev 1: 4 x Seagate ST4000VN000-1H4168 NAS HDD at 4TB
• vdev 2: 4x Samsung HD204UI SpinPoint F4EG (AF) at 2 TB
• Boot drive: Sandisk Cruzer Fit 8GB
• Intel PRO/1000 dual PCIe NIC
• Back-UPS RS 1500G
• Some generic case, not enough airflow.
• Latest version of NAS4Free.

Soon to be the above with the following changes:

• Moving to FreeNAS 9.3
• TWO x Sandisk Cruzer Fit 32GB in mirror
• Fractal R4 with 3 added 140mm fans giving 2 front, 1 rear, 1 side, 1 bottom.
• Replacing the Samsung drives with 4x Seagate ST6000VN0001-1SF17Z at 6TB.

Future upgrade plans:

• Supermicro X10SL7-F or X10SLH-F (Possibly X9SRH-7TF, X9SRH-7F or X9SRL-F-O)
• Intel Xeon E3-1231 or 1230 v3 (of if the boards change Xeon E5-1620v2)
• SeaSonic Platinum SS-860XP2 860W PSU (Is this too much power?)
• Add an additional drive to each vdev and rebuild both vdevs to RAIDZ2.
• Cold spare, probably just one 6TB.
• SATA SSD boot drives. (Yes/No?)

I cannot do this all at once, but by the end of 2016, I’d like this list to be checked off (and anything else suggested).

Goals:

• Storage and streaming of lossless audio including 5.1 DSD audio; 720p and 1080p MKV video files including bluray rips with lossless audio to three Kodi HTPCs and three PCs (this is max case scenario, generally only 1 or 2 of any is streaming at one time).
• Occasional system backup/images.
• General file storage.
• I’d like ability to share audio files with friends and family securely via internet.
• BitTorrent client downloads to a share on the NAS (using a separate Ubuntu system to run Tranmission with Moblock [I have reasons]).
• Possibly Jails for MySQL for the Kodi systems, the database is well under 50MB, honestly, I’d prefer an extra Intel NUC for that as I am not a believer in multi-use systems. Currently this runs on my BitTorrent system.
• Rsync client for MySQL backup (50 MB is about the size of the total backup for this).

My questions:

• What order should I take for my upgrade plans, what should take priority? Motherboard? PSU? RAIDZ2?
• What are my immediate dangers/red flags (please don't say "everything!" :) )?
• Is the PSU I'm looking at just way over spec? Should I bring that down some keeping in mind I'm going to add two more HDDs and possibly 2 SSDs?

Additional question I’ve had a hard time finding answers to:

• Does anyone have a good source for quality MiniSAS to SATA breakout cables? The one’s I’ve been getting fail if you breathe on them. With the tight quarters in my case, I’ve needed to bend the cables sharply at which point they start losing communication with the HBA. Are there any recommended cable brands/sources?

Thank you in advance for your time and advice/suggestions.

 

[Ericloewe]

• SeaSonic Platinum SS-860XP2 860W PSU (Is this too much power?)

For 10 drives, you can more than safely go with a Seasonic Platinum 660. You could even go with a 550W PSU, but it would get a bit tight and I won't be the one to recommend against Seasonic Platinum. :p

In all seriousness, if using a Seasonic G-650 or an X-650 instead of the Platinum 650 is enough to facilitate a faster motherboard+CPU replacement, it's perfectly alright. All three are very good PSUs (X-Series and Platinum are excellent).

SATA SSD boot drives. (Yes/No?)

Definitely leave this for last. It's a nice-to-have feature, but decent USB drives won't cause you any trouble.

What order should I take for my upgrade plans, what should take priority? Motherboard? PSU? RAIDZ2?

Your most immediate need is to ensure all pools are healthy. Silent corruption due to bad RAM isn't much of a concern if your pool is a goner.

Depending on how awful your PSU is (let's assume it's absolute garbage that's ready to catch fire if you stare at it just the right way), I'd take care of that next.
Then, finally, motherboard+CPU.

 

[Fuganater]

Check out the X10SRL-F instead of the X9. The E5-1620V3 will be able to take care of anything you are running at home.

I never recommend it but for you, get a M1015 since the ports face 'up' and you won't need to worry about bending the cable too much.

2x 16GB Sandisks is more than enough. 32GB is totally unnecessary.

 

[Ericloewe]

X10SRL-F instead of the X9

OP already has 32GB of DDR3 on hand.;)

 

[Fuganater]

OP already has 32GB of DDR3 on hand.;)

So sell it?

 

[Grewterd]

Yeah, not sure SWMBO will authorize MB/CPU and RAM, particularly as it looks like the X10SRL-F requires registered RAM, which seems a bit more costly. We'll see when the time gets closer, never know. I'll present the budget both ways during the family budget meeting.

The cable issues I'm having are not on the HBA side, it's on the drive side where the cable needs to be bent. I have enough room on the HBA side to route things comfortably. I'm hoping the Fractal R4 will solve most of these problems with the extra width.

The 32GB USB drives were on sale for $10, it was like $2 more than the 16GB so I figured why not.

Thanks for the advise.

 

[Grewterd]

So sell it?

You offering? Because I'm really not liking you right now for suggesting that board, gosh it's amazing. I hope you stub your toe for that! ;) :p :D

I'm going to really have to plan the budget presentation carefully...It would future proof the build more that's a big plus. Good thing is the MB/CPU upgrade will be a while so that gives me time to save and maybe prices will come down some.

All joking above aside, I really do appreciate the suggestion and am seriously considering it, you got the wheels turning.

 

[Ericloewe]

X10SRL-F requires registered RAM

No, UDIMMs are fine. Depends on your expectations of future growth - you could go with fewer RDIMMs allowing for easy future expansion or more (say 4) UDIMMs.

 

[Grewterd]

No, UDIMMs are fine. Depends on your expectations of future growth - you could go with fewer RDIMMs allowing for easy future expansion or more (say 4) UDIMMs.

Scott's confused...The only options on Supermicro's site for the X10SRL-F are the following:

DDR4 1.2V-2133 Registered ECC
DDR4 1.2V-2400 Registered ECC
DDR4 LRDIMM-2133 Registered ECC

 

[Ericloewe]

Scott's confused...The only options on Supermicro's site for the X10SRL-F are the following:

DDR4 1.2V-2133 Registered ECC
DDR4 1.2V-2400 Registered ECC
DDR4 LRDIMM-2133 Registered ECC

Probably no ECC UDIMMs as of when they last updated the QVL. Even the puny 4GB DIMMs in my workstation are RDIMMs, there simply weren't any DDR4 ECC UDIMMs around this time last year.