BUILD FreeNAS - the next step

[c0d1st]

Hello,

I have cruising the FreeNAS forums for a while. I’ve been reading the guides and posts. I think I’ve started to grasp the concepts presented. I’ve out grown my current solution and it’s time to move on. This will be a fun challenge, but I would much rather take a hit on the forums than in the wallet.

Background:

I recently graduated college with an IT related degree. During my college adventures, I picked up a small Synology 210J to back up homework and projects. A friend from school introduced me to FreeNAS. I played around with it in a VM and eventually some old hardware. I was impressed with the hardware test. I’m not afraid of research, hard work, the word “No.”, system building or CLI. I’m a BSD noob, but willing to learn. Now it’s time to take the leap.

Project:

Current Needs - My Synology is near full. I require more space, greater reliability and better throughput. Living in a rural area, my internet service is poor; therefore, I tend to stash a lot of stuff on my NAS for future use. I have media (video, e-books, music and photos), ISO files, installed game folders, work and school files, etc.

I will repurpose my Synology as a backup. All of my project files, work files, archived homework, financials, etc. will be stored there. The only media I will be archiving will be my books. The rest is an acceptable risk.

I will be setting up an ESXi environment with leftover parts as a preliminary test bed. I’ll be hosting some development projects and testing software. If the system loses all of its smoke, I will not be out a whole lot.

Future Needs – Based on my testing with the above consumer grade ESXi box, I will be building a beefier ESXi box with sever grade components. I would like to leverage the FreeNAS storage as much as reasonable. I have no qualms in investing into 10 GB networking. I may be working with large data sets for research at some point. I need to be mindful of this requirement. The data workflow will need some thought.

Hardware:

I have an old Antec 1200. I plan on using 5.25” to 3.5” (4-in-3) hot-swap bays.

I need to Flash LSI 2308 to IT mode.

I'll need to update the MOBO firmware.

Parts –

HDD Bays: ICY Dock 4-in-3 Hot-Swap Module

Power Supply: Not 100% here. Seasonic ~500W Bronze rated?

UPS: Eaton 5S (seemed viable per NUT)

MOBO: Supermicro MBD-X10SL7—F-O

RAM: Crucial 16GB (2 x 8GB) ECC Unbuffered x2 (32 GB total)

CPU: Intel Xeon E3-1230V3 - read about BIOS issues with newer 1231V3 (I don't have an extra CPU to update.)

SATA Connectors: SATA Connector x11

OS: 8GB Sandisk Cruzer Low Profile (Cheaper than 4GB)

Data Storage: WD40EFRX 4 TB WD Red (have 2 + 9) x11 Total

Storage –

I thought I read somewhere that it was a bad idea to exceed 80% of your available storage. If this is true, I was planning 4TB * 11 disks in RAIDZ3. I think my evaluation below is correct.

11 – 3 = 8 disks | 8 * 4TB = 32 TB | 32 * .8 = 25.6 TB MAX | 32 * .65 = 20.8 TB WARNING

Networking --

Based on the posts in the forums, Link Aggregation isn't going to get me anywhere. I'd be better off getting a larger pipe if I need the bandwidth.

Testing:

CPU - Load Testing

Flash Storage – Memtest86?

RAM – Memtest86

HDDs – testing methodology per the forum.

Network - Load Testing

Thanks in advance for any assistance. Wall of text crits for over 9K.

C0d1st

 

[c0d1st]

UPDATE:

OS used:

• FreeNAS 9.3.0 Stable

I ordered from above:

• Motherboard
• CPU
• RAM
• Cages
• UPS
• PSU
• USB for OS

I had:

• x2 MLC SSDs
• x2 4TB WD REDs

On the way:

• Replacement 16GB ECC ram (bring up to 32 GB)
• Rest of the data disks.

So far:

• I put the system together and installed the cages. I stressed the USB, CPU and RAM. I found x1 stick of x4 sticks to be bad. I returned the kit of x2 sticks.
• I did an initial test of all cages. Drives were recognized. I have been conducting more thorough tests in each cage. I have 2 extra bays that I will be using for cold spares.
• I have been testing Volume configurations and throughput with what I have. Writes with x2 SSDs in mirror 100+ Mb/s and x2 HDDs in morror 50 - 65 mb/s (sometimes hitting 70 mb/s). The source is a single 7200 RPM HDD.
• I setup the Plex plugin. It's been working quite well. I have jails on the SSDs and Media on the HDDs. I used a xbmc client to stream the video. I need to try it on a cell phone next to see how well that works.

Current Issue:

• I have a Synology 210j as my "production" NAS until this box is implemented. Once all hardware/software configuration is validated, I would like to use that as my backup. However, I am having a heck of a time figuring out how this is going to work. From what I can gather, RSYNC from FreeNAS (client/push) to Synology (server/pull) is my best option. I'm having an issue working out how to configure it. Modules vs SSH? This will require additional research.

Lessons Learned:

• I am hooked on IPMI. I am enjoying the headless management of my server right from start up.
• The Comodo browser is NOT your friend. I had been playing with several browsers. I found out the hard way it is not compatible. It will work for some things such as logging in, but when using certain controls it completely flakes. The mode selection checkboxes (CHMOD) for permissions are missing. Also, during some updates, it refuses to commit the change. I then tried Firefox and the issues were resolved.

 

[c0d1st]

I've encountered an odd issue:

When I attempt to add an Rsync Task, upon clicking "OK", the button changes to "please wait" and reverts back, but no entry is made in the Rsync Tasks. The only option I have is to cancel. The Rsync service is on.

Update:

Validate Remote Path was checked. Which means what I've been working is incorrect!

 

[c0d1st]

I got it! I hung in there and learned much. Both manual and Rsync tasks work properly.

• SSH Key gen (not sure why it took so long, but this is the tutorial where it "clicked" for me.
• Another missing piece.
• The "final missing piece" rsync -av /mnt/yourvolume/youmedia rsync@[Synology IP]:/volume1/NetBackup/test/