Greetings...

[rvassar]

Just wrapping up my first FreeNAS build/deploy. Thought I'd introduce myself...

Geologist turned Software QA Engineer. 22 years in software, including Sun, Dell, HGST, and DXC. I'm basicly a Unix curmudgeon. I have a soft spot for the BSD's because of SunOS pre 5.x, but I'm quite at home with Linux & a variety of SVR4 derived commercial offerings. I have extensive knowledge of telco scale email systems, exposure to HA clusters, Enterprise storage appliances, and a smattering of Cloud Computing.

I've always kept a small *nix server at home. The last several years I've been running ESXi at home, and been pretty unhappy with storage riding on the hypervisor. I tried a small QNAP box, and made the mistake of choosing an ARM model on sale. I can tip it over pretty easily, and it needs to be rebooted every 2 weeks. So I tried FreeNAS on an old Dell SC1430, and it seemed intriguing, but far too loud for home office use. So I picked up an off-lease Optiplex 790 with an i3 proc, and have been working it into a pretty useful little FreeNAS box. It only has a 265 watt PSU, so I'm probably pushing it with 4 internal drives. I'm hoping to extend this with a eSATA enclosure of some sort in the near future.

So far I've preformed a disk replacement, with resilvering, a config backup, boot drive replacement, installed a mirrored boot pool, config restore, and moved some 2 Tb of data in. It's good to be on ZFS again!


Rob

 

[Chris Moore]

I'm hoping to extend this with a eSATA enclosure of some sort in the near future.

Don't do it. I have been down that road and it is better left untraveled.
The eSATA hardware is all very unreliable and drives will suddenly drop out for no discernable reason. The most reliable connection is a SAS HBA. I use SATA drives on mine and get fine performance.
There is a wealth of knowledge on the forum, many are from hard lessons learned. If you have an idea, throw it out and someone will give you some tips. Speaking of which, you might want to brows the links in my signature under the "Useful Links" button.

 

[rvassar]

Don't do it. I have been down that road and it is better left untraveled.
The eSATA hardware is all very unreliable and drives will suddenly drop out for no discernable reason. The most reliable connection is a SAS HBA. I use SATA drives on mine and get fine performance.

Interesting... Now that you mention it, I've only used SAS & FC/AL arrays outside the host in the past. I've been ignoring eSATA for years, and just assumed someone had the bugs worked out. I'll chase the thought, but I'll need non-rack mount equipment. I've had enough of 25mm fans trying to push 30 cfm...

To be honest, there's probably not a lot of point in external enclosures until all the internal drives are 8 - 10 Tb. I'm a big fan of HGST drives, for obvious reasons. Though I don't have much experience with the post WD aquisition era drives. Hopefully they're still up to snuff!

 

[Chris Moore]

I'm a big fan of HGST drives, for obvious reasons. Though I don't have much experience with the post WD aquisition era drives. Hopefully they're still up to snuff!

From what I have seen, WD looks to be reaping the reward of HGST innovation. I have a rack at work with 80 of the WD Red Pro 4TB drives and 60 more of the WD Red Pro 6TB drives. The 4TB drives have been online for right around 3 years and 8 have failed and 6TB drives have been online for about 18 months and we have only had 4 of them fail. Not too bad. I don't have significant numbers of the HGST drives, only 16 in one server, but I have no failures from them.
You can find some real good deals on surplus datacenter gear on eBay and put it in any chassis you are happy with. The Fractal Design Define R5 is really popular because it can mount a decent number of drives and it is quiet.
http://www.nextwarehouse.com/item/?1657209_g10e
I like these for a system board because it allows use of registered RAM which is cheaper right now than regular UDIMM memory because of the way the market has gone nuts lately. In addition to that it has 10 SATA ports so you could use 2 for a mirrored boot pool and 8 for a nice RAIDz2 storage pool.
https://www.ebay.com/itm/NEW-Unopened-Super-Micro-Motherboard-LGA-2011-MBD-X9SRL-F-O/253300204890
I saved over $200 using a Xeon E5 and RDIMM memory vs buying a system board that would take a Xeon E3 because of the higher cost of DDR-4 or DDR-3 UDIMM memory.
These Xeon E5 CPUs are powerful enough to do most anything you need in FreeNAS unless you want to run a bunch of VMs and they are relatively cheap:
https://www.ebay.com/itm/Intel-Xeon...-s-Hex-Core-Server-CPU-Processor/382393160646
Then you can choose the amount of memory you like but here is an example:
https://www.ebay.com/itm/202280394199
I didn't use a calculator but I think that is around $600 for hardware that will make a pretty decent server, just add drives.

 

[Chris Moore]

If you just want to add more drives by SAS, you can get a good used enclosure on eBay also:
https://www.ebay.com/itm/Enhancebox...Enclosure-Mini-SAS-SFF-8088-JBOD/162844719513
Then you just need an interface card and cable.
Controller:
https://www.ebay.com/itm/LSI-SAS920...-SAS-HBA-Controller-H3-25140-QTY/372102178384
Cable:
https://www.ebay.com/itm/External-2...8-407344-003-408767-001-Warranty/382293563900

 

[Chris Moore]

I've been ignoring eSATA for years, and just assumed someone had the bugs worked out.

I used eSATA with single drives and it works pretty reliably, but if you use a SATA port multiplier, that is where you can run into problems. I used an external 4 drive enclosure for several years to house a backup pool but it would give me problems every time I needed to restart the server. I had to down the server, disconnect it, bring the server back up, then reconnect it and manually import the pool and ensure it was mounted on the right mount point. It is doable, but it could really cause problems if you have a single pool with one vdev internal and one in the external enclosure.

 

[rvassar]

Chris,

There's a lot of good info in those links! I do appreciate it! I suspect you're used to working a larger IOPS/sec problem that I have here at home. Counting the two new 4Tb drives I imported from the QNAP attempt, I have less than $350 into this little chassis at the moment.

I was originally thinking about a 4-bay eSATA enclosure more to offload the power supply than expand the number of bays. I think I'll be OK with a 4-drive solution in the short term, provided I move up the drive density ladder as needed. I still need to confirm this chassis is stable with 4xSATA drives in it, both from a PSU load standpoint, and airflow/temps. Two of the drives are in the 5-1/4 bays. I do have a USB 3.x PCIe card in the box that appears to work with the kernel, so I may have some ability to shuffle stuff in and out of the main ZFS pool locally, and/or keep a "USB pool", single spindle w/lesser trust. I do have two GbE networks to feed... A "house network", and something local to my desk that mostly serves as a SAN / PXE Boot network, to isolate the heavy storage traffic... 4 drives should be OK, or at least I keep telling myself that! :)

Rob

 

[Chris Moore]

I was originally thinking about a 4-bay eSATA enclosure more to offload the power supply

Dell usually puts pretty good supplies in their systems. The first two FreeNAS systems I built for home were using Dell Precision workstations and I used them for a couple years before the desire for more drives made me consider a larger chassis.
I have tried a good number of things at home that I would never trust for the production environment at work, and I have found that there is a good reason not to trust some of those things. I had one of the 4 drive SATA port multiplier enclosures connected and besides what I mentioned above, I also had trouble with it letting the drives be too hot to suit me. It is a challenge getting decent performance from some of the lower priced hardware.

I still need to confirm this chassis is stable with 4xSATA drives in it, both from a PSU load standpoint, and airflow/temps.

It sounds like you have a good start on this, but if you have any questions, just ask, someone will give advice.