New to FreeNAS and looking for advice on what I've chosen so far....

[Danwalsh]

First off, I am new to FreeNAS and am looking forward to building a box that I will be able to use as disk to disk backup and non-essential data storage for my corporate network.

I am looking at spending roughly 2-3K on the box, and if all works well, I may investigate building an enterprise worthy system running FreeNAS as a large SAN essentially.

Thoughts and suggestions are more than welcome. Here is my goal and then I'll paste my hardware that I'm purchasing. Keep in mind, I have the ability to do things right, so I'm looking for constructive criticism too.

Total storage will be 24TB with 8 3TB drives, and I'm anticipating running 16GB of RAM. Of course, ZFS is a big reason why I'm looking at FreeNAS among other things.
2 SATA cards plus onboard ports for storage, and prob 2 SSD or sata drives for the OS/cache. I will also use NIC's on both my backup server and FreeNAS to do a storage network between them. I want to make sure I can 'dual-home' my machine without issue.

Here is the hardware I have picked so far (using NewEgg)

Qty. Product Description Savings Total Price
2
Mushkin Enhanced Silverline 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10666) Desktop Memory
Item #:N82E16820226095
Return Policy: Memory Standard Return Policy $99.98
1
AMD Phenom II X4 970 Black Edition 3.5GHz Socket AM3 125W Quad-Core Desktop Processor
Item #:N82E16819103894
Return Policy: CPU Replacement Only Return Policy $149.99
2
Western Digital Caviar Blue 160GB 3.5" SATA 3.0Gb/s Internal Hard Drive -Bare Drive
Item #:N82E16822136075
Return Policy: Standard Return Policy $93.98
$79.98
1
Antec EA750 750W Continuous Power "compatible with Core i7/Core i5" Power Supply
Item #:N82E16817371026
Return Policy: Standard Return Policy $139.99
$89.99
3
Rosewill 8" Sata Power Splitter Cable
Item #:N82E16812119010
Return Policy: Standard Return Policy $11.97
8
Western Digital AV-GP 3TB 3.5" SATA 3.0Gb/s Internal Hard Drive -Retail
Item #:N82E16822136926
Return Policy: Standard Return Policy $1,519.92
$1,199.92
1
MSI 890FXA-GD70 ATX AMD Motherboard
Item #:N82E16813130274
Return Policy: Limited Replacement Only Return Policy $199.99
$194.99
2
SYBA SY-PEX40008 PCI Express SATA II (3.0Gb/s) Software RAID Controller Card
Item #:N82E16816124027
Return Policy: Standard Return Policy $119.98
$109.98
1
OKGEAR 24" SATA 6 Gbps Cable, Straight to Straight W/ Metal Latch, Black
Item #:N82E16812123169
Return Policy: Standard Return Policy $1.99
1
Supermicro AOC-SG-i2 PCI-Express High-performance, Cost-effective Dual-port Gigabit Ethernet Card
Item #:N82E16833268002
Return Policy: Standard Return Policy $84.99
Grand Total: $2,023.78



I don't have a case on here yet, but I'm looking at a 16 or 20 hotswap rackmountable case.

I want to make sure what I've picked was going to be suitable, as well as compliant with FreeBSD as far as drivers go. I'm not locked into any one thing on here right now. Let me know, and I can't wait to get this bad boy built!

 

[ProtoSD]

Just a couple questions.
Why the SYBA SY-PEX40008 PCI Express SATA II (3.0Gb/s) Software RAID Controller Card

Why the 750W power supply? I don't see anything that's going to use THAT much or even half that much power?? (8 disks is only ~96w peak!)

Whats the 160GB drive for?

You don't need a hardware RAID controller with ZFS, it's better if you don't. There's a couple of discussions about that here if you search.

The SSD cache disks are probably a waste for a low/mid end system like this, maybe for an Enterprise server with a big data base.

Also, even though FreeNAS is reasonably stable for basic stuff, it's still new and doesn't have all the kinks worked out yet, it may not be the best choice for an enterprise environment yet....

Those are just some initial thoughts.

Note: FreeNAS does not run from a hard drive, it's designed to run from a small flash drive. You can't use the extra space for your other files.

 

[matthewowen01]

If you're looking for an enterprise level system, buy it. a support contract that will ship you parts next day with a 4 am call is essential. you'll save money building by doing it yourself but if you'll loose the first time you have any hardware issues. if your motherboard decided to fail, it's going to be 3-10 days for a replacement from MSI. can your company afford that kind of downtime.

if you do still decide to built it yourself, i LOVE my norco case. it's a 4U chassis with 20 hot swap SAS bays, 2 internal bays and a spot for a slimline optical device.

http://www.newegg.com/Store/BrandSu...12&name=Norco-Technologies-Inc-Server-Chassis

 

[Danwalsh]

The 160's were for FreeNAS. Would I be better served if I just used one or two flash drives? How big if so? I want to make sure if a USB disk/SATA boot drive went down, I could bring it back up quickly.
If I don't need the cache for the data, I don't want to just waste it.

I'm not using the raid on the cards, just simply SATA bus cards. If I only need 8 for the drives, I wanted to split up drives on each bus for speed.
Good enough on RAM?
I liked the USB 3.0 as well as dual ports on the board, but I'm not sold on anything.

I was looking at that EXACT case too! I guess the 20 bay Norco it is....

 

[Danwalsh]

Also, as a long time administrator, believe me, I have no intention to run production on this as that's at my colocation, but I do plan on running some test and dev VM's on here. Like I said, this is mainly D2D2T backups and nonessential data. Eventually if this does work, I'll go with a worthwhile iSCSI SAN solution, or JBOD and perhaps FreeNAS at that point. If you guys had to build a solid solution what would you do?

 

[ProtoSD]

What you have planned sounds pretty solid. I wouldn't forget a decent sized UPS. I'd also keep a few key spare parts on hand, like a spare RAID/disk controller, an extra disk or two, a spare flash drive with FreeNAS pre-installed, and a copy of your config file.

 

[Danwalsh]

I've got good UPS's for my other systems, so that's well covered. Thanks though.
Good idea on the controllers, etc. I think I'm going to order extra cables as well. Looks like I need some special breakout cables for this Norco case as well.

 

[matthewowen01]

i run my system off of a 8 GB CF card. i got a 'fancy' expansion bay card holder for it so i can swap the card without opening the case.
http://www.newegg.com/Product/Product.aspx?Item=N82E16812186002

i have a SYBA SY-PEX40008 in my server and it works great. only thing to mention, the 1x pcie bus cannot handle the max throughput of all 4 drives. but i only use it for my test's. i can saturate it with 4 drives, anything from the disk cache can't go as fast as it should.

i don't know that SATA 3 works with FreeBSD hopefully someone here can clarify that for us.

other than those things, you look like you've got yourself a nice system.

one more note on VM's, i run my VM's on a Solaris box. ZFS is blazing fast for them and they are local to the box so my disk i/o isn't capped by the network. some VM's load in seconds.

 

[survive]

Hi Danwalsh,

Here are my thoughts:



CPU: I think you might want to consider looking at the Thuban 6 core processors, not because you need the horsepower but for the improved power management. The 970 is AMD's top of the line quad core so it needs the most power to run at it's rated speed. The 6 cores have a "power boost" function like Intel's "Turbo Mode" in their Core chips that will boost a single processor a couple of hundred MHz when the other cores are lightly loaded.

Motherboard: I see you are looking at the MSI 890FX board. That thing's a beast as far as PCI-e slots go! I would confirm with MSI that you can run non-video cards in them.

Controller card: I would look for something with a little more class than a Syba controller card. Take a look at the Supermicro controllers here:

http://www.supermicro.com/products/accessories/index.cfm

They make a number of HBA's using both the LSi 1068 (3Gb) and 2008 (6Gb) chips. The cards come in a weird form factor called "UIO" this is basically a mirror image of a PCI-e card...the components are on the "other side" so they require a bit of creative engineering connecting the brackets. That said, for the money they can't be beat. I know the 1068 chips work in FN8...those controllers work in everything, but double check that the 2008 chips are supported.

Ont thing I don't understand is why you are looking for a 16 or 20 slot rackmount case to run 8 drives. Fractal Designs makes a very nice case that would fit your build's needs perfectly:

http://www.newegg.com/Product/Product.aspx?Item=N82E16811352002

Personally I would look at something like that for this initial project, and then I would take a look at these guys:

http://www.ixsystems.com/ix/storage/titan-truenas-pro/truenas-4u-pro

When it comes time to get yourself a serious enterprise-level system!

-Will

 

[Danwalsh]

I'm probably going to go with 10 drives to start off, since RAID-Z will pull off some storage for overhead. I'm not locked into that case, but it's a good option, and if we did want to expand it or improve, it's there.
I submitted a quote to iXSystems, but I'm curious for the price ranges on their systems mainly.
I'm looking at the Supermicro controllers, but I want to go ahead and have controllers in the system for all the drives if I decide to expand.
Trying to figure out the exact cabling tonight and cards. Love it.

 

[Danwalsh]

Thoughts on a AOC-SAT2-MV8 card? http://www.newegg.com/Product/Product.aspx?Item=N82E16815121009

So, would I be best served to go with CF or something else? Thinking about SD instead since I always have SD around.....

 

[matthewowen01]

Thoughts on a AOC-SAT2-MV8 card? http://www.newegg.com/Product/Product.aspx?Item=N82E16815121009

So, would I be best served to go with CF or something else? Thinking about SD instead since I always have SD around.....

Terrible. That's a pcix card. You have none. It will work in your pci slot bu it will be increadably slow. Cf.is cheep but so is sd. Either is good as long as you have a controler to support it. I don't recall if your bord has an ide port or not. So make sure you have a way to plug in your device.

 

[Danwalsh]

Here is the last list:


Qty. Product Description Savings Total Price
1
AMD Phenom II X6 1075T 3.0GHz Socket AM3 125W Six-Core Desktop Processor
Item #:N82E16819103858
Return Policy: CPU Replacement Only Return Policy $169.99
4
NORCO C-SFF8087-4S Discrete to SFF-8087 (Reverse breakout) Cable
Item #:N82E16816133033
Return Policy: Standard Return Policy $59.96
1
MSI 890FXA-GD70 ATX AMD Motherboard
Item #:N82E16813130274
Return Policy: Limited Replacement Only Return Policy $199.99
$194.99
1
BYTECC Bracket-525 3.5" Drive/Device Transfer Bracket For 5.25" Drive Bay (Black)
Item #:N82E16817994080
Return Policy: Standard Return Policy $4.99
2
Mushkin Enhanced Silverline 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10666) Desktop Memory
Item #:N82E16820226095
Return Policy: Memory Standard Return Policy $99.98
1
nMEDIAPC ZE-C218 All-in-one USB Card Reader with USB 3.0 & e-SATA Ports
Item #:N82E16820132028
Return Policy: Standard Return Policy $16.99
1
NORCO RL-26 26" 3-sections Ball Bearing Sliding Rail kit for rackmount chassis
Item #:N82E16811997301
Return Policy: Standard Return Policy $42.99
$39.99
1
Antec EA750 750W Continuous Power "compatible with Core i7/Core i5" Power Supply
Item #:N82E16817371026
Return Policy: Standard Return Policy $139.99
$89.99
1
NORCO RPC-4216 4U Rackmount Server Case w/16 Hot-Swappable SATA/SAS Drive Bays
Item #:N82E16811219041
Return Policy: Standard Return Policy $349.99
3
Adaptec RAID 1430SA 2240900-R SATAII PCI Express x4, 4 ports Controller Card, Kit
Item #:N82E16816103058
Return Policy: Standard Return Policy $314.97
3
SanDisk Extreme III 4GB Secure Digital High-Capacity (SDHC) Flash Card
Item #:N82E16820171259
Return Policy: Standard Return Policy $41.97
10
Western Digital AV-GP 3TB 3.5" SATA 3.0Gb/s Internal Hard Drive -Retail
Item #:N82E16822136926
Return Policy: Standard Return Policy $1,899.90
$1,499.90


I scaled back to 16 drives for expansion, mainly for the front loaded SD card I put on there versus a CF card.
Am I missing anything? I MAY change the power supply, but I want to make sure to have enough for this unit when I load it up with 16 drives.
It looks good to me. I think the CPU, board, etc is all great.

 

[Danwalsh]

Any parting thoughts on this configuration? I'm also trying to look and see what I need to do to extend RAID Z after it's set up to use the other 6 disks eventually. Looks like it's not really meant to be 'extended'

 

[matthewowen01]

looks good. only thing i can see is the CPU is a bit overkill. 125 watts is a lot to power and cool, not to mention all those drives. should be one impressive build.