Need Build Advice.

[Wrench]

I would love to buy an IX system but I don't really have that cash atm. Its for my Vmware lab.

Any who my build is going to be ZFS based

Mobo
TYAN S5510GM3NR Micro ATX Server Motherboard LGA 1155 Intel C204 DDR3 ECC UDIMM
Ram
Kingston 8GB (2 x 4GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 (PC3 10600) Server Memory Model KVR1333D3E9SK2/8G

Intel Xeon E3-1220 Sandy Bridge 3.1GHz LGA 1155 80W

NORCO RPC-2212 Black 2U Rackmount Server Case with 12 Hot-Swappable SATA/SAS Drive Bays - OEM

Part I am stuck on is Hard Drives and a controller card that will work with Freenas

 

[ProtoSD]

Hard drives shouldn't be a problem, you don't need 'Black' drives. It's better to have a variety of different manufactures, this way if you get a bad batch or a certain manufacturer has problems, all your drives aren't at risk.

Take a look at this link, there's a hardware thread I started to help people find parts that work with FreeNAS.

FAQS-Documentation-are-HERE

Hopefully there's a controller in there that will work for you. If there's not and you find one that does, let me know and I'll update that post and add what you got.

 

[Wrench]

Ok Looks like two 9211 8i would do.

Did also reading on ZIL. I was thinking of 10 and 2 Log drive. Should I also have 2 Cache drives? and should the Log file drives and or Cache drives be fast AKA SSD? Do all drives need to be the same?

Drives I will use are HITACHI Deskstar 7K3000 Or Western Digital Caviar Black 2TB 64MB Cache 7200RPM

 

[survive]

Hi Wrench,

I have seen the FreeNAS guys suggest controllers based on the LSi 2008 chip as the "cards to get" so the 9211 should do just fine. You might want to consider a single SAS 9201-16i card...it looks like it's exactly double the money but you would end up saving an 8x slot. Take a look at this, it's a little dated, but is a good rundown of cards and chips you might want to consider:

http://blog.zorinaq.com/?e=10

As far as ZIL\ARC (log) drives go you want to add them in pairs. Currently, with ZFS v15 (which is what FreeNAS uses) the loss of the cache drive is fatal to your volume, so you want to add them 2 at a time & mirror them.
There are actually 2 types of caches, the ZIL, which is for writes, and the ARC which is for reads. You want fast drives, at least as fast as your network interface(s), so if you have gig-e, you want to see 125MB/s write speed if you go full bore. The ZIL doesn't have to be very big as ZFS flushes to disk every 30 seconds so I think 10GB of ZIL cache would be the most you need for gig-e (check my math!). Ideally you want SLC (enterprise) for the ZIL cache since all it does is writes and you can eventually "wear out" a cheaper MLC based device. I think you can partition the cache drive so both the ZIL & ARC are on the same device, but that means you run the risk of overloading the device with requests. With 16 ports in a 12 bay case it might be best to add 4 SSD's, 2 smaller SLC drives for ZIL and 2 bigger MLC drives for ARC if you can find room to stash them. I think the ZIL cache will particularly help out with NFS since it wants to see writes properly acknowledged\committed to disk.

I don't think there's anything wrong with either choice of drives but I would lean to the Blacks if it were my money.

-Will

 

[Wrench]

I was looking at that card. I just didn't see it on the BSD HCL.It doesn't look low profile tho. Oh well I guess I can build a 3U Nas with 16 drives :)

 

[Wrench]

I was looking at that card. I just didn't see it on the BSD HCL.It doesn't look low profile tho. Oh well I guess I can build a 3U Nas with 16 drives :)

I found an intel 20gig SLC (but if I bond nics then I will definetley need something bigger.)
and a
OCZ Agility 3 60GB MLC drives

Now when you meant mirror does that mean Software or Hardware ?

Can I just get 15krpm sas drives to those jobs? Can I upgrade the ZIL and Cache drives at a later date?

 

[Milhouse]

Now when you meant mirror does that mean Software or Hardware ?

Can I just get 15krpm sas drives to those jobs? Can I upgrade the ZIL and Cache drives at a later date?

Software (ZFS) mirroring - two drives (SSD or fast HD).

Yes you can upgrade the 15K RPM drives to perform as ZIL and ARC (Cache) drives, but to get the most benefit you really want to consider using SSDs.

If you go with two 9211-8i controllers and a 16-drive 3-U case, you may also want to give some thought to how you configure your drives, backplanes and vdevs so that you don't lose your entire array should a single 9211-8i controller fail. If you were to configure your 12-16 drives using three vdevs as follows (RAIDZ1 or RAIDZ2), you should have controller redundancy:

(where Cx are the controllers, Bm.n is Backplane m port n on your Norco, and Vx are your vdevs):

Code:

V1	V2	V3	ZIL/ARC
B1.1	B1.2	B1.3	B1.4
B2.1	B2.2	B2.3	B2.4
B3.1	B3.2	B3.3
B4.1	B4.2	B4.3

C1	C2		
B1	B2		
B3	B4		

With this configuration, if you "lost" controller C1 you would lose all drives connected to the "odd" numbered backplanes (ie. B1 and B3) and while this would leave all vdevs and ZIL/ARC degraded, the pool would remain available.

If however you have more than two disks in any vdev connected to the same controller, which might be the obvious means of configuring the disks/backplanes/controllers, then you run the risk of losing the entire pool should a single controller fail.

As likely as a controller failure is, it can happen and if you have the benefit of multiple controller cards it makes sense to eliminate any single point of failure at the outset. :)

For this reason two 9211-8i's may be offer advantages over a single 9201-16i, and the former will also utilise two 8x PCIe slots rather than squeezing the same 16-disk bandwidth through the latters single 8x PCIe slot.

 

[Wrench]

Its a valid point with two controllers rather then 1. Awesome advice thank you.