8 Disks Recommended ZFS Storage Pool

[Oko]

Hi All,

This is my first post of this forum so I will bore you little with the background story. I am new to ZFS and FreeNAS but not new to FreeBSD. I used to run FreeBSD as my only OS many years ago before switching to OpenBSD.

We purchased new shiny Super Micro file server for our university research group. 2xXeon E5-2620v2 (6 cores). 64GB= 8x8GB DDR-1600 ECC Registered 2R) LSI 9207-8i 6Gb/s SAS HBA 8port and 8x3TB HDD. X540 10 Gigabit dual Intel controller. FreeNAS is installed on 64GB SATA DOM.

Server is running FreeNAS 9.2 stable release. I tried nightly snapshot 9.2.1 because of significantly better NFS performance but installer was miss behaving (reboot button would not work). I opted for FreeNAS over vanilla FreeBSD primarily due to the fact that I have strict deadlines to get this thing up and running properly and no time for a steep learning curve required to configure FreeBSD 10.0 to the same standards.

One of the things I like the best is FreeNAS documentation and Dru has done magnificent job as usual. I am reading that stuff back and forth right now.
I just discovered this
http://forums.freenas.org/threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/
Really useful!


I was wondering if you could give me a quick advise how to use 8 disks. I realized that many would advise to create RAIDZ3 with 7x3TB=10TB and use one as a hot spear. I would like to get little more storage out of it. What would be a good alternative to above short of buying two new HDD and creating RAIDZ2 with 10 HDD or buying 3 more and creating a RAIDZ3 with 11 HDD. I am not going even to consider RAIDZ1. Is RAIDZ3 so superior to RAIDZ2 that is worth of scarifying space.

Thank you for your advise!
Predrag

P.S. I got another file server little bit more modest (1 XEON, 32 GB of RAM, 4x1 Gigabit Intel) but still nice and I created RAIDZ2 pool our to 6 HDD. So far works as a charm.

 

[Rand]

Well it *totally* depends on your needs;)
If you need the space and thats more important than keeping the data save go for Z2.
If safety is the primary concern go for Z3 or Mirrors.
Hot spare is not working in FreeNas atm btw

Additionally your not saying anything about your usage patterns (pure storage, large files, small files, VMs, amount of users etc pp)

 

[joeschmuck]

Rand hit the nail on the head. If the data is very critical then Z3, you could even get by with a Z2 providing someone is monitoring your system and will have the opportunity to replace the failed drive in a reasonable time. And with 3TB drives, well it depends on how full they are as to how long the resilvering process will take.

The other factor is how fast you expect your throughput since you stated you have 10Gb Ethernet ports. You need to select your hard drives for the intended use and then I assume you have read Cyberjocks little guide on the proper configuration of a pool. You should have the proper multiple of drives in order to maximize ZFS performance.

Make sure you turn on the tunables so all your RAM is used, since you have a ton of it and that would put it to good use.

 

[Oko]

I appreciate all answers I got. I would like to fill in on the details I left in my original post.
We are using or file servers primarily as a storage space. No virtual machines. Files, well
we have files of all sizes going including really big (is 100-150 GB for a single file really big?)
I have have close to 100 users but only 50 active. They access file systems via NFS primarily. The
data is however spread over 9 files servers. I am monitoring all systems and in particular
our file servers 24/7 and typically HDD are replaced in at most 24h. Everything is backed up
and then also archived. In the light of that info since the storage space is very important
to me it looks like RAID-Z2 is the way to go for me. I found out the info about lack of hot
spares in Dru's documentation but I really appreciate that you confirmed that.

The main remaining question for me is should I configure those 8 drives as RAID-Z2 which
is not optimal according to GUI or get two more. I would really like to get 2 more. I figured
out 10 drives will give us close to 21 TB of effective space which is really cool.

On the final note I would like to commend developers of FreeNAS and of course of FreeBSD
for magnificent work they have done. I am in the process of migrating and decommissioning
file servers (I should end up with about 7-8) and while I concur that I will have to keep few
of those running RedHat and hardware RAID 6 due to the fact that they also used as
computing nodes (God I hate the fact that there is no MATLAB for FreeBSD) all Linux software
RAIDs will be wiped out and replaced with FreeNAS.


Can somebody tell me how does FreeBSD work configured as hardware RAID 6 (MegaRAID SAS 9271-4)?