Advice regarding zpool setup options for a noob

[Pseudobolt]

Hi,
I'm about to start playing around with a server which I eventually intend to set up as a home NAS file and media server. I plan to test it out a fair bit before I put any real data on it, because I have read more than enough warnings in these forums that if you set ZFS up wrong, you are basically screwed.

My server has 8 drives in it currently, plus I can add at least three, and I think probably four, more to it. (It hasn't actually arrived yet -- I will check all this out once it gets here in the next day or two.) Essentially, I am considering two options: 11 drives in 8+3 RAIDZ, or 12 drives in two vdevs of 4+2. I'm looking for advice from experienced people on the tradeoffs involved.

Firstly, upgrading: the drives it comes with are 500G, and I plan to slowly upgrade them as my budget allows. Doing 2x(4+2) means I can upgrade half of them and get an immediate size increase, whereas (8+3) will require upgrading all of them to expand the size. I also seem to recall reading in a thread here (which I now can't find) that two vdevs is generally faster than a single large vdev because the data is striped across them. (On the other hand, I expect a relatively light load on the server and am limited to gigabit ethernet or wireless-n speeds anyway, so the performance difference may be irrelevant.)

On the other hand, I also remember reading (and again I can't find it) that if you enlarge one vdev in a zpool, ZFS will start writing most(/all?) its new data in that vdev because most of the pool's free space is there. If so, this sounds like it might limit any potential performance benefit I could get from having two vdevs. But will this end up actually being slower than a single vdev (because the data's spread over only half the number of disks)?


I realise some or all of these questions might not have good answers apart from "try it and see". But if anyone with some knowledge of the subject can tell me that one's a better idea, it would be good to know.

Thanks!
~ a FreeNAS noob
(Although with a fair bit of general IT background)

 

[cyberjock]

Firstly, upgrading: the drives it comes with are 500G, and I plan to slowly upgrade them as my budget allows. Doing 2x(4+2) means I can upgrade half of them and get an immediate size increase, whereas (8+3) will require upgrading all of them to expand the size. I also seem to recall reading in a thread here (which I now can't find) that two vdevs is generally faster than a single large vdev because the data is striped across them. (On the other hand, I expect a relatively light load on the server and am limited to gigabit ethernet or wireless-n speeds anyway, so the performance difference may be irrelevant.)

A well built machine will almost always be limited to Gb speeds. In your case, 500GB drives are relatively slow, but 8 of them should be able to do GbE speeds without breaking a sweat in almost any configuration you chose.

On the other hand, I also remember reading (and again I can't find it) that if you enlarge one vdev in a zpool, ZFS will start writing most(/all?) its new data in that vdev because most of the pool's free space is there. If so, this sounds like it might limit any potential performance benefit I could get from having two vdevs. But will this end up actually being slower than a single vdev (because the data's spread over only half the number of disks)?

"Enlarge" as in start using larger disks in a pre-existing vdev? That's the *only* way to make a vdev bigger. ZFS has a smart algorithm it uses to optimize writes as appropriate. If you add a new data it will preferentially chose the new vdev to a point. Things will balance itself out because of this algorithm and does not require any action on your part. As for your concern about some potential limitation(or being slower than a single vdev which I do not understand the logic at all) if you are that concerned about speeds and that limited you've already made bigger mistakes and being upset because a second vdev isn't being striped with the first in a 1:1 ratio is the least of your problems.

 

[Pseudobolt]

Yes, enlarge by replacing the drives.

When I said 'slower than a single vdev', I meant the following: with one vdev, the data goes across 8 drives (+3 parity). With two vdevs the data would go across 8 drives as well (+4 parity), but if I upgraded the drives in one vdev and it started to prefer that one, wouldn't that mean the data would (mostly) go across only 4 of the drives instead of all 8? Thus reducing the speed. Or do I have that all wrong?

I know that more spindles != more IOPS necessarily with RAIDZ (I followed a link from the forums here to some blogs about it), but it should still mean increased MB/sec for sequential read/write, correct? Again, not that it matters... if as you said it will max out on a Gb ethernet anyway, then it sounds like it doesn't really matter which option I choose from a performance point of view.

 

[cyberjock]

You have it right(kind of), but like I said.. if you are so short on spare performance that you're having to sweat the difference between using one vdev or two, you've already failed to properly size your server for the workload. If one vdev can't keep up ZFS is supposed to use the other vdev(unless you've filled it to 100% capacity).