ZFS Can of worms

[Terry Wilson]

So I have three 4TB SAS drives and need:
Redundancy
Speed
Faster/Safe recovery if a drive(s) fail.
Capacity is not important as 4TB is more than enough

I was going to do Z2 but GUI say's nope, Z1 or mirror is all I will allow (Freenas 9.10)

So for the fun of it I did a 3 way mirror (which I had never done before coming from the hardware RAID world where even number of drives are required).

So it appears the pool is good and has a 3 way mirror.

Will this accomplish the goal above or a bad idea and need to be revisited as Z?

 

[danb35]

ZFS supports mirrors of any number of devices (I'm not sure if there is a maximum number of devices, but if there is, it's big), so you have a perfectly valid configuration. However, you're using 2/3 of your total capacity for redundancy. If you're OK with that, there's no problem.

 

[Spearfoot]

So I have three 4TB SAS drives and need:
Redundancy
Speed
Faster/Safe recovery if a drive(s) fail.
Capacity is not important as 4TB is more than enough

I was going to do Z2 but GUI say's nope, Z1 or mirror is all I will allow (Freenas 9.10)

So for the fun of it I did a 3 way mirror (which I had never done before coming from the hardware RAID world where even number of drives are required).

So it appears the pool is good and has a 3 way mirror.

Will this accomplish the goal above or a bad idea and need to be revisited as Z?

A 3-way mirror like you've built is much safer than a RAIDZ1 array. But the cost is high, as @danb35 pointed out.

 

[Terry Wilson]

ZFS supports mirrors of any number of devices (I'm not sure if there is a maximum number of devices, but if there is, it's big), so you have a perfectly valid configuration. However, you're using 2/3 of your total capacity for redundancy. If you're OK with that, there's no problem.

Yeah, I am fine with that, just so happens I had these drives laying around, only needed 1 TB so the rest is wasted space (did I say that outloud?), just wanted to make sure I was not off base on the ZFS RAID configuration as coming from the old school hardware based RAID this was a WTX? moment... odd number of drives, mirror? ZFS.. gotta love it!

 

[jgreco]

Some hardware RAID controllers can pull that trick too.

Mirrors will tend to have better performance for heavy workloads, but as others have pointed out, this comes at a cost in terms of capacity. On the other hand, you can develop three different data faults and survive them all...

 

[Terry Wilson]

Thanks again for the info!

 

[diehard]

The downside of a 3 way mirror is that with only 3 drives, you will sill only get the IOPS and throughput of a single drive so i don't know if it really fulfills your speed requirement.

 

[SweetAndLow]

Streaming reads well be super fast since it will read from all disks. Personally as I use my Nas more I'm a fan of more vdevs it makes random stuff so much faster. If you can gather up one more disk you could do 2 sets of mirrors which would be a good performance improvement.

Sent from my Nexus 5X using Tapatalk

 

[diehard]

I'm about 95% sure Sequential reads on single 3 way mirror vdev will still only be the speed of 1 drive. However, RAIDZ can increase throughput (not IOPS) from multiple disks on a single vdev.

 

[Stux]

I'm about 95% sure Sequential reads on single 3 way mirror vdev will still only be the speed of 1 drive. However, RAIDZ can increase throughput (not IOPS) from multiple disks on a single vdev.

I'm fairly certain that you get double throughput on sequential reads on a dual mirror and single throughput on sequential writes.

I would expect, but have never tested, triple read throughput from a triple mirror

 

[Arwen]

I'm fairly certain that you get double throughput on sequential reads on a dual mirror and single throughput on sequential writes.

I would expect, but have never tested, triple read throughput from a triple mirror

Yes, that is also my understanding.

 

[Ericloewe]

ZFS is smart enough to be able to read from several elements of a mirror at once to improve performance, and that is sure to scale as far as you can scale mirrors.

 

[Terry Wilson]

The downside of a 3 way mirror is that with only 3 drives, you will sill only get the IOPS and throughput of a single drive so i don't know if it really fulfills your speed requirement.

Good Point, I didn't think of that but in this case since its a 15K SAS I "should" be ok.

 

[diehard]

As others have pointed out in this thread, that "downside" is a misconception. Reading from a mirror is going to be faster than reading from a single disk, in cases where the single disk would have been a bottleneck. You'll be fine.

It is very possibly i am wrong about the sequential throughput , but im absolutely positive that each vdev gives the write iops of 1 drive.

 

[diehard]

Yes that is true for writing. After all, you have to write all the data to each disk in the mirror.

Right, was more trying to point out the benefit of in this case getting another 4TB drive and using mirrored vdevs might be worth it for him. I probably should have just said that.

 

[Terry Wilson]

Right, was more trying to point out the benefit of in this case getting another 4TB drive and using mirrored vdevs might be worth it for him. I probably should have just said that.

Yeah, I thought about that as well, 2 mirrored vdevs but I didn't have a spare 4TB drive laying around and this was a donation anyway to a non profit group that only needed 1-15.TB usable.

While I am thinking about it, didn't someone create a spreadsheet one time that you could use to calculate the theoretical risk of adding more drives to a vdev based on a certain ZFS RAID type?

 

[Stux]

In this case, a 3 way mirror is probably a good idea.

The Non-profit will probably run it into the ground, and a 3 way mirror will be able to take a lot of abuse. Ie if a drive fails, they can wait quite a while for a donation of another 4TB drive or greater.

Another thing, you can remove the 3rd mirror trivially at a later stage. For example, they need more than 4TB eventually. Detach the 3rd disk, add it and another disk as another mirror.

 

[Terry Wilson]

In this case, a 3 way mirror is probably a good idea.

The Non-profit will probably run it into the ground, and a 3 way mirror will be able to take a lot of abuse. Ie if a drive fails, they can wait quite a while for a donation of another 4TB drive or greater.

Another thing, you can remove the 3rd mirror trivially at a later stage. For example, they need more than 4TB eventually. Detach the 3rd disk, add it and another disk as another mirror.

Not following you here on the last paragraph but sounds very interesting.... so you are saying remove the third drive and it automatically becomes a 2 way mirror, then use that drive along with another and you have 2 vdevs each as a Z1? Sort of like in the old days a RAID 10 ?

 

[snaptec]

2 vdevs as mirrors. Not a z1.
Yes something like old raid 10.
Bit if you need more space or performance you can simply add as many mirrors as vdev as you want


Gesendet von iPhone mit Tapatalk

 

[Stux]

Yes.

3 way mirror, remove one drive. Take that drive and a new drive. Add as a new mirror.

Now you have twice as much space and performance.