Striped Mirrors (RAID 1+0) with ZFS

[TravisT]

I've used hardware RAID 10 before and had awesome performance with it. I understand the limitations of ZFS being software RAID, as well as the pro's of using it for preventing bit rot, etc. I thought that striped zfs mirrors would perform similarly to hardware RAID 10, but in my performance testing, it is resulting in almost the exact same speeds as a single disk.

Testing single disk:

dd if=/dev/zero of=/dev/da0 bs=2048k count=50k results in about 130MB/s transfers

Testing striped mirrors:

dd if=/dev/zero of=/mnt/tank/test.dat bs=2048k count=50k results in about 120MB/s transfers

zpool status:

Code:

[root@freenas] ~# zpool status
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME                                            STATE     READ WRITE CKSUM
        tank                                            ONLINE       0     0     0
          mirror                                        ONLINE       0     0     0
            gptid/b5daa73b-f3d8-11e1-a39d-406186f5e31a  ONLINE       0     0     0
            gptid/91f9d565-f3da-11e1-a39d-406186f5e31a  ONLINE       0     0     0
          mirror                                        ONLINE       0     0     0
            gptid/caec8ac9-f3d8-11e1-a39d-406186f5e31a  ONLINE       0     0     0
            gptid/cb4c8631-f3d8-11e1-a39d-406186f5e31a  ONLINE       0     0     0

 

[TravisT]

Am I missing something here? Shouldn't performance increase in a ZFS RAID 10 configuration over a single disk?

Could something else be causing ZFS shares to run slow, and if so, what is the best way to diagnose?

 

[paleoN]

I've used hardware RAID 10 before and had awesome performance with it.

Am I missing something here?

You tell me. What's the write penalty for RAID 10 anyway?

 

[TravisT]

The write penalty for RAID 10 is 2x.

I'm not going to say that I completely understand this (my background is in networking, but I'm picking up a lot from having all these problems), but from what I understand, because each bit of striped data is mirrored across 2 disks, you should essentially get the performance of 2 striped disks even though you are using 4, but with the added benefit of redundancy of each drive (by adding a mirror).

How I understand it, my writes should be twice as fast in a RAID 10, 4 disk array as they are with a single disk.

 

[jgreco]

That really depends on things like stripe size, etc. For example, for what we do here, we tend to build for parallelism, and use a giant stripe size. The result is that a single task won't see any noticeable speed increase over a single drive, but multiple tasks will each tend to get a lot more speed, at least up until all drives are busy. Some caveats and all that... I'm not sure what ZFS does for stripe size, but my guess is that it may be optimizing for parallelism rather than optimizing for speed.