Reconfigure my backup storage? 8 disk, currently RaidZ2

[beezel]

We've been using an 8wide RaidZ2 for a while now, and it performs "ok." We have been running into issues with full restores taking forever, and our backups are only getting bigger (duh). We currently have a chance to rebuild our array, and I'm curious if going 4wide RaidZ1 striped would be preferred? I understand I lose a bit of redundancy, as I can only lose one disk per vdev before total failure, whereas the RaidZ2 provides any 2 disk failure recovery. As our backups are also replicated off-site, this isn't a HUGE worry for me, but if it's considered reckless I'd love to know.

So what do you guys think? Stick with 8wide RaidZ2 or would I get a lot more IOPS and read/write out of 4wide RaidZ1 x2?

Specs are: Supermicro Xeon, 32gb ram, 10gbE directly connected (no switch). Currently I can only hit about 400MB/s reads and 130MBs writes to my 8x RaidZ2. I'd love to further saturate my 10GBe and (more importantly) provide quicker full bare-metal restores of our larger (1+ TB) backups.

Thanks!

 

[MatthewSteinhoff]

Roughly speaking, give or take, you'll get twice the throughput and twice the IOPS with a two VDEV pool versus a single VDEV pool.

RAIDZ2 is highly recommended compared to RAIDZ1. Depending on your level of risk aversion and the importance of your data, you may be perfectly happy with RAIDZ1.

Cheers,
Matt

 

[beezel]

Roughly speaking, give or take, you'll get twice the throughput and twice the IOPS with a two VDEV pool versus a single VDEV pool.

RAIDZ2 is highly recommended compared to RAIDZ1. Depending on your level of risk aversion and the importance of your data, you may be perfectly happy with RAIDZ1.

Cheers,
Matt

Thanks Matt,

That is what I understood, doubling your vdevs should roughly double your performance. I guess I was looking for any gotchas or serious reasons to not stripe two raidz1s (other than the obvious one disk per pool failure scenario). So far, so good.

 

[Robert Trevellyan]

doubling your vdevs should roughly double your performance

That's true if your goal is to increase IOPS. If your goal is to increase sequential read or write performance, you need to think again: http://open-zfs.org/wiki/Performance_tuning

 

[rs225]

Can you do higher network speeds on smaller things that fit entirely in RAM? Try copying a 1GB file three times in a row, and watch that disk activity on the server doesn't happen on the repeats.

 

[beezel]

Can you do higher network speeds on smaller things that fit entirely in RAM? Try copying a 1GB file three times in a row, and watch that disk activity on the server doesn't happen on the repeats.

Yes, caching is working fine. I can get up to 7gbps on small files that fit into the ZIL, but being our backup target this is very rarely the case. Our backup solution does a ton of small reads and writes as it compares and dedupes data, very IOPS intensive. We don't have the l2arc to fit it in, with about a 15TB working dataset. Due to the nature of rollups and integrity checks, we hit all of the data fairly often in non-sequential patterns.

In an ideal world we'd use raid 10, but we just don't have more than 8 bays available and we need to maximize our total storage.