Best practice

[Gandalf Corvotempesta]

I have to create a new storage for VMs image hosting

Any suggestion to archieve Max reliability?

I was thinking to create a stripes mirrored (with 3 disks) zvols (6 disks in total)

This should allow me to survive to 2 disk failures on each mirror.

Probably this storage will be created with only SSDs (Intel 3616 i think)

Any advice? A raidz2 would be better?

 

[m0nkey_]

For VM images, you want mirrored vdevs.

 

[tvsjr]

How many VMs? Screwing around at home, or a production workload?

3-way mirrors with SSD seems like a pretty significant waste of expensive storage. 2-way mirrors should be quite sufficient... combine that with proper SMART testing/monitoring, and a good backup strategy.

 

[Deleted47050]

Any advice? A raidz2 would be better?

No, not for VMs. Not for anything that requires high IOPS, for that matter. In these cases, mirrors are the way to go.

 

[Gandalf Corvotempesta]

I really hate mirrors because you can loose only one disk for each pair and Murphy is always watching. I've lost (the last one 2/3 days ago) multiple disks at the same time from the same pair.

What if i create mirrors with 3 disks?

 

[Gandalf Corvotempesta]

keep in mind that currently all of my servers are made with raid6 and sas 15k disks

Replacing sas with ssd the performance gain would be sensible but i don't like the write amplification due to raidz2 on ssd disks

 

[Deleted47050]

3-disk mirrors would be good. You would have to have all three drives fail in the same vdev to lose all your pool.

Implement a backup strategy anyway.


Sent from my iPhone using Tapatalk

 

[Gandalf Corvotempesta]

What about raidz2 and ssd reliability?
By replacing the current raid6 made with 6 sas disks with a raidz2 made with 6ssds, performance would be better regardless the double parity

What is scaring me is the write amplification because ssd tend to fails on writes and in a raid6 almost all disks are subject to the same write pattern

 

[Stux]

If it were all ssd, I'd consider Raidz2. Ssd IOPS are an order of magnitude (or two) better than spinning rust, and thus the mere doubling of IOPS from 2 3-way mirrors is not worth it vs the loss of capacity to redundnsncy to get the same reliability as 6-way Raidz2.

 

[Jailer]

What is scaring me is the write amplification because ssd tend to fails on writes and in a raid6 almost all disks are subject to the same write pattern

You mitigate that with close monitoring and a good backup strategy and replace the disks before the end of their write cycle life.

 

[joeschmuck]

Do you need to use SSDs? If not then go for some 15k drives instead, lots of RAM, an SLOG, and a large L2ARC. Of course you could just use the SSDs but they may not live a long life.

 

[Deleted47050]

What about raidz2 and ssd reliability?
By replacing the current raid6 made with 6 sas disks with a raidz2 made with 6ssds, performance would be better regardless the double parity

What is scaring me is the write amplification because ssd tend to fails on writes and in a raid6 almost all disks are subject to the same write pattern

Write amplification would definitely be an issue if you go for raidz2.

However, I suppose that you could at least partially mitigate that by using larger SSD drives, which can normally withstand a higher amount of data written to them compared to smaller SSDs.

 

[Gandalf Corvotempesta]

Currently i have multiple servers with raid6 made with 6 or 8 sas 15k (300gb or 600gb)

Rebuild time for the 6x600gb is way too long, about 25 hours

i would like to lower this a lot but a raid6 made with ssd could lead to multiple failures at the same time because all disks are written almost in the same way at the same time.

 

[Deleted47050]

Well, if rebuild times are also a concern, mirrors are also much faster to resilver.


Sent from my iPhone using Tapatalk

 

[Gandalf Corvotempesta]

That's why i was talking at 3way mirrors.
They should also be safer as you have to loose 3 disks from the same vdev to loose the pool

With raidz2 you have to loose any 3 disks, so, in this case, a raidz2 is more insecure than multiple 3way mirrors

Anyone has experience with resilvering times for a full ssd raidz2?

(I could also try a raidz3 :) )

 

[Ericloewe]

Write amplification would definitely be an issue if you go for raidz2.

How do you figure? Beyond the obvious extra 50% of writes, that is.

 

[Deleted47050]

How do you figure? Beyond the obvious extra 50% of writes, that is.

Oh that's what I meant. Perhaps "issue" is not the right word, "concern" might have been a better one :)

I meant that the additional writes could be more of a problem in this scenario since we are dealing with SSDs. Especially if they use smaller capacity ones.


Sent from my iPhone using Tapatalk

 

[Gandalf Corvotempesta]

I meant that the additional writes could be more of a problem in this scenario since we are dealing with SSDs. Especially if they use smaller capacity ones.

Exactly

 

[Ericloewe]

Oh that's what I meant. Perhaps "issue" is not the right word, "concern" might have been a better one :)

I meant that the additional writes could be more of a problem in this scenario since we are dealing with SSDs. Especially if they use smaller capacity ones.


Sent from my iPhone using Tapatalk

It's not something I would worry about.

 

[Robert Trevellyan]

Rebuild time for the 6x600gb is way too long, about 25 hours

One benefit of ZFS is that is only resilvers allocated blocks. Combine this with the requirement to keep very high free space percentage for a block storage application.