ZFS RAID size and reliability calculator

ZFS RAID size and reliability calculator 2017-05-17

Status
Not open for further replies.

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
This is the discussion thread for this Resource. To find the actual Resource, please use the Overview tab above to navigate to it.

The original post follows below, in the spoiler tags:

I've developed a JS web app to calculate the exact usable space and the reliability specially for the ZFS RAID types. You can find it here ;)

It's stupid-proof but not idiot-proof. For example you can put only one drive and select a mirror RAID (even if the app gives to you the minimum number of drives for the chosen RAID...). It's that way because I don't want to clutter the code with a ton of useless checks on the values, the user must know a minimum what he's doing :)

The app assumes ashift=12, recordsize=128k and no compression.

The app is released under the GPL license.


Size

It supports striped mirrors (but not striped RAID-Zs). For example you can put 12 drives then select "3 Drives Mirror" and the result will be calculated for 4 striped 3 drives mirrors.

The blocks overhead is disabled for now. I now know how to calculate it but I need some time to update the app :)

The parity and data spaces percentages are relative to the RAID total space. The overheads percentages are relative to the total data space. The minimum free and usable spaces percentages are relative to the total data space minus the total overhead.

The minimum recommended free space uses the 80% rule.

The usable data space is the total data space minus the total overhead and the minimum recommended free space.


Reliability

The MTBF is asked in Mh which stands for Megahours to avoid the need to input many zeros. It's the only other field (the other one is the drive size field) where you can put a decimal number.

The MTTPR (Mean Time To Physical Replacement) is the time needed to detect and physically replace a failed drive.

The MTTR (Mean Time To Recovery) is the sum of the MTTPR and the usable data space divided by the rebuild speed (this is the worst case, if you have a 50% full pool for example the rebuild time will be lower). It's not displayed for a Stripe RAID for obvious reasons...

The MTTDL (Mean Time To Data Loss) is expressed with the scientific notation because it's easier to read and to compare for very large or small numbers. It's not displayed for striped mirrors because it's a pain in the *** to implement :p


Misc

If you don't need the reliability stats you can just fill the first 3 inputs.

If you don't need the size stats you need to fill all the inputs anyway (the reliability stats use them).

You can suggest me some improvements of course ;)


Changelog

[2015-09-29]
# Changed the drive size field type to decimal numbers (you can now put drive sizes lower than 1 TB)

[2015-09-14]
# Changed the link of the app for a static link to negate the problem of the app's URL changes with each update of the app

[2015-03-12]
# Added the reliability stats
# Disabled the blocks overhead temporarily
# Reduced the inputs height (now equal to select height)

[2015-03-10]
# Remade the UI to use a table
# Added the percentages
# Zoom friendly

[2015-03-08]
# Added the minimum recommended free space
# Added the total overhead

[2015-03-07]
# Added the minimum number of drives


TODO

# Move the app to my website
# Integrate the manual to the app
# Rename "Metadata overhead" to "CoW overhead" and change the value from 1/64 to 1/32 (see https://forums.freenas.org/index.ph...-8-or-10-with-raidz2.38334/page-2#post-449814)
# Add "Minimum recommended free space" % input field (so the user can change the default 80 %)
# Add RAM minimum size recommendation (add checkbox for dedup)
# Rename "Usable data space" to "Recommended usable data space"
# Rename "Minimum recommended free space" to "Recommended minimum free space"
# Rename "Drive size" to "Drive space" (in the results, not the input)
# Add the total data space minus the total overhead (label: Usable data space)
# Add the swap size input (in GB, before RAID type); add swap space after drive space; add drive space - swap space (label: Usable drive space); add total swap space before total parity and data spaces; correct total data space to be total data space - total swap space
# Add Number of vdevs (after RAID type)
# Change the display block/none to be in each RAID type functions instead of a global if() (to allow to temporary not implement the MTTDL, MTTR, ... calculus for a RAID type if too complex)
# Add the DL probability percentage because of URE during resilver (take into account thet for multiple parity it must be URE on the same sector of each drive)
# Add the NOMDL and BHL if possible
# Add the DL probability percentages (for 1, 2, 3, 5, 10, 15, 20 and 30 y)
# Calculate and display the MTTDL for striped mirrors
# Fix the blocks overhead
# Add the allocation overhead
# Display the MTTDL on a logarithmic scale (e.g. 10^x.yz)
 
Last edited:

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
I've developed a JS web app to calculate the exact usable space for each ZFS RAID types, you can find it here ;)

It's stupid-proof but not idiot-proof. For example you can't put a negative value but you can put only one drive and select a mirror RAID. It's that way because I don't want to clutter the code with a ton of useless checks on the values, the user must know what he is doing (and, really, there's only 2 fields and one select...) :D

The checksum and block overheads are more or less experimental. I can't find a lot of info on the exact value or on how to calculate them. I made a thread so if someone knows the answer he is welcome :)

Of course you can suggest me some improvements ;)

NB: if you want to bookmark the app don't bookmark it directly because the URL will change for each modification I make on it, bookmark this thread instead.
this is actually really useful. Thank you. I was wondering how much usable space i will have once i add the fourth drive,,

Thanks for this.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
You're welcome ;)
 

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
You're welcome ;)
hey by the way, it shows RAID-Z2 and RAID-z3 in there.. i am wondering, is there a minimum amount of hard drives needed to enable those? because i have 3 drives and i dont see them on the drop menu when i create the volume.
if they do have drive requirement it would be cool to mention on the site or at least not allow the option when someone puts 3 drives?
just a thought .. .
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Yeah, it's part of the "It's stupid-proof but not idiot-proof" in the first post.

I guess I can add a text like "x drives minimum" next to the select ;)
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
I upsetted you? because it wasn't the intention at all.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Ah ok, no problem (I miss some language details sometimes so I'm not sure if it's a joke or not) :)

I'll do the change on the app ;)
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,458
hey by the way, it shows RAID-Z2 and RAID-z3 in there.. i am wondering, is there a minimum amount of hard drives needed to enable those?
RAIDZ2 needs at least four disks; RAIDZ3 needs at least five disks.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Modification done ;)
 

TinTIn

Contributor
Joined
Feb 9, 2015
Messages
142
This was very helpful indeed. Many thanks for the effort.


Sent from my iPhone using Tapatalk
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
You're welcome ;)

NB: I've modified the app this morning to add some stats (hopefully it's more clear and helpful now), clear some calculation ambiguities and clean the source code.
 

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
You're welcome ;)

NB: I've modified the app this morning to add some stats (hopefully it's more clear and helpful now), clear some calculation ambiguities and clean the source code.
and you also broke it :p
because its not doing any calculation for me now
 

diedrichg

Wizard
Joined
Dec 4, 2012
Messages
1,319
Nice job. I guess I will have to use it on my desktop because I can't get it to work on my phone. It's there a "go" button or something?
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
I tested after I did the changes and I just tested right now and it works. At least on Firefox 36.

@diedrichg No, it automatically recalculate the values each time you modify the inputs. It seems it's the last changes that degraded the compatibility, see below.

NB: maybe I know what's the problem, I used a feature not supported on every browser that are a bit old. I'll change that ;)

Edit: changes done. It should be good this time ;) (remember to use the link in the first post to get the latest version because the URL changes)
 
Last edited:

yis

Dabbler
Joined
Jan 24, 2015
Messages
30
I tested after I did the changes and I just tested right now and it works. At least on Firefox 36.

@diedrichg No, it automatically recalculate the values each time you modify the inputs. It seems it's the last changes that degraded the compatibility, see below.

NB: maybe I know what's the problem, I used a feature not supported on every browser that are a bit old. I'll change that ;)

Edit: changes done. It should be good this time ;) (remember to use the link in the first post to get the last version because the URL changes)
yeah much better now
 

diedrichg

Wizard
Joined
Dec 4, 2012
Messages
1,319
Working now, awesome job! Tyvm.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Good ;)

Thanks for the testing.
 

SwampRabbit

Explorer
Joined
Apr 25, 2014
Messages
61
Thank you for taking the time to make this, its pretty awesome.

I like how you included the breakdown of:
Data space minus checksums overhead
Data space minus blocks overhead
Data space minus total overhead
Minimum free space recommended
Recommended usable data space

I usually use the wintelguy or serverthehome calculators but those do not give you the
recommended free and usable space.

One thing I like about the wintelguy calculator is that you can set how many RAID groups you have.
It also shows Space efficiency and Fault tolerance (disk drives per RAID group) and I like that.

The STH RAID calculator is very very basic, but they have a cool RAID Reliability calculator, but it doesn't
include RAIDZ1 or RAIDZ2.

Those are some options to look at including, but what you have is simple and to the point, which is perfect sometimes.
But good job! I'll test it out more when I have some time.
 
Status
Not open for further replies.
Top