Effective storage space

[only1miller]

Hi everyone,

So a quick back of the napkin question here - I have a freenas server with 8 4TB drives setup into one raidz2 volume. Now, some very rough calculations would give the machine 32TB of raw storage space to begin with, then subtract 8TB for the raidz2 - leaving 24TB. Again, rough - I know the drives don't give you a full 4TB, but still - when all is said and done, I get barely 20TB of storage space. Seems like I'm missing some space there somewhere. Even with the swap folders, formatting, etc. does 20TB sound right to everyone? I just want to be sure I'm not configuring things incorrectly.

Thanks and apologies if i'm being daft.

-J

 

[garm]

FreeNAS and ZFS shows TiB, not TB.
Wanna know what it’s doing, try @Bidule0hm ’s calculator https://jsfiddle.net/Biduleohm/paq5u7z5/1/embedded/result/

 

[jgreco]

It's also worth noting that RAIDZ parity does not consume a fixed amount of overhead.

https://extranet.www.sol.net/files/freenas/fragmentation/RAIDZ-small.png

A block that happens to work out, with parity, to the width of the RAIDZ vdev (such as the first two rows) consumes the "expected" amount of space, if we define "expected" to be what people familiar with RAID5 to expect. Smaller sizes (including the brown single sector data write at LBA 3 and the light blue starting at LBA 8). For RAIDZ2 or Z3 the situation can be even worse than that. The number that ZFS provides is not actually a guarantee of the amount of stuff you can store.

 

[only1miller]

It's also worth noting that RAIDZ parity does not consume a fixed amount of overhead.

https://extranet.www.sol.net/files/freenas/fragmentation/RAIDZ-small.png

A block that happens to work out, with parity, to the width of the RAIDZ vdev (such as the first two rows) consumes the "expected" amount of space, if we define "expected" to be what people familiar with RAID5 to expect. Smaller sizes (including the brown single sector data write at LBA 3 and the light blue starting at LBA 8). For RAIDZ2 or Z3 the situation can be even worse than that. The number that ZFS provides is not actually a guarantee of the amount of stuff you can store.

Ok, that is the first time I've seen that and it's quite enlightening! Thanks jgreco

 

[only1miller]

FreeNAS and ZFS shows TiB, not TB.
Wanna know what it’s doing, try @Bidule0hm ’s calculator https://jsfiddle.net/Biduleohm/paq5u7z5/1/embedded/result/

That's a great tool. Thanks garm. One sure loses a lot of space to freenas but I think it's worth it. My the calculator it seems I'm getting the storage space that I should.

 

[jgreco]

Ok, that is the first time I've seen that and it's quite enlightening! Thanks jgreco

So the big takeaway there is that ZFS will generally be less efficient if it is storing "small" things on RAIDZ. This includes small files and also block storage (i.e. iSCSI/VM/etc). In the particular case of block storage, you can work out your particular variables to get optimal storage, but if you don't, there are horrifyingly inefficient configurations also possible.

If you mostly use RAIDZ for storing big sequential files, a task it was designed for, you won't be likely to notice any inefficiency, as the vast majority of your data will be optimal.