Misaligned Pools and Lost Space...

[jgreco]

No, you won't be running into problems, as long as you've got compression enabled and aren't doing any other funny/stupid/etc stuff.

 

[SirMaster]

So @jgreco, in a mixed environment with a lot of big files like photos, videos and movies but also smaller files like documents and source code etc. and a place to put VMs and jails on, which recordsize would you choose? Because we all have to make a choice. ;)

I think you missed a point. The beauty is you don't have to choose one recordsize. Recordsize is a dataset property, not a pool property.

You should split your data into datasets and set your various datasets to different recordsizes. Set your large multi-GB media files to 1M, set your documents and source code to 128K or so, set your DB and VMs smaller if it make sense.

 

[RichTJ99]

So for an 8x 6TiB raidz2 - your saying dont do it as one large dataset but use multiple with varying record sizes? Does setting the record size to 1M mean that each file is recorded at a minium of 1M even if its 100k?

I am in the process of moving data off the pool so i can destroy & recreate it. Just wanted some last minute hand holding to make sure i am doing this the right way.

 

[Bidule0hm]

"Remember when I talked about the recordsize value? well the block size can be any power of two between the size of a sector (512 or 4k) as determined by the ashift value, and the recordsize value." quoted from this post.

The TL;DR version is no, the smallest block size will be 4 kB, not 1 MB, even with a 1 MB recordsize.

 

[jgreco]

Plus, the block size actually allocated on disk is after compression.

 

[RichTJ99]

When creating the new volume, where would the option to change the record size to 1m be?

 

[Bidule0hm]

Please read the manual, especially the 9.1.3 part.

 

[RichTJ99]

OK, so I have the 8x 6tb drive setup. I chose a 1024k record size when making the 'storage' pool.

If i go into settings it doesnt show that it is 1024k - it has no mention of record size at all.

I am pretty sure it is setup the way you all said to do it. For my 8 drive setup should i be good to go?

Am i ready to dump data on this large dataset?

 

[Bidule0hm]

Use the command zfs list -o name,recsize to know what the recordsize value is on each dataset ;)

 

[snicke]

I've just created a RAIDZ3 pool with 8x 4TB drives. Hence 5x4=20TB should be available for storage, which is 18.18TiB which was also what the volume manager said when I was about to create the pool: "Capacity: 18.18 TiB". However, when the pool is created and empty the available space is only 16.0TiB if you look under "View Volumes". Where did 18.18-16.0 = 2.18 TiB space go? The pool is empty.

 

[Bidule0hm]

Have you read the first posts of this thread?

 

[snicke]

Have you read the first posts of this thread?

Yes I have. That's why I ask the question because from what I can see nothing there says anything about how you can get (18.18-16.0)/18.18=12% less space. I doesn't either say anything about how "available space" is calculated under "view volumes".

 

[Bidule0hm]

Ok, I wasn't sure. So, an 8 drives RAID-Z3 will have a 1.56 % misalignment overhead (32 * 3 / 5 = 19.2; 32 + 19.2 = 51.2; next multiple of 4 = 52; (52 - 51.2) / 51.2 = 1.56 %). There's also a 1.6 % metadata overhead to add.

Now 5 * 4 TB = 20 TB = 18.19 TiB. With the 3.2 % total overhead we lose 0.57 TiB so the available space should be 17.62 TiB.

So actually you're right, there's space missing here, 1.6 TiB to be exact. What is the used space in the GUI? Is this pool brand new? Do you have any jails? snapshots?

NB: with my 8x 3 TB RAID-Z3 I have 13.1 TiB available in the GUI for 13.20 TiB calculated so the calculation is definitely right for an 8 drives RAID-Z3.

The available space in the GUI include all the overheads and assume recordsize = 128 k so even when you change it to 1 M the available space will not change (however the used space will diminish).

 

[snicke]

So actually you're right, there's space missing here, 1.6 TiB to be exact. What is the used space in the GUI? Is this pool brand new? Do you have any jails? snapshots?

* Used space: 1012.9 KiB = "empty" (no data saved to it)
* Brand new. Actually tried to recreate it like 5 times by detaching the pool and mark both "Mark the disks as new (destroy data):" and "Also delete the share's configuration:" but with the same result each time. Upon creation the volume manager says capacity: 18.18 TiB". However, when the pool is created and empty the available space is only 16.0TiB.
* No jails
* No snapshots
* Totally new and recreated several times.

 

[Bidule0hm]

Well, I have no explication, sorry.

Maybe try to put some data on it to see if it's because the pool is totally empty that the usable space is wrong.

I any case this is just a number in the GUI, don't worry too much ;)

 

[snicke]

Nothing happens when writing data. However, I don't like "wrong numbers" in the GUI...

If using command line "zfs list" says:
NAME USED AVAIL REFER MOUNTPOINT
<mypoolname> 1013K 16.0T 219K /mnt/<mypoolname>

Why would it be "just a number in the GUI" and not something else?

 

[Bidule0hm]

A real test would be to fill it to 100 % (do not do that if you care about the data on the pool) to see what quantity of data you can really put on it. It should take less than a dozen hours if you use dd with /dev/random and a bs of 128 k ;)

 

[snicke]

Yes. But I want an explanation to the problem. Seems like a bug to me. I deleted the pool and booted from "initial install" (don't remember the exact name) for FreeNas. Should be like "fresh install" I guess. Everything i default settings when booted up (time zone etc). But when created RaidZ3 pool. Still like 1.6TB missing (16.0 TiB). Same problem when creating a RaidZ2 pool.

 

[Bidule0hm]

How many space is missing when you do an 8 drives RAID-Z2 pool?

 

[snicke]

Everything seems ok when using 7 instead of 8 disks in RAIDZ3 giving:
"Capacity: 14.55 TiB" (before volume creation)
"Available: 14 TiB" (after volume creation)
Giving only 0.55 TiB of "lost space"