Hard drives usage and disk space

[RobGoss]

Hello everyone, so I just finished getting TrueNas Core installed
After trying to configuring the system and getting things all setup, I was able to login to my files share folder on my PC. I saved a few of my YouTube videos in a folder problem is I don't see that disk space being used up when I look at the system stats

I have 5 hard drives install on my home server total of drive space is 10 TB, are all these drives working as one unit, meaning can I see the 10 TB of drives on my server UI some were. I hope I'm saying this correctly

Thanks everyone

 

[sretalla]

when I look at the system stats

Maybe you mean the Dashboard?

You will usually see your pool there with Used % and Free space shown in TB...

Depending on the size of your videos, you may not have "moved the needle" to have 1% used yet.

1% of 10TB is 100GB

Perhaps you're looking for the kind of details you would see under Storage | Pools ?

 

[RobGoss]

Hello Sretalla thanks so much for your fast reply, Yes I meant dashboard you are correct
I have five 2-TB drives installed on my home server that means I should be seeing 10-TB of free space correct?
I uploaded a few of my YouTube videos just to see how much space it would use but I'm not seeing the full amount of my installed hard drives on my server. It shows Free space. 5.23 TB and Total Disk 5 (data )

Shouldn't I be seeing 10 - TB of Free space? I did not upload anything except those few videos


Hope I'm explaining this correctly and Thank you so much for your time I attached a screen shot of what I'm seeing in my dash board

 

[sretalla]

What would help us to see the details of your pool would be if you go to the Storage | Pools page, click the cogwheel to the right of the screen on your pool and select Status.

There we will see the pool layout, which I imagine from your description of the hardware and what we see above will show us you selected to have RAIDZ2, which is a perfectly reasonable choice, but gives you only the available storage from 3 of the disks (which is also measured in TiB, so shows as smaller than you see written on the packet, which is TB).

 

[RobGoss]

Wow you're are correct sretalla it is RaidZ2 but I believe it choose that for me when setting up the pools. I'm at work right now but will do as suggested as soon as I get to my machines

How would I correct this issue in order to see the full amount of my drives.

Thanks so much

 

[sretalla]

In order to use all of your drives' capacity, you would need to run with no redundancy (selecting the Stripe VDEV type when creating the pool... which you would need to re-do to put that change into place).

Be aware that running without redundancy means you will lose access to all of the data on all of the disks (the entire pool) immediately upon losing (through failure or other removal) any one of your 5 disks.

You also won't have the possibility to recover a file that ZFS would detect as corrupt (due to the checksums it keeps for all blocks) even if you can detect that corruption, the disks won't contain any alternate data to rebuild the corrupt data.

If you're fine with that and you just want a bunch of storage that doesn't matter if it's lost and can be re-built from scratch, then great, rebuild your pool and go ahead.

 

[RobGoss]

In order to use all of your drives' capacity, you would need to run with no redundancy (selecting the Stripe VDEV type when creating the pool... which you would need to re-do to put that change into place).

Be aware that running without redundancy means you will lose access to all of the data on all of the disks (the entire pool) immediately upon losing (through failure or other removal) any one of your 5 disks.

You also won't have the possibility to recover a file that ZFS would detect as corrupt (due to the checksums it keeps for all blocks) even if you can detect that corruption, the disks won't contain any alternate data to rebuild the corrupt data.

If you're fine with that and you just want a bunch of storage that doesn't matter if it's lost and can be re-built from scratch, then great, rebuild your pool and go ahead.

Sounds good I don't really mind loosing any data on those drives they are all new with nothing really on them except a few videos I uploaded just to try out the system Thank you so much I will try rebuilding my pools as suggested. A big thanks to you for your knowledge on this issue much appreciated

 

[RobGoss]

I'm assuming I would have to delete the current pools and add the new one's correct?

 

[sretalla]

I would have to delete the current pools and add the new one's correct?

The official ZFS term is destroy (in case delete didn't sound like enough of a warning).

That way, all of your disks are freed (if you select the option to quick wipe them) to be part of another pool (which you can then create).

 

[RobGoss]

The official ZFS term is destroy (in case delete didn't sound like enough of a warning).

That way, all of your disks are freed (if you select the option to quick wipe them) to be part of another pool (which you can then create).

So instead of deleting them are you saying I'll have a option to quick wipe them?

 

[sretalla]

You get to destroying a pool via the Export/Disconnect option from the cogwheel.

If you select the option to destroy the data, the disks will be wiped.

I'm sure there was an option in a previous version to select if that wipe would be a full erase/overwrite or a quick one, but seems no more.

 

[RobGoss]

You get to destroying a pool via the Export/Disconnect option from the cogwheel.

View attachment 58067

If you select the option to destroy the data, the disks will be wiped.

I'm sure there was an option in a previous version to select if that wipe would be a full erase/overwrite or a quick one, but seems no more.

Thank you much for your help its much appreciated I will look into it

 

[RobGoss]

In order to use all of your drives' capacity, you would need to run with no redundancy (selecting the Stripe VDEV type when creating the pool... which you would need to re-do to put that change into place).

Be aware that running without redundancy means you will lose access to all of the data on all of the disks (the entire pool) immediately upon losing (through failure or other removal) any one of your 5 disks.

You also won't have the possibility to recover a file that ZFS would detect as corrupt (due to the checksums it keeps for all blocks) even if you can detect that corruption, the disks won't contain any alternate data to rebuild the corrupt data.

If you're fine with that and you just want a bunch of storage that doesn't matter if it's lost and can be re-built from scratch, then great, rebuild your pool and go ahead.

When I tried using the Stripe VDEV type
this is the message I am getting see the attached image below

 

[sretalla]

That's exactly the stuff I was mentioning above.

If you're OK with the risks like you seem to have been, tick the box and continue.

 

[RobGoss]

That's exactly the stuff I was mentioning above.

If you're OK with the risks like you seem to have been, tick the box and continue.

LOL I understand is this s good thing to do?

 

[RobGoss]

LOL I understand is this s good thing to do?

That's exactly the stuff I was mentioning above.

If you're OK with the risks like you seem to have been, tick the box and continue.

This server will only be accessed by me it's just to store my YouTube content on it. I would like to to be safe if anything should happy on the server. What's is your advice?

 

[sretalla]

I would like to to be safe if anything should happy on the server. What's is your advice?

Do you mean that you want to be able to lose one or two disks without the pool being lost?

In that case you need to give one or two disks to redundancy (so choose RAIDZ1 or RAIDZ2 for your VDEV type).

Are you saying that your content is stored first on YouTube and then copied to your NAS? or is the "primary location" on the NAS (which would suggest maybe some redundancy and even a backup strategy should be considered).

Think about things like "How long could I continue to work if I didn't have access to the files on my NAS?" or "is my NAS the only place that some files are stored in the original quality?".

Then think about what would happen if you lost access to the files on your NAS due to a disk failure... is it worth your time unable to work to pay for an additional disk or 2? Would you be able to get the original files back somehow? (or do you need another kind of backup to ensure you would?)

 

[RobGoss]

Do you mean that you want to be able to lose one or two disks without the pool being lost?

In that case you need to give one or two disks to redundancy (so choose RAIDZ1 or RAIDZ2 for your VDEV type).

Are you saying that your content is stored first on YouTube and then copied to your NAS? or is the "primary location" on the NAS (which would suggest maybe some redundancy and even a backup strategy should be considered).

Think about things like "How long could I continue to work if I didn't have access to the files on my NAS?" or "is my NAS the only place that some files are stored in the original quality?".

Then think about what would happen if you lost access to the files on your NAS due to a disk failure... is it worth your time unable to work to pay for an additional disk or 2? Would you be able to get the original files back somehow? (or do you need another kind of backup to ensure you would?)

Yes my content is stored on YouTube and on my PC. I just wanted to have some were else to store my work as well. is redundancy something that allows me to retrieve my data if one of my disk fails? I'm new to all this server stuff so forgive me for all the questions

Question? if something were to happen to my server / Nas would I be able to access that data on the hard drives I have on my home server?

 

[sretalla]

is redundancy something that allows me to retrieve my data if one of my disk fails?

Yes (simple answer).

To explain fully, a ZFS pool uses VDEVs (at least one) to provide the storage. Since the VDEV(s) are joined together in a stripe, if any one VDEV is lost, the pool won't have the required information to continue operating and all content will be unavailable.

So what's recommended in most cases is to use redundancy within a VDEV to ensure that a single disk loss (which is one of the more common things to happen to a spinning HDD, although relatively rare during the first 4 years or so of a disk's life) doesn't render the pool unavailable.

Redundancy can take the form of Mirrors (1 disk for data and additional disk(s) for redundancy), RAIDZ1 (3 or more disks where 1 disk worth of capacity is given to storing "parity" - sufficient information to rebuild the block based on all remaining disks if one is missing) RAIDZ2 (4 or more disks where 2 disks capacity are given to parity) or RAIDZ3... 5 or more disks, 3 to parity.

In each of those cases, losing 1 disk will result in the pool continuing to operate with full access to read and write files as normal (although the pool will show the administrator that it's in "degraded condition" due to the missing disk).

Obviously there are positives and negatives to each type of redundancy, which can be weighed in terms of cost of the parity versus performance and data security...

Mirrors are costly in that you need to buy 2 disks to have 1 disk worth of capacity. But for that, you get good performance (particularly in terms of write IOPS). You run the risk of a failure of the remaining disk or data corruption when one fails until you are able to add a replacement disk, but that risk is reduced due to the relatively low load of "resilvering" (copying/reconstructing back the content to the replacement disk) as you're just copying blocks 1-1.

RAIDZ1 costs you 1 disk out of your VDEV in terms of capacity, so it's not too bad if you have 5 disks (you still get the capacity of 4). Throughput for writes can be probematic for large amounts of smaller files and for things like Virtual machine disks or iSCSI as the IOPS of a RAIDZ1 VDEV is about the same as that of only 1 of the member disks (about 1-300 IOPS for most HDDs). With a failed disk, you can continue operating, but the resilvering process is hard on the remaining disks, increasing the likelihood of a second failure and/or corruption of data before you can get back to normal with the replacement. For that reason, it's not encouraged to use RAIDZ1 with disks larger than 1TB (you could stretch it to 2TB).

RAIDZ2 is more of the same as RAIDZ1, but with the difference that with one disk lost, you are still protected against corruption and even against a further disk failure during the resilver process. Highly recommended if data integrity and security is a priority.

RAIDZ3 is even more paranoid, allowing for the pool to operate with 3 disks missing and providing protection even in the case of 2 missing.

Generally, if a pool is to have more than one VDEV, you would try to match the sizes and VDEV types in order to ensure you're not taking risks for the whole pool that you hadn't decided to take for one of the VDEVs.

Anyway, that's probably information overload for you at this point, so I'll suggest that you look over in the Resources section in the Fundamentals category for more reading material if you're so inclined.

 

[sretalla]

Question? if something were to happen to my server / Nas would I be able to access that data on the hard drives I have on my home server?

I'm not sure that question is very clear... is your NAS also your home server? Is your home server running a unix or linux based Operating System?

To put it simply...

You can use the disks in a ZFS pool in another system that supports ZFS and you will be able to import and use your pool data. (a few caveats around the version of ZFS that you're using, but if you're using recent versions of OpenZFS, no problem generally)