potential data loss -- how to know?

[Avion]

Hey there,

I am running TRUENAS Core on a 60TB stack of drives.
I have three users and root access. When I have setup the main storage folder I just dumped all Footage from Years of Film Agency work into it.
After one year I realised I should have setup proper (wiser) user management and permissions and structured the pools etc. That will have to be done when I re-work it all.
BUT Now I am looking into the main storage folder and lots of sub-folders are empty, missing or are way to smal.

Could it be that I lost access to a large chunk of data due to user access? (i think i uploaded parts under the root user and parts ander the later installed normal user)
How would I know? Is there a way to access ALL DATA?
Like ignore or overwrite all user access data and have root access to it all?
I feels like a huge part is missing or hiding from me.

How would you trouble shoot something like this?

Thanks a lot for any help.

PS: I live in constant fear of data loss as this data is my whole asset, my treasure, my personal and company capital so to say.

 

[joeschmuck]

As far as I'm aware, if a user deleted the data, the data is gone UNLESS you are using snapshots from which you could restore the data.

 

[Ericloewe]

PS: I live in constant fear of data loss as this data is my whole asset, my treasure, my personal and company capital so to say.

In that case, you need a lot more in terms of backups, but that's an orthogonal discussion.

BUT Now I am looking into the main storage folder and lots of sub-folders are empty, missing or are way to smal.

Let's start with the very basics: how much storage is actually being used? What's the output of zfs list -o space?

 

[Avion]

Hey there, so the main data block should be "MainData and a large chunk should be in RAW footage, Kasimir and Projects." it should be rather close to 10 TB of used space.

I did not intentionally delete anything of course. Is there a log file that goes back to the initial setup? --> Where I could look and see if something got deleted?

Code:

root@truenas[~]# zfs list -o space
NAME                                                       AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
MainData                                                   22.6T  4.16T        0B    185K             0B      4.16T
MainData/.system                                           22.6T  45.8M        0B    213K             0B      45.5M
MainData/.system/configs-cadb1ce96f8c4a01a65be3ef8f5cb996  22.6T  1.30M        0B   1.30M             0B         0B
MainData/.system/cores                                     1024M   170K        0B    170K             0B         0B
MainData/.system/rrd-cadb1ce96f8c4a01a65be3ef8f5cb996      22.6T  20.5M        0B   20.5M             0B         0B
MainData/.system/samba4                                    22.6T  20.1M     19.5M    689K             0B         0B
MainData/.system/services                                  22.6T   170K        0B    170K             0B         0B
MainData/.system/syslog-cadb1ce96f8c4a01a65be3ef8f5cb996   22.6T  3.14M        0B   3.14M             0B         0B
MainData/.system/webui                                     22.6T   170K        0B    170K             0B         0B
MainData/iocage                                            22.6T  4.72G        0B   10.0M             0B      4.71G
MainData/iocage/download                                   22.6T   878M        0B    170K             0B       877M
MainData/iocage/download/13.1-RELEASE                      22.6T   434M        0B    434M             0B         0B
MainData/iocage/download/13.2-RELEASE                      22.6T   443M        0B    443M             0B         0B
MainData/iocage/images                                     22.6T   170K        0B    170K             0B         0B
MainData/iocage/jails                                      22.6T   170K        0B    170K             0B         0B
MainData/iocage/log                                        22.6T   170K        0B    170K             0B         0B
MainData/iocage/releases                                   22.6T  3.85G        0B    170K             0B      3.85G
MainData/iocage/releases/13.1-RELEASE                      22.6T  1.88G        0B    170K             0B      1.88G
MainData/iocage/releases/13.1-RELEASE/root                 22.6T  1.88G        0B   1.88G             0B         0B
MainData/iocage/releases/13.2-RELEASE                      22.6T  1.97G        0B    170K             0B      1.97G
MainData/iocage/releases/13.2-RELEASE/root                 22.6T  1.97G        0B   1.97G             0B         0B
MainData/iocage/templates                                  22.6T   170K        0B    170K             0B         0B
MainData/mainStorage                                       22.6T  4.15T        0B   1.25T             0B      2.90T
MainData/mainStorage/RAW footage                           22.6T  1.33T        0B   1.33T             0B         0B
MainData/mainStorage/documents                             22.6T  1.42G        0B   1.42G             0B         0B
MainData/mainStorage/kasimir                               22.6T   170K        0B    170K             0B         0B
MainData/mainStorage/projects                              22.6T  1.13T        0B   1.13T             0B         0B
MainData/mainStorage/sophie                                22.6T   170K        0B    170K             0B         0B
boot-pool                                                   443G  2.59G        0B     24K             0B      2.59G
boot-pool/ROOT                                              443G  2.58G        0B     24K             0B      2.58G
boot-pool/ROOT/13.0-U6.1                                    443G  2.58G     1.29

 

[Ericloewe]

10 TB of used space.

So, some 6 TB missing.

Is there a log file that goes back to the initial setup? --> Where I could look and see if something got deleted?

Simple answer is no, for a variety of reasons.

 

[joeschmuck]

That is very unfortunate. I didn't know if there was a log of file deletions and user name/date/time or not. Some sort of audit history.

 

[joeschmuck]

It appears that SCALE does offer Auditing but CORE does not. That does not help now. Look into snapshots to prevent future issues like this.