SOLVED TrueNAS destroyed all my data? What happened to "data safety"? :thumbs down:

[winnielinnie]

I ran some recursive "rm" commands from an SSH session as the root user to delete entire directories in a handful of my datasets. I wanted to see just how reliable (and practical) these so-called "snapshots" are. (Apparently ZFS prides itself on data integrity and safety.)

I forgot to take snapshots of these datasets before doing my tests, and now everything is gone. No files, no folders. All my archived documents and sentimental photos and videos. Gone. Just like that? Really, TrueNAS?

On Windows you can buy software (or try "freeware") that is user-friendly, which can recover accidentally deleted files. I can't find anything like that in TrueNAS or even ZFS. Did the developers even think this through before releasing this to the public?

Why would TrueNAS allow the admin to login as root, and run the "rm" commands, if it doesn't ship with a file-deletion recovery tool?

Wow.

At least this command works (for now): date "+%B%e"st

 

[Kris Moore]

 

[awasb]

1. “Unix gives you just enough rope to hang yourself – and then a couple of more feet, just to be sure.” ~ Eric Allman
2. “Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.” ~ Douglas Gwyn

 

[HoneyBadger]

My TrueNAS just deleted all the files by itself, I didn't do anything. Just woke up and all my files were gone.

I'm using an ISP-provided modem, and I port-forwarded everything to it so that I can access my files remotely. Don't worry though, I have Windows Defender so it can't be a security issue, and my password was a nice strong one with a capital, lowercase, number, and a symbol. And it's ten characters long!

Password1!

check the date of posting

 

[NugentS]

You could at least have made that highly secure with Pa55w0rd1!

[shudder]

 

[HoneyBadger]

You could at least have made that highly secure with Pa55w0rd1!

[shudder]

I've always gone with old faithful hunter2

Also I'm going to shout out @jgreco because he'll get the reference to that, as well as the one in my previous post. The fond memory will probably make the Grinch's heart grow at least half a size.

 

[jgreco]

Why would TrueNAS allow the admin to login as root, and run the "rm" commands, if it doesn't ship with a file-deletion recovery tool?

I recommend you use the emergency recovery mode. Now, this is dangerous magic, and we don't normally recommend this, but you seem pretty clueful.

Shut down your filer. Remove each disk tray, one at a time. Now unscrew the drive, rotate it 180 degrees, and screw it back into the tray. This engages emergency hardware undo mode. Now reinsert the drive. Note that you will encounter some resistance as you reinsert the drive. Most drives have never been put into hardware undo mode, so the insertion may be stiff and you might even hear cracking or snapping noises. This is A-OK, it just means good things ahead. Once you have done this to all drives, turn on the filer again and it will begin restoring your data. Once you are happy with what you see, it is important to disable hardware undo mode. Be sure to shut down and repeat the process in order to place the filer back in normal operating mode.

Good luck, enjoy.

Grinchy

 

[Etorix]

@winnielinnie Nice impersonation!

Allow me to take this opportunity to congratulate @JoshDW19 and his young and talented consultant for the new TrueNAS logo (horny penguin…).

 

[TheUsD]

Did the developers even think this through before releasing this to the public?

Why would TrueNAS allow the admin to login as root, and run the "rm" commands, if it doesn't ship with a file-deletion recovery tool?

TrueNAS is working on proof this. However, they are short-handed with individuals like yourself and I'm sure they could use someone of your skill level. Continue to keep finding new ways to destroy data without backup. Report back here with ay updates, please.

 

[morganL]

I ran some recursive "rm" commands from an SSH session as the root user to delete entire directories in a handful of my datasets. I wanted to see just how reliable (and practical) these so-called "snapshots" are. (Apparently ZFS prides itself on data integrity and safety.)

I forgot to take snapshots of these datasets before doing my tests, and now everything is gone. No files, no folders. All my archived documents and sentimental photos and videos. Gone. Just like that? Really, TrueNAS?

On Windows you can buy software (or try "freeware") that is user-friendly, which can recover accidentally deleted files. I can't find anything like that in TrueNAS or even ZFS. Did the developers even think this through before releasing this to the public?

Why would TrueNAS allow the admin to login as root, and run the "rm" commands, if it doesn't ship with a file-deletion recovery tool?

Wow.

At least this command works (for now): date "+%B%e"st

Its Ok you can use Apple Time machine to go back to March 31st...

 

[winnielinnie]

That's one of the best versions of this meme that I've come across.

 

[Etorix]

Its Ok you can use Apple Time machine to go back to March 31st...

…but don't forget to overclock the CPU to 88 mph! Time Machine needs that.

 

[sretalla]

I can't find anything like that in TrueNAS or even ZFS

Interestingly (and being a bit more serious now it's April 2nd), I noticed a while ago that testdisk mentions it can find lost ZFS partitions... https://www.cgsecurity.org/wiki/TestDisk

There is, of course, Klennet and also this one which may be cheaper (= free) and I think can run directly (or in a jail) on TrueNAS (not on Winows like Klennet): https://github.com/Stefan311/ZfsSpy.

 

[Arwen]

Wait til you have application support with full root do something silly like;

chown -R app:appgrp /usr/lib​

on a Solaris 11 machine. TRUE STORY.

The only saving grace was that the server was patched 2 weeks before, so I had an alternate boot environment to boot off.

 

[Etorix]

There is, of course, Klennet and also this one which may be cheaper (= free) and I think can run directly (or in a jail) on TrueNAS (not on Winows like Klennet): https://github.com/Stefan311/ZfsSpy.

Hum…

WARNING​

- This project is abandoned. It will not be developed any further.
- It is very likely that it will not work with pools from newer ZFS versions.

and also

The ZfsSpy is in an early stage of development, so many important features are still missing:

• All Compressions except LZ4
• All Checksumms except Fletcher4
• Display of file/directory attributes
• Complex pool geometries like RAIDZ or mixed STRIPE+MIRROR
• usefull help pages on every step
• more fault tolerance
• encryption support
• gangblock support

 

[sretalla]

I never said it was perfect.

 

[kherr]

Restore from your Backups .,,........

Raid/ZFS or any other mechanism is NOT 100% foolproof/fault tolerant. The more individual backups (physical location other than your server) you have ... the better one sleeps .....

Snapshots stored on the server ..... while convenient to restore data on the server it's self .......is not considered a true backup as it too can be corrupted or die with the box if destroyed or catastrophic failure.

Don't ask me how I know ...... been there done that .......

 

[morganL]

This issue seems to be solved with the following advice:

"never run these types of tests before noon on April 1st"'