TrueNAS Scale 22.02.4 boot disk dead and more related

[Kingo]

Let me start with saying what a great software/system, Thank you, loving it.

I have put together, some 8 months ago, a machine with a 2GB boot/OS disk and 6pc 4TB disks with RAID6. It has been running smoothly, as far as I know and feel. I am not a specialist in this, but feels like a good system. I used to have an old Iomega StoreCenter ix4 running RAID5 for many years and I loved that little box. Building my own system I decited to go for RAID6 since I had lots of disks laying around, ..but I forgot to mirror the boot disk

A few days ago I noticed that I lost directory connection to my TrueNAS / running Plex server. Also the server got rather noisy, some fan went full speed...
The Plex server seemed to be still functional, at least I could still watch content via Plex app. Server IP would also respond to ping, but WebUI would not responde. So I decided to connect a screen and keyboard to it. Now my mistake I connected the screen to the wrong VGA port and got no picture... not realizing this mistake, I turned off the server to restart it again... And now it's gone ! it will not boot. I took the server onto a table to open it up, to check for possible motherboard failure or broken powersupply. I notice the other VGA port, connected to it, powered on and got picture, got into bios. Bios showed disk 0 is missing, rest of the disks are ok. Not to panic, wait a little, try to turn on again, but disk will still not start, so I take it out, pack it in antistatic bag and I am keeping it in the freezer, in faint hope it might possibly start once again... I will wait until I get all info I need before trying to restoring the system. I am holding on to the hope.

I do have a bootable USB with TrueNAS Scale 22.02.4 at hand. I originally installed from this USB.

Now I have a few questions;
1. If I am lucky and the dead (now frozen) disk starts again, What is the best way to quickly get a good config backup of the system, and how do I go about that ?, and save the backup where ? USB, pool or ???
2. How do I get/reconnect the pool again ? I have read that the pool/data should be ok and can be reconnected/imported... ?
I figure I can just install a new HDD, I will put a purple disk this time, and boot from USB and install TrueNAS Scale again, and restore from backup, if lucky.
3. Would that be the right way to go ? What is the best/safest way to do this ? Am I thinking this right ?

Next I need to setup some kind of config backup, preferably automatic / per eMail ? I guess I could find some threads for that.

Any help will be much appreciated.

Many thanks in advance.

 

[sretalla]

need to setup some kind of config backup, preferably automatic / per eMail ? I guess I could find some threads for that.

This script will give you a mail as often as you set it with a summary of your pools and SMART status for all disks and will attach the config.

Multi_report.sh version for Core and Scale

This resource was originally created by user: @joeschmuck on the TrueNAS Community Forums Archive. https://www.truenas.com/community/resources/multi_report-sh-version-for-core-and-scale.179/download Multi-Report Tool to chart and record statistical data on your drives. This will email the...

www.truenas.com

 

[sretalla]

If I am lucky and the dead (now frozen) disk starts again, What is the best way to quickly get a good config backup of the system, and how do I go about that ?, and save the backup where ? USB, pool or ???

You download it to your computer (Windows/Mac/linux, whatever you use).

How do I get/reconnect the pool again ? I have read that the pool/data should be ok and can be reconnected/imported... ?

If you restore your config to afresh install, your pools will already import based on that config where they were there... if not, import the pool(s).

Would that be the right way to go ? What is the best/safest way to do this ? Am I thinking this right ?

Unless you do something silly and actually write data to your pool disks with some software that isn't ZFS, you can't really "break it".

Just get a running system and either import your config or import the pool.

 

[melloa]

I will wait until I get all info I need before trying to restoring the system. I am holding on to the hope.

Bit picture here is that he needs to get his system up again, as I didn´t see anything on the OP saying its running. So please @Kingo provide some info about your system so the folks here can better help you.

 

[NugentS]

My 2p worth.

First - please, as @melloa says post complete hardware specs

Second - 2GB for boot is very light. 8GB is the recomended minimum. But I would suggest 16GB at least (I am using 9 atm with 3 environments). You can buy 16GB Optane M.2 drives dirt cheap which work really well in an adapter

Third - I imagine that the 2GB Boot disk was a USB Thumb Drive or eUSB type device. These are NOT reccomended for (probably) exactly the reason that you have come across. Either use a SATA Port (a waste) or a proper SSD in a USB to SATA or USB to NVMe adapter.

Fourth - Mirroring the boot is less useful than you might think. All you actually need is a copy of the config and a rebuild and setup takes 10 minutes (probably longer to find, download and burn the ISO, along with finding a suitable stick to use)

Fifth - Your data is almost certainly good. The system dataset contains a copy of the config - unless you moved it to the boot-pool (I doubt it) then it will be on the first pool that you created AFTER building TN. Its in somewhere like /var/db/system/...... but someone else will have to help with more details. This can be imported and all should come back to life. Get this file by:
1. Replace Boot disk with something more appropriate
2. Rebuild from ISO - be careful to use the new Boot Disk. The new system dataset is on the boot disk - not the existing ZFS Data Pool.
3. Import the Data Pool
4. Find and import config
5. Bob's your uncle

Then implement @joeschmuck's multi-report script - which will email you regular copies of the config along with lots of other details about disk health.

Lastly make sure you schedule:
1. regular scrubs
2. regular short smart tests
3. regular long smart tests




[Pedant Mode]: Its not RAID6 - its RAIDZ2 in ZFS Land. They are not (quite) the same albeit they achieve the same objective on the surface.

 

[Kingo]

Thank you guys for all the answers/info.

Upsy.... a tiny little typo... My system had a 2TB HDD as boot, not 2GB my bad...

@sretalla I'm trying not to do silly things that's why I'm posting questions. Like your suggestions.

@melloa , yes you are right, my system is down at the moment. I am collecting info so I won't mess things up when I rebuild.

@NuggetS my boot was a 2TB HDD, Seagate if I remember right (sitting at work now, not in front of system). Also, are you saying that the config might be sitting on the pool ? In witch case I can just reinstall TN (same version) and pick the old config from the pool after importing it ? That would just be great.

Always learning ...or remembering old things

 

[NugentS]

There is a copy of your config on the system dataset which, if you didn't move it, is on the first pool that you created after installing TrueNAS. So install on new boot disk, find the config on the imported pool and load that and you SHOULD be good to go

 

[sretalla]

find the config on the imported pool and load that and you SHOULD be good to go

Not quite as easily done as said.

I was just looking at this last week, so have the post in mind... look directly at the linked post and the couple after it for the full instructions.

How to recover and upload a config file from an imported pool?

Hi Folks, First off if this has been asked and answered I apologize, I've been looking at the threads for the last hour and can't seem to find the answer. The short version of the story is that the SSD containing my TruenNAS OS bit the dust, I have gotten a fresh SSD, re-installed TrueNAS...

www.truenas.com

 

[NugentS]

Yeah I did gloss over the difficulties with "find the config on the imported pool"

 

[Kingo]

Status update:

I installed another 1TB HDD (Purple this time) as boot, using the USB I had originally used. I managed to import the pool without problems. Next I checked Apps, and it rolled for a few seconds and puff, my Plex Server was Active again
Actually I didn't see any other available apps, but that seems to be all back now...

I tried to find and locate the config backups using Shell. I had no problems finding all the files, one for each day since install, ...but I could not by any means get access to (mount and use) external usb I was planing to copy to so I could restore the setup. After quite a few hours and nights reading and trying, I decided to skip this since I only had a small setup. I have created the 2 users I had again and SMB shares, and connected again from my laptop and pc.

I would say this case is now SOLVED Thank all you guys for the help and directions

My next mission will be to set up automatic backup and tests with eMail. Over'n'Out for now.

 

[NugentS]

see @joeschmuck's multi-report script.