How to restore boot pool from backup after uncorrectable I/O failure ?

[EPU]

Looking for some advice please.

Due to current pandemic, I haven't had physical access to my TrueNAS server for a number of months. Note that before this, I made a backup which is saved to another machine. Unfortunately whilst away, I received 2 email notifications in relatively quick succession to indicate that both the USB boot drives had failed.

The console is currently showing Solaris: WARNING: Pool 'freenas-boot' has encountered an uncorrectable I/O failure and has been suspended, and I've no access to SSH or the GUI/UI. The server still appears to be running however, as all Jails are operational.

I now have physical access to the server again so need to get the backup restored. My understanding is I need to do the following in order to get the server fully back online and operational.

• Download same ISO as I'm currently running (12.0-U2.1)
• Use the ISO to install TrueNAS to a new USB drive
• Use that USB to boot to the GUI on the TrueNAS server (with the data on it)
• Use the GUI to restore the backup to the working USB drive
• Reboot, and (assuming no other issues) the system should be back working

If that is the case, I've a couple of questions on this.

1. Given no access to SSH or GUI, are my only shutdown options press the power button (and hope it does a graceful shutdown), or pull the plug ? ie, without SSH or the GUI, there's no other way to shut the server down ?
2. Given the pool was mirrored, do I have to setup a mirrored config before restoring the backup ?
3. Do I have to perform the install (and restore) using the TrueNAS box, or can this be done on any machine ? ie, will the above process of installing a new server work on any machine, and is then transferable to a new box, or does it have to be done on the target server ?
4. Does the USB drive have to be same make/model/capacity as the existing, or would any variant work ?
5. Finally, I've recently purchased a couple of SuperMicro 16Gb SATA DOM's. Can I install them and install/restore the backup to those, or do I need to build using the USB first, then move the boot pool to that ? (ie, is TrueNAS expecting a mirrored USB boot pool, or would a single SATA DOM suffice ?)

Sorry for the myriad of questions, but obviously I want to make sure I get the process right so I don't torch the current dataset by mistake.

 

[Arwen]

First off, do you have a copy / backup of your TrueNAS Core configuration?
You say backup of your boot pool, but is that the configuration or the boot pool itself?

You don't restore your boot pool. You re-install to good media and restore the configuration backup. It will restore your TrueNAS to exactly where it was before, pool configuration for shares and all. (The ZFS pool configuration is stored on the data disks.) The make, model or size of the boot media is irrelevant, (other than being good media and >=16GBs.)

Their are tools out their that can use cron to automatically store a copy of your configuration on a data pool. So that is where one configuration is stored for me.

I think there is a copy in the system info, which generally goes to the data pool as well. But, I've not had to research / use that method.


Using mirrored USB flash drives is better than one USB flash drive. But, as the last few years have shown, most USB flash drives are of questionable quality. So if you have a spare SATA / SAS port, using a cheap SATA SSD, like your SATA DOM, (or even hard drive), is generally a better option.


Last, if you use encryption, you need the information to decrypt your data pool. Lots of people have created self destroying data pools by not properly backing up the encryption keys and or passphrase(s).

 

[EPU]

Yes, I do have a backup of the config (sorry, I meant to say that, not boot pool).

My real concern is the process, and ensuring that the existing data isn't wiped. A lot of the backup/restore process guides assume you haven't lost access to the GUI (or SSH). If they were accessible, I'd be a lot less concerned, as I know I can use the built-in tools, or even copy a newer version of the config.

Have I got the restore process correct, and do I have to do this on the server itself, or can I build a basic install (then restore the config) on any machine ?

 

[Redcoat]

My real concern is the process, and ensuring that the existing data isn't wiped.

One way to ensure that is to disconnect the data drives before working on the install, do the install, reconnect them and import the pool(s).