How To: Boot, Replicate, Scub, SMART, Shutdown?

[Electr0]

Hi All,

I have two systems running TrueNAS Core: TrueNAS (Main) and TrueNAS (Backup).

My Main system is up 24/7 however I just want to have my Backup system online for a few days a month.

I would like to do the following:

1. Remotely power on the Backup system
2. Replicate a Snapshot from my Main system
3. Scrub the pool on the Backup system (only if the last scrub was >35 days ago)
4. Perform SMART tests on all disks
5. Shutdown the Backup system

My plan was to wrap the commands below (2- 5) in a Bash script, and set it to run on Boot.
Each command would be on a new line, so successive commands should only complete once the previous command has finished. Correct?

1. Power On
I can do this by using the following command from my Main system (they're on the same network):
ipmitool -I lanplus -H <IP-Address> -U <Username> -P <Password> power on

2. Replicate
I already have a replication task setup with an SSH connection to the Main system, so I belive I can just run it by using:
midclt call replication.run 1

3. Scrub
Normally I could do this by running: zpool scrub
However, that would scrub the pool everytime the script was run. Is there a way via the CLI to check when the last scrub was run?
Then I could just use an IF command...
It seems like you can do this in TrueNAS SCALE through this CLI command scrub query, but that command doesn't exist in CORE.
I checked the CORE API and it doesn't seem possible through that either.

4. SMART Tests
I believe I can run these for all disks in parallel by using the smartctl command with the appropriate options.

5. Shutdown
shutdown -h now

Am I missing anything?

 

[artlessknave]

in most cases this will kill your drives. you are likely to loose money doing this due to having to replace the drives more often. monthly might not be too bad, dunno, but you would have to save enough on power to pay for any drive it kills.
that said, here are some thoughts:

1. Power On
I can do this by using the following command from my Main system (they're on the same network):
ipmitool -I lanplus -H <IP-Address> -U <Username> -P <Password> power on

I believe you can put the password in a file, then make the file only readable by root, or whatever user your cron job uses.
putting passwords in the CLI anywhere is not a good idea.

2. Replicate
I already have a replication task setup with an SSH connection to the Main system, so I belive I can just run it by using:
midclt call replication.run 1

do a pull replication from the backup system. obviously this will only occur when its online.

3. Scrub
Normally I could do this by running: zpool scrub
However, that would scrub the pool everytime the script was run. Is there a way via the CLI to check when the last scrub was run?
Then I could just use an IF command...

just schedule a scrub to run every day week, and set it to only run if the last scrub was more than x days. it will run automatically. as you will be stressing your harddrives, you will want to run this more often than usually (35 days is the default)

4. SMART Tests
I believe I can run these for all disks in parallel by using the smartctl command with the appropriate options.

simil.ar to the scrubs, just schedule it to run basically every day since you'll probably be killing your drives anyway...

. Shutdown
shutdown -h now

poweroff works irrc, no realy difference.

 

[thomas-hn]

in most cases this will kill your drives.

I understand where this statement comes from and why it is beneficial to let the drives run continuously.
However, a usual desktop system with harddrives is used - maybe - once a day and the drives last for five years or more. If a TrueNAS backup system is only powered once a week, it has less power cycles than a usual desktop system. So I would expect that it last at least the same time, if not even longer.
Even that the power cycles stress the drives, this will only happen once a week and not on a daily or hourly base.

So, is it really as worse as stated in the citation above?

 

[artlessknave]

monthly might not be too bad, dunno, but you would have to save enough on power to pay for any drive it kills.

unsure. most of what I have seen is weekly. the workload on a desktop is very different. for one thing, zfs expects drives to online and readable pretty constantly for scrubs and health checks and running services. it simply isnt designed for a desktop, so expecting it to behave like a desktop filesystem doesnt make sense; its designed for enterprise 24/7 with backups and spares running mission critical databases.

you would be volunteering yourself to test it, which is why i added the part about one of the things you would have to calculate out; power savings vs drive replacements. maybe you get lucky and have immortal drives, but you could also get unlucky and have drives die on every startup. the biggest problem with this is if enough drives die to make the pool offline.

 

[Etorix]

Spinning the drives up and then down once a week should not put an excessive stress.
The issue is to time the shutdown when replication, (long) SMART tests and scrub are done, given that you don't want to have everything occuring at the same time and only SMART tests have a defined;, known, duration.

 

[Electr0]

... and set it to only run if the last scrub was more than x days.

Thanks, but like I asked... How can I find out from the system (using the CLI) when the last scrub was run?

Everything else seems pretty straight forward.

 

[Electr0]

Spinning the drives up and then down once a week should not put an excessive stress.
The issue is to time the shutdown when replication, (long) SMART tests and scrub are done, given that you don't want to have everything occuring at the same time and only SMART tests have a defined;, known, duration.

True.

My plan was to boot the server once or twice a month.

And like I said in my original post, I'll just make all the task run successively. If it takes 24h or so to run everything once a month, that's fine in my use case.

 

[chuck32]

Thanks, but like I asked... How can I find out from the system (using the CLI) when the last scrub was run?

GUI under Storage, it even displays the last duration.
I know you asked for CLI, don't you have GUI access?

Maybe give this a read I started investigating the exact same issue but in the end decided to just let the second machine run 24/7 and save myself the headache.

 

[jlpellet]

Not sure if this will work for you but in the cli zpool status "poolname" returns the results of the last scrub & adding -x will just display any pools with issue(s). Hope this helps.
John

 

[Electr0]

GUI under Storage, it even displays the last duration.
I know you asked for CLI, don't you have GUI access?

Yes, I have GUI access, but as per my original post, I'm trying to wrap all these commands in a Bash script, and getting Bash to read a GUI is no easy task.