Auto shutdown and powerup for backup node

[JimKusz]

Hello:

I have a customer who has two FreeNAS systems in a replication system currently. Today, they have space in two different buildings across the street and down a block, linked by private microwave, and this system has worked well. They have over 50TB of data on the system currently, and of course, this data is the life of the business; loosing it will end the business (no recovery possible). We have snapshots with replication enabled (no limits on when replication runs).

They're getting ready to move to a new space that is nearly next door to their primary space. Good news is it will be connected by 10GB fiber; bad news is its just a few brick walls and other businesses away (thinking fire risk here...There is no "air gap" between the new building and their current main building). Hence, too much risk. Plus, they'd like to have a 3rd copy just for safety and to help the owners sleep better at night. So, I'm proposing building a 3rd identical FreeNAS system, and placing it in a data center a city away. Unfortunately, this is a small community, so such arrangements come with good and bad parts: Good, its inexpensive, bad, it comes with stipulations; replication can only run during the night so their bandwidth is available during the day for their other uses (its an ISP). They also are hesitant about the 24/7 power consumption.

I got to thinking, since the node is only going to be actively doing stuff for an hour or two a day, can I have it shut down during the rest of the day and only boot up when the day's backup is about to begin? I have done similar arrangements in the past with different system types (MythTV is the most prominent example). The hardware should support it; in theory, set the RTC to wake every day at 1am, for example, and have the OS shut down when the replication is compete. This would ease the hosting provider's concerns about bandwidth consumption as well.

I realize FreeNAS is normally a 24/7 online tool, but it seems to make sense in this case to consider a power down option. It also offers additional protection against hacking attacks and the like (nobody can hack it if its powered off). Is there a way to do this on FreeNAS currently? How would I go about setting it up (I assume the power-on is the BIOS's PM RTC wakeup option; not sure how best to do the shutdown).

Thanks!
--Jim

 

[kdragon75]

They also are hesitant about the 24/7 power consumption.

If your client is paying for the space in the DC, tell the ISP to build it into the contract price (they have one right?).

I got to thinking, since the node is only going to be actively doing stuff for an hour or two a day

No, it should have long smart tests and ZFS scrubs running. This would take several hours on 50TB of data. Unless you don't care to check the integrity of the data.
Your talking about a lot of work for something I wouldn't trust to save $50 (on the high end) in month in power. If you client is willing to pay for a new 50TB server, they can afford a few dollars in power...

 

[danb35]

How would I go about setting it up (I assume the power-on is the BIOS's PM RTC wakeup option; not sure how best to do the shutdown).

Leaving aside the question of whether this is a good idea (and I tend to agree with @kdragon75 there), a cron job to run poweroff at the desired time would take care of the shutdown issue.

 

[kdragon75]

Leaving aside the question of whether this is a good idea (and I tend to agree with @kdragon75 there), a cron job to run poweroff at the desired time would take care of the shutdown issue.

You should test if the replication process is still running before shutdown. I dont know if a pid file is created or if you would have to ps -aux | grep to find it but I don't think you want to shut down before the replication is done. Do you have a monitoring plan in place for this system? I'm sure you would want to know if replication is working as expected. Also if you do this shutdown thing and the system does not shut down, how will the ISP react? Will they pull the cord? Just hard shutdown? Will they be willing to power it on at request if it does not boot? Will you have IPMI access to the box? Physical access? I just want to make sure you cover this on all sides.

 

[JimKusz]

I was looking for if there was a way to shut down only if there are no running jobs, eg., the replication has finished and there are no scrubs or smart tests. Something like, "if after 4am and no jobs are running, shut down".

--Jim

 

[Elliot Dierksen]

I am going to throw another voice in on the 'leave it running' side. In 30+ years in I.T., I can tell you with absolute certainly that there is no guarantee that something will come back to life after having been powered down. If something fails while it is running, there is at least a chance that it will try to let you know (assuming you have alarms/logging properly configured) before it dies. If something fails when trying to come back on, you may have no way to know except going to the console. I do put my lab machines at home into standby when I am not using them, but I understand the risk. If I were running a business on them, I would NEVER do that. All IMHO, of course.

 

[kdragon75]

I was looking for if there was a way to shut down only if there are no running jobs, eg., the replication has finished and there are no scrubs or smart tests. Something like, "if after 4am and no jobs are running, shut down".

--Jim

It would be a cron job to check every hr from 4am to <pick_a_time> and then run your check. If the ISP is TOTALLY UNREASONABLE as it sounds they may be and imposes a hard limit on the run time, have cron just shut down at the specified time. Again, this sounds simple, just shutdown and start up at x time but it's actually MUCH more involved on the management front. Make sure all responsibilities, costs, bandwidth, remote assistance, physical access is written in a legally binding contract. This is also one of the only times I would recommend using encryption on FreeNAS (and on the wire, SSH based replication should suffice for this.). Just be sure to backup the keys and test/document the process to restore access the the disks on a different system.