Scrub 1st Saturday/Month

[JayG30]

I'm altering my scrub/SMART/replication/rsync schedules and looking for some clarification on this.

I want my Scrub to run once a month, but I want to have more control over the day of the week it runs on to make sure it doesn't conflict with other things. So I'm thinking I'd like to to run on the first Saturday on each month. I think I have it right but the "threshold days" makes my brain hurt when I try visualize it.

So here is what I'm thinking:

Threshold days: 28
Minute: 00 (each selected minute tab)
Hour: 02 (each selected hour tab)
Day of Month: 01, 02, 03, 04, 05, 06, 07 (Each selected day of month)
Month: ALL
Day of week: Saturday

My thought process is as follows;

The first Saturday of a month HAS to fall within the days 01-07.

Threshold days needs to change to 28 instead of 35 because there are months without 5 full weeks, which would result in the following Saturday falling before the threshold hit 35 days and the scrub being skipped.As an example, with this schedule it would run on November 4th 2017 and you would expect the next run on December 2nd 2017, however because November only has 30 days it will only be 28 days when Dec 2nd comes around.

However, I'm not sure in that situation I pointed out if the scrub would actually run or if it would STILL consider that "within" the threshold.

Any help is appreciated, Thanks.

 

[danb35]

Set the threshold to 22 days.

 

[JayG30]

Thanks. Any reason 22 days specifically? I thought 35 and 28 were used commonly due to the divisible by 7 "rule". Any reason not to use 21 days for instance?

 

[danb35]

21 would likely work as well. Actually, in your case, anything from 7-27 would probably work, but 28 would miss the scrub for March. My pool's big enough that my scrubs regularly take 36+ hours, and the threshold is from the completion of the last scrub, so I like to set mine at least a couple of days less than the interval I want.

 

[JayG30]

So it seems like my understanding of the "day of week" and "day of month" is wrong. I thought it was a AND statement but appears to be an OR statement. Which means my settings aren't right.

See THIS THREAD on the forums.

Does anyone have a working set of settings to run on the 1st Saturday/Month?

 

[SweetAndLow]

So it seems like my understanding of the "day of week" and "day of month" is wrong. I thought it was a AND statement but appears to be an OR statement. Which means my settings aren't right.

Does anyone have a working set of settings to run on the 1st Saturday/Month?

Basically every line needs to be able to be true for a scrub to run. So you don't care about the date of the month you just want Saturday at 2am every 28 days. So it will run every 28 days or more on Saturday.

Depending on how long your scrub of you might miss a Saturday if it hasn't been 28 days since your last one finished.

Sent from my Nexus 5X using Tapatalk

 

[JayG30]

Basically every line needs to be able to be true for a scrub to run. So you don't care about the date of the month you just want Saturday at 2am every 28 days. So it will run every 28 days or more on Saturday.

Depending on how long your scrub of you might miss a Saturday if it hasn't been 28 days since your last one finished.

Sent from my Nexus 5X using Tapatalk

But that doesn't mean it will always be the FIRST Saturday of the month from my understanding.

Those settings could have it first run this Saturday, the 24th and then it would always try to run on the 24th of the month as long as the threshold is met. And if the threshold isn't meet even once that would cause the day it would fall on to change as well.

Please see this thread which I forgot to originally link to HERE.

 

[SweetAndLow]

But that doesn't mean it will always be the FIRST Saturday of the month from my understanding.

Those settings could have it first run this Saturday, the 24th and then it would always try to run on the 24th of the month as long as the threshold is met. And if the threshold isn't meet even once that would cause the day it would fall on to change as well.

Please see this thread which I forgot to originally link to HERE.

Correct it might not run on the first Saturday.

Sent from my Nexus 5X using Tapatalk

 

[JayG30]

Which is the problem. The way I have it I don't want it to have the possibility to ever interfere with my SMART long test, ZFS replications, or rsync tasks. But due to limitations in my "time windows" the first Saturday of the month is the best time for this to occur ALWAYS. And with the way this currently works I don't know if it is even possible (which would be unfortunate).

 

[SweetAndLow]

Set it up using dates instead of days of the week. Dates will always be the same very month. Or stick with days of the week and do smart short tests on Monday every week, smart long every other Wednesday, replication on Thursdays and scrubs on Sundays.

 

[JayG30]

I guess this simply can't be done using the GUI.

I've found two cron jobs that I BELIEVE would accomplish what I want;

Code:

00 02 * * 6 [ $(date +\%d) -le 07 ] && /run/your/script

Code:

0 2 1-7 * *	test $(date +%u) -eq 6 && echo /run/your/script

That can't be replicated using the GUI.

So, as mentioned in the linked thread, you could create a script that is stored on your pool and enter it into the GUI cron job section (since the GUI cron job section will incorrectly script characters in that command).

So I disable the only scrub listed in the Web GUI, but look what gets added into /etc/crontab when you do this;

Code:

45	  3	   *	   *	   *	   root	/usr/local/libexec/nas/scrub -t 35 freenas-boot

Am I mistaken, or is this adding the default scrub job even though I've disabled it? I get why they might want to do that (protecting users) but doesn't this make it impossible to turn this off and enter your own through cron job?

 

[BigDave]

I submit this as a solution...
If the first Saturday is an absolute must, you will have to schedule by this example:
Threshold = 26 (there might be a better choice here)
Min/Hour = Your Choice
Days Of Month = 1,2,3,4,5,6,7
Month = All 12
Day Of The Week = Saturday

 

[JayG30]

I submit this as a solution...
If the first Saturday is an absolute must, you will have to schedule by this example:
Threshold = 26 (there might be a better choice here)
Min/Hour = Your Choice
Days Of Month = 1,2,3,4,5,6,7
Month = All 12
Day Of The Week = Saturday

Please see my above posts and the linked thread. This will not work because of misconceptions around how Days on Month and Day of Week work in conjunction with each other.

https://www.freebsd.org/cgi/man.cgi?crontab(5)

The day of a command's execution can be specified by two fields - day of month, and day of week. If both fields are restricted (i.e., aren't *), the command will be run when either field matches the current time. For example, 30 4 1,15 * 5 would cause a command to be run at 4:30 am on the 1st and 15th of each month, plus every Friday.

 

[BigDave]

@JayG30
Maybe I'm too thick for this...
Does not the Threshold setting circumvent both the day and date?

If you had started (with the settings below) on Jan 7th of this year,
the scrub/cron would only fire (if I counted correctly) on the first Saturday of each month.

Threshold = 27
Min/Hour = Your Choice
Days Of Month = 8th (only)
Month = All 12
Day Of The Week = Saturday

And yes, the periods of Apr/May and Sept/Oct would have 34 days between scrubs all others
count up as 27 days. April was the only month in which the 8th day was also a Saturday, but
fell only 7 days from the last scrub, therefore it would be passed up. along with all the other 8th days.

I'm only guessing (as I have not looked at 2018), but I think the monthly date (the 8th)
would have to be adjusted every year to make it go.

EDIT: I looked at 2018 and I believe that it works for that year as well...

 

[JayG30]

@JayG30
Maybe I'm too thick for this...
Does not the Threshold setting circumvent both the day and date?

If you had started (with the settings below) on Jan 7th of this year,
the scrub/cron would only fire (if I counted correctly) on the first Saturday of each month.

Threshold = 27
Min/Hour = Your Choice
Days Of Month = 8th (only)
Month = All 12
Day Of The Week = Saturday

And yes, the periods of Apr/May and Sept/Oct would have 34 days between scrubs all others
count up as 27 days. April was the only month in which the 8th day was also a Saturday, but
fell only 7 days from the last scrub, therefore it would be passed up. along with all the other 8th days.

I'm only guessing (as I have not looked at 2018), but I think the monthly date (the 8th)
would have to be adjusted every year to make it go.

EDIT: I looked at 2018 and I believe that it works for that year as well...

Thanks, I'll sit down and work through your idea and see if it will work. Have some other things to deal with right now. :)

 

[JayG30]

So I disable the only scrub listed in the Web GUI, but look what gets added into /etc/crontab when you do this;

Code:

45	  3	   *	   *	   *	   root	/usr/local/libexec/nas/scrub -t 35 freenas-boot

Am I mistaken, or is this adding the default scrub job even though I've disabled it? I get why they might want to do that (protecting users) but doesn't this make it impossible to turn this off and enter your own through cron job?

I'm dense, this is entered because it adds a default scrub of the BOOT DRIVE, not the pool (ie. freenas-boot). So you can disable scrub on the pool and setup via a cron job.

 

[JayG30]

@JayG30
Maybe I'm too thick for this...
Does not the Threshold setting circumvent both the day and date?

If you had started (with the settings below) on Jan 7th of this year,
the scrub/cron would only fire (if I counted correctly) on the first Saturday of each month.

Threshold = 27
Min/Hour = Your Choice
Days Of Month = 8th (only)
Month = All 12
Day Of The Week = Saturday

And yes, the periods of Apr/May and Sept/Oct would have 34 days between scrubs all others
count up as 27 days. April was the only month in which the 8th day was also a Saturday, but
fell only 7 days from the last scrub, therefore it would be passed up. along with all the other 8th days.

I'm only guessing (as I have not looked at 2018), but I think the monthly date (the 8th)
would have to be adjusted every year to make it go.

EDIT: I looked at 2018 and I believe that it works for that year as well...

I don't see this working. Let's start in July 2017 and this is how I see your settings playing out.

July 1st, 2017 - 1st scrib. First Sat of Month
---July 8th/15th/22nd Skipped due to 27 day threshold
July 29th, 2017 - 2nd scrub. Sat & 28 days since July 1st
---Aug 5th/12th/19th Skipped due to 27 day threshold
Aug 26th, 2017 - 3rd scrub. Sat & 28 days since July 29th

So right off the bat this doesn't look right. The second scrub should have been on Aug 5th not the 29th and therefore that makes everything go out of wack.

 

[BigDave]

The second scrub should have been on Aug 5th not the 29th and therefore that makes everything go out of wack.

Right you are...

I thought it was a AND statement but appears to be an OR statement.

I wonder how difficult this would be to change?

 

[Ericloewe]

There's another recent thread about cron and its less-than-intuitive operation. Search for @HolyK's recent post's and you'll find it.

I wonder how difficult this would be to change?

Programatically? Easy. Bureaucratically? Nightmarish.

 

[JayG30]

There's another recent thread about cron and its less-than-intuitive operation. Search for @HolyK's recent post's and you'll find it.


Programatically? Easy. Bureaucratically? Nightmarish.

Yea, that was the thread I linked to above. I found that thread after trying to solve my problem and realized what everyone had told me about how setting the SCRUB settings worked was wrong. It all comes around to that OR condition for cron, but that is how cron was made and it is clearly documented.

It's not that cron can't be setup to do this (in fact that is what I'm doing now by just adding it to the Web GUI cron job(s)). You can find a plethora of examples of people trying to accomplish this stuff and how to construct the cron job. However, as you can see with my cron job and @HolyK cron job(s), they are more complex to create and I don't think there is any standard way you could cover all the potential combinations of Day of Month AND Day of Week selections a user could make. There are a lot of case by case combinations you would have to consider and try to cover.

Trying to change cron itself, I doubt anyone would even entertain the idea. It's worked for a long time and, as explain, it can be written to cover these types of conditions. The potential to break things would be to high I'd imagine.

The biggest issue from the FreeNAS side is that the srub web GUI is laid out in a way that doesn't indication to the uninitiated that (1) This will be handled by a cron job once created (2) That day of month & day of week are linked in some fashion and what that fashion of linking is (ie. an OR condition). Most people would default to assuming it is an AND condition, which is wrong.

What makes this even worse IMO is that when setting up SMART tasks you are given the same settings screen. However because those aren't cron but instead handled by smartmontools (smartd in particular) they don't behave the same way. In those cases it is handled as an AND condition. Obviously that adds confusion.