Any downside to aggressive scrubs schedule for flash drives?

[RoboKaren]

It's often recommended to not scrub rotating disks too frequently as the scrub operation itself is rather intensive in terms of wear and load on the head mechanisms.

Does this still hold for flash-based media such as SSDs? My assumption is that the zfs scrub only does read operations and so there is no wear caused by them to the flash ssd media (unless of course they trigger a rebuild).

Thus, assuming that one can handle the reduced performance downtime during scrubs, is there any downtime to a more aggressive scrub schedule (every 2-3 days; daily) of flash based pools, including the boot drive?


p.s. Let's leave aside the question of whether scrubs are even needed for flash based pools given solid state's lower propensity for bit rot/bit flipping and internal error checking mechanisms.

 

[DrKK]

I think you can scrub SSD's or flash drives as much as you want. I wouldn't worry about that too badly.

Also, you are dead wrong about "scrubs not being needed for flash based pools". Maybe for SSD's it's not a big deal, but, for USB flash sticks, even good ones, on the boot devices? We have users LEFT AND RIGHT that are getting plenty of filesystem errors turning up during ZFS scrubs. I myself have had to replace several in my system (the best way to solve this would appear to be to throw anything with "Kingston" on it in the garbage, and replace it with Sandisk, I have found), and in other systems that I have built for associates, when the ZFS scrub on the boot pool starting turning up problems.

I conclude that problems and corruption to the operating system boot drives have been plaguing many of us in the pre-9.3 days, and we had no idea.

God bless ZFS scrubbing of mirrored boot pools!

 

[RoboKaren]

Maybe for SSD's it's not a big deal, but, for USB flash sticks, even good ones, on the boot devices? We have users LEFT AND RIGHT that are getting plenty of filesystem errors turning up during ZFS scrubs. I myself have had to replace several in my system (the best way to solve this would appear to be to throw anything with "Kingston" on it in the garbage, and replace it with Sandisk, I have found), and in other systems that I have built for associates, when the ZFS scrub on the boot pool starting turning up problems.

I conclude that problems and corruption to the operating system boot drives have been plaguing many of us in the pre-9.3 days, and we had no idea.!

Interesting. I thought I was the only one with boot pool scrub errors. All of them have been corrected as I'm mirroring.

I've put in a feature request for a more aggressive scrub schedule for the boot pool (Issue #8409).

 

[DrKK]

You won't get anywhere with that. It was all we could do to get them to put in the user-settable quantity.

:)

In fact, I led that charge myself: https://bugs.freenas.org/issues/7301

 

[Bidule0hm]

Any reason to not have the same pop-up (with minutes, hours, days, months, ...) than for the storage scrubs and the CRON tasks for example?

 

[DrKK]

Any reason to not have the same pop-up (with minutes, hours, days, months, ...) than for the storage scrubs and the CRON tasks for example?

Yes. If you look at my original bug ticket, you'll see that Jordan has declared that any "additional granularity" over the number of days is moot. I tend to agree. Also, you can see in there as well, that they want the boot pool treated TOTALLY differently than the data pools. There are some good ZFS reasons for that.

So this is as good as it gets.

I think the main argument for leaving the defaults high is that the higher setting is probably appropriate for SATA DOMs, which the professional users have in lieu of USB sticks.

 

[RoboKaren]

Glad that I made the request as a "feature" and not a "bug" then.

 

[Bidule0hm]

Ok.

Personally I don't want to schedule a scrub every minute but the possibility to schedule the scrub on a fixed day of the month (for example the 1 and 15th of the month).

 

[titan_rw]

If you really want it to scrub the boot pool on the 1st and 15th, simply setup a cronjob to do so.

 

[Bidule0hm]

OMG, I didn't think about it, must be really tired or just stupid

But now I'm curious to know if the automatic scrub every x day will be a problem. Also what's the maximum value for this field? because if I can disabled it by putting 9999... :P

 

[cyberjock]

Ok.

Personally I don't want to schedule a scrub every minute but the possibility to schedule the scrub on a fixed day of the month (for example the 1 and 15th of the month).

We're talking about scrubbing a boot device. It literally has no impact on the performance of your server while it is running. I've been doing daily scrubs for quite a while and never found a problem.

 

[Bidule0hm]

It's not because of the performance but because of my email reporting scripts that are executed on fixed day of month after the SMART tests and storage scrubs.

 

[titan_rw]

OMG, I didn't think about it, must be really tired or just stupid

But now I'm curious to know if the automatic scrub every x day will be a problem. Also what's the maximum value for this field? because if I can disabled it by putting 9999... :p

No, because the automatic scheduled scrub won't scrub unless it's been > THRESHOLD days, which I think is 35. So if you scrub twice a month forceably via cron, then every time the 'zfs scrub schedule' checks to see if it should run, it should abort because the last scrub was always < 35 days ago. You'll still get the email on those days saying it's been less than X days, not scrubbing. I know my boot pool scrubs in <1 minute, as even old 1st gen ssd's can read fairly quickly when it's only 3-4 gigs of data.

 

[Bidule0hm]

Ah yeah, it's what I was thinking about when I said "I'm curious to know if the automatic scrub every x day will be a problem." Ok now I'm curious to know if there is a -f (for --force...), I'll read the man page :)

 

[titan_rw]

Ah yeah, it's what I was thinking about when I said "I'm curious to know if the automatic scrub every x day will be a problem." Ok now I'm curious to know if there is a -f (for --force...), I'll read the man page :)

Just call zpool manually. This is the automated one:

cat /etc/crontab:

Code:

45  3  *  *  *  root  /usr/local/sbin/scrub -t 35 freenas-boot

That's the automatic scrub schedule.

Just make a cron entry with "/sbin/zpool scrub freenas-boot".

I don't see why that wouldn't work.

 

[Bidule0hm]

-t ? hmm, there is only a -s in the FreeBSD zpool man page https://www.freebsd.org/cgi/man.cgi?zpool(8) Edit: I see, it's a custom script.

"I don't see why that wouldn't work." because of the threshold. But I can see that in fact the GUI don't set a scrub with an minimum interval but rather execute a scrub regularly and then the scrub checks if it executes or not. So a custom CRON should work. Thanks, I'll try that ;)

Edit: CRON added, there is only to wait now. BTW putting 9999 in the threshold field works... :D

 

[cyberjock]

I would just leave it at 30 days or something in the WebGUI. That way if your manual cronjob were to go badly the worst case is it would scrub every 30 days. But that's just me.

 

[Bidule0hm]

Even if I keep an eye on the result of each scrub (including the last exec date) it's not a bad idea actually ;)

But at least now we know we can put 9999 in that field even if it's not useful :D

 

[cyberjock]

Even if I keep an eye on the result of each scrub (including the last exec date) it's not a bad idea actually ;)

But at least now we know we can put 9999 in that field even if it's not useful :D

F*ck yeah! One scrub every 27 years! Do or die!

 

[danb35]

/usr/local/sbin/scrub is an iXSystems-created script, not a native part of FreeBSD's ZFS. AFAIK, there isn't a manpage for it.