Is there a way to limit the number of concurrent scrubs?

[bollar]

It appears that SCALE allows ten concurrent scrubs. I'd like to limit that to a lower number. Is there a sysctl or other way to set the specific number I want?

 

[Davvo]

As far as I know, OpenZFS allows you to have only a single scrub/resilver per pool.

Because scrubbing and resilvering are I/O-intensive operations, ZFS only allows one at a time.

Source.

Anyway, look at this and this.

 

[bollar]

As far as I know, OpenZFS allows you to have only a single scrub/resilver (either per pool or per system, don't remember).

That's by pool. In Cobia, it's possible to have ten run at the same time. In Bluefin release, the number was unlimited, which killed my system.

 

[Davvo]

That's by pool. In Cobia, it's possible to have ten run at the same time. In Bluefin release, the number was unlimited, which killed my system.

Yup, just checked... that's painful. Look at the links I added; there doesn't seem to be a tunable for limiting the number of concurrent scrubs on the system... Just don't schedule them in an overlapping way I guess.

 

[bollar]

Thanks. It's possible that zfs_scrub_delay will do what I want, but I don't really want the scrubs to be slower -- I just want fewer to run at once. This feels like a SCALE setting, not an OpenZFS setting.

 

[bollar]

For those who care...

I wound up just changing the code to limit the number of scrubs to 2. In pool_actions.py edit the line below from 10 to whatever is required (note the caveat that changing the code could hose up the middleware and require a reinstall of TrueNAS), but for me, this is much better than allowing ten concurrent scrubs:

Code:

                        if running_scrubs >= 2:

 

[Ericloewe]

How many pools do you have to make this an issue at all? I can't imagine more than three pools in a single system, and even that is weird edge case stuff.

 

[bollar]

How many pools do you have to make this an issue at all? I can't imagine more than three pools in a single system, and even that is weird edge case stuff.

I guess I’m an edge case. I have up to 50 pools for various reasons.

A key one is that I keep client data on separate pools so I can detach the pool and remove the drives to put their data into cold storage for a retention period.

 

[Ericloewe]

Holy crap. That's all I have, I've never heard of 50 pools on a single system. Not even as a weird publicity stunt.