Is there a way to temporarily flush or disable ARC for testing purposes?

[JamesNisly]

I’m in the middle of a new build and I”m currently trying to get a handle on how much of an impact a metadata special vdev will have in our particular workflow on a sample of actual client data.

To this end, I have three pools set up with the same sample data on all three. One with no special vdev, one with a metadata only special vdev, and one with a vdev with metadata and small files.

I have a small battery of common operations that I‘m trying to run some repeated tests on in order to get some semi-reliable timing numbers between the three special vdev scenarios.

The only problem is that after each test, the ARC kicks in as it should and I have to reboot between every test to ensure that I’m actually pulling data from the pools and not just getting it from cache.

Is there a way to temporarily disable ARC just to help speed this process up a little?

 

[sretalla]

Pick your dataset(s).

zfs set primarycache=none pool/dataset

zfs set secondarycache=none pool/dataset for good measure.

Don't forget to set it back to all at the end.

 

[JamesNisly]

Thank you so much; you just saved me a ton of time. To confirm, to set it back, the opposite of “none” is ”all” in this case, right?

As per: https://docs.oracle.com/cd/E19253-01/819-5461/ghbxt/index.html

 

[sretalla]

In theory, whatever was set before (default is all). zfs get primarycache pool/dataset before you set it.

 

[sretalla]

It may be necessary to reboot once after the set operation as I don't think that setting will hunt for already cached items and remove them.

If you're on SCALE, maybe this would get around that:

zfs: flush cache without exporting

For testing I need zfs' cache to be cold. I can flush caching from the pool by removing the cache disks, exporting and importing the pool. This gives the effect I want. But would really like not to

serverfault.com

If you're on CORE (seems to be the case from your sig), then you could (probably not very attractive) export/disconnect the pool and import it again rather than reboot.