L2Arc Latency

[kspare]

we are running Intel P3700 2TB, OP to 500GB, set to 4K, also have another 800GB OP to 500GB with 512.

Tried setting our blocksize to 64 and back to default.

But our latency always looks like this.

The middle one is before data, the other two have been running for some time.

If I move the data off the server, the latency goes down....

What am I missing?

running 11.2-U6

 

[HoneyBadger]

I think we need to have a FreeNAS dev talk about what metric/sysctl/kstat that graph is pulling from. If you were actually seeing hundreds of milliseconds of read latency from your L2ARC device I imagine your workloads and users would be screaming bloody murder.

What do the stats look like from an external storage consumer (eg: if you're using hypervisors, what does it say for datastore/disk latency?)

 

[kspare]

Everything seems normal to be honest. but the disk wait, disk busy and disk latency seem high as well. So i'm not sure if this is believable?

 

[HoneyBadger]

Everything seems normal to be honest. but the disk wait, disk busy and disk latency seem high as well. So i'm not sure if this is believable?

I'm leaning towards the UI misrepresenting things. You'd definitely notice if you were waiting 400ms for a block of data.

What does a snapshot of gstat -df nvd look like?*