High arc hit ratio on arc, low l2arc hit ratio after update to 11.1U5

[francomartin33]

Hey Im having some issues after updating from u4 to u5. After a power outage my usb drive malfunctioned and was bootlooping, after I got it working everything was ok until two days ago the system went unresponsive, no ssh, no gui, nothing. Had to reboot it and the bootloop started again. I tried updating to u5 and it came back to life with the same usb drive.
Everything was working fine, but I noticed that my arc hit ratio is way up. It used to be around 80% and now its between 90 and 95 percent. So far so good, but I also noticed that the l2arc hit ratio is both low and unstable. It used to be stable at 40%, which is reasonable.

My use case is virtualization with vmware.

im running an i3 4170, 8gb non ecc ddr3, 5x1tb 7200rpm raidz1 mix-match drives, 120gb ssd for log and cache.

Thank you!

 

[sretalla]

8GB of RAM is not enough to do L2ARC, you should add a lot more RAM rather than worrying about L2ARC under-performance.

After a reboot, both ARC and L2ARC are reset, so your working set of cached blocks will need to rebuild over time for you to see the higher hit rates you would have had after running for some days/weeks.

 

[francomartin33]

8GB of RAM is not enough to do L2ARC, you should add a lot more RAM rather than worrying about L2ARC under-performance.

After a reboot, both ARC and L2ARC are reset, so your working set of cached blocks will need to rebuild over time for you to see the higher hit rates you would have had after running for some days/weeks.

Thank you for your response!
I thought they were reset, but when I got the ssd it only took a couple of hours to crawl up to 30% and a day or two to crawl to 40. besides it used to be relatively stable(a flat line).
now its been running for 2,5 days and its not only low but also all over the place.

 

[sretalla]

It's much more likely that there is/was some problem with the reporting that has now been fixed.

If you think about it, of the 3 or so TB you might have on the spinning disks, only about 5% of the stored data is already 150GB (bigger than the whole disk you're running L2ARC on).

So how often do you access that exact same 5%... for stuff that isn't already in ARC... ? 40% constant sounds like some cooked books to me.

 

[francomartin33]

It's much more likely that there is/was some problem with the reporting that has now been fixed.

If you think about it, of the 3 or so TB you might have on the spinning disks, only about 5% of the stored data is already 150GB (bigger than the whole disk you're running L2ARC on).

So how often do you access that exact same 5%... for stuff that isn't already in ARC... ? 40% constant sounds like some cooked books to me.

I only have 700G used on the pool, and (i dont know why) my l2arc size is 150GB (im guessing compression). 40% sounded reasonable since im mosly reading operating systems, not much else.

This new hit ratio tells me not the same data is being accessed but i think it should be since im not doing anything fancy on the pool, only iscsi with vms that are 90% of the time idle.

 

[sretalla]

Maybe you're right, but a constant hit rate is fishy in my opinion.

 

[pro lamer]

Has the performance dropped?

On the other hand ZFS when having lots of RAM (not the OP case) tends to eat more and more RAM fir ARC and then there are some parameters limiting the ARC size.

As a test you may try those parameters or autotuning/tunables...

Sent from my mobile phone

 

[sretalla]

Actually, it smacks to me of a change from charting the overall average hit rate to charting the actual hit rate (probably still an average, but across 5 min or shorter). As mentioned by @pro lamer maybe nothing to worry about if performance hasn't dropped away on the VM side.