ARC stats questions/problems thread

Status
Not open for further replies.

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
I open this thread to avoid cluttering the original thread on the ARC stats with the questions about the ARC health :)

You can post your ARC stats on the other thread and put a link to it here with your questions, or you can post your stats on both but, please, don't ask your questions on the other thread as it's only for sharing your stats, thanks ;)

Please remember that it's stats analysis and that every server and use case is different so it's an art as much as a science. Only you can know if your perfs are ok or not, regardless of the ARC (or other) stats.

Some facts:
  • The ARC can takes days (and even weeks with light loads) to warm-up.
  • The Hit ratio should be over 90 %, if it's lower then your ARC might be too small and if it's under 85 % then you can be pretty sure your ARC is too small.
  • The Hit MFU:MRU ratio should be over 10:1 (and ideally over 100:1), if it's lower then your ARC might be too small and if it's under about 1:1 then you can be pretty sure your ARC is too small.
  • If your ARC stats are worse and worse during the warm-up (especially the Hit ratio) your ARC is probably too small.
 
Last edited:

WordlyBedouin

Dabbler
Joined
Mar 30, 2015
Messages
10
Thanks for setting up this thread Bidule0hm.

Here's my stats and questions again.
  • Media, personal files (photos, PDFs, etc.), TimeMachine backups
  • 2.18TiB / 14.5TiB (Pool1)
  • 39.4GiB / 928GiB (TimeMachine)
  • 966MiB / 14.6GiB (freenas-boot)
  • 11.20GiB (MRU: 10.50GiB, MFU: 717.07MiB) / 16.00GiB
  • Hit ratio -> 85.19% (higher is better)
  • Prefetch -> 11.73% (higher is better)
  • Hit MFU:MRU -> 77.25%:22.06% (higher ratio is better)
  • Hit MRU Ghost -> 4.12% (lower is better)
  • Hit MFU Ghost -> 0.64% (lower is better)
1. Do the above stats seem ok for home use? A quick glance at the stats makes me think there's some room for improvment/re-configuration.
2. Is there anything I can/should do to further performance tune this? Though, I just built this system 2 days ago and am currently still loading up data onto it from my backup copies...so I may need more time for ARC to "warm up".
3. I don't think adding an L2ARC would help considering the type of data/content I'm storing on my NAS. Nor would ZIL help either from what I've read.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
1. Well, those stats aren't very good, but as the ARC is a read cache and because of your point 2, we can't say that you've a too small ARC or not. You need to use the server the way you'll use it normally for at least a few days in order to have some useful stats ;)

3. Very probably not, especially if you have less than 32 or 64 GB of RAM. The rule is to max out the system with RAM before adding a L2ARC (because a SSD is many orders of magnitude slower than RAM and because the L2ARC uses some RAM (which isn't used for the ARC anymore...) for his index) :)
 

WordlyBedouin

Dabbler
Joined
Mar 30, 2015
Messages
10
1. Well, those stats aren't very good, but as the ARC is a read cache and because of your point 2, we can't say that you've a too small ARC or not. You need to use the server the way you'll use it normally for at least a few days in order to have some useful stats ;)

3. Very probably not, especially if you have less than 32 or 64 GB of RAM. The rule is to max out the system with RAM before adding a L2ARC (because a SSD is many orders of magnitude slower than RAM and because the L2ARC uses some RAM (which isn't used for the ARC anymore...) for his index) :)


1. & 2. )Makes sense. It will take me a couple of days to copy over all the content from my backups over to the FreeNAS box. After that's done is when I figure I'll be at a point where I start using it in "normal" fashion. So perhaps next week sometime I'll re-post some stats and see where we're at.

3. Yep, I've maxed the RAM on this mini-ITX mobo @ 16GB (only 2 slots). I figured 4x4TB drives, @ 1GB RAM/1TB HD, meant I'd be in a "good enough" place RAM-wise. Regardless I just bought/built this rig, so not much of a chance of buying new parts 2 days later to upgrade it. :smile: My wife would just kill me if I even suggested it as a remote possibility.

Thanks for your help and I'll follow up in a bit after things have "warmed up" a bit on my new box.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Given the size of your HDDs and RAM, and your usage case, I can say right now it'll be ok ;)

So, hopefully, your wife will not kill you :D
 

WordlyBedouin

Dabbler
Joined
Mar 30, 2015
Messages
10
Sounds good, thanks for the feedback. Incidentally this FreeNAS box will be a glorified media store for my TV's OpenElec miniPC. My wife's Mac will do the typical Time Machine backups... So while I don't need tons of performance, the nerdy side of my brain like to tinker, learn, and play with tech.
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
I fell like my hit ratio should be better, or do you think its just because its an Avoton?

  • Mostly Media, some backups
  • 838MiB / 14.5GiB (freenas-boot)
  • 7.51TiB / 14.5TiB (volume)
  • 24.46GiB (MRU: 19.79GiB, MFU: 4.70GiB) / 32.00GiB
  • Hit ratio -> 86.45% (higher is better)
  • Prefetch -> 18.12% (higher is better)
  • Hit MFU:MRU -> 80.60%:16.75% (higher ratio is better)
  • Hit MRU Ghost -> 0.17% (lower is better)
  • Hit MFU Ghost -> 0.70% (lower is better)
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
It's not excellent but not that bad either. What's the uptime?
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
4 days, arc is settled
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
I PM'd a few Avoton people here to post their arcstats for me to compare in the other thread.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Ok ;)

After all, the question is: are the perfs ok for your needs? if yes then don't bother :)
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
Yea, I hear ya, and of course its plenty.

But you know how it is, if it can be better, then it NEEDS to be better! LOL
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Yeah, I know this feeling too :D

At least it's simple here: you want higher perfs? then put more RAM :P
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
Yea well at $300 per 16GB stick, I think its safe to say im maxed out on RAM.
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
I feel with 32GB of RAM, my server should beast through most tasks with ease. So either Im really overestimating the Avoton, or something's wrong. Shrugging my shoulders and saying oh well "Avoton sucks" and doing nothing, doesnt sit right with me. So I shall approach this with the assumption something is wrong.

So I removed 2 RAM sticks to go back down to 16GB of memory. After reboot, I artificially maxed out my ARC and Wired memory usage by running 3, full 1080p video streams with the "Make my CPU hurt" option ticked in Plex.

generate


Before with 32GB of RAM:

  • Mostly Media, some backups
  • 838MiB / 14.5GiB (freenas-boot)
  • 7.51TiB / 14.5TiB (volume)
  • 24.46GiB (MRU: 19.79GiB, MFU: 4.70GiB) / 32.00GiB
  • Hit ratio -> 86.45% (higher is better)
  • Prefetch -> 18.12% (higher is better)
  • Hit MFU:MRU -> 80.60%:16.75% (higher ratio is better)
  • Hit MRU Ghost -> 0.17% (lower is better)
  • Hit MFU Ghost -> 0.70% (lower is better)

And now with 16GB of RAM:

  • Put your data type(s) here...
  • 838MiB / 14.5GiB (freenas-boot)
  • 7.54TiB / 14.5TiB (volume)
  • 11.19GiB (MRU: 10.24GiB, MFU: 978.91MiB) / 16.00GiB
    [*]Hit ratio -> 99.54% (higher is better)
  • Prefetch -> 3.16% (higher is better)
  • Hit MFU:MRU -> 24.96%:0.96% (higher ratio is better)
  • Hit MRU Ghost -> 0.02% (lower is better)
  • Hit MFU Ghost -> 0.04% (lower is better)

How does this make any sense?

More testing to follow, but I think I know what the problem may be.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
It makes sense to me. It's not as it seems, and it's not supposed to.

There's various aspects of this that are being totally ignored, such as uptime, other potential workloads such as other shared files, other windows machines trying to do crap like thumbnails, scrubs, resilvers.

Those numbers mean nothing without the server having uptime measured in days or weeks. Those numbers are nothing more than a short snapshot, and it is easy to make that number really high. I can make mine 100% if I wished.
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
Yea I hear ya, but I believe the problem is I have 2 very slightly diff model Kingston RAM modules in my system. I bought them seperately a year ago and didnt notice the difference until recently when troubleshooting an unrelated issue.

Kingston 8GB 204-Pin DDR3 SO-DIMM ECC Unbuffered DDR3 1600 (PC3 12800) Server Memory Model KVR16LSE11/8
http://www.newegg.com/Product/Product.aspx?Item=N82E16820239702

Kingston 8GB 204-Pin DDR3 SO-DIMM ECC Unbuffered DDR3 1600 (PC3 12800) Server Memory Model KVR16LSE11/8KF
http://www.newegg.com/Product/Product.aspx?Item=N82E16820239958

My last arcstat (16GB, hit ratio 99%) was with 2 of the same modules KVR16LSE11/8. That was 4 hrs after reboot.

I just swapped back in the pair that is on of each 11/8 and 11/8KF.

Here are my arcstats 5 mins after reboot

  • Put your data type(s) here...
  • 839MiB / 14.5GiB (freenas-boot)
  • 7.52TiB / 14.5TiB (volume)
  • 810.40MiB (MRU: 7.22GiB, MFU: 7.22GiB) / 16.00GiB
  • Hit ratio -> 93.12% (higher is better)
  • Prefetch -> 18.03% (higher is better)
  • Hit MFU:MRU -> 83.26%:14.15% (higher ratio is better)
  • Hit MRU Ghost -> 0.00% (lower is better)
  • Hit MFU Ghost -> 0.00% (lower is better)

From my experience, it starts at 100% and works it way down.

Im about to reboot again, but with only the 11/8KF and then once more with 11/8. I think that should say something one way or another.
 

JJT211

Patron
Joined
Jul 4, 2014
Messages
323

JJT211

Patron
Joined
Jul 4, 2014
Messages
323
OK Cyberjock, you're right. I did as above, and even put the 99% pair back in and they all seem to start at archit rate 93% which means, as you said, not much. Im chasing ghosts here. Thanks for the heads up.
 
Last edited:

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
If you want to compare between 16 and 32 GB you need to have everything else equal; same uptime, same files accessed in the same order, etc... ;)

But yeah, if it ain't broke, don't fix it :p
 
Last edited:
Status
Not open for further replies.
Top