We have a FreeNAS system that is used for NFS file storage and is kept busy with a lot of random read and writes.
It is using about 27TB of it's 36TB of storage however I have noticed only a small percentage (12.5%) of RAM is being used for the ARC.
Does anyone have any ideas why this may be? I thought it generally used all the memory that was available.
Most of the time there is about 80Mbps of read/write traffic going through.
This is the output from arc_summary.py
System Memory:
0.16% 77.24 MiB Active, 1.47% 704.11 MiB Inact
16.13% 7.52 GiB Wired, 0.00% 0 Bytes Cache
82.23% 38.36 GiB Free, 0.00% 0 Bytes Gap
Real Installed: 48.00 GiB
Real Available: 99.78% 47.89 GiB
Real Managed: 97.39% 46.65 GiB
Logical Total: 48.00 GiB
Logical Used: 18.65% 8.95 GiB
Logical Free: 81.35% 39.05 GiB
Kernel Memory: 488.46 MiB
Data: 94.59% 462.05 MiB
Text: 5.41% 26.41 MiB
Kernel Memory Map: 46.65 GiB
Size: 12.26% 5.72 GiB
Free: 87.74% 40.93 GiB
Page: 1
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Storage pool Version: 5000
Filesystem Version: 5
Memory Throttle Count: 0
ARC Misc:
Deleted: 230.96m
Mutex Misses: 39.09k
Evict Skips: 39.09k
ARC Size: 12.51% 5.71 GiB
Target Size: (Adaptive) 12.50% 5.71 GiB
Min Size (Hard Limit): 12.50% 5.71 GiB
Max Size (High Water): 8:1 45.65 GiB
ARC Size Breakdown:
Recently Used Cache Size: 93.69% 5.35 GiB
Frequently Used Cache Size: 6.31% 368.94 MiB
ARC Hash Breakdown:
Elements Max: 2.95m
Elements Current: 16.34% 482.05k
Collisions: 75.43m
Chain Max: 7
Chains: 13.28k
Page: 2
------------------------------------------------------------------------
ARC Total accesses: 2.85b
Cache Hit Ratio: 85.28% 2.43b
[root@tga-backup] ~# arc_summary.py | more
System Memory:
0.16% 77.37 MiB Active, 1.47% 703.98 MiB Inact
16.08% 7.50 GiB Wired, 0.00% 0 Bytes Cache
82.29% 38.38 GiB Free, 0.00% 0 Bytes Gap
Real Installed: 48.00 GiB
Real Available: 99.78% 47.89 GiB
Real Managed: 97.39% 46.65 GiB
Logical Total: 48.00 GiB
Logical Used: 18.60% 8.93 GiB
Logical Free: 81.40% 39.07 GiB
Kernel Memory: 488.77 MiB
Data: 94.60% 462.35 MiB
Text: 5.40% 26.41 MiB
Kernel Memory Map: 46.65 GiB
Size: 12.22% 5.70 GiB
Free: 87.78% 40.95 GiB
Page: 1
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Storage pool Version: 5000
Filesystem Version: 5
Memory Throttle Count: 0
ARC Misc:
Deleted: 230.97m
Mutex Misses: 39.11k
Evict Skips: 39.11k
ARC Size: 12.49% 5.70 GiB
Target Size: (Adaptive) 12.50% 5.71 GiB
Min Size (Hard Limit): 12.50% 5.71 GiB
Max Size (High Water): 8:1 45.65 GiB
ARC Size Breakdown:
Recently Used Cache Size: 93.75% 5.35 GiB
Frequently Used Cache Size: 6.25% 365.17 MiB
ARC Hash Breakdown:
Elements Max: 2.95m
Elements Current: 16.34% 481.98k
Collisions: 75.43m
Chain Max: 7
Chains: 13.27k
Page: 2
------------------------------------------------------------------------
ARC Total accesses: 2.85b
Cache Hit Ratio: 85.28% 2.43b
Cache Miss Ratio: 14.72% 418.96m
Actual Hit Ratio: 84.46% 2.40b
Data Demand Efficiency: 89.21% 1.70b
Data Prefetch Efficiency: 34.52% 1.37m
CACHE HITS BY CACHE LIST:
Anonymously Used: 0.79% 19.08m
Most Recently Used: 39.07% 947.83m
Most Frequently Used: 59.98% 1.46b
Most Recently Used Ghost: 0.12% 3.03m
Most Frequently Used Ghost: 0.04% 1.02m
CACHE HITS BY DATA TYPE:
Demand Data: 62.50% 1.52b
Prefetch Data: 0.02% 472.74k
Demand Metadata: 36.10% 875.83m
Prefetch Metadata: 1.38% 33.56m
CACHE MISSES BY DATA TYPE:
Demand Data: 43.79% 183.47m
Prefetch Data: 0.21% 896.62k
Demand Metadata: 52.93% 221.75m
Prefetch Metadata: 3.07% 12.84m
Page: 3
------------------------------------------------------------------------
Page: 4
------------------------------------------------------------------------
DMU Prefetch Efficiency: 6.50m
Hit Ratio: 10.26% 667.17k
Miss Ratio: 89.74% 5.83m
It is using about 27TB of it's 36TB of storage however I have noticed only a small percentage (12.5%) of RAM is being used for the ARC.
Does anyone have any ideas why this may be? I thought it generally used all the memory that was available.
Most of the time there is about 80Mbps of read/write traffic going through.
This is the output from arc_summary.py
System Memory:
0.16% 77.24 MiB Active, 1.47% 704.11 MiB Inact
16.13% 7.52 GiB Wired, 0.00% 0 Bytes Cache
82.23% 38.36 GiB Free, 0.00% 0 Bytes Gap
Real Installed: 48.00 GiB
Real Available: 99.78% 47.89 GiB
Real Managed: 97.39% 46.65 GiB
Logical Total: 48.00 GiB
Logical Used: 18.65% 8.95 GiB
Logical Free: 81.35% 39.05 GiB
Kernel Memory: 488.46 MiB
Data: 94.59% 462.05 MiB
Text: 5.41% 26.41 MiB
Kernel Memory Map: 46.65 GiB
Size: 12.26% 5.72 GiB
Free: 87.74% 40.93 GiB
Page: 1
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Storage pool Version: 5000
Filesystem Version: 5
Memory Throttle Count: 0
ARC Misc:
Deleted: 230.96m
Mutex Misses: 39.09k
Evict Skips: 39.09k
ARC Size: 12.51% 5.71 GiB
Target Size: (Adaptive) 12.50% 5.71 GiB
Min Size (Hard Limit): 12.50% 5.71 GiB
Max Size (High Water): 8:1 45.65 GiB
ARC Size Breakdown:
Recently Used Cache Size: 93.69% 5.35 GiB
Frequently Used Cache Size: 6.31% 368.94 MiB
ARC Hash Breakdown:
Elements Max: 2.95m
Elements Current: 16.34% 482.05k
Collisions: 75.43m
Chain Max: 7
Chains: 13.28k
Page: 2
------------------------------------------------------------------------
ARC Total accesses: 2.85b
Cache Hit Ratio: 85.28% 2.43b
[root@tga-backup] ~# arc_summary.py | more
System Memory:
0.16% 77.37 MiB Active, 1.47% 703.98 MiB Inact
16.08% 7.50 GiB Wired, 0.00% 0 Bytes Cache
82.29% 38.38 GiB Free, 0.00% 0 Bytes Gap
Real Installed: 48.00 GiB
Real Available: 99.78% 47.89 GiB
Real Managed: 97.39% 46.65 GiB
Logical Total: 48.00 GiB
Logical Used: 18.60% 8.93 GiB
Logical Free: 81.40% 39.07 GiB
Kernel Memory: 488.77 MiB
Data: 94.60% 462.35 MiB
Text: 5.40% 26.41 MiB
Kernel Memory Map: 46.65 GiB
Size: 12.22% 5.70 GiB
Free: 87.78% 40.95 GiB
Page: 1
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Storage pool Version: 5000
Filesystem Version: 5
Memory Throttle Count: 0
ARC Misc:
Deleted: 230.97m
Mutex Misses: 39.11k
Evict Skips: 39.11k
ARC Size: 12.49% 5.70 GiB
Target Size: (Adaptive) 12.50% 5.71 GiB
Min Size (Hard Limit): 12.50% 5.71 GiB
Max Size (High Water): 8:1 45.65 GiB
ARC Size Breakdown:
Recently Used Cache Size: 93.75% 5.35 GiB
Frequently Used Cache Size: 6.25% 365.17 MiB
ARC Hash Breakdown:
Elements Max: 2.95m
Elements Current: 16.34% 481.98k
Collisions: 75.43m
Chain Max: 7
Chains: 13.27k
Page: 2
------------------------------------------------------------------------
ARC Total accesses: 2.85b
Cache Hit Ratio: 85.28% 2.43b
Cache Miss Ratio: 14.72% 418.96m
Actual Hit Ratio: 84.46% 2.40b
Data Demand Efficiency: 89.21% 1.70b
Data Prefetch Efficiency: 34.52% 1.37m
CACHE HITS BY CACHE LIST:
Anonymously Used: 0.79% 19.08m
Most Recently Used: 39.07% 947.83m
Most Frequently Used: 59.98% 1.46b
Most Recently Used Ghost: 0.12% 3.03m
Most Frequently Used Ghost: 0.04% 1.02m
CACHE HITS BY DATA TYPE:
Demand Data: 62.50% 1.52b
Prefetch Data: 0.02% 472.74k
Demand Metadata: 36.10% 875.83m
Prefetch Metadata: 1.38% 33.56m
CACHE MISSES BY DATA TYPE:
Demand Data: 43.79% 183.47m
Prefetch Data: 0.21% 896.62k
Demand Metadata: 52.93% 221.75m
Prefetch Metadata: 3.07% 12.84m
Page: 3
------------------------------------------------------------------------
Page: 4
------------------------------------------------------------------------
DMU Prefetch Efficiency: 6.50m
Hit Ratio: 10.26% 667.17k
Miss Ratio: 89.74% 5.83m