I'm on FreeNAS 11.1-U7.
I previously had autotune enabled, then disabled it a long time ago when I read someone on this forum saying it was rubbish and only ever messed things up, while chasing down an issue with qBittorrent.
I recently added some RAM to the machine, and notices the ARC was maxing out in size WAY below what I would expect. I read up, and `sysctl -a vfs.zfs.arc_max` gave me back ~12.6G (in bytes). I found that there was an old tunable left by autotune, and changed it to 59G in bytes.
That did nothing, and more reading later, I found that a reboot was required, even though the manual says the exact opposite. I rebooted, and the value changed, but not to what I set. `sysctl -a vfs.zfs.arc_max` now outputs 20251258880, (roughly 18.9G in bytes). My tunable is still the number I set, it just appears that something along the line has decided "haha no dude you don't need that much" and taken control away from me. 18.9G is not my max RAM minus 1G, nor is it 5/8 of my max RAM.
I tried changing the tunable from a `sysctl` to a `Loader` type as https://www.freebsd.org/doc/handbook/zfs-advanced.html states it can be set in "/boot/loader.conf or /etc/sysctl.conf", and I don't know what I'm doing.
To hammer home the point that I don't know what I'm doing, I then did this:
I've seen other people get told off for not posting system specs, but I don't know what would be relevant here, nor am I even sure how to get it. https://i.imgur.com/OHKm5XS.png is my system information page, the motherboard is an ASUS P10S-M, the PSU is a Corsair 450W something or other (from memory), the hard drives are 2TB Seagate Barracuda, the boot drives are a pair of 8GiB SanDisk Cruzer blades (also from memory).
EDIT: Just gonna add more shell output as I find commands that look like they might be useful. First up:
I previously had autotune enabled, then disabled it a long time ago when I read someone on this forum saying it was rubbish and only ever messed things up, while chasing down an issue with qBittorrent.
I recently added some RAM to the machine, and notices the ARC was maxing out in size WAY below what I would expect. I read up, and `sysctl -a vfs.zfs.arc_max` gave me back ~12.6G (in bytes). I found that there was an old tunable left by autotune, and changed it to 59G in bytes.
That did nothing, and more reading later, I found that a reboot was required, even though the manual says the exact opposite. I rebooted, and the value changed, but not to what I set. `sysctl -a vfs.zfs.arc_max` now outputs 20251258880, (roughly 18.9G in bytes). My tunable is still the number I set, it just appears that something along the line has decided "haha no dude you don't need that much" and taken control away from me. 18.9G is not my max RAM minus 1G, nor is it 5/8 of my max RAM.
I tried changing the tunable from a `sysctl` to a `Loader` type as https://www.freebsd.org/doc/handbook/zfs-advanced.html states it can be set in "/boot/loader.conf or /etc/sysctl.conf", and I don't know what I'm doing.
To hammer home the point that I don't know what I'm doing, I then did this:
Code:
[asday@freenas ~]$ sysctl vfs.zfs.arc_max=63350767616 vfs.zfs.arc_max: 20251258880 sysctl: vfs.zfs.arc_max=63350767616: Operation not permitted [asday@freenas ~]$ sudo sysctl vfs.zfs.arc_max=63350767616 We trust you have received the usual lecture from the local System Administrator. It usually boils down to these three things: #1) Respect the privacy of others. #2) Think before you type. #3) With great power comes great responsibility. Password: hunter2 vfs.zfs.arc_max: 20251258880 sysctl: vfs.zfs.arc_max=63350767616: Invalid argument [asday@freenas ~]$
I've seen other people get told off for not posting system specs, but I don't know what would be relevant here, nor am I even sure how to get it. https://i.imgur.com/OHKm5XS.png is my system information page, the motherboard is an ASUS P10S-M, the PSU is a Corsair 450W something or other (from memory), the hard drives are 2TB Seagate Barracuda, the boot drives are a pair of 8GiB SanDisk Cruzer blades (also from memory).
EDIT: Just gonna add more shell output as I find commands that look like they might be useful. First up:
Code:
[asday@freenas ~]$ zfs-stats -M -s -A ------------------------------------------------------------------------ sysctl: unknown oid 'kstat.zfs.misc.arcstats.l2_writes_hdr_miss' sysctl: unknown oid 'kstat.zfs.misc.arcstats.recycle_miss' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.bogus_streams' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.colinear_hits' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.colinear_misses' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.reclaim_failures' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.reclaim_successes' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.streams_noresets' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.streams_resets' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.stride_hits' sysctl: unknown oid 'kstat.zfs.misc.zfetchstats.stride_misses' ZFS Subsystem Report Sun May 19 01:35:10 2019 ------------------------------------------------------------------------ System Memory Statistics: Physical Memory: 65429.81M Kernel Memory: 3754.64M DATA: 98.97% 3716.25M TEXT: 1.02% 38.39M ------------------------------------------------------------------------ ARC Misc: Deleted: 31 Recycle Misses: 0 Mutex Misses: 0 Evict Skips: 0 ARC Size: Current Size (arcsize): 87.08% 16817.89M Target Size (Adaptive, c): 100.00% 19313.10M Min Size (Hard Limit, c_min): 12.50% 2414.13M Max Size (High Water, c_max): ~8:1 19313.10M ARC Size Breakdown: Recently Used Cache Size (p): 69.62% 13446.31M Freq. Used Cache Size (c-p): 30.37% 5866.79M ARC Hash Breakdown: Elements Max: 294159 Elements Current: 100.00% 294159 Collisions: 5647 Chain Max: 0 Chains: 5297 ARC Eviction Statistics: Evicts Total: 336384 Evicts Eligible for L2: 89.64% 301568 Evicts Ineligible for L2: 10.35% 34816 Evicts Cached to L2: 0 ARC Efficiency Cache Access Total: 677185 Cache Hit Ratio: 42.12% 285277 Cache Miss Ratio: 57.87% 391908 Actual Hit Ratio: 38.97% 263918 Data Demand Efficiency: 39.57% Data Prefetch Efficiency: 10.16% CACHE HITS BY CACHE LIST: Anonymously Used: 7.48% 21359 Most Recently Used (mru): 54.50% 155477 Most Frequently Used (mfu): 38.01% 108441 MRU Ghost (mru_ghost): 0.00% 0 MFU Ghost (mfu_ghost): 0.00% 0 CACHE HITS BY DATA TYPE: Demand Data: 7.42% 21175 Prefetch Data: 4.89% 13963 Demand Metadata: 84.89% 242190 Prefetch Metadata: 2.78% 7949 CACHE MISSES BY DATA TYPE: Demand Data: 8.25% 32337 Prefetch Data: 31.49% 123439 Demand Metadata: 25.11% 98418 Prefetch Metadata: 35.13% 137714 ------------------------------------------------------------------------ ZFS Tunable (sysctl): kern.maxusers=4425 vfs.zfs.vol.immediate_write_sz=32768 vfs.zfs.vol.unmap_sync_enabled=0 vfs.zfs.vol.unmap_enabled=1 vfs.zfs.vol.recursive=0 vfs.zfs.vol.mode=2 vfs.zfs.sync_pass_rewrite=2 vfs.zfs.sync_pass_dont_compress=5 vfs.zfs.sync_pass_deferred_free=2 vfs.zfs.zio.dva_throttle_enabled=1 vfs.zfs.zio.exclude_metadata=0 vfs.zfs.zio.use_uma=1 vfs.zfs.zil_slog_bulk=786432 vfs.zfs.cache_flush_disable=0 vfs.zfs.zil_replay_disable=0 vfs.zfs.version.zpl=5 vfs.zfs.version.spa=5000 vfs.zfs.version.acl=1 vfs.zfs.version.ioctl=7 vfs.zfs.debug=0 vfs.zfs.super_owner=0 vfs.zfs.immediate_write_sz=32768 vfs.zfs.min_auto_ashift=12 vfs.zfs.max_auto_ashift=13 vfs.zfs.vdev.queue_depth_pct=1000 vfs.zfs.vdev.write_gap_limit=4096 vfs.zfs.vdev.read_gap_limit=32768 vfs.zfs.vdev.aggregation_limit=1048576 vfs.zfs.vdev.trim_max_active=64 vfs.zfs.vdev.trim_min_active=1 vfs.zfs.vdev.scrub_max_active=2 vfs.zfs.vdev.scrub_min_active=1 vfs.zfs.vdev.async_write_max_active=10 vfs.zfs.vdev.async_write_min_active=1 vfs.zfs.vdev.async_read_max_active=3 vfs.zfs.vdev.async_read_min_active=1 vfs.zfs.vdev.sync_write_max_active=10 vfs.zfs.vdev.sync_write_min_active=10 vfs.zfs.vdev.sync_read_max_active=10 vfs.zfs.vdev.sync_read_min_active=10 vfs.zfs.vdev.max_active=1000 vfs.zfs.vdev.async_write_active_max_dirty_percent=60 vfs.zfs.vdev.async_write_active_min_dirty_percent=30 vfs.zfs.vdev.mirror.non_rotating_seek_inc=1 vfs.zfs.vdev.mirror.non_rotating_inc=0 vfs.zfs.vdev.mirror.rotating_seek_offset=1048576 vfs.zfs.vdev.mirror.rotating_seek_inc=5 vfs.zfs.vdev.mirror.rotating_inc=0 vfs.zfs.vdev.trim_on_init=1 vfs.zfs.vdev.bio_delete_disable=0 vfs.zfs.vdev.bio_flush_disable=0 vfs.zfs.vdev.cache.bshift=16 vfs.zfs.vdev.cache.size=0 vfs.zfs.vdev.cache.max=16384 vfs.zfs.vdev.metaslabs_per_vdev=200 vfs.zfs.vdev.trim_max_pending=10000 vfs.zfs.txg.timeout=5 vfs.zfs.trim.enabled=1 vfs.zfs.trim.max_interval=1 vfs.zfs.trim.timeout=30 vfs.zfs.trim.txg_delay=32 vfs.zfs.space_map_blksz=4096 vfs.zfs.spa_min_slop=134217728 vfs.zfs.spa_slop_shift=5 vfs.zfs.spa_asize_inflation=24 vfs.zfs.deadman_enabled=1 vfs.zfs.deadman_checktime_ms=5000 vfs.zfs.deadman_synctime_ms=1000000 vfs.zfs.debug_flags=0 vfs.zfs.debugflags=0 vfs.zfs.recover=0 vfs.zfs.spa_load_verify_data=1 vfs.zfs.spa_load_verify_metadata=1 vfs.zfs.spa_load_verify_maxinflight=10000 vfs.zfs.ccw_retry_interval=300 vfs.zfs.check_hostid=1 vfs.zfs.mg_fragmentation_threshold=85 vfs.zfs.mg_noalloc_threshold=0 vfs.zfs.condense_pct=200 vfs.zfs.metaslab.bias_enabled=1 vfs.zfs.metaslab.lba_weighting_enabled=1 vfs.zfs.metaslab.fragmentation_factor_enabled=1 vfs.zfs.metaslab.preload_enabled=1 vfs.zfs.metaslab.preload_limit=3 vfs.zfs.metaslab.unload_delay=8 vfs.zfs.metaslab.load_pct=50 vfs.zfs.metaslab.min_alloc_size=33554432 vfs.zfs.metaslab.df_free_pct=4 vfs.zfs.metaslab.df_alloc_threshold=131072 vfs.zfs.metaslab.debug_unload=0 vfs.zfs.metaslab.debug_load=0 vfs.zfs.metaslab.fragmentation_threshold=70 vfs.zfs.metaslab.gang_bang=16777217 vfs.zfs.free_bpobj_enabled=1 vfs.zfs.free_max_blocks=18446744073709551615 vfs.zfs.zfs_scan_checkpoint_interval=7200 vfs.zfs.zfs_scan_legacy=0 vfs.zfs.no_scrub_prefetch=0 vfs.zfs.no_scrub_io=0 vfs.zfs.resilver_min_time_ms=3000 vfs.zfs.free_min_time_ms=1000 vfs.zfs.scan_min_time_ms=1000 vfs.zfs.scan_idle=50 vfs.zfs.scrub_delay=4 vfs.zfs.resilver_delay=2 vfs.zfs.top_maxinflight=32 vfs.zfs.delay_scale=500000 vfs.zfs.delay_min_dirty_percent=60 vfs.zfs.dirty_data_sync=67108864 vfs.zfs.dirty_data_max_percent=10 vfs.zfs.dirty_data_max_max=4294967296 vfs.zfs.dirty_data_max=4294967296 vfs.zfs.max_recordsize=1048576 vfs.zfs.default_ibs=17 vfs.zfs.default_bs=9 vfs.zfs.zfetch.array_rd_sz=1048576 vfs.zfs.zfetch.max_idistance=67108864 vfs.zfs.zfetch.max_distance=33554432 vfs.zfs.zfetch.min_sec_reap=2 vfs.zfs.zfetch.max_streams=8 vfs.zfs.prefetch_disable=0 vfs.zfs.send_holes_without_birth_time=1 vfs.zfs.mdcomp_disable=0 vfs.zfs.per_txg_dirty_frees_percent=30 vfs.zfs.nopwrite_enabled=1 vfs.zfs.dedup.prefetch=1 vfs.zfs.dbuf_cache_lowater_pct=10 vfs.zfs.dbuf_cache_hiwater_pct=10 vfs.zfs.dbuf_cache_shift=5 vfs.zfs.dbuf_cache_max_bytes=632851840 vfs.zfs.arc_min_prescient_prefetch_ms=6 vfs.zfs.arc_min_prfetch_ms=1 vfs.zfs.l2c_only_size=0 vfs.zfs.mfu_ghost_data_esize=0 vfs.zfs.mfu_ghost_metadata_esize=0 vfs.zfs.mfu_ghost_size=0 vfs.zfs.mfu_data_esize=1013981184 vfs.zfs.mfu_metadata_esize=1543413248 vfs.zfs.mfu_size=3404158976 vfs.zfs.mru_ghost_data_esize=0 vfs.zfs.mru_ghost_metadata_esize=0 vfs.zfs.mru_ghost_size=0 vfs.zfs.mru_data_esize=13438197760 vfs.zfs.mru_metadata_esize=208108544 vfs.zfs.mru_size=14097149440 vfs.zfs.anon_data_esize=0 vfs.zfs.anon_metadata_esize=0 vfs.zfs.anon_size=2409472 vfs.zfs.l2arc_norw=0 vfs.zfs.l2arc_feed_again=1 vfs.zfs.l2arc_noprefetch=0 vfs.zfs.l2arc_feed_min_ms=200 vfs.zfs.l2arc_feed_secs=1 vfs.zfs.l2arc_headroom=2 vfs.zfs.l2arc_write_boost=40000000 vfs.zfs.l2arc_write_max=10000000 vfs.zfs.arc_meta_limit=5062814720 vfs.zfs.arc_free_target=113212 vfs.zfs.compressed_arc_enabled=1 vfs.zfs.arc_grow_retry=60 vfs.zfs.arc_shrink_shift=7 vfs.zfs.arc_average_blocksize=8192 vfs.zfs.arc_no_grow_shift=5 vfs.zfs.arc_min=2531407360 vfs.zfs.arc_max=20251258880 vm.kmem_size=21325000704 vm.kmem_size_scale=1 vm.kmem_size_min=0 vm.kmem_size_max=1319413950874 ------------------------------------------------------------------------