- Joined
- Jan 11, 2021
- Messages
- 415
Meanwhile at Villabajo — things magically dropped over night by 100GB. Never saw anything like this before. Neither with Solaris/OmniOS, nor with freeBSD. The use case is constantly the same: Hourly backups from 3 up to 5 MacOS clients (TimeMachine vis SMB), daily backups from 2 Linux clients (rsync via NFS); daily replication to a remote (vanilla) freeBSD ZFS file server (zfs send via SSH). It's all about scientific data. No DVD-archives, no video editing. And of course: no reboots/service daemon restarts:
This is the corresponding part of the reporting section. Seems awfully wrong:
Code:
root@truenas ~ # arc_summary [0]
------------------------------------------------------------------------
ZFS Subsystem Report Sun Mar 19 15:09:32 2023
FreeBSD 13.1-RELEASE-p7 zpl version 5
Machine: truenas.fritz.box (amd64) spa version 5000
ARC status: HEALTHY
Memory throttle count: 0
ARC size (current): 9.9 % 12.5 GiB
Target size (adaptive): 10.1 % 12.8 GiB
Min size (hard limit): 3.1 % 4.0 GiB
Max size (high water): 31:1 126.9 GiB
Most Frequently Used (MFU) cache size: 37.2 % 4.4 GiB
Most Recently Used (MRU) cache size: 62.8 % 7.5 GiB
Metadata cache size (hard limit): 75.0 % 95.2 GiB
Metadata cache size (current): 1.9 % 1.8 GiB
Dnode cache size (hard limit): 10.0 % 9.5 GiB
Dnode cache size (current): 3.1 % 305.9 MiB
ARC hash breakdown:
Elements max: 2.1M
Elements current: 10.6 % 221.6k
Collisions: 813.6k
Chain max: 5
Chains: 1.5k
ARC misc:
Deleted: 4.1M
Mutex misses: 732
Eviction skips: 12.1M
Eviction skips due to L2 writes: 0
L2 cached evictions: 0 Bytes
L2 eligible evictions: 5.8 GiB
L2 eligible MFU evictions: 59.9 % 3.5 GiB
L2 eligible MRU evictions: 40.1 % 2.3 GiB
L2 ineligible evictions: 593.1 GiB
ARC total accesses (hits + misses): 1.2G
Cache hit ratio: 99.8 % 1.2G
Cache miss ratio: 0.2 % 3.1M
Actual hit ratio (MFU + MRU hits): 99.6 % 1.2G
Data demand efficiency: 98.4 % 25.7M
Data prefetch efficiency: 22.7 % 1.9M
Cache hits by cache type:
Most frequently used (MFU): 98.2 % 1.2G
Most recently used (MRU): 1.7 % 21.2M
Most frequently used (MFU) ghost: < 0.1 % 356.1k
Most recently used (MRU) ghost: 0.1 % 865.4k
Anonymously used: < 0.1 % 193.5k
Cache hits by data type:
Demand data: 2.1 % 25.3M
Prefetch data: < 0.1 % 421.1k
Demand metadata: 92.3 % 1.1G
Prefetch metadata: 5.6 % 68.6M
Cache misses by data type:
Demand data: 13.5 % 414.1k
Prefetch data: 46.6 % 1.4M
Demand metadata: 12.8 % 392.6k
Prefetch metadata: 27.2 % 835.2k
DMU prefetch efficiency: 19.1M
Hit ratio: 27.6 % 5.3M
Miss ratio: 72.4 % 13.8M
L2ARC not detected, skipping section
Tunables:
abd_scatter_enabled 1
abd_scatter_min_size 4097
allow_redacted_dataset_mount 0
anon_data_esize 0
anon_metadata_esize 0
anon_size 416256
arc.average_blocksize 8192
arc.dnode_limit 0
arc.dnode_limit_percent 10
arc.dnode_reduce_percent 10
arc.evict_batch_limit 10
arc.eviction_pct 200
arc.grow_retry 0
arc.lotsfree_percent 10
arc.max 0
arc.meta_adjust_restarts 4096
arc.meta_limit 0
arc.meta_limit_percent 75
arc.meta_min 0
arc.meta_prune 10000
arc.meta_strategy 1
arc.min 0
arc.min_prefetch_ms 0
arc.min_prescient_prefetch_ms 0
arc.p_dampener_disable 1
arc.p_min_shift 0
arc.pc_percent 0
arc.prune_task_threads 1
arc.shrink_shift 0
arc.sys_free 0
arc_free_target 695855
arc_max 0
arc_min 0
arc_no_grow_shift 5
async_block_max_blocks 18446744073709551615
autoimport_disable 1
btree_verify_intensity 0
ccw_retry_interval 300
checksum_events_per_second 20
commit_timeout_pct 5
compressed_arc_enabled 1
condense.indirect_commit_entry_delay_ms 0
condense.indirect_obsolete_pct 25
condense.indirect_vdevs_enable 1
condense.max_obsolete_bytes 1073741824
condense.min_mapping_bytes 131072
condense_pct 200
crypt_sessions 2046165
dbgmsg_enable 1
dbgmsg_maxsize 4194304
dbuf.cache_shift 5
dbuf.metadata_cache_max_bytes 18446744073709551615
dbuf.metadata_cache_shift 6
dbuf_cache.hiwater_pct 10
dbuf_cache.lowater_pct 10
dbuf_cache.max_bytes 18446744073709551615
dbuf_state_index 0
ddt_data_is_special 1
deadman.checktime_ms 60000
deadman.enabled 1
deadman.failmode wait
deadman.synctime_ms 600000
deadman.ziotime_ms 300000
debug 0
debugflags 0
dedup.prefetch 0
default_bs 9
default_ibs 15
delay_min_dirty_percent 60
delay_scale 500000
dirty_data_max 4294967296
dirty_data_max_max 4294967296
dirty_data_max_max_percent 25
dirty_data_max_percent 10
dirty_data_sync_percent 20
disable_ivset_guid_check 0
dmu_object_alloc_chunk_shift 7
dmu_offset_next_sync 1
dmu_prefetch_max 134217728
dtl_sm_blksz 4096
embedded_slog_min_ms 64
flags 0
fletcher_4_impl [fastest] scalar superscalar superscalar4 sse2 ssse3
free_bpobj_enabled 1
free_leak_on_eio 0
free_min_time_ms 1000
history_output_max 1048576
immediate_write_sz 32768
initialize_chunk_size 1048576
initialize_value 16045690984833335022
keep_log_spacemaps_at_export 0
l2arc.exclude_special 0
l2arc.feed_again 1
l2arc.feed_min_ms 200
l2arc.feed_secs 1
l2arc.headroom 2
l2arc.headroom_boost 200
l2arc.meta_percent 33
l2arc.mfuonly 0
l2arc.noprefetch 1
l2arc.norw 0
l2arc.rebuild_blocks_min_l2size 1073741824
l2arc.rebuild_enabled 0
l2arc.trim_ahead 0
l2arc.write_boost 8388608
l2arc.write_max 8388608
l2arc_feed_again 1
l2arc_feed_min_ms 200
l2arc_feed_secs 1
l2arc_headroom 2
l2arc_noprefetch 1
l2arc_norw 0
l2arc_write_boost 8388608
l2arc_write_max 8388608
l2c_only_size 0
livelist.condense.new_alloc 0
livelist.condense.sync_cancel 0
livelist.condense.sync_pause 0
livelist.condense.zthr_cancel 0
livelist.condense.zthr_pause 0
livelist.max_entries 500000
livelist.min_percent_shared 75
lua.max_instrlimit 100000000
lua.max_memlimit 104857600
max_async_dedup_frees 100000
max_auto_ashift 14
max_dataset_nesting 50
max_log_walking 5
max_logsm_summary_length 10
max_missing_tvds 0
max_missing_tvds_cachefile 2
max_missing_tvds_scan 0
max_nvlist_src_size 0
max_recordsize 1048576
metaslab.aliquot 1048576
metaslab.bias_enabled 1
metaslab.debug_load 0
metaslab.debug_unload 0
metaslab.df_alloc_threshold 131072
metaslab.df_free_pct 4
metaslab.df_max_search 16777216
metaslab.df_use_largest_segment 0
metaslab.find_max_tries 100
metaslab.force_ganging 16777217
metaslab.fragmentation_factor_enabled 1
metaslab.fragmentation_threshold 70
metaslab.lba_weighting_enabled 1
metaslab.load_pct 50
metaslab.max_size_cache_sec 3600
metaslab.mem_limit 25
metaslab.preload_enabled 1
metaslab.preload_limit 10
metaslab.segment_weight_enabled 1
metaslab.sm_blksz_no_log 16384
metaslab.sm_blksz_with_log 131072
metaslab.switch_threshold 2
metaslab.try_hard_before_gang 0
metaslab.unload_delay 32
metaslab.unload_delay_ms 600000
mfu_data_esize 3622574080
mfu_ghost_data_esize 0
mfu_ghost_metadata_esize 868868096
mfu_ghost_size 868868096
mfu_metadata_esize 221680640
mfu_size 4772314112
mg.fragmentation_threshold 95
mg.noalloc_threshold 0
min_auto_ashift 9
min_metaslabs_to_flush 1
mru_data_esize 7435692032
mru_ghost_data_esize 2107219456
mru_ghost_metadata_esize 0
mru_ghost_size 2107219456
mru_metadata_esize 173265408
mru_size 8064858112
multihost.fail_intervals 10
multihost.history 0
multihost.import_intervals 20
multihost.interval 1000
multilist_num_sublists 0
no_scrub_io 0
no_scrub_prefetch 0
nocacheflush 0
nopwrite_enabled 1
obsolete_min_time_ms 500
pd_bytes_max 52428800
per_txg_dirty_frees_percent 30
prefetch.array_rd_sz 1048576
prefetch.disable 0
prefetch.max_distance 67108864
prefetch.max_idistance 67108864
prefetch.max_sec_reap 2
prefetch.max_streams 8
prefetch.min_distance 4194304
prefetch.min_sec_reap 1
read_history 0
read_history_hits 0
rebuild_max_segment 1048576
rebuild_scrub_enabled 1
rebuild_vdev_limit 33554432
reconstruct.indirect_combinations_max 4096
recover 0
recv.queue_ff 20
recv.queue_length 16777216
recv.write_batch_size 1048576
removal_suspend_progress 0
remove_max_segment 16777216
resilver_disable_defer 0
resilver_min_time_ms 3000
scan_blkstats 0
scan_checkpoint_intval 7200
scan_fill_weight 3
scan_ignore_errors 0
scan_issue_strategy 0
scan_legacy 0
scan_max_ext_gap 2097152
scan_mem_lim_fact 20
scan_mem_lim_soft_fact 20
scan_strict_mem_lim 0
scan_suspend_progress 0
scan_vdev_limit 4194304
scrub_min_time_ms 1000
send.corrupt_data 0
send.no_prefetch_queue_ff 20
send.no_prefetch_queue_length 1048576
send.override_estimate_recordsize 0
send.queue_ff 20
send.queue_length 16777216
send.unmodified_spill_blocks 1
send_holes_without_birth_time 1
slow_io_events_per_second 20
spa.asize_inflation 24
spa.discard_memory_limit 16777216
spa.load_print_vdev_tree 0
spa.load_verify_data 1
spa.load_verify_metadata 1
spa.load_verify_shift 4
spa.slop_shift 5
space_map_ibs 14
special_class_metadata_reserve_pct 25
standard_sm_blksz 131072
super_owner 0
sync_pass_deferred_free 2
sync_pass_dont_compress 8
sync_pass_rewrite 2
sync_taskq_batch_pct 75
top_maxinflight 1000
traverse_indirect_prefetch_limit 32
trim.extent_bytes_max 134217728
trim.extent_bytes_min 32768
trim.metaslab_skip 0
trim.queue_limit 10
trim.txg_batch 32
txg.history 100
txg.timeout 5
unflushed_log_block_max 131072
unflushed_log_block_min 1000
unflushed_log_block_pct 400
unflushed_log_txg_max 1000
unflushed_max_mem_amt 1073741824
unflushed_max_mem_ppm 1000
user_indirect_is_special 1
validate_skip 0
vdev.aggregate_trim 0
vdev.aggregation_limit 1048576
vdev.aggregation_limit_non_rotating 131072
vdev.async_read_max_active 3
vdev.async_read_min_active 1
vdev.async_write_active_max_dirty_percent 60
vdev.async_write_active_min_dirty_percent 30
vdev.async_write_max_active 5
vdev.async_write_min_active 1
vdev.bio_delete_disable 0
vdev.bio_flush_disable 0
vdev.cache_bshift 16
vdev.cache_max 16384
vdev.cache_size 0
vdev.def_queue_depth 32
vdev.default_ms_count 200
vdev.default_ms_shift 29
vdev.file.logical_ashift 9
vdev.file.physical_ashift 9
vdev.initializing_max_active 1
vdev.initializing_min_active 1
vdev.max_active 1000
vdev.max_auto_ashift 14
vdev.min_auto_ashift 9
vdev.min_ms_count 16
vdev.mirror.non_rotating_inc 0
vdev.mirror.non_rotating_seek_inc 1
vdev.mirror.rotating_inc 0
vdev.mirror.rotating_seek_inc 5
vdev.mirror.rotating_seek_offset 1048576
vdev.ms_count_limit 131072
vdev.nia_credit 5
vdev.nia_delay 5
vdev.queue_depth_pct 1000
vdev.read_gap_limit 32768
vdev.rebuild_max_active 3
vdev.rebuild_min_active 1
vdev.removal_ignore_errors 0
vdev.removal_max_active 2
vdev.removal_max_span 32768
vdev.removal_min_active 1
vdev.removal_suspend_progress 0
vdev.remove_max_segment 16777216
vdev.scrub_max_active 3
vdev.scrub_min_active 1
vdev.sync_read_max_active 10
vdev.sync_read_min_active 10
vdev.sync_write_max_active 10
vdev.sync_write_min_active 10
vdev.trim_max_active 2
vdev.trim_min_active 1
vdev.validate_skip 0
vdev.write_gap_limit 4096
version.acl 1
version.ioctl 15
version.module v2023012500-zfs_9ef0b67f8
version.spa 5000
version.zpl 5
vnops.read_chunk_size 1048576
vol.mode 2
vol.recursive 0
vol.unmap_enabled 1
wrlog_data_max 8589934592
xattr_compat 1
zap_iterate_prefetch 1
zevent.len_max 512
zevent.retain_expire_secs 900
zevent.retain_max 2000
zfetch.max_distance 67108864
zfetch.max_idistance 67108864
zil.clean_taskq_maxalloc 1048576
zil.clean_taskq_minalloc 1024
zil.clean_taskq_nthr_pct 100
zil.maxblocksize 131072
zil.nocacheflush 0
zil.replay_disable 0
zil.slog_bulk 786432
zio.deadman_log_all 0
zio.dva_throttle_enabled 1
zio.exclude_metadata 0
zio.requeue_io_start_cut_in_line 1
zio.slow_io_ms 30000
zio.taskq_batch_pct 80
zio.taskq_batch_tpq 0
zio.use_uma 1
VDEV cache disabled, skipping section
ZIL committed transactions: 8.1M
Commit requests: 1.6M
Flushes to stable storage: 1.5M
Transactions to SLOG storage pool: 187.4 GiB 2.7M
Transactions to non-SLOG storage pool: 70.7 MiB 10.8k
This is the corresponding part of the reporting section. Seems awfully wrong:
Last edited:
