I recently upgraded my Z1 Pool to 4 20TB Seagate Exos drives (manufacture recertified).
Over the past few weeks, I would occasionally get a fault amongst the drives. I've been trying to look for solutions but have been unable to find any with my limited knowledge and experience. Another drive just faulted again so I've been getting a bit worried. The drives have all passed the SMART long test with no errors (I think), and rebooting/offlining the drive promptly dismisses the fault. I learned besides smartctl, I can also check for possible errors with dmesg; from what I see, it could be a possible issue with the controller I added when I bought the drives: I added a new Lsi Logic 179356 Controller Card 05-25703-00 9305-16i with these cables in order to connect more drives and used those instead of the sata cables directly to motherboard I was using before. I will try replacing the 4 connectors with sata cables directly to the motherboard for now and hopefully that would be the end of it, but I'm worried that multiple drives will fault at the same time causing me to lose all my data (if that's how Z1 works I think) if the issue isn't with the LSI card (If it is, I would assume I just got a faulty card and would need to replace it?).
In the meanwhile, could anyone who has better expertise sift through these logs and help me find out what's causing it?
zpool status -v (finally learned this command to check the status before clearing the fault to see the issue):
smartctl -a /dev/da2 (drive that faulted this time, *SMART test was ran before fault but recently. Previously another one of the drives faulted and after running a SMART long it also concluded with "No Errors Logged":
dmesg (Think the important bits are at the bottom):
Over the past few weeks, I would occasionally get a fault amongst the drives. I've been trying to look for solutions but have been unable to find any with my limited knowledge and experience. Another drive just faulted again so I've been getting a bit worried. The drives have all passed the SMART long test with no errors (I think), and rebooting/offlining the drive promptly dismisses the fault. I learned besides smartctl, I can also check for possible errors with dmesg; from what I see, it could be a possible issue with the controller I added when I bought the drives: I added a new Lsi Logic 179356 Controller Card 05-25703-00 9305-16i with these cables in order to connect more drives and used those instead of the sata cables directly to motherboard I was using before. I will try replacing the 4 connectors with sata cables directly to the motherboard for now and hopefully that would be the end of it, but I'm worried that multiple drives will fault at the same time causing me to lose all my data (if that's how Z1 works I think) if the issue isn't with the LSI card (If it is, I would assume I just got a faulty card and would need to replace it?).
In the meanwhile, could anyone who has better expertise sift through these logs and help me find out what's causing it?
zpool status -v (finally learned this command to check the status before clearing the fault to see the issue):
Code:
root@ManasMk2[~]# zpool status -v
pool: Main
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: resilvered 22.1G in 00:03:08 with 0 errors on Tue Feb 13 18:45:12 2024
config:
NAME STATE READ WRITE CKSUM
Main DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
gptid/126e142b-c4b3-11ee-8111-f02f74210da7 ONLINE 0 0 0
gptid/9e7fcbbe-c6f9-11ee-90e4-f02f74210da7 FAULTED 3 1.29K 0 too many errors
gptid/0f9709c1-c52b-11ee-aaf5-f02f74210da7 ONLINE 0 0 0
gptid/b1c4bb7b-c5df-11ee-90e4-f02f74210da7 ONLINE 0 0 0
cache
gptid/323800c9-5180-11ec-8ffc-6805cac949fd ONLINE 0 0 0
errors: No known data errors
pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:00:05 with 0 errors on Sun Feb 25 03:45:05 2024
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
nvd0p2 ONLINE 0 0 0
errors: No known data errorssmartctl -a /dev/da2 (drive that faulted this time, *SMART test was ran before fault but recently. Previously another one of the drives faulted and after running a SMART long it also concluded with "No Errors Logged":
Code:
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: ST20000NM007D-3DJ103
Serial Number: ZVT7HAVG
LU WWN Device Id: 5 000c50 0e643cac0
Firmware Version: SN03
User Capacity: 20,000,588,955,648 bytes [20.0 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-4 (minor revision not indicated)
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Feb 25 17:06:06 2024 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 559) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: (1725) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 083 064 044 Pre-fail Always - 201892048
3 Spin_Up_Time 0x0003 097 097 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 3
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 078 061 045 Pre-fail Always - 64139021
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 465
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 3
18 Unknown_Attribute 0x000b 100 100 050 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 059 048 000 Old_age Always - 41 (Min/Max 35/47)
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 2
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 44
194 Temperature_Celsius 0x0022 041 051 000 Old_age Always - 41 (0 24 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0023 100 100 001 Pre-fail Always - 0
240 Head_Flying_Hours 0x0000 100 100 000 Old_age Offline - 451 (204 65 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 39743673234
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 66243051233
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 130 -
# 2 Extended offline Interrupted (host reset) 70% 99 -
# 3 Short offline Completed without error 00% 58 -
# 4 Extended offline Interrupted (host reset) 90% 56 -
# 5 Conveyance offline Completed without error 00% 26 -
# 6 Short offline Completed without error 00% 0 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
clear [-nF] <pool> [device]dmesg (Think the important bits are at the bottom):
Code:
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE-p9 n245429-296d095698e TRUENAS amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: AMD Ryzen 7 5800X 8-Core Processor (3793.08-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0xa20f10 Family=0x19 Model=0x21 Stepping=0
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
AMD Features2=0x75c237ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX,ADMSKX>
Structured Extended Features=0x219c97a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA>
Structured Extended Features2=0x40068c<UMIP,PKU,VAES,VPCLMULQDQ,RDPID>
Structured Extended Features3=0x10<FSRM>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
AMD Extended Feature Extensions ID EBX=0x111ef657<CLZERO,IRPerf,XSaveErPtr,RDPRU,WBNOINVD,IBPB,IBRS,STIBP,STIBP_ALWAYSON,PREFER_IBRS,SSBD>
SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
TSC: P-state invariant, performance statistics
real memory = 68719476736 (65536 MB)
avail memory = 66715742208 (63625 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 1 package(s) x 8 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.1> irqs 0-23
ioapic1 <Version 2.1> irqs 24-55
Launching APs: 1 13 11 6 8 4 3 15 12 2 5 9 10 7 14
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
padlock0: No ACE support.
acpi0: <ALASKA A M I >
acpi0: Power Button (fixed)
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 350
Event timer "HPET2" frequency 14318180 Hz quality 350
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
amdsmn0: <AMD Family 19h System Management Network> on hostb0
amdtemp0: <AMD CPU On-Die Thermal Sensors> on hostb0
pci0: <base peripheral, IOMMU> at device 0.2 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 1.1 on pci0
pci1: <ACPI PCI bus> on pcib1
nvme0: <Generic NVMe Device> mem 0xfcb00000-0xfcb03fff at device 0.0 on pci1
pcib2: <ACPI PCI-PCI bridge> at device 1.2 on pci0
pci2: <ACPI PCI bus> on pcib2
xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfc7a0000-0xfc7a7fff at device 0.0 on pci2
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ahci0: <AHCI SATA controller> mem 0xfc780000-0xfc79ffff at device 0.1 on pci2
ahci0: AHCI v1.31 with 6 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
mpr0: <Avago Technologies (LSI) SAS3224> port 0xf000-0xf0ff mem 0xfc500000-0xfc50ffff at device 0.0 on pci4
mpr0: Firmware: 09.00.100.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
pcib5: <ACPI PCI-PCI bridge> at device 1.0 on pci3
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 2.0 on pci3
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 3.0 on pci3
pci7: <ACPI PCI bus> on pcib7
igb0: <Intel(R) I210 (Copper)> at device 0.0 on pci7
igb0: Setup of Shared code failed, error -2
igb0: IFDI_ATTACH_PRE failed 6
device_attach: igb0 attach returned 6
pcib8: <ACPI PCI-PCI bridge> at device 8.0 on pci3
pci8: <ACPI PCI bus> on pcib8
pci8: <network> at device 0.0 (no driver attached)
pcib9: <ACPI PCI-PCI bridge> at device 9.0 on pci3
pcib9: failed to allocate initial memory window: 0xfc200000-0xfc3fffff
pci9: <ACPI PCI bus> on pcib9
igc0: <Intel(R) Ethernet Controller I225-V> mem 0xfc300000-0xfc3fffff at device0.0 on pci9
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: 0x4000 bytes of rid 0x1c res 3 failed (0, 0xffffffffffffffff).
igc0: Unable to map MSI-X table
igc0: Using an MSI interrupt
igc0: Ethernet address: f0:2f:74:21:0d:a7
pcib10: <ACPI PCI-PCI bridge> at device 3.1 on pci0
pci10: <ACPI PCI bus> on pcib10
vgapci0: <VGA-compatible display> port 0xe000-0xe07f mem 0xfb000000-0xfbffffff,0xd0000000-0xdfffffff,0xe0000000-0xe1ffffff at device 0.0 on pci10
pci10: <multimedia, HDA> at device 0.1 (no driver attached)
xhci1: <NVIDIA TU106 USB 3.1 controller> mem 0xe2000000-0xe203ffff,0xe2040000-0xe204ffff at device 0.2 on pci10
xhci1: 64 bytes context size, 64-bit DMA
usbus1 on xhci1
usbus1: 5.0Gbps Super Speed USB v3.0
pci10: <serial bus> at device 0.3 (no driver attached)
pcib11: <ACPI PCI-PCI bridge> at device 7.1 on pci0
pci11: <ACPI PCI bus> on pcib11
pci11: <unknown> at device 0.0 (no driver attached)
pcib12: <ACPI PCI-PCI bridge> at device 8.1 on pci0
pci12: <ACPI PCI bus> on pcib12
pci12: <unknown> at device 0.0 (no driver attached)
pci12: <encrypt/decrypt> at device 0.1 (no driver attached)
xhci2: <AMD Matisse USB 3.0 controller> mem 0xfc800000-0xfc8fffff at device 0.3on pci12
xhci2: 64 bytes context size, 64-bit DMA
usbus2 on xhci2
usbus2: 5.0Gbps Super Speed USB v3.0
pci12: <multimedia, HDA> at device 0.4 (no driver attached)
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
acpi_button0: <Power Button> on acpi0
acpi_syscontainer0: <System Container> on acpi0
cpu0: <ACPI CPU> on acpi0
amdsbwd0: <AMD FCH Rev 41h+ Watchdog Timer> at iomem 0xfed80b00-0xfed80b03,0xfed80b04-0xfed80b07 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> at port 0x3f8 irq 4 flags 0x10 on isa0
hwpstate0: <Cool`n'Quiet 2.0> on cpu0
Timecounter "TSC-low" frequency 1896436841 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
ugen2.1: <AMD XHCI root HUB> at usbus2
ugen0.1: <AMD XHCI root HUB> at usbus0
ugen1.1: <(0x10de) XHCI root HUB> at usbus1
uhub0 on usbus2
uhub0: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2
uhub1 on usbus1
uhub1: <(0x10de) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
uhub2 on usbus0
uhub2: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
nvd0: <Samsung SSD 970 EVO Plus 500GB> NVMe namespace
nvd0: 476940MB (976773168 512 byte sectors)
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001f> enclosureHandle<0x0001> slot 11
mpr0: At enclosure level 0 and connector name ( )
igb0: <Intel(R) I210 (Copper)> mem 0xfc200000-0xfc2fffff at device 0.0 on pci7
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001c> enclosureHandle<0x0001> slot 10
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001e> enclosureHandle<0x0001> slot 8
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001d> enclosureHandle<0x0001> slot 9
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001a> enclosureHandle<0x0001> slot 15
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x0019> enclosureHandle<0x0001> slot 14
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x0020> enclosureHandle<0x0001> slot 12
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x001b> enclosureHandle<0x0001> slot 13
mpr0: At enclosure level 0 and connector name ( )
igb0: EEPROM V3.30-0 eTrack 0x800007a9
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 4 RX queues 4 TX queues
igb0: Using MSI-X interrupts with 5 vectors
igb0: Ethernet address: 68:05:ca:c9:49:fd
ada0 at ahcich4 bus 0 scbus4 target 0 lun 0
ada0: <Samsung SSD 870 EVO 1TB SVT01B6Q> ACS-4 ATA SATA 3.x device
ada0: Serial Number S625NZ0R921874X
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 953869MB (1953525168 512 byte sectors)
cd0 at ahcich1 bus 0 scbus1 target 0 lun 0
cd0: <HL-DT-ST BD-RE WH10LS30 1.00> Removable CD-ROM SCSI device
cd0: Serial Number K92A5DL2743
cd0: 150.000MB/s transfers (SATA 1.x, UDMA6, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
da2 at mpr0 bus 0 scbus6 target 10 lun 0
da2: <ATA ST20000NM007D-3D SN03> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number ZVT7HAVG
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 19074048MB (39063650304 512 byte sectors)
da5 at mpr0 bus 0 scbus6 target 13 lun 0
da5: <ATA ST8000NE001-2M71 EN01> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number WSD09H62
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 7630885MB (15628053168 512 byte sectors)
da6 at mpr0 bus 0 scbus6 target 14 lun 0
da6: <ATA ST8000NE001-2M71 EN01> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number WSD0EWX2
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 7630885MB (15628053168 512 byte sectors)
da4 at mpr0 bus 0 scbus6 target 12 lun 0
da4: <ATA ST16000NT001-3LV EN01> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number ZR70WVNJ
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 15259648MB (31251759104 512 byte sectors)
da7 at mpr0 bus 0 scbus6 target 15 lun 0
da7: <ATA ST8000NE001-2M71 EN01> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number WSD0P20Y
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 7630885MB (15628053168 512 byte sectors)
da0 at mpr0 bus 0 scbus6 target 8 lun 0
da0: <ATA ST20000NM007D-3D SN03> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number ZVT71GXE
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 19074048MB (39063650304 512 byte sectors)
da1 at mpr0 bus 0 scbus6 target 9 lun 0
da1: <ATA ST20000NM007D-3D SN03> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number ZVTALWBF
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 19074048MB (39063650304 512 byte sectors)
da3 at mpr0 bus 0 scbus6 target 11 lun 0
da3: <ATA ST20000NM007D-3D SN03> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number ZVT6K4BS
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 19074048MB (39063650304 512 byte sectors)
Trying to mount root from zfs:boot-pool/ROOT/13.0-U6.1 []...
uhub1: 6 ports with 6 removable, self powered
uhub0: 8 ports with 8 removable, self powered
uhub2: 14 ports with 14 removable, self powered
ugen0.2: <vendor 0x8087 product 0x0029> at usbus0
ugen0.3: <AsusTek Computer Inc. AURA LED Controller> at usbus0
uhid0 on uhub2
uhid0: <AsusTek Computer Inc. AURA LED Controller, class 0/0, rev 2.00/1.00, addr 2> on usbus0
Root mount waiting for: usbus0
ugen0.4: <vendor 0x05e3 USB2.0 Hub> at usbus0
uhub3 on uhub2
uhub3: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/60.60, addr 3> on usbus0
uhub3: MTT enabled
Root mount waiting for: usbus0
uhub3: 4 ports with 4 removable, self powered
ugen0.5: <SeaSonic. Usb2I2C HID Device> at usbus0
uhid1 on uhub3
uhid1: <SeaSonic. Usb2I2C HID Device, class 0/0, rev 2.00/1.00, addr 4> on usbus0
intsmb0: <AMD FCH SMBus Controller> at device 20.0 on pci0
smbus0: <System Management Bus> on intsmb0
lo0: link state changed to UP
SVM: disabled by BIOS.
module_register_init: MOD_LOAD (vmm, 0xffffffff83604200, 0) error 6
amdiommu0 at device 0.2 on pci0
AMD-Vi: IVRS Info VAsize = 64 PAsize = 48 GVAsize = 2 flags:0
ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0
ivhd0: Flag:b0<IotlbSup,Coherent>
ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0
ivhd0: Extended features[31:0]:22294ade<PPRSup,<b2>,NXSup,GTSup,IASup,GASup,PCSup> HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2
ivhd0: Extended features[62:32]:58f77ef<USSup,PprOvrflwEarlySup,PPRAutoRspSup,BlKStopMrkSup,PerfOptSup,MsiCapMmioSup,GIOSup,HASup,EPHSup,AttrFWSup> Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1
ivhd0: supported paging level:7, will use only: 4
ivhd0: device [0x8 - 0xfffe] config:0
ivhd0: device [0xff00 - 0xffff] config:0
ivhd0: PCI cap 0x190b640f@0x40 feature:19<IOTLB,EFR,CapExt>
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp 0xfffff800031c0800), ignoring.
GEOM_MIRROR: Device mirror/swap0 launched (2/2).
GEOM_MIRROR: Device mirror/swap1 launched (2/2).
GEOM_ELI: Device mirror/swap0.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: accelerated software
GEOM_ELI: Device mirror/swap1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: accelerated software
CPU: AMD Ryzen 7 5800X 8-Core Processor (3792.87-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0xa20f10 Family=0x19 Model=0x21 Stepping=0
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
AMD Features2=0x75c237ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX,ADMSKX>
Structured Extended Features=0x219c97a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA>
Structured Extended Features2=0x40069c<UMIP,PKU,OSPKE,VAES,VPCLMULQDQ,RDPID>
Structured Extended Features3=0x10<FSRM>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
AMD Extended Feature Extensions ID EBX=0x111ef657<CLZERO,IRPerf,XSaveErPtr,RDPRU,WBNOINVD,IBPB,IBRS,STIBP,STIBP_ALWAYSON,PREFER_IBRS,SSBD>
SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
TSC: P-state invariant, performance statistics
igb0: link state changed to DOWN
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> K8/16/48/0x1ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA>
nfsd: can't register svc name
Security policy loaded: MAC/ntpd (mac_ntpd)
igb0: link state changed to UP
bridge0: Ethernet address: 58:9c:fc:10:ff:cb
epair0a: Ethernet address: 02:c0:f8:9e:7e:0a
epair0b: Ethernet address: 02:c0:f8:9e:7e:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: changing name to 'vnet0.1'
igb0: link state changed to DOWN
bridge0: link state changed to UP
igb0: promiscuous mode enabled
vnet0.1: promiscuous mode enabled
lo0: link state changed to UP
igb0: link state changed to UP
epair1a: Ethernet address: 02:63:41:82:41:0a
epair1b: Ethernet address: 02:63:41:82:41:0b
epair1a: link state changed to UP
epair1b: link state changed to UP
epair1a: changing name to 'vnet0.2'
epair1b: changing name to 'epair0b'
vnet0.2: promiscuous mode enabled
lo0: link state changed to UP
epair2a: Ethernet address: 02:7a:63:48:2d:0a
epair2b: Ethernet address: 02:7a:63:48:2d:0b
epair2a: link state changed to UP
epair2b: link state changed to UP
epair2a: changing name to 'vnet0.3'
epair2b: changing name to 'epair0b'
igb0: link state changed to DOWN
lo0: link state changed to UP
igb0: link state changed to UP
epair3a: Ethernet address: 02:27:80:b5:d8:0a
epair3b: Ethernet address: 02:27:80:b5:d8:0b
epair3a: link state changed to UP
epair3b: link state changed to UP
epair3a: changing name to 'vnet0.4'
epair3b: changing name to 'epair0b'
vnet0.4: promiscuous mode enabled
lo0: link state changed to UP
epair4a: Ethernet address: 02:3b:a9:a4:27:0a
epair4b: Ethernet address: 02:3b:a9:a4:27:0b
epair4a: link state changed to UP
epair4b: link state changed to UP
epair4a: changing name to 'vnet0.5'
epair4b: changing name to 'epair0b'
vnet0.5: promiscuous mode enabled
lo0: link state changed to UP
sonewconn: pcb 0xfffff8048f9947c0 ([::]:32400 (proto 6)): Listen queue overflow: 193 already in queue awaiting acceptance (1 occurrences)
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 04 28 7d c3 68 00 00 00 48 00 00 length 36864 SMID 102 Command timeout on target 10(0x001c), 60000 set, 60.753994427 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
mpr0: Sending abort to target 10 for SMID 102
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 04 28 7d c3 68 00 00 00 48 00 00 length 36864 SMID 102 Aborting command 0xfffffe015b25a5d0
(da2:mpr0:0:10:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 length 0 SMID 1835 Command timeout on target 10(0x001c), 60000 set, 60.754585751 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 38 48 00 00 00 58 00 00 length 45056 SMID 1318 Command timeout on target 10(0x001c), 60000 set, 60.77998205 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 f0 00 00 00 58 00 00 length 45056 SMID 1741 Command timeout on target 10(0x001c), 60000 set, 60.78174586 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 98 00 00 00 58 00 00 length 45056 SMID 1215 Command timeout on target 10(0x001c), 60000 set, 60.78359789 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 c0 3d 18 00 00 00 28 00 00 length 20480 SMID 1538 Command timeout on target 10(0x001c), 60000 set, 60.78489525 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 80 00 00 00 18 00 00 length 12288 SMID 633 Command timeout on target 10(0x001c), 60000 set, 60.78637550 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 78 00 00 00 08 00 00 length 4096 SMID 565 Command timeout on target 10(0x001c), 60000 set, 60.78721329 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 38 a0 00 00 00 58 00 00 length 45056 SMID 2020 Command timeout on target 10(0x001c), 60000 set, 60.73963892 elapsed
mpr0: At enclosure level 0, slot 10, connector name ( )
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1567 loginfo 31130000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 2020 loginfo 31130000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1318 loginfo 31130000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1741 loginfo 31130000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1215 loginfo 31130000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1538 loginfo 31130000
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 32 80 00 00 00 18 0000
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 633 loginfo 31130000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 565 loginfo 31130000
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
mpr0: Controller reported scsi ioc terminated tgt 10 SMID 1835 loginfo 31130000
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 38 a0 00 00 00 58 0000
mpr0: Finished abort recovery for target 10
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 38 48 00 00 00 58 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 f0 00 00 00 58 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 98 00 00 00 58 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 c0 3d 18 00 00 00 28 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 80 00 00 00 18 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 03 d0 fa 37 78 00 00 00 08 0000
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): WRITE(16). CDB: 8a 00 00 00 00 04 28 7d c3 68 00 00 00 48 0000
(da2:mpr0:0:10:0): CAM status: Command timeout
(da2:mpr0:0:10:0): Retrying command, 3 more tries remain
(da2:mpr0:0:10:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00
(da2:mpr0:0:10:0): CAM status: CCB request completed with an error
(da2:mpr0:0:10:0): Retrying command, 0 more tries remain
(da2:mpr0:0:10:0): SYNCHRONIZE CACHE(10
