zio read error

Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Rosin0416 said:
maybe one thing that can help:
_ my SSD (OS) is connected to a SATA port
_ my 4 data drives are each connected to a SAS port
Click to expand...
According to supermicro, the SAS controller is a Broadcom 2308.
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
Hello,

@Alecmascot
I managed to find out how to output the dmesg logs. Personally, nothing shocked me, but I don't know how to interpret the logs :
root@truenas[~]# dmesg -a
---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE-p2 n245412-484f039b1d0 TRUENAS amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.20-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 34359738368 (32768 MB)
avail memory = 33316122624 (31772 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23
Launching APs: 1 2 3
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
padlock0: No ACE support.
acpi0: <SUPERM SMCI--MB>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
apei0: <ACPI Platform Error Interface> on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 1.1 on pci0
pci2: <ACPI PCI bus> on pcib2
mps0: <Avago Technologies (LSI) SAS2308> port 0xe000-0xe0ff mem 0xf7240000-0xf724ffff,0xf7200000-0xf723ffff irq 17 at device 0.0 on pci2
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xf7500000-0xf750ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7514000-0xf75143ff irq 16 at device 26.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
usbus1: 480Mbps High Speed USB v2.0
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
vgapci0: <VGA-compatible display> port 0xd000-0xd07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci4
vgapci0: Boot video device
pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci5: <ACPI PCI bus> on pcib5
igb0: <Intel(R) I210 (Copper)> port 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18 at device 0.0 on pci5
igb0: EEPROM V3.16-0 eTrack 0x800004d7
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 2 RX queues 2 TX queues
igb0: Using MSI-X interrupts with 3 vectors
igb0: Ethernet address: 11:77:80:f6:a8:77
pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci6: <ACPI PCI bus> on pcib6
igb1: <Intel(R) I210 (Copper)> port 0xb000-0xb01f mem 0xf7300000-0xf737ffff,0xf7380000-0xf7383fff irq 19 at device 0.0 on pci6
igb1: EEPROM V3.16-0 eTrack 0x800004d9
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 2 RX queues 2 TX queues
igb1: Using MSI-X interrupts with 3 vectors
igb1: Ethernet address: 44:52:09:f7:b8:11
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7513000-0xf75133ff irq 22 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
usbus2: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xf7512000-0xf75127ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 5 on acpi0
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ichwd0: <Intel Lynx Point watchdog timer> on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
ichwd0: <Intel Lynx Point watchdog timer> at port 0x1830-0x1837,0x1860-0x187f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xce000-0xcefff pnpid ORM0000 on isa0
superio0: <Nuvoton NCT6776> at port 0x2e-0x2f on isa0
wbwd0: <Nuvoton NCT6776 (0xc3/0x33) Watchdog Timer> at WDT ldn 0x08 on superio0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
Timecounter "TSC-low" frequency 1699998462 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
ugen2.1: <Intel EHCI root HUB> at usbus2
ugen1.1: <Intel EHCI root HUB> at usbus1
ugen0.1: <Intel XHCI root HUB> at usbus0
uhub0 on usbus2
uhub1 on usbus0
uhub1: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub2 on usbus1
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ipmi0: IPMI device rev. 1, firmware rev. 3.88, version 2.0, device support mask 0xbf
ipmi0: Number of channels 2
ipmi0: Attached watchdog
ipmi0: Establishing power cycle handler
ses0 at ahciem0 bus 0 scbus7 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <SATA SSD HAFEB2.0> ACS-4 ATA SATA 3.x device
ada0: Serial Number D21090881A05636
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 122104MB (250069680 512 byte sectors)
ses0: pass4,ada0 in 'Slot 00', SATA Slot: scbus1 target 0
da0 at mps0 bus 0 scbus0 target 0 lun 0
da0: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number WD-WCC4N4JPZ41B
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 2861588MB (5860533168 512 byte sectors)
da0: quirks=0x8<4K>
da2 at mps0 bus 0 scbus0 target 2 lun 0
da2: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number WD-WCC4N0RSC0IV
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 2861588MB (5860533168 512 byte sectors)
da2: quirks=0x8<4K>
da1 at mps0 bus 0 scbus0 target 1 lun 0
da1: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number WD-WCC4N7LS2CFL
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 2861588MB (5860533168 512 byte sectors)
da1: quirks=0x8<4K>
da3 at mps0 bus 0 scbus0 target 3 lun 0
da3: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number WD-WCC4N5DVH913
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 2861588MB (5860533168 512 byte sectors)
da3: quirks=0x8<4K>
Trying to mount root from zfs:boot-pool/ROOT/default []...
uhub2: 2 ports with 2 removable, self powered
uhub1: 17 ports with 17 removable, self powered
uhub0: 2 ports with 2 removable, self powered
Root mount waiting for: usbus0 usbus1 usbus2
ugen0.2: <vendor 0x0557 product 0x7000> at usbus0
uhub3 on uhub1
uhub3: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 1> on usbus0
ugen1.2: <vendor 0x8087 product 0x8008> at usbus1
uhub4 on uhub2
uhub4: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen2.2: <vendor 0x8087 product 0x8000> at usbus2
uhub5 on uhub0
uhub5: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus2
uhub3: 4 ports with 3 removable, self powered
uhub4: 4 ports with 4 removable, self powered
uhub5: 6 ports with 6 removable, self powered
ugen0.3: <vendor 0x0557 product 0x2419> at usbus0
ukbd0 on uhub3
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd2 at ukbd0
ums0 on uhub3
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
Root mount waiting for: usbus0
ugen0.4: <MGE UPS SYSTEMS Evolution> at usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Starting devd.
Autoloading module: ichsmb
ichsmb0: <Intel Lynx Point SMBus controller> port 0x580-0x59f mem 0xf7511000-0xf75110ff irq 18 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
Starting ums0 moused.
Autoloading module: uhid
Autoloading module: usbhid
Autoloading module: wmt
Starting zfsd.
Setting hostuuid: 00000000-0000-0000-0000-002590f5a654.
Setting hostid: 0x7dd148b3.
lo0: link state changed to UP
<118>middlewared: starting middlewared: loaded plugin middlewared.plugins.service middlewared: loaded plugin middlewared.plugins.ec2 middlewared: loaded plugin middlewared.plugins.cache middlewared: loaded plugin middlewared.plugins.s3 middlewared: loaded plugin middlewared.plugins.rsync middlewared: loaded plugin middlewared.plugins.replication middlewared: loaded plugin middlewared.plugins.ssh middlewared: loaded plugin middlewared.plugins.api_key middlewared: loaded plugin middlewared.plugins.cron middlewared: loaded plugin middlewared.plugins.migration middlewared: loaded plugin middlewared.plugins.ldap middlewared: loaded plugin middlewared.plugins.activedirectomiddlewared: loaded plugin middlewared.plugins.afp middlewared: loaded plugin middlewared.plugins.config middlewared: loaded plugin middlewared.plugins.network middlewared: loaded plugin middlewared.plugins.keychain middlewared: loaded plugin middlewared.plugins.webdav middlewared: loaded plugin middlewared.plugins.stats middlewared: loaded plugin middlewared.plugins.trueview middlewared: loaded plugin middlewared.plugins.directoryservmiddlewared: loaded plugin middlewared.plugins.keyvalue middlewared: loaded plugin middlewared.plugins.jail_freebsd middlewared: loaded plugin middlewared.plugins.etc middlewared: loaded plugin middlewared.plugins.cloud_sync middlewared: loaded plugin middlewared.plugins.smb middlewared: loaded plugin middlewared.plugins.disk middlewared: loaded plugin middlewared.plugins.geom middlewared: loaded plugin middlewared.plugins.device middlewared: loaded plugin middlewared.plugins.smart middlewared: loaded plugin middlewared.plugins.update middlewared: loaded plugin middlewared.plugins.kerberos middlewared: loaded plugin middlewared.plugins.usage middlewared: loaded plugin middlewared.plugins.snmp middlewared: loaded plugin middlewared.plugins.auth middlewared: loaded plugin middlewared.plugins.sysdataset middlewared: loaded plugin middlewared.plugins.acme_protocolmiddlewared: loaded plugin middlewared.plugins.nfs middlewared: loaded plugin middlewared.plugins.ftp middlewared: loaded plugin middlewared.plugins.filesystem middlewared: loaded plugin middlewared.plugins.support middlewared: loaded plugin middlewared.plugins.bootenv middlewared: loaded plugin middlewared.plugins.boot middlewared: loaded plugin middlewared.plugins.tunables middlewared: loaded plugin middlewared.plugins.ups middlewared: loaded plugin middlewared.plugins.dyndns middlewared: loaded plugin middlewared.plugins.account middlewared: loaded plugin middlewared.plugins.multipath middlewared: loaded plugin middlewared.plugins.unscheduled_rmiddlewared: loaded plugin middlewared.plugins.system middlewared: loaded plugin middlewared.plugins.lldp middlewared: loaded plugin middlewared.plugins.hardware middlewared: loaded plugin middlewared.plugins.ntp middlewared: loaded plugin middlewared.plugins.zettarepl middlewared: loaded plugin middlewared.plugins.alert middlewared: loaded plugin middlewared.plugins.ipmi middlewared: loaded plugin middlewared.plugins.zfs middlewared: loaded plugin middlewared.plugins.crypto middlewared: loaded plugin middlewared.plugins.vpn middlewared: loaded plugin middlewared.plugins.idmap middlewared: loaded plugin middlewared.plugins.pwenc middlewared: loaded plugin middlewared.plugins middlewared: loaded plugin middlewared.plugins.vmware middlewared: loaded plugin middlewared.plugins.webui middlewared: loaded plugin middlewared.plugins.nis middlewared: loaded plugin middlewared.plugins.snapshot middlewared: loaded plugin middlewared.plugins.tftp middlewared: loaded plugin middlewared.plugins.pool middlewared: loaded plugin middlewared.plugins.iscsi middlewared: loaded plugin middlewared.plugins.vm middlewared: loaded plugin middlewared.plugins.init_shutdownmiddlewared: loaded plugin middlewared.plugins.mail middlewared: loaded plugin middlewared.plugins.hardware_.mcamiddlewared: loaded plugin middlewared.plugins.hardware_ middlewared: loaded plugin middlewared.plugins.disk_.overpromiddlewared: loaded plugin middlewared.plugins.disk_.remove_middlewared: loaded plugin middlewared.plugins.disk_ middlewared: loaded plugin middlewared.plugins.disk_.temperamiddlewared: loaded plugin middlewared.plugins.disk_.smartctmiddlewared: loaded plugin middlewared.plugins.disk_.identifmiddlewared: loaded plugin middlewared.plugins.disk_.dumpdevmiddlewared: loaded plugin middlewared.plugins.disk_.encryptmiddlewared: loaded plugin middlewared.plugins.disk_.disk_inmiddlewared: loaded plugin middlewared.plugins.disk_.swap_mimiddlewared: loaded plugin middlewared.plugins.disk_.smart_amiddlewared: loaded plugin middlewared.plugins.disk_.format middlewared: loaded plugin middlewared.plugins.disk_.zfs_guimiddlewared: loaded plugin middlewared.plugins.disk_.availabmiddlewared: loaded plugin middlewared.plugins.disk_.disk_evmiddlewared: loaded plugin middlewared.plugins.disk_.wipe middlewared: loaded plugin middlewared.plugins.disk_.swap_comiddlewared: loaded plugin middlewared.plugins.disk_.sync middlewared: loaded plugin middlewared.plugins.disk_.nvme_frmiddlewared: loaded plugin middlewared.plugins.disk_.swap_remiddlewared: loaded plugin middlewared.plugins.disk_.hddstanmiddlewared: loaded plugin middlewared.plugins.disk_.info middlewared: loaded plugin middlewared.plugins.disk_.smart middlewared: loaded plugin middlewared.plugins.disk_.label_fmiddlewared: loaded plugin middlewared.plugins.mail_.gmail middlewared: loaded plugin middlewared.plugins.mail_ middlewared: loaded plugin middlewared.plugins.system_.dmi middlewared: loaded plugin middlewared.plugins.system_ middlewared: loaded plugin middlewared.plugins.service_ middlewared: loaded plugin middlewared.plugins.zettarepl_.utmiddlewared: loaded plugin middlewared.plugins.zettarepl_.stmiddlewared: loaded plugin middlewared.plugins.zettarepl_ middlewared: loaded plugin middlewared.plugins.interface.capmiddlewared: loaded plugin middlewared.plugins.interface.typmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface middlewared: loaded plugin middlewared.plugins.interface.dhcmiddlewared: loaded plugin middlewared.plugins.interface.vlamiddlewared: loaded plugin middlewared.plugins.interface.netmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.brimiddlewared: loaded plugin middlewared.plugins.interface.linmiddlewared: loaded plugin middlewared.plugins.zfs_.events middlewared: loaded plugin middlewared.plugins.zfs_.disks_frmiddlewared: loaded plugin middlewared.plugins.zfs_ middlewared: loaded plugin middlewared.plugins.zfs_.zfs_evenmiddlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.pool_.replacemiddlewared: loaded plugin middlewared.plugins.pool_.attach_middlewared: loaded plugin middlewared.plugins.pool_.expand middlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.info middlewared: loaded plugin middlewared.plugins.pool_.format_middlewared: loaded plugin middlewared.plugins.pool_ middlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.enterprise middlewared: loaded plugin middlewared.plugins.enterprise.m_middlewared: loaded plugin middlewared.plugins.nfs_.sec middlewared: loaded plugin middlewared.plugins.nfs_.v4_freebmiddlewared: loaded plugin middlewared.plugins.nfs_ middlewared: loaded plugin middlewared.plugins.nfs_.krb5 middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_ middlewared: loaded plugin middlewared.plugins.boot_.format middlewared: loaded plugin middlewared.plugins.boot_.boot_lomiddlewared: loaded plugin middlewared.plugins.boot_ middlewared: loaded plugin middlewared.plugins.geom_ middlewared: loaded plugin middlewared.plugins.geom_.cache middlewared: loaded plugin middlewared.plugins.iscsi_ middlewared: loaded plugin middlewared.plugins.iscsi_.iscsi_middlewared: loaded plugin middlewared.plugins.iscsi_.globalmiddlewared: loaded plugin middlewared.plugins.kmip.connectimiddlewared: loaded plugin middlewared.plugins.kmip.sync middlewared: loaded plugin middlewared.plugins.kmip middlewared: loaded plugin middlewared.plugins.kmip.update middlewared: loaded plugin middlewared.plugins.kmip.zfs_keysmiddlewared: loaded plugin middlewared.plugins.kmip.sed_keysmiddlewared: loaded plugin middlewared.plugins.s3_ middlewared: loaded plugin middlewared.plugins.s3_.attachmenmiddlewared: loaded plugin middlewared.plugins.vm_.connectiomiddlewared: loaded plugin middlewared.plugins.vm_ middlewared: loaded plugin middlewared.plugins.vm_.events middlewared: loaded plugin middlewared.plugins.reporting.conmiddlewared: loaded plugin middlewared.plugins.reporting.evemiddlewared: loaded plugin middlewared.plugins.reporting.promiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting.rrdmiddlewared: loaded plugin middlewared.plugins.reporting.iosmiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting middlewared: loaded plugin middlewared.plugins.reporting.updmiddlewared: loaded plugin middlewared.plugins.datastore middlewared: loaded plugin middlewared.plugins.datastore.evemiddlewared: loaded plugin middlewared.plugins.datastore.wrimiddlewared: loaded plugin middlewared.plugins.datastore.utimiddlewared: loaded plugin middlewared.plugins.datastore.schmiddlewared: loaded plugin middlewared.plugins.datastore.filmiddlewared: loaded plugin middlewared.plugins.datastore.conmiddlewared: loaded plugin middlewared.plugins.datastore.reamiddlewared: loaded plugin middlewared.plugins.device_ middlewared: loaded plugin middlewared.plugins.device_.devd_middlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.pendimiddlewared: loaded plugin middlewared.plugins.update_ middlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.uploamiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.sysctl.hint_imiddlewared: loaded plugin middlewared.plugins.sysctl middlewared: loaded plugin middlewared.plugins.truecommand.emiddlewared: loaded plugin middlewared.plugins.truecommand middlewared: loaded plugin middlewared.plugins.truecommand.wmiddlewared: loaded plugin middlewared.plugins.truecommand.pmiddlewared: loaded plugin middlewared.plugins.truecommand.cmiddlewared: loaded plugin middlewared.plugins.truecommand.umiddlewared: loaded plugin middlewared.plugins.account_ middlewared: loaded plugin middlewared.plugins.account_.syncmiddlewared: loaded plugin middlewared.plugins.cloud_sync_.cmiddlewared: loaded plugin middlewared.plugins.cloud_sync_ middlewared: loaded plugin middlewared.plugins.smb_.util_sd middlewared: loaded plugin middlewared.plugins.smb_.admin_grmiddlewared: loaded plugin middlewared.plugins.smb_.passdb middlewared: loaded plugin middlewared.plugins.smb_.registrymiddlewared: loaded plugin middlewared.plugins.smb_.sid middlewared: loaded plugin middlewared.plugins.smb_ middlewared: loaded plugin middlewared.plugins.smb_.groupmapmiddlewared: loaded plugin middlewared.plugins.smb_.sharesecmiddlewared: loaded plugin middlewared.plugins.smb_.status middlewared: loaded plugin middlewared_truenas.plugins.truenmiddlewared: loaded plugin middlewared_truenas.plugins.sensomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.fcpormiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: resolving plugins schemas middlewared: registering services middlewared: setting up plugins (datastore) [1/60] middlewared: setting up plugins (auth) [2/60] middlewared: setting up plugins (service) [3/60] middlewared: setting up plugins (pwenc) [4/60] middlewared: setting up plugins (boot) [5/60] middlewared: setting up plugins (system) [6/60] middlewared: setting up plugins (alert) [7/60] middlewared: setting up plugins (account) [8/60] middlewared: setting up plugins (network) [9/60] middlewared: setting up plugins (ec2) [10/60] middlewared: setting up plugins (cache) [11/60] middlewared: setting up plugins (rsync) [12/60] middlewared: setting up plugins (replication) [13/60] middlewared: setting up plugins (activedirectory) [14/60] middlewared: setting up plugins (afp) [15/60] middlewared: setting up plugins (webdav) [16/60] middlewared: setting up plugins (trueview) [17/60] middlewared: setting up plugins (directoryservices) [18/60] middlewared: setting up plugins (jail_freebsd) [19/60] middlewared: setting up plugins (etc) [20/60] middlewared: setting up plugins (cloud_sync) [21/60] middlewared: setting up plugins (smb) [22/60] middlewared: setting up plugins (disk) [23/60] middlewared: setting up plugins (usage) [24/60] middlewared: setting up plugins (sysdataset) [25/60] middlewared: setting up plugins (nfs) [26/60] middlewared: setting up plugins (ftp) [27/60] middlewared: setting up plugins (filesystem) [28/60] middlewared: setting up plugins (ups) [29/60] middlewared: setting up plugins (unscheduled_reboot_alert) [middlewared: setting up plugins (zettarepl) [31/60] middlewared: setting up plugins (ipmi) [32/60] middlewared: setting up plugins (crypto) [33/60] middlewared: setting up plugins (vpn) [34/60] middlewared: setting up plugins (snapshot) [35/60] middlewared: setting up plugins (pool) [36/60] middlewared: setting up plugins (iscsi) [37/60] middlewared: setting up plugins (vm) [38/60] middlewared: setting up plugins (disk_.smartctl) [39/60] middlewared: setting up plugins (disk_.zfs_guid) [40/60] middlewared: setting up plugins (disk_.disk_events) [41/60] middlewared: setting up plugins (disk_.sync) [42/60] middlewared: setting up plugins (disk_.hddstandby_force) [43middlewared: setting up plugins (mail_.gmail) [44/60] middlewared: setting up plugins (interface.link_address) [45
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp 0xfffff80004f84800), ignoring.
igb1: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb1 (ifp 0xfffff80004f86800), ignoring.
<118>middlewared: setting up plugins (zfs_.events) [46/60] middlewared: setting up plugins (zfs_.zfs_events) [47/60] middlewared: setting up plugins (enterprise.m_series_nvdimm_middlewared: setting up plugins (kmip) [49/60] middlewared: setting up plugins (s3_.attachments) [50/60] middlewared: setting up plugins (reporting.events) [51/60] middlewared: setting up plugins (reporting.processes) [52/60middlewared: setting up plugins (device_.devd_events) [53/60middlewared: setting up plugins (sysctl.hint_isp_role_freebsmiddlewared: setting up plugins (truecommand) [55/60] middlewared: setting up plugins (failover) [56/60] middlewared: setting up plugins (enclosure) [57/60] middlewared: setting up plugins (failover_.remote) [58/60] middlewared: setting up plugins (failover_.fenced) [59/60] middlewared: setting up plugins (failover_.event) [60/60] middlewared: loading completed
Loading early kernel modules:
Syncing disks...
Waiting on devd connection
Enumerating system disks
Enumerating geom disk XML information
Enumerating disk information from database
Syncing disk 1/5
Syncing disk 2/5
Syncing disk 3/5
Syncing disk 4/5
Syncing disk 5/5
Click to expand...

to be continued
 
Last edited:
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
Next "dmesg -a"
Syncing all disks complete!
Alarm clock
Starting file system checks:
Mounting local filesystems:.
Beginning pools import
Importing MyData
spa.c:8367:spa_async_request(): spa=$import async request task=2048
spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADED
spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): UNLOADING
spa.c:6107:spa_import(): spa_import: importing MyData
spa_misc.c:419:spa_load_note(): spa_load(MyData, config trusted): LOADING
vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/98246037-8d17-11ed-aac4-002590f5a654': best uberblock found for spa MyData. txg 132
spa_misc.c:419:spa_load_note(): spa_load(MyData, config untrusted): using uberblock with txg=132
spa_misc.c:419:spa_load_note(): spa_load(MyData, config trusted): read 9 log space maps (9 total blocks - blksz = 131072 bytes) in 72 ms
mmp.c:241:mmp_thread_start(): MMP thread started pool 'MyData' gethrtime 25058084169
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 10, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 11, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 12, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 13, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 14, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 15, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 133, spa MyData, vdev_id 0, ms_id 16, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25058 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
spa.c:8367:spa_async_request(): spa=MyData async request task=1
spa.c:8367:spa_async_request(): spa=MyData async request task=2048
spa_misc.c:419:spa_load_note(): spa_load(MyData, config trusted): LOADED
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 17, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25174 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 18, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25174 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 19, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25174 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
spa_history.c:311:spa_history_log_sync(): txg 134 open pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 20, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 21, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 23, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 0, smp_length 32, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179783168, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 22, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 24, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 25, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 26, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 27, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 134, spa MyData, vdev_id 0, ms_id 28, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25442 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717
spa.c:8367:spa_async_request(): spa=MyData async request task=32
spa_history.c:311:spa_history_log_sync(): txg 136 import pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
GEOM_MIRROR: Device mirror/swap0 launched (3/3).
GEOM_ELI: Device mirror/swap0.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: accelerated software
Pools import completed
Updating CPU Microcode...
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.00-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
Done.
Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/perl5/5.32/mach/CORE /usr/local/lib/samba4
32-bit compatibility ldconfig path: /usr/local/lib32/compat
Setting hostname: truenas.local.
Syncing multipaths...
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Starting interfaces...
igb0: link state changed to DOWN
igb1: link state changed to DOWN
igb0: link state changed to UP
igb1: link state changed to UP
Generating configuration for interface_sync checkpoint
Loading kernel modules:
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating /var/run/os-release done.
Creating and/or trimming log files.
Updating motd:.
Starting syslog_ng.
Starting watchdogd.
Security policy loaded: MAC/ntpd (mac_ntpd)
Starting ntpd.
Mounting late filesystems:.
Starting dbus.
Starting rrdcached.
Starting avahi-daemon.
starting wsdd
Starting cron.
Starting smartd_daemon.
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.
Starting collectd_daemon.
Configuring vt: blanktime.
net.inet.carp.allow: 0 -> 1
Click to expand...


I won't be able to do much tonight, the disks are doing a long smart test.


@Davvo
According to supermicro, the SAS controller is a Broadcom 2308.
Click to expand...
yes. I have 8 SAS ports.
I have recently updated the program (https://www.supermicro.com/wdl/driver/SAS/Broadcom/2308/Firmware/IT/)
The controller is well configured in IT mode
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
Here are the results of the smart tests:

ADA0 (Truenas OS - SSD connected on sata port)
Code:
root@truenas[~]# smartctl -a /dev/ada0
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     SATA SSD
Serial Number:    D21090881A05626
Firmware Version: HAFEB2.0
User Capacity:    128,035,676,160 bytes [128 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Jan  6 23:24:28 2023 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (65535) seconds.
Offline data collection
capabilities:                    (0x79) SMART execute Offline immediate.
                                        No Auto Offline data collection support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (  30) minutes.
Conveyance self-test routine
recommended polling time:        (   6) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   050    Pre-fail  Always       -       0
  5 Reallocated_Sector_Ct   0x0013   100   100   050    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       2193
 12 Power_Cycle_Count       0x0012   100   100   000    Old_age   Always       -       94
161 Unknown_Attribute       0x0012   100   100   000    Old_age   Always       -       183
163 Unknown_Attribute       0x0003   100   100   050    Pre-fail  Always       -       57
165 Unknown_Attribute       0x0000   100   100   000    Old_age   Offline      -       7
166 Unknown_Attribute       0x0000   100   100   000    Old_age   Offline      -       2
167 Unknown_Attribute       0x0000   100   100   000    Old_age   Offline      -       5
172 Unknown_Attribute       0x0012   100   100   000    Old_age   Always       -       0
173 Unknown_Attribute       0x0022   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0012   100   100   000    Old_age   Always       -       42
194 Temperature_Celsius     0x0023   060   060   000    Pre-fail  Always       -       40 (Min/Max 33/40)
196 Reallocated_Event_Count 0x0000   100   100   000    Old_age   Offline      -       0
198 Offline_Uncorrectable   0x0012   100   100   000    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x000b   100   100   050    Pre-fail  Always       -       5
241 Total_LBAs_Written      0x0012   100   100   000    Old_age   Always       -       200
242 Total_LBAs_Read         0x0012   100   100   000    Old_age   Always       -       188
249 Unknown_Attribute       0x0012   100   100   000    Old_age   Always       -       354

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      2183         -
# 2  Extended offline    Completed without error       00%      2180         -
# 3  Extended offline    Completed without error       00%      1850         -
# 4  Short offline       Completed without error       00%      1847         -
# 5  Short offline       Completed without error       00%      1628         -
# 6  Short offline       Completed without error       00%      1460         -
# 7  Extended offline    Completed without error       00%      1414         -
# 8  Short offline       Completed without error       00%      1292         -
# 9  Short offline       Completed without error       00%      1124         -
#10  Short offline       Completed without error       00%       956         -
#11  Short offline       Completed without error       00%       877         -
#12  Short offline       Completed without error       00%       671         -
#13  Conveyance offline  Completed without error       00%       354         -
#14  Extended offline    Completed without error       00%       346         -
#15  Short offline       Completed without error       00%       344         -
#16  Short offline       Completed without error       00%       321         -
#17  Short offline       Completed without error       00%       243         -
#18  Conveyance offline  Completed without error       00%         2         -
#19  Short offline       Completed without error       00%         2         -

SMART Selective self-test log data structure revision number 0
Note: revision number not 1 implies that no selective self-test has ever been run
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]#


da0 (disk1 of raidz2 data pool - HDD connected on SAS port)
Code:
root@truenas[~]# smartctl -a /dev/da0
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N4JPZ41B
LU WWN Device Id: 5 0014ee 2b75faadc
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Jan  6 23:32:11 2023 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (39720) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 399) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   183   179   021    Pre-fail  Always       -       5816
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       190
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   056   056   000    Old_age   Always       -       32611
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       142
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       47
193 Load_Cycle_Count        0x0032   186   186   000    Old_age   Always       -       43919
194 Temperature_Celsius     0x0022   125   106   000    Old_age   Always       -       25
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       1
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     32605         -
# 2  Short offline       Completed without error       00%     32507         -
# 3  Extended offline    Completed without error       00%     32472         -
# 4  Conveyance offline  Completed without error       00%     32451         -
# 5  Short offline       Completed without error       00%     32439         -
# 6  Short offline       Completed without error       00%     32416         -
# 7  Short offline       Completed without error       00%     32338         -
# 8  Extended offline    Completed without error       00%     32190         -
# 9  Short offline       Completed without error       00%     32183         -
#10  Short offline       Completed without error       00%     25278         -
#11  Extended offline    Completed without error       00%     25274         -
#12  Short offline       Completed without error       00%     25134         -
#13  Short offline       Completed without error       00%     24990         -
#14  Short offline       Completed without error       00%     24846         -
#15  Short offline       Completed without error       00%     24702         -
#16  Short offline       Completed without error       00%     24534         -
#17  Short offline       Completed without error       00%     24391         -
#18  Short offline       Completed without error       00%     24168         -
#19  Short offline       Completed without error       00%     24024         -
#20  Short offline       Completed without error       00%     23880         -
#21  Short offline       Completed without error       00%     23736         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]#


da1 (disk2 of raidz2 data pool - HDD connected on SAS port)
Code:
root@truenas[~]# smartctl -a /dev/da1
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N7LS2CFL
LU WWN Device Id: 5 0014ee 2b67b88c7
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Jan  6 23:34:46 2023 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (39720) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 399) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   181   177   021    Pre-fail  Always       -       5950
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       149
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   054   054   000    Old_age   Always       -       33788
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       123
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       41
193 Load_Cycle_Count        0x0032   186   186   000    Old_age   Always       -       43764
194 Temperature_Celsius     0x0022   123   104   000    Old_age   Always       -       27
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     33782         -
# 2  Short offline       Completed without error       00%     33684         -
# 3  Extended offline    Completed without error       00%     33673         -
# 4  Short offline       Completed without error       00%     33616         -
# 5  Short offline       Completed without error       00%     33593         -
# 6  Extended offline    Completed without error       00%     33365         -
# 7  Short offline       Completed without error       00%     33357         -
# 8  Short offline       Completed without error       00%     25260         -
# 9  Extended offline    Completed without error       00%     25256         -
#10  Short offline       Completed without error       00%     25116         -
#11  Short offline       Completed without error       00%     24972         -
#12  Short offline       Completed without error       00%     24828         -
#13  Short offline       Completed without error       00%     24685         -
#14  Short offline       Completed without error       00%     24517         -
#15  Short offline       Completed without error       00%     24373         -
#16  Short offline       Completed without error       00%     24150         -
#17  Short offline       Completed without error       00%     24006         -
#18  Short offline       Completed without error       00%     23862         -
#19  Short offline       Completed without error       00%     23718         -
#20  Short offline       Completed without error       00%     23575         -
#21  Short offline       Completed without error       00%     23431         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]#



da2 (disk3 of raidz2 data pool - HDD connected on SAS port)
Code:
root@truenas[~]# smartctl -a /dev/da2
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N0RSC0IV
LU WWN Device Id: 5 0014ee 20d3bf7d8
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Jan  6 23:36:31 2023 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (39060) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 392) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   177   171   021    Pre-fail  Always       -       6150
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       150
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   056   056   000    Old_age   Always       -       32602
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       126
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       43
193 Load_Cycle_Count        0x0032   186   186   000    Old_age   Always       -       43762
194 Temperature_Celsius     0x0022   124   104   000    Old_age   Always       -       26
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     32596         -
# 2  Short offline       Completed without error       00%     32498         -
# 3  Short offline       Completed without error       00%     32429         -
# 4  Short offline       Completed without error       00%     32407         -
# 5  Short offline       Completed without error       00%     32372         -
# 6  Extended offline    Completed without error       00%     32176         -
# 7  Extended offline    Completed without error       00%     32165         -
# 8  Short offline       Completed without error       00%     32158         -
# 9  Short offline       Completed without error       00%     25254         -
#10  Extended offline    Completed without error       00%     25250         -
#11  Short offline       Completed without error       00%     25110         -
#12  Short offline       Completed without error       00%     24966         -
#13  Short offline       Completed without error       00%     24822         -
#14  Short offline       Completed without error       00%     24679         -
#15  Short offline       Completed without error       00%     24511         -
#16  Short offline       Completed without error       00%     24367         -
#17  Short offline       Completed without error       00%     24144         -
#18  Short offline       Completed without error       00%     24000         -
#19  Short offline       Completed without error       00%     23856         -
#20  Short offline       Completed without error       00%     23712         -
#21  Short offline       Completed without error       00%     23568         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]#




da3 (disk4 of raidz2 data pool - HDD connected on SAS port)
Code:
root@truenas[~]# smartctl -a /dev/da3
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N5DVH913
LU WWN Device Id: 5 0014ee 20d954c65
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Jan  6 23:36:59 2023 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (38760) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 389) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   176   171   021    Pre-fail  Always       -       6183
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       90
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   055   055   000    Old_age   Always       -       33478
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       83
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       35
193 Load_Cycle_Count        0x0032   186   186   000    Old_age   Always       -       43641
194 Temperature_Celsius     0x0022   125   107   000    Old_age   Always       -       25
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     33471         -
# 2  Short offline       Completed without error       00%     33373         -
# 3  Short offline       Completed without error       00%     33306         -
# 4  Short offline       Completed without error       00%     33283         -
# 5  Short offline       Completed without error       00%     33248         -
# 6  Short offline       Completed without error       00%     33204         -
# 7  Extended offline    Completed without error       00%     33049         -
# 8  Short offline       Completed without error       00%     33042         -
# 9  Short offline       Completed without error       00%     24944         -
#10  Extended offline    Completed without error       00%     24940         -
#11  Short offline       Completed without error       00%     24800         -
#12  Short offline       Completed without error       00%     24656         -
#13  Short offline       Completed without error       00%     24512         -
#14  Short offline       Completed without error       00%     24368         -
#15  Short offline       Completed without error       00%     24201         -
#16  Short offline       Completed without error       00%     24057         -
#17  Short offline       Completed without error       00%     23834         -
#18  Short offline       Completed without error       00%     23690         -
#19  Short offline       Completed without error       00%     23546         -
#20  Short offline       Completed without error       00%     23402         -
#21  Short offline       Completed without error       00%     23258         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]#


Do you see anything wrong?
I will take advantage of the weekend to do the other tests that @Alecmascot suggested.
Regards
 
A

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,177
Nothing in the dmesg. One crc error in the smart data. Nothing to worry about.
The first thing I would do is to test with all drives connected to the SATA ports.

Edit. I don't usually see all the "metaslab" messages during the pool mount. Looks a bit strange.
when you exported the pool did you tick the destroy box ?
 
Last edited:
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Rosin0416 said:
Do you see anything wrong?
Click to expand...
Besides your HDDs being exceptionally cool I see a thing that I deem strange but nor wrong per se: your SSD is really hot! 40°C when your other drives are around 27°C is worrying.

EDIT: your SSD has 5 UDMA_CRC errors. That's no bueno. da0 looks like it has one as well so keep your eyes open. Check your SATA and power cables, then run a zpool clear. See if those errors return.
 
Last edited:
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
@Alecmascot
Edit. I don't usually see all the "metaslab" messages during the pool mount. Looks a bit strange.
when you exported the pool did you tick the destroy box ?
Click to expand...
Yes I checked the box "destroy the pool" like this.
1673099596163.png

Then I recreated the pool.
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
@Davvo
The hard drives have been working for 5 years.
However, I bought the SSD this year because I was switching from freenas 9.3 to Truenas 13. Freenas was installed on a usb stick. Now it is recommended to install Truenas on an SSD.
So the SSD is very new.
Since I plugged it in, Truenas has always indicated a temperature of 40°C.
When I put my fingers on the SSD, the temperature seems to be much lower than 40°C (having turned off the fans). Maybe the temperature is higher on the inside.
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
I performed several tests:

TEST 1 :
I deleted the volume raidz2 (with data destruction)
I have disconnected (power + cable) my 4 data disks. I had two small laptop hard drives that I plugged into the SAS ports.
I rebooted and built a mirror pool

Resultat : I didn't have any mistakes


TEST 2 :
I deleted the volume mirror (with data destruction)
I reconnected my 4 hard drives to the SAS ports.
I rebuilt a raidz2 pool.
On the next reboot : this image below shows is at the beginning of the system boot (after the BIOS invocation but before the OS boot)
1673100910904.png

Here is what I got :
1673101029490.png



TEST 3 :
I deleted the volume raidz2 (with data destruction)
In this test, I intertwined / mixed the SATA cables
And I create a pool mirror with 4x Disks
On the next reboot, I got this :
1673101353576.png

1673101402662.png

In this test, I had no errors


TEST 4 :
As I didn't know if it was the mirrored pool or the moving of the sata cables that gave this result, I deleted the mirrored pool (with data destruction) and created a new raidz2.
On the next reboot, I got this :
1673101664885.png


1673101721058.png

The errors have returned here.
In addition there is the line "Consoles : internal vide/keyboard" which appeared.



TEST 5 :
Next I deleted the raidz2 pool (with data destruction).
I created a pool in raidz1.
On the next reboot, I got this :
1673101947612.png

1673101990641.png

Other types of errors have appeared here (see the last two lines)



TEST 6 :
I deleted the raidz1 pool. I shutdown the server.
I then disconnected the 4x cables from the SAS ports and connected them to the 4x SATA ports on the motherboard
On the next reboot, I got this :
1673102252989.png

We can see that there is no more disk detected on the SAS ports (see picture above)
1673102659938.png

Here no error, but there is no pool configured yet


TEST 7 :
I create a pool raidz2
On the next reboot, I got this :
1673102664300.png

No errors with pool in raidz2.


TEST 8 :
I connected the 4 hard drives to the other 4 SAS connectors in case one connector was the problem.
(The motherboard has 8 SAS connectors)

Here I kept the raidz2 pool created in the previous test.
On the next reboot, I got this :
1673102990973.png

1673103026011.png

From here, I realized that I had initially connected my disks to SAS ports 0, 2, 4, 6.
(Except in my previous test (TEST 8), where I moved them to ports 1, 3, 5, 7).
I was convinced that they were in the right order
1673103221943.png



TEST 9 :
I have reconnected the hard disks on the SAS ports 0,1,2,3
Here I kept the raidz2 pool created in the previous test.
On the next reboot, I got this :
1673103337425.png

1673103366342.png

Always the same mistakes


TEST 10 :
here, I simply deleted the raidz2 pool (with data destruction) and recreated the raidz2 pool
On the next reboot, I got this :
1673103443883.png

1673103533267.png
 

Attachments

  • 1673102326597.png
    1673102326597.png
    44.1 KB · Views: 127
Last edited:
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
Following the last test (10), I went into Avago configuration (at system startup):
avago_2.jpg

avago_screen.jpg
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
For AHCI, it is well activated I think
iKVM_capture_.jpg
 
A

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,177
So it is connected with the use of the SAS ports ?
With the RaidZ2 connected to SATA can you post dmesg ?
The sata raid option in the bios should be off/disabled
Thanks
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
A priori, my problems happen when it is a raidz1 or raidz2, but not in mirroring
 
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
I think there is an issue with the SAS ports/controller.
Take a look at this link, it might help.
h30434.www3.hp.com

Z820 - Flash onboard LSI 2308 controller to IT-Mode

Hi all, firmware for the onboard LSI 2308 controller is available as (default) IR mode (Raid mode) thru the downloads section. I need to flash the Z820 onboard LSI 2308 controller to IT-Mode (instead of the default IR-Mode). I cannot find the IT-Mode firmware (only the IR-Mode firmware...
h30434.www3.hp.com h30434.www3.hp.com
 
Last edited:
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
@Alecmascot
So it is connected with the use of the SAS ports ?
Click to expand...
In my post #31, I have my SSD connected on SATA and my 4 HDD on SAS port.

Before making the change (SAS > SATA), here is the return of the dmesg command when my 4 HDD are connected on SAS ports (0,1,2,3)
root@truenas[~]# dmesg -a
---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE-p2 n245412-484f039b1d0 TRUENAS amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.07-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 34359738368 (32768 MB)
avail memory = 33314963456 (31771 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23
Launching APs: 1 2 3
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
padlock0: No ACE support.
acpi0: <SUPERM SMCI--MB>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
apei0: <ACPI Platform Error Interface> on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 1.1 on pci0
pci2: <ACPI PCI bus> on pcib2
mps0: <Avago Technologies (LSI) SAS2308> port 0xe000-0xe0ff mem 0xf7240000-0xf724ffff,0xf7200000-0xf723ffff irq 17 at device 0.0 on pci2
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xf7500000-0xf750ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7514000-0xf75143ff irq 16 at device 26.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
usbus1: 480Mbps High Speed USB v2.0
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
vgapci0: <VGA-compatible display> port 0xd000-0xd07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci4
vgapci0: Boot video device
pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci5: <ACPI PCI bus> on pcib5
igb0: <Intel(R) I210 (Copper)> port 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18 at device 0.0 on pci5
igb0: EEPROM V3.16-0 eTrack 0x800004d7
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 2 RX queues 2 TX queues
igb0: Using MSI-X interrupts with 3 vectors
igb0: Ethernet address: 00:25:90:f5:a6:54
pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci6: <ACPI PCI bus> on pcib6
igb1: <Intel(R) I210 (Copper)> port 0xb000-0xb01f mem 0xf7300000-0xf737ffff,0xf7380000-0xf7383fff irq 19 at device 0.0 on pci6
igb1: EEPROM V3.16-0 eTrack 0x800004d9
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 2 RX queues 2 TX queues
igb1: Using MSI-X interrupts with 3 vectors
igb1: Ethernet address: 00:25:90:f5:a6:55
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7513000-0xf75133ff irq 22 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
usbus2: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xf7512000-0xf75127ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 5 on acpi0
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ichwd0: <Intel Lynx Point watchdog timer> on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
ichwd0: <Intel Lynx Point watchdog timer> at port 0x1830-0x1837,0x1860-0x187f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xce000-0xcefff pnpid ORM0000 on isa0
superio0: <Nuvoton NCT6776> at port 0x2e-0x2f on isa0
wbwd0: <Nuvoton NCT6776 (0xc3/0x33) Watchdog Timer> at WDT ldn 0x08 on superio0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
Timecounter "TSC-low" frequency 1699998857 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ugen1.1: <Intel EHCI root HUB> at usbus1
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
ugen0.1: <Intel XHCI root HUB> at usbus0
uhub0 on usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen2.1: <Intel EHCI root HUB> at usbus2
uhub1 on usbus0
uhub2 on usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub1: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ipmi0: IPMI device rev. 1, firmware rev. 3.88, version 2.0, device support mask 0xbf
ipmi0: Number of channels 2
ipmi0: Attached watchdog
ipmi0: Establishing power cycle handler
ses0 at ahciem0 bus 0 scbus7 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <SATA SSD HAFEB2.0> ACS-4 ATA SATA 3.x device
ada0: Serial Number D21090881A05626
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 122104MB (250069680 512 byte sectors)
ses0: pass4,ada0 in 'Slot 00', SATA Slot: scbus1 target 0
da0 at mps0 bus 0 scbus0 target 0 lun 0
da0: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number WD-WCC4N4JPZ41B
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 2861588MB (5860533168 512 byte sectors)
da0: quirks=0x8<4K>
da1 at mps0 bus 0 scbus0 target 1 lun 0
da1: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number WD-WCC4N0RSC0IV
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 2861588MB (5860533168 512 byte sectors)
da1: quirks=0x8<4K>
da2 at mps0 bus 0 scbus0 target 2 lun 0
da2: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number WD-WCC4N7LS2CFL
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 2861588MB (5860533168 512 byte sectors)
da2: quirks=0x8<4K>
da3 at mps0 bus 0 scbus0 target 3 lun 0
da3: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number WD-WCC4N5DVH902
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 2861588MB (5860533168 512 byte sectors)
da3: quirks=0x8<4K>
Trying to mount root from zfs:boot-pool/ROOT/default []...
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub1: 17 ports with 17 removable, self powered
Root mount waiting for: usbus0 usbus1 usbus2
ugen0.2: <vendor 0x0557 product 0x7000> at usbus0
uhub3 on uhub1
uhub3: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 1> on usbus0
ugen1.2: <vendor 0x8087 product 0x8008> at usbus1
uhub4 on uhub0
uhub4: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen2.2: <vendor 0x8087 product 0x8000> at usbus2
uhub5 on uhub2
uhub5: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus2
uhub3: 4 ports with 3 removable, self powered
uhub4: 4 ports with 4 removable, self powered
uhub5: 6 ports with 6 removable, self powered
ugen0.3: <vendor 0x0557 product 0x2419> at usbus0
ukbd0 on uhub3
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd2 at ukbd0
ums0 on uhub3
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
Root mount waiting for: usbus0
ugen0.4: <MGE UPS SYSTEMS Evolution> at usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Starting devd.
Autoloading module: ichsmb
ichsmb0: <Intel Lynx Point SMBus controller> port 0x580-0x59f mem 0xf7511000-0xf75110ff irq 18 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
Starting ums0 moused.
Autoloading module: uhid
Autoloading module: usbhid
Autoloading module: wmt
Starting zfsd.
Setting hostuuid: 00000000-0000-0000-0000-002590f5a654.
Setting hostid: 0x7dd148b3.
lo0: link state changed to UP
<118>middlewared: starting middlewared: loaded plugin middlewared.plugins.service middlewared: loaded plugin middlewared.plugins.ec2 middlewared: loaded plugin middlewared.plugins.cache middlewared: loaded plugin middlewared.plugins.s3 middlewared: loaded plugin middlewared.plugins.rsync middlewared: loaded plugin middlewared.plugins.replication middlewared: loaded plugin middlewared.plugins.ssh middlewared: loaded plugin middlewared.plugins.api_key middlewared: loaded plugin middlewared.plugins.cron middlewared: loaded plugin middlewared.plugins.migration middlewared: loaded plugin middlewared.plugins.ldap middlewared: loaded plugin middlewared.plugins.activedirectomiddlewared: loaded plugin middlewared.plugins.afp middlewared: loaded plugin middlewared.plugins.config middlewared: loaded plugin middlewared.plugins.network middlewared: loaded plugin middlewared.plugins.keychain middlewared: loaded plugin middlewared.plugins.webdav middlewared: loaded plugin middlewared.plugins.stats middlewared: loaded plugin middlewared.plugins.trueview middlewared: loaded plugin middlewared.plugins.directoryservmiddlewared: loaded plugin middlewared.plugins.keyvalue middlewared: loaded plugin middlewared.plugins.jail_freebsd middlewared: loaded plugin middlewared.plugins.etc middlewared: loaded plugin middlewared.plugins.cloud_sync middlewared: loaded plugin middlewared.plugins.smb middlewared: loaded plugin middlewared.plugins.disk middlewared: loaded plugin middlewared.plugins.geom middlewared: loaded plugin middlewared.plugins.device middlewared: loaded plugin middlewared.plugins.smart middlewared: loaded plugin middlewared.plugins.update middlewared: loaded plugin middlewared.plugins.kerberos middlewared: loaded plugin middlewared.plugins.usage middlewared: loaded plugin middlewared.plugins.snmp middlewared: loaded plugin middlewared.plugins.auth middlewared: loaded plugin middlewared.plugins.sysdataset middlewared: loaded plugin middlewared.plugins.acme_protocolmiddlewared: loaded plugin middlewared.plugins.nfs middlewared: loaded plugin middlewared.plugins.ftp middlewared: loaded plugin middlewared.plugins.filesystem middlewared: loaded plugin middlewared.plugins.support middlewared: loaded plugin middlewared.plugins.bootenv middlewared: loaded plugin middlewared.plugins.boot middlewared: loaded plugin middlewared.plugins.tunables middlewared: loaded plugin middlewared.plugins.ups middlewared: loaded plugin middlewared.plugins.dyndns middlewared: loaded plugin middlewared.plugins.account middlewared: loaded plugin middlewared.plugins.multipath middlewared: loaded plugin middlewared.plugins.unscheduled_rmiddlewared: loaded plugin middlewared.plugins.system middlewared: loaded plugin middlewared.plugins.lldp middlewared: loaded plugin middlewared.plugins.hardware middlewared: loaded plugin middlewared.plugins.ntp middlewared: loaded plugin middlewared.plugins.zettarepl middlewared: loaded plugin middlewared.plugins.alert middlewared: loaded plugin middlewared.plugins.ipmi middlewared: loaded plugin middlewared.plugins.zfs middlewared: loaded plugin middlewared.plugins.crypto middlewared: loaded plugin middlewared.plugins.vpn middlewared: loaded plugin middlewared.plugins.idmap middlewared: loaded plugin middlewared.plugins.pwenc middlewared: loaded plugin middlewared.plugins middlewared: loaded plugin middlewared.plugins.vmware middlewared: loaded plugin middlewared.plugins.webui middlewared: loaded plugin middlewared.plugins.nis middlewared: loaded plugin middlewared.plugins.snapshot middlewared: loaded plugin middlewared.plugins.tftp middlewared: loaded plugin middlewared.plugins.pool middlewared: loaded plugin middlewared.plugins.iscsi middlewared: loaded plugin middlewared.plugins.vm middlewared: loaded plugin middlewared.plugins.init_shutdownmiddlewared: loaded plugin middlewared.plugins.mail middlewared: loaded plugin middlewared.plugins.hardware_.mcamiddlewared: loaded plugin middlewared.plugins.hardware_ middlewared: loaded plugin middlewared.plugins.disk_.overpromiddlewared: loaded plugin middlewared.plugins.disk_.remove_middlewared: loaded plugin middlewared.plugins.disk_ middlewared: loaded plugin middlewared.plugins.disk_.temperamiddlewared: loaded plugin middlewared.plugins.disk_.smartctmiddlewared: loaded plugin middlewared.plugins.disk_.identifmiddlewared: loaded plugin middlewared.plugins.disk_.dumpdevmiddlewared: loaded plugin middlewared.plugins.disk_.encryptmiddlewared: loaded plugin middlewared.plugins.disk_.disk_inmiddlewared: loaded plugin middlewared.plugins.disk_.swap_mimiddlewared: loaded plugin middlewared.plugins.disk_.smart_amiddlewared: loaded plugin middlewared.plugins.disk_.format middlewared: loaded plugin middlewared.plugins.disk_.zfs_guimiddlewared: loaded plugin middlewared.plugins.disk_.availabmiddlewared: loaded plugin middlewared.plugins.disk_.disk_evmiddlewared: loaded plugin middlewared.plugins.disk_.wipe middlewared: loaded plugin middlewared.plugins.disk_.swap_comiddlewared: loaded plugin middlewared.plugins.disk_.sync middlewared: loaded plugin middlewared.plugins.disk_.nvme_frmiddlewared: loaded plugin middlewared.plugins.disk_.swap_remiddlewared: loaded plugin middlewared.plugins.disk_.hddstanmiddlewared: loaded plugin middlewared.plugins.disk_.info middlewared: loaded plugin middlewared.plugins.disk_.smart middlewared: loaded plugin middlewared.plugins.disk_.label_fmiddlewared: loaded plugin middlewared.plugins.mail_.gmail middlewared: loaded plugin middlewared.plugins.mail_ middlewared: loaded plugin middlewared.plugins.system_.dmi middlewared: loaded plugin middlewared.plugins.system_ middlewared: loaded plugin middlewared.plugins.service_ middlewared: loaded plugin middlewared.plugins.zettarepl_.utmiddlewared: loaded plugin middlewared.plugins.zettarepl_.stmiddlewared: loaded plugin middlewared.plugins.zettarepl_ middlewared: loaded plugin middlewared.plugins.interface.capmiddlewared: loaded plugin middlewared.plugins.interface.typmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface middlewared: loaded plugin middlewared.plugins.interface.dhcmiddlewared: loaded plugin middlewared.plugins.interface.vlamiddlewared: loaded plugin middlewared.plugins.interface.netmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.brimiddlewared: loaded plugin middlewared.plugins.interface.linmiddlewared: loaded plugin middlewared.plugins.zfs_.events middlewared: loaded plugin middlewared.plugins.zfs_.disks_frmiddlewared: loaded plugin middlewared.plugins.zfs_ middlewared: loaded plugin middlewared.plugins.zfs_.zfs_evenmiddlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.pool_.replacemiddlewared: loaded plugin middlewared.plugins.pool_.attach_middlewared: loaded plugin middlewared.plugins.pool_.expand middlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.info middlewared: loaded plugin middlewared.plugins.pool_.format_middlewared: loaded plugin middlewared.plugins.pool_ middlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.enterprise middlewared: loaded plugin middlewared.plugins.enterprise.m_middlewared: loaded plugin middlewared.plugins.nfs_.sec middlewared: loaded plugin middlewared.plugins.nfs_.v4_freebmiddlewared: loaded plugin middlewared.plugins.nfs_ middlewared: loaded plugin middlewared.plugins.nfs_.krb5 middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_ middlewared: loaded plugin middlewared.plugins.boot_.format middlewared: loaded plugin middlewared.plugins.boot_.boot_lomiddlewared: loaded plugin middlewared.plugins.boot_ middlewared: loaded plugin middlewared.plugins.geom_ middlewared: loaded plugin middlewared.plugins.geom_.cache middlewared: loaded plugin middlewared.plugins.iscsi_ middlewared: loaded plugin middlewared.plugins.iscsi_.iscsi_middlewared: loaded plugin middlewared.plugins.iscsi_.globalmiddlewared: loaded plugin middlewared.plugins.kmip.connectimiddlewared: loaded plugin middlewared.plugins.kmip.sync middlewared: loaded plugin middlewared.plugins.kmip middlewared: loaded plugin middlewared.plugins.kmip.update middlewared: loaded plugin middlewared.plugins.kmip.zfs_keysmiddlewared: loaded plugin middlewared.plugins.kmip.sed_keysmiddlewared: loaded plugin middlewared.plugins.s3_ middlewared: loaded plugin middlewared.plugins.s3_.attachmenmiddlewared: loaded plugin middlewared.plugins.vm_.connectiomiddlewared: loaded plugin middlewared.plugins.vm_ middlewared: loaded plugin middlewared.plugins.vm_.events middlewared: loaded plugin middlewared.plugins.reporting.conmiddlewared: loaded plugin middlewared.plugins.reporting.evemiddlewared: loaded plugin middlewared.plugins.reporting.promiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting.rrdmiddlewared: loaded plugin middlewared.plugins.reporting.iosmiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting middlewared: loaded plugin middlewared.plugins.reporting.updmiddlewared: loaded plugin middlewared.plugins.datastore middlewared: loaded plugin middlewared.plugins.datastore.evemiddlewared: loaded plugin middlewared.plugins.datastore.wrimiddlewared: loaded plugin middlewared.plugins.datastore.utimiddlewared: loaded plugin middlewared.plugins.datastore.schmiddlewared: loaded plugin middlewared.plugins.datastore.filmiddlewared: loaded plugin middlewared.plugins.datastore.conmiddlewared: loaded plugin middlewared.plugins.datastore.reamiddlewared: loaded plugin middlewared.plugins.device_ middlewared: loaded plugin middlewared.plugins.device_.devd_middlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.pendimiddlewared: loaded plugin middlewared.plugins.update_ middlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.uploamiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.sysctl.hint_imiddlewared: loaded plugin middlewared.plugins.sysctl middlewared: loaded plugin middlewared.plugins.truecommand.emiddlewared: loaded plugin middlewared.plugins.truecommand middlewared: loaded plugin middlewared.plugins.truecommand.wmiddlewared: loaded plugin middlewared.plugins.truecommand.pmiddlewared: loaded plugin middlewared.plugins.truecommand.cmiddlewared: loaded plugin middlewared.plugins.truecommand.umiddlewared: loaded plugin middlewared.plugins.account_ middlewared: loaded plugin middlewared.plugins.account_.syncmiddlewared: loaded plugin middlewared.plugins.cloud_sync_.cmiddlewared: loaded plugin middlewared.plugins.cloud_sync_ middlewared: loaded plugin middlewared.plugins.smb_.util_sd middlewared: loaded plugin middlewared.plugins.smb_.admin_grmiddlewared: loaded plugin middlewared.plugins.smb_.passdb middlewared: loaded plugin middlewared.plugins.smb_.registrymiddlewared: loaded plugin middlewared.plugins.smb_.sid middlewared: loaded plugin middlewared.plugins.smb_ middlewared: loaded plugin middlewared.plugins.smb_.groupmapmiddlewared: loaded plugin middlewared.plugins.smb_.sharesecmiddlewared: loaded plugin middlewared.plugins.smb_.status middlewared: loaded plugin middlewared_truenas.plugins.truenmiddlewared: loaded plugin middlewared_truenas.plugins.sensomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.fcpormiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: resolving plugins schemas middlewared: registering services middlewared: setting up plugins (datastore) [1/60] middlewared: setting up plugins (auth) [2/60] middlewared: setting up plugins (service) [3/60] middlewared: setting up plugins (pwenc) [4/60] middlewared: setting up plugins (boot) [5/60] middlewared: setting up plugins (system) [6/60] middlewared: setting up plugins (alert) [7/60] middlewared: setting up plugins (account) [8/60] middlewared: setting up plugins (network) [9/60] middlewared: setting up plugins (ec2) [10/60] middlewared: setting up plugins (cache) [11/60] middlewared: setting up plugins (rsync) [12/60] middlewared: setting up plugins (replication) [13/60] middlewared: setting up plugins (activedirectory) [14/60] middlewared: setting up plugins (afp) [15/60] middlewared: setting up plugins (webdav) [16/60] middlewared: setting up plugins (trueview) [17/60] middlewared: setting up plugins (directoryservices) [18/60] middlewared: setting up plugins (jail_freebsd) [19/60] middlewared: setting up plugins (etc) [20/60] middlewared: setting up plugins (cloud_sync) [21/60] middlewared: setting up plugins (smb) [22/60] middlewared: setting up plugins (disk) [23/60] middlewared: setting up plugins (usage) [24/60] middlewared: setting up plugins (sysdataset) [25/60] middlewared: setting up plugins (nfs) [26/60] middlewared: setting up plugins (ftp) [27/60] middlewared: setting up plugins (filesystem) [28/60] middlewared: setting up plugins (ups) [29/60] middlewared: setting up plugins (unscheduled_reboot_alert) [middlewared: setting up plugins (zettarepl) [31/60] middlewared: setting up plugins (ipmi) [32/60] middlewared: setting up plugins (crypto) [33/60] middlewared: setting up plugins (vpn) [34/60] middlewared: setting up plugins (snapshot) [35/60] middlewared: setting up plugins (pool) [36/60] middlewared: setting up plugins (iscsi) [37/60] middlewared: setting up plugins (vm) [38/60] middlewared: setting up plugins (disk_.smartctl) [39/60] middlewared: setting up plugins (disk_.zfs_guid) [40/60] middlewared: setting up plugins (disk_.disk_events) [41/60] middlewared: setting up plugins (disk_.sync) [42/60] middlewared: setting up plugins (disk_.hddstandby_force) [43middlewared: setting up plugins (mail_.gmail) [44/60] middlewared: setting up plugins (interface.link_address) [45
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp 0xfffff80004c03000), ignoring.
igb1: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb1 (ifp 0xfffff80004ed4800), ignoring.
<118>middlewared: setting up plugins (zfs_.events) [46/60] middlewared: setting up plugins (zfs_.zfs_events) [47/60] middlewared: setting up plugins (enterprise.m_series_nvdimm_middlewared: setting up plugins (kmip) [49/60] middlewared: setting up plugins (s3_.attachments) [50/60] middlewared: setting up plugins (reporting.events) [51/60] middlewared: setting up plugins (reporting.processes) [52/60middlewared: setting up plugins (device_.devd_events) [53/60middlewared: setting up plugins (sysctl.hint_isp_role_freebsmiddlewared: setting up plugins (truecommand) [55/60] middlewared: setting up plugins (failover) [56/60] middlewared: setting up plugins (enclosure) [57/60] middlewared: setting up plugins (failover_.remote) [58/60] middlewared: setting up plugins (failover_.fenced) [59/60] middlewared: setting up plugins (failover_.event) [60/60] middlewared: loading completed
Loading early kernel modules:
Syncing disks...
Waiting on devd connection
Enumerating system disks
Enumerating geom disk XML information
Enumerating disk information from database
Syncing disk 1/5
Syncing disk 2/5
Syncing disk 3/5
Syncing disk 4/5
Syncing disk 5/5
Syncing disk 6/5
Syncing disk 7/5
Syncing disks with enclosures
Emitting disk events
Syncing all disks complete!
Alarm clock
Starting file system checks:
Mounting local filesystems:.
Beginning pools import
Importing MDT
vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/73cc04d3-8e8c-11ed-a04e-002590f5a654': best uberblock found for spa MDT. txg 13
spa_misc.c:419:spa_load_note(): spa_load(MDT, config untrusted): using uberblock with txg=13
spa_misc.c:419:spa_load_note(): spa_load(MDT, config trusted): read 7 log space maps (7 total blocks - blksz = 131072 bytes) in 62 ms
mmp.c:241:mmp_thread_start(): MMP thread started pool 'MDT' gethrtime 25379739145
spa.c:8367:spa_async_request(): spa=MDT async request task=1
spa.c:8367:spa_async_request(): spa=MDT async request task=2048
spa_misc.c:419:spa_load_note(): spa_load(MDT, config trusted): LOADED
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 19, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 20, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 21, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 22, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 23, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 24, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 25, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 26, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 27, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25380 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
spa_history.c:311:spa_history_log_sync(): txg 15 open pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 28, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 29, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 0, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 17179869
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 30, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 31, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 32, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 33, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 34, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 35, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 15, spa MDT, vdev_id 0, ms_id 36, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25606 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
spa.c:8367:spa_async_request(): spa=MDT async request task=32
spa_history.c:311:spa_history_log_sync(): txg 17 import pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
GEOM_MIRROR: Device mirror/swap0 launched (3/3).
GEOM_ELI: Device mirror/swap0.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: accelerated software
Pools import completed
Updating CPU Microcode...
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.00-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
Done.
Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/perl5/5.32/mach/CORE /usr/local/lib/samba4
32-bit compatibility ldconfig path: /usr/local/lib32/compat
Setting hostname: truenas.local.
Syncing multipaths...
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Starting interfaces...
igb0: link state changed to DOWN
igb1: link state changed to DOWN
igb0: link state changed to UP
igb1: link state changed to UP
Generating configuration for interface_sync checkpoint
Loading kernel modules:
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating /var/run/os-release done.
Creating and/or trimming log files.
Updating motd:.
Starting syslog_ng.
Starting watchdogd.
Security policy loaded: MAC/ntpd (mac_ntpd)
Starting ntpd.
Mounting late filesystems:.
Starting dbus.
Starting rrdcached.
Performing sanity check on openssh configuration.
Starting openssh.
Starting avahi-daemon.
starting wsdd
Starting cron.
Starting smartd_daemon.
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.
Starting collectd_daemon.
Configuring vt: blanktime.
net.inet.carp.allow: 0 -> 1
root@truenas[~]#
Click to expand...

what are the "metaslab" messages ?
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
I think there is an issue with the SAS ports/controller.
Take a look at this link, it might help.
Click to expand...
The controller is well flashed in IT mode.
If I remember correctly:
IR (Integrated Raid) mode is the mode used when using a physical card to manage the raid.
When the raid is managed by software (like Freenas/Truenas), you have to use the IT mode (initiator target)
See my picture below, there is LSI SAS2308-IT 20.00.07.00.
1673103443883-png.62078
 
A

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,177
The SAS controller has the correct firmware... but you could always try a reflash with code from Supermicro.
Re the Metaslab messages, I can find "stuff" in google but it does not make sense to me except that the messages are associated with the failing mode of using the SAS controller and RAIDZX.
Can I suggest that you continue the build using the SATA ports.
This may complicate things if you ever want to expand the number of disks you want to use.
 
R

Rosin0416

Patron
Joined
Apr 11, 2016
Messages
214
So,

After connecting the disks to SATA.
(The Raiz2 pool is still existing)
1673110037328.png


root@truenas[~]# dmesg -a
---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE-p2 n245412-484f039b1d0 TRUENAS amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.06-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 34359738368 (32768 MB)
avail memory = 33314967552 (31771 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23
Launching APs: 1 2 3
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
padlock0: No ACE support.
acpi0: <SUPERM SMCI--MB>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
apei0: <ACPI Platform Error Interface> on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 1.1 on pci0
pci2: <ACPI PCI bus> on pcib2
mps0: <Avago Technologies (LSI) SAS2308> port 0xe000-0xe0ff mem 0xf7240000-0xf724ffff,0xf7200000-0xf723ffff irq 17 at device 0.0 on pci2
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xf7500000-0xf750ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7514000-0xf75143ff irq 16 at device 26.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
usbus1: 480Mbps High Speed USB v2.0
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
vgapci0: <VGA-compatible display> port 0xd000-0xd07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci4
vgapci0: Boot video device
pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci5: <ACPI PCI bus> on pcib5
igb0: <Intel(R) I210 (Copper)> port 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18 at device 0.0 on pci5
igb0: EEPROM V3.16-0 eTrack 0x800004d7
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 2 RX queues 2 TX queues
igb0: Using MSI-X interrupts with 3 vectors
igb0: Ethernet address: 00:25:90:f5:a6:65
pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci6: <ACPI PCI bus> on pcib6
igb1: <Intel(R) I210 (Copper)> port 0xb000-0xb01f mem 0xf7300000-0xf737ffff,0xf7380000-0xf7383fff irq 19 at device 0.0 on pci6
igb1: EEPROM V3.16-0 eTrack 0x800004d9
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 2 RX queues 2 TX queues
igb1: Using MSI-X interrupts with 3 vectors
igb1: Ethernet address: 00:25:90:f5:a6:66
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7513000-0xf75133ff irq 22 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
usbus2: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xf7512000-0xf75127ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 5 on acpi0
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ichwd0: <Intel Lynx Point watchdog timer> on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
ichwd0: <Intel Lynx Point watchdog timer> at port 0x1830-0x1837,0x1860-0x187f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc87ff,0xc8800-0xc97ff pnpid ORM0000 on isa0
superio0: <Nuvoton NCT6776> at port 0x2e-0x2f on isa0
wbwd0: <Nuvoton NCT6776 (0xc3/0x33) Watchdog Timer> at WDT ldn 0x08 on superio0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
Timecounter "TSC-low" frequency 1699998558 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
ugen0.1: <Intel XHCI root HUB> at usbus0
ugen2.1: <Intel EHCI root HUB> at usbus2
ugen1.1: <Intel EHCI root HUB> at usbus1
uhub0 on usbus0
uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub1 on usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub2 on usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ipmi0: IPMI device rev. 1, firmware rev. 3.88, version 2.0, device support mask 0xbf
ipmi0: Number of channels 2
ipmi0: Attached watchdog
ipmi0: Establishing power cycle handler
ses0 at ahciem0 bus 0 scbus7 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <SATA SSD HAFEB2.0> ACS-4 ATA SATA 3.x device
ada0: Serial Number D21090881A05636
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 122104MB (250069680 512 byte sectors)
ses0: pass0,ada0 in 'Slot 00', SATA Slot: scbus1 target 0
ada1 at ahcich2 bus 0 scbus3 target 0 lun 0
ada1: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada1: Serial Number WD-WCC4N4JPZ41B
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2861588MB (5860533168 512 byte sectors)
ada1: quirks=0x1<4K>
ses0: pass1,ada1 in 'Slot 02', SATA Slot: scbus3 target 0
ada2 at ahcich3 bus 0 scbus4 target 0 lun 0
ada2: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada2: Serial Number WD-WCC4N0RSC0IV
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 2861588MB (5860533168 512 byte sectors)
ada2: quirks=0x1<4K>
ses0: pass2,ada2 in 'Slot 03', SATA Slot: scbus4 target 0
ada3 at ahcich4 bus 0 scbus5 target 0 lun 0
ada3: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada3: Serial Number WD-WCC4N7LS2CFL
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 2861588MB (5860533168 512 byte sectors)
ada3: quirks=0x1<4K>
ses0: pass3,ada3 in 'Slot 04', SATA Slot: scbus5 target 0
ada4 at ahcich5 bus 0 scbus6 target 0 lun 0
ada4: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada4: Serial Number WD-WCC4N5DVH913
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 2861588MB (5860533168 512 byte sectors)
ada4: quirks=0x1<4K>
ses0: pass4,ada4 in 'Slot 05', SATA Slot: scbus6 target 0
Trying to mount root from zfs:boot-pool/ROOT/default []...
uhub0: 17 ports with 17 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
ugen0.2: <vendor 0x0557 product 0x7000> at usbus0
uhub3 on uhub0
uhub3: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 1> on usbus0
Root mount waiting for: usbus0 usbus1 usbus2
ugen1.2: <vendor 0x8087 product 0x8008> at usbus1
uhub4 on uhub1
uhub4: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen2.2: <vendor 0x8087 product 0x8000> at usbus2
uhub5 on uhub2
uhub5: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus2
uhub3: 4 ports with 3 removable, self powered
uhub4: 4 ports with 4 removable, self powered
uhub5: 6 ports with 6 removable, self powered
ugen0.3: <vendor 0x0557 product 0x2419> at usbus0
ukbd0 on uhub3
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd2 at ukbd0
ums0 on uhub3
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
Root mount waiting for: usbus0
ugen0.4: <MGE UPS SYSTEMS Evolution> at usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Starting devd.
Autoloading module: ichsmb
ichsmb0: <Intel Lynx Point SMBus controller> port 0x580-0x59f mem 0xf7511000-0xf75110ff irq 18 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
Starting ums0 moused.
Autoloading module: uhid
Autoloading module: usbhid
Autoloading module: wmt
Starting zfsd.
Setting hostuuid: 00000000-0000-0000-0000-002590f5a654.
Setting hostid: 0x7dd148b3.
lo0: link state changed to UP
<118>middlewared: starting middlewared: loaded plugin middlewared.plugins.service middlewared: loaded plugin middlewared.plugins.ec2 middlewared: loaded plugin middlewared.plugins.cache middlewared: loaded plugin middlewared.plugins.s3 middlewared: loaded plugin middlewared.plugins.rsync middlewared: loaded plugin middlewared.plugins.replication middlewared: loaded plugin middlewared.plugins.ssh middlewared: loaded plugin middlewared.plugins.api_key middlewared: loaded plugin middlewared.plugins.cron middlewared: loaded plugin middlewared.plugins.migration middlewared: loaded plugin middlewared.plugins.ldap middlewared: loaded plugin middlewared.plugins.activedirectomiddlewared: loaded plugin middlewared.plugins.afp middlewared: loaded plugin middlewared.plugins.config middlewared: loaded plugin middlewared.plugins.network middlewared: loaded plugin middlewared.plugins.keychain middlewared: loaded plugin middlewared.plugins.webdav middlewared: loaded plugin middlewared.plugins.stats middlewared: loaded plugin middlewared.plugins.trueview middlewared: loaded plugin middlewared.plugins.directoryservmiddlewared: loaded plugin middlewared.plugins.keyvalue middlewared: loaded plugin middlewared.plugins.jail_freebsd middlewared: loaded plugin middlewared.plugins.etc middlewared: loaded plugin middlewared.plugins.cloud_sync middlewared: loaded plugin middlewared.plugins.smb middlewared: loaded plugin middlewared.plugins.disk middlewared: loaded plugin middlewared.plugins.geom middlewared: loaded plugin middlewared.plugins.device middlewared: loaded plugin middlewared.plugins.smart middlewared: loaded plugin middlewared.plugins.update middlewared: loaded plugin middlewared.plugins.kerberos middlewared: loaded plugin middlewared.plugins.usage middlewared: loaded plugin middlewared.plugins.snmp middlewared: loaded plugin middlewared.plugins.auth middlewared: loaded plugin middlewared.plugins.sysdataset middlewared: loaded plugin middlewared.plugins.acme_protocolmiddlewared: loaded plugin middlewared.plugins.nfs middlewared: loaded plugin middlewared.plugins.ftp middlewared: loaded plugin middlewared.plugins.filesystem middlewared: loaded plugin middlewared.plugins.support middlewared: loaded plugin middlewared.plugins.bootenv middlewared: loaded plugin middlewared.plugins.boot middlewared: loaded plugin middlewared.plugins.tunables middlewared: loaded plugin middlewared.plugins.ups middlewared: loaded plugin middlewared.plugins.dyndns middlewared: loaded plugin middlewared.plugins.account middlewared: loaded plugin middlewared.plugins.multipath middlewared: loaded plugin middlewared.plugins.unscheduled_rmiddlewared: loaded plugin middlewared.plugins.system middlewared: loaded plugin middlewared.plugins.lldp middlewared: loaded plugin middlewared.plugins.hardware middlewared: loaded plugin middlewared.plugins.ntp middlewared: loaded plugin middlewared.plugins.zettarepl middlewared: loaded plugin middlewared.plugins.alert middlewared: loaded plugin middlewared.plugins.ipmi middlewared: loaded plugin middlewared.plugins.zfs middlewared: loaded plugin middlewared.plugins.crypto middlewared: loaded plugin middlewared.plugins.vpn middlewared: loaded plugin middlewared.plugins.idmap middlewared: loaded plugin middlewared.plugins.pwenc middlewared: loaded plugin middlewared.plugins middlewared: loaded plugin middlewared.plugins.vmware middlewared: loaded plugin middlewared.plugins.webui middlewared: loaded plugin middlewared.plugins.nis middlewared: loaded plugin middlewared.plugins.snapshot middlewared: loaded plugin middlewared.plugins.tftp middlewared: loaded plugin middlewared.plugins.pool middlewared: loaded plugin middlewared.plugins.iscsi middlewared: loaded plugin middlewared.plugins.vm middlewared: loaded plugin middlewared.plugins.init_shutdownmiddlewared: loaded plugin middlewared.plugins.mail middlewared: loaded plugin middlewared.plugins.hardware_.mcamiddlewared: loaded plugin middlewared.plugins.hardware_ middlewared: loaded plugin middlewared.plugins.disk_.overpromiddlewared: loaded plugin middlewared.plugins.disk_.remove_middlewared: loaded plugin middlewared.plugins.disk_ middlewared: loaded plugin middlewared.plugins.disk_.temperamiddlewared: loaded plugin middlewared.plugins.disk_.smartctmiddlewared: loaded plugin middlewared.plugins.disk_.identifmiddlewared: loaded plugin middlewared.plugins.disk_.dumpdevmiddlewared: loaded plugin middlewared.plugins.disk_.encryptmiddlewared: loaded plugin middlewared.plugins.disk_.disk_inmiddlewared: loaded plugin middlewared.plugins.disk_.swap_mimiddlewared: loaded plugin middlewared.plugins.disk_.smart_amiddlewared: loaded plugin middlewared.plugins.disk_.format middlewared: loaded plugin middlewared.plugins.disk_.zfs_guimiddlewared: loaded plugin middlewared.plugins.disk_.availabmiddlewared: loaded plugin middlewared.plugins.disk_.disk_evmiddlewared: loaded plugin middlewared.plugins.disk_.wipe middlewared: loaded plugin middlewared.plugins.disk_.swap_comiddlewared: loaded plugin middlewared.plugins.disk_.sync middlewared: loaded plugin middlewared.plugins.disk_.nvme_frmiddlewared: loaded plugin middlewared.plugins.disk_.swap_remiddlewared: loaded plugin middlewared.plugins.disk_.hddstanmiddlewared: loaded plugin middlewared.plugins.disk_.info middlewared: loaded plugin middlewared.plugins.disk_.smart middlewared: loaded plugin middlewared.plugins.disk_.label_fmiddlewared: loaded plugin middlewared.plugins.mail_.gmail middlewared: loaded plugin middlewared.plugins.mail_ middlewared: loaded plugin middlewared.plugins.system_.dmi middlewared: loaded plugin middlewared.plugins.system_ middlewared: loaded plugin middlewared.plugins.service_ middlewared: loaded plugin middlewared.plugins.zettarepl_.utmiddlewared: loaded plugin middlewared.plugins.zettarepl_.stmiddlewared: loaded plugin middlewared.plugins.zettarepl_ middlewared: loaded plugin middlewared.plugins.interface.capmiddlewared: loaded plugin middlewared.plugins.interface.typmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface middlewared: loaded plugin middlewared.plugins.interface.dhcmiddlewared: loaded plugin middlewared.plugins.interface.vlamiddlewared: loaded plugin middlewared.plugins.interface.netmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.conmiddlewared: loaded plugin middlewared.plugins.interface.lagmiddlewared: loaded plugin middlewared.plugins.interface.brimiddlewared: loaded plugin middlewared.plugins.interface.linmiddlewared: loaded plugin middlewared.plugins.zfs_.events middlewared: loaded plugin middlewared.plugins.zfs_.disks_frmiddlewared: loaded plugin middlewared.plugins.zfs_ middlewared: loaded plugin middlewared.plugins.zfs_.zfs_evenmiddlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.pool_.replacemiddlewared: loaded plugin middlewared.plugins.pool_.attach_middlewared: loaded plugin middlewared.plugins.pool_.expand middlewared: loaded plugin middlewared.plugins.pool_.encryptmiddlewared: loaded plugin middlewared.plugins.pool_.info middlewared: loaded plugin middlewared.plugins.pool_.format_middlewared: loaded plugin middlewared.plugins.pool_ middlewared: loaded plugin middlewared.plugins.pool_.import_middlewared: loaded plugin middlewared.plugins.enterprise middlewared: loaded plugin middlewared.plugins.enterprise.m_middlewared: loaded plugin middlewared.plugins.nfs_.sec middlewared: loaded plugin middlewared.plugins.nfs_.v4_freebmiddlewared: loaded plugin middlewared.plugins.nfs_ middlewared: loaded plugin middlewared.plugins.nfs_.krb5 middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_.middlewared: loaded plugin middlewared.plugins.replication_ middlewared: loaded plugin middlewared.plugins.boot_.format middlewared: loaded plugin middlewared.plugins.boot_.boot_lomiddlewared: loaded plugin middlewared.plugins.boot_ middlewared: loaded plugin middlewared.plugins.geom_ middlewared: loaded plugin middlewared.plugins.geom_.cache middlewared: loaded plugin middlewared.plugins.iscsi_ middlewared: loaded plugin middlewared.plugins.iscsi_.iscsi_middlewared: loaded plugin middlewared.plugins.iscsi_.globalmiddlewared: loaded plugin middlewared.plugins.kmip.connectimiddlewared: loaded plugin middlewared.plugins.kmip.sync middlewared: loaded plugin middlewared.plugins.kmip middlewared: loaded plugin middlewared.plugins.kmip.update middlewared: loaded plugin middlewared.plugins.kmip.zfs_keysmiddlewared: loaded plugin middlewared.plugins.kmip.sed_keysmiddlewared: loaded plugin middlewared.plugins.s3_ middlewared: loaded plugin middlewared.plugins.s3_.attachmenmiddlewared: loaded plugin middlewared.plugins.vm_.connectiomiddlewared: loaded plugin middlewared.plugins.vm_ middlewared: loaded plugin middlewared.plugins.vm_.events middlewared: loaded plugin middlewared.plugins.reporting.conmiddlewared: loaded plugin middlewared.plugins.reporting.evemiddlewared: loaded plugin middlewared.plugins.reporting.promiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting.rrdmiddlewared: loaded plugin middlewared.plugins.reporting.iosmiddlewared: loaded plugin middlewared.plugins.reporting.plumiddlewared: loaded plugin middlewared.plugins.reporting middlewared: loaded plugin middlewared.plugins.reporting.updmiddlewared: loaded plugin middlewared.plugins.datastore middlewared: loaded plugin middlewared.plugins.datastore.evemiddlewared: loaded plugin middlewared.plugins.datastore.wrimiddlewared: loaded plugin middlewared.plugins.datastore.utimiddlewared: loaded plugin middlewared.plugins.datastore.schmiddlewared: loaded plugin middlewared.plugins.datastore.filmiddlewared: loaded plugin middlewared.plugins.datastore.conmiddlewared: loaded plugin middlewared.plugins.datastore.reamiddlewared: loaded plugin middlewared.plugins.device_ middlewared: loaded plugin middlewared.plugins.device_.devd_middlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.pendimiddlewared: loaded plugin middlewared.plugins.update_ middlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.instamiddlewared: loaded plugin middlewared.plugins.update_.uploamiddlewared: loaded plugin middlewared.plugins.update_.downlmiddlewared: loaded plugin middlewared.plugins.update_.trainmiddlewared: loaded plugin middlewared.plugins.update_.utilsmiddlewared: loaded plugin middlewared.plugins.sysctl.hint_imiddlewared: loaded plugin middlewared.plugins.sysctl middlewared: loaded plugin middlewared.plugins.truecommand.emiddlewared: loaded plugin middlewared.plugins.truecommand middlewared: loaded plugin middlewared.plugins.truecommand.wmiddlewared: loaded plugin middlewared.plugins.truecommand.pmiddlewared: loaded plugin middlewared.plugins.truecommand.cmiddlewared: loaded plugin middlewared.plugins.truecommand.umiddlewared: loaded plugin middlewared.plugins.account_ middlewared: loaded plugin middlewared.plugins.account_.syncmiddlewared: loaded plugin middlewared.plugins.cloud_sync_.cmiddlewared: loaded plugin middlewared.plugins.cloud_sync_ middlewared: loaded plugin middlewared.plugins.smb_.util_sd middlewared: loaded plugin middlewared.plugins.smb_.admin_grmiddlewared: loaded plugin middlewared.plugins.smb_.passdb middlewared: loaded plugin middlewared.plugins.smb_.registrymiddlewared: loaded plugin middlewared.plugins.smb_.sid middlewared: loaded plugin middlewared.plugins.smb_ middlewared: loaded plugin middlewared.plugins.smb_.groupmapmiddlewared: loaded plugin middlewared.plugins.smb_.sharesecmiddlewared: loaded plugin middlewared.plugins.smb_.status middlewared: loaded plugin middlewared_truenas.plugins.truenmiddlewared: loaded plugin middlewared_truenas.plugins.sensomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.fcpormiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.enclomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: loaded plugin middlewared_truenas.plugins.failomiddlewared: resolving plugins schemas middlewared: registering services middlewared: setting up plugins (datastore) [1/60] middlewared: setting up plugins (auth) [2/60] middlewared: setting up plugins (service) [3/60] middlewared: setting up plugins (pwenc) [4/60] middlewared: setting up plugins (boot) [5/60] middlewared: setting up plugins (system) [6/60] middlewared: setting up plugins (alert) [7/60] middlewared: setting up plugins (account) [8/60] middlewared: setting up plugins (network) [9/60] middlewared: setting up plugins (ec2) [10/60] middlewared: setting up plugins (cache) [11/60] middlewared: setting up plugins (rsync) [12/60] middlewared: setting up plugins (replication) [13/60] middlewared: setting up plugins (activedirectory) [14/60] middlewared: setting up plugins (afp) [15/60] middlewared: setting up plugins (webdav) [16/60] middlewared: setting up plugins (trueview) [17/60] middlewared: setting up plugins (directoryservices) [18/60] middlewared: setting up plugins (jail_freebsd) [19/60] middlewared: setting up plugins (etc) [20/60] middlewared: setting up plugins (cloud_sync) [21/60] middlewared: setting up plugins (smb) [22/60] middlewared: setting up plugins (disk) [23/60] middlewared: setting up plugins (usage) [24/60] middlewared: setting up plugins (sysdataset) [25/60] middlewared: setting up plugins (nfs) [26/60] middlewared: setting up plugins (ftp) [27/60] middlewared: setting up plugins (filesystem) [28/60] middlewared: setting up plugins (ups) [29/60] middlewared: setting up plugins (unscheduled_reboot_alert) [middlewared: setting up plugins (zettarepl) [31/60] middlewared: setting up plugins (ipmi) [32/60] middlewared: setting up plugins (crypto) [33/60] middlewared: setting up plugins (vpn) [34/60] middlewared: setting up plugins (snapshot) [35/60] middlewared: setting up plugins (pool) [36/60] middlewared: setting up plugins (iscsi) [37/60] middlewared: setting up plugins (vm) [38/60] middlewared: setting up plugins (disk_.smartctl) [39/60] middlewared: setting up plugins (disk_.zfs_guid) [40/60] middlewared: setting up plugins (disk_.disk_events) [41/60] middlewared: setting up plugins (disk_.sync) [42/60] middlewared: setting up plugins (disk_.hddstandby_force) [43middlewared: setting up plugins (mail_.gmail) [44/60] middlewared: setting up plugins (interface.link_address) [45
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp 0xfffff80004e79800), ignoring.
igb1: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb1 (ifp 0xfffff80004e7b800), ignoring.
<118>middlewared: setting up plugins (zfs_.events) [46/60] middlewared: setting up plugins (zfs_.zfs_events) [47/60] middlewared: setting up plugins (enterprise.m_series_nvdimm_middlewared: setting up plugins (kmip) [49/60] middlewared: setting up plugins (s3_.attachments) [50/60] middlewared: setting up plugins (reporting.events) [51/60] middlewared: setting up plugins (reporting.processes) [52/60middlewared: setting up plugins (device_.devd_events) [53/60middlewared: setting up plugins (sysctl.hint_isp_role_freebsmiddlewared: setting up plugins (truecommand) [55/60] middlewared: setting up plugins (failover) [56/60] middlewared: setting up plugins (enclosure) [57/60] middlewared: setting up plugins (failover_.remote) [58/60] middlewared: setting up plugins (failover_.fenced) [59/60] middlewared: setting up plugins (failover_.event) [60/60] middlewared: loading completed
Loading early kernel modules:
Syncing disks...
Waiting on devd connection
Enumerating system disks
Enumerating geom disk XML information
Enumerating disk information from database
Syncing disk 1/5
Syncing disk 2/5
Syncing disk 3/5
Syncing disk 4/5
Syncing disk 5/5
Syncing disk 6/5
Syncing disk 7/5
Syncing disks with enclosures
Emitting disk events
Syncing all disks complete!
Alarm clock
Starting file system checks:
Mounting local filesystems:.
Beginning pools import
Importing MDT
spa.c:8367:spa_async_request(): spa=$import async request task=2048
spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADED
spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): UNLOADING
spa.c:6107:spa_import(): spa_import: importing MDT
spa_misc.c:419:spa_load_note(): spa_load(MDT, config trusted): LOADING
vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/73cc04d3-8e8c-11ed-a04e-002590f5a654': best uberblock found for spa MDT. txg 19
spa_misc.c:419:spa_load_note(): spa_load(MDT, config untrusted): using uberblock with txg=19
spa.c:8367:spa_async_request(): spa=MDT async request task=1
spa_misc.c:419:spa_load_note(): spa_load(MDT, config trusted): read 9 log space maps (9 total blocks - blksz = 131072 bytes) in 97 ms
mmp.c:241:mmp_thread_start(): MMP thread started pool 'MDT' gethrtime 25087652988
spa.c:8367:spa_async_request(): spa=MDT async request task=1
spa.c:8367:spa_async_request(): spa=MDT async request task=2048
spa_misc.c:419:spa_load_note(): spa_load(MDT, config trusted): LOADED
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 28, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 29, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 30, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 31, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 32, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 33, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 34, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 35, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 36, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25088 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
spa_history.c:311:spa_history_log_sync(): txg 21 open pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 38, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 0, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 17179869
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 37, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 39, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 40, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 41, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 42, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 43, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 21, spa MDT, vdev_id 0, ms_id 44, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
metaslab.c:2457:metaslab_load_impl(): metaslab_load: txg 22, spa MDT, vdev_id 0, ms_id 45, smp_length 0, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 25323 ms, loading_time 0 ms, ms_max_size 17179869184, max size error 1717986
spa.c:8367:spa_async_request(): spa=MDT async request task=32
spa_history.c:311:spa_history_log_sync(): txg 23 import pool version 5000; software version zfs-2a0dae1b7; uts 13.1-RELEASE-p2 1301000 amd64
GEOM_MIRROR: Device mirror/swap0 launched (3/3).
GEOM_ELI: Device mirror/swap0.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: accelerated software
Pools import completed
Updating CPU Microcode...
CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3400.00-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
Structured Extended Features3=0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
Done.
Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/e2fsprogs /usr/local/lib/gcc10 /usr/local/lib/perl5/5.32/mach/CORE /usr/local/lib/samba4
32-bit compatibility ldconfig path: /usr/local/lib32/compat
Setting hostname: truenas.local.
Syncing multipaths...
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
Starting interfaces...
igb1: link state changed to DOWN
igb0: link state changed to DOWN
igb1: link state changed to UP
igb0: link state changed to UP
Generating configuration for interface_sync checkpoint
Loading kernel modules:
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
Error: vmware-checkvm must be run inside a virtual machine on a VMware hypervisor product.
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating /var/run/os-release done.
Creating and/or trimming log files.
Updating motd:.
Starting syslog_ng.
Starting watchdogd.
Security policy loaded: MAC/ntpd (mac_ntpd)
Starting ntpd.
Mounting late filesystems:.
Starting dbus.
Starting rrdcached.
Performing sanity check on openssh configuration.
Starting openssh.
Starting avahi-daemon.
starting wsdd
Starting cron.
Starting smartd_daemon.
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.
Starting collectd_daemon.
Configuring vt: blanktime.
net.inet.carp.allow: 0 -> 1
root@truenas[~]#
Click to expand...
Connected in SATA the problem does not really seem to occur.
 
Last edited:
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Try a clean install using the UEFI route, see if something changes... for the better.
 
A

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,177
Alecmascot said:
Re the Metaslab messages, I can find "stuff" in google but it does not make sense to me except that the messages are associated with the failing mode of using the SAS controller and RAIDZX.
Click to expand...
After your last post, this is not true!
Let's stick with the zio errors....
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "zio read error"

Similar threads

S
error in resilvering
Replies
1
Views
842
Arwen
Arwen
dewhite04
Unhealthy pool, read errors, device reset.
Replies
5
Views
7K
dewhite04
dewhite04
S
checksum errors on u.2 disks connected to nvme after upgrade
Replies
1
Views
1K
souporman
S
D
Will I finally have a reliable server?
Replies
7
Views
1K
daweed
D
EvilRSA
Sanity check - Just got an alert about a failing drive that was replaced over a month ago.
Replies
5
Views
2K
NugentS
NugentS
Top