Hello Everyone,
So I had some major issues upgrading from 11.2U8 to 11.3U1, and I was put in a position where I had to wipe my OS SSDs, and install a clean version of 11.3U1 on them. After doing so, one of my two pools came back just fine. The other one didn't. Both pools are connected to the same LSI SAS9211-8i HBA. Any help getting the pool back online would be greatly appreciated. All of my VMs live on this pool.
HBA Info:
The pool that came back fine was a Raid 10 of 4x8TB Sata HDDs. (SpinningRust).
The pool that didn't come back was a Raid 10 of 4x400GB SAS SSDs (SSD). The drives having issues are SanDisk LT0400MO drives, and are da0-da3.
CAMCONTROL Shows the disks.
The drives show up in /dev
diskinfo shows that only one of the disks are present.
geom disk list shows the drives having 0B.
So I had some major issues upgrading from 11.2U8 to 11.3U1, and I was put in a position where I had to wipe my OS SSDs, and install a clean version of 11.3U1 on them. After doing so, one of my two pools came back just fine. The other one didn't. Both pools are connected to the same LSI SAS9211-8i HBA. Any help getting the pool back online would be greatly appreciated. All of my VMs live on this pool.
HBA Info:
Code:
Exiting SAS2Flash.
root@NAS1[~]# sas2flash -c 1 -list
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved
Adapter Selected is a LSI SAS: SAS2008(B2)
Controller Number : 1
Controller : SAS2008(B2)
PCI Address : 00:04:00:00
SAS Address : 500605b-0-04e1-3e60
NVDATA Version (Default) : 14.01.00.08
NVDATA Version (Persistent) : 14.01.00.08
Firmware Product ID : 0x2213 (IT)
Firmware Version : 20.00.07.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9211-8i
BIOS Version : 07.39.02.00
UEFI BSD Version : N/A
FCODE Version The pool that came back fine was a Raid 10 of 4x8TB Sata HDDs. (SpinningRust).
The pool that didn't come back was a Raid 10 of 4x400GB SAS SSDs (SSD). The drives having issues are SanDisk LT0400MO drives, and are da0-da3.
Code:
root@NAS1[/]# zpool import
pool: SSD
id: 14792298917610002332
state: UNAVAIL
status: The pool was last accessed by another system.
action: The pool cannot be imported due to damaged devices or data.
see: http://illumos.org/msg/ZFS-8000-EY
config:
SSD UNAVAIL insufficient replicas
mirror-0 UNAVAIL insufficient replicas
2560435512279638578 UNAVAIL cannot open
7379458405662909057 UNAVAIL cannot open
mirror-1 DEGRADED
11272517624822511865 UNAVAIL cannot open
gptid/3004cd18-2695-11ea-9a68-002590e465b0 ONLINE
logs
gptid/0e29fabb-2739-11ea-9a68-002590e465b0 ONLINE
root@NAS1[/]#
root@NAS1[/]#
root@NAS1[/]# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
SpinningRust 14.5T 4.65T 9.85T - - 2% 32% 1.00x ONLINE /mnt
freenas-boot 103G 1.00G 102G - - 0% 0% 1.00x ONLINE -
root@NAS1[/]#
CAMCONTROL Shows the disks.
Code:
root@NAS1[/]# camcontrol devlist <SanDisk LT0400MO C401> at scbus1 target 4 lun 0 (pass0,da0) <SanDisk LT0400MO C401> at scbus1 target 5 lun 0 (pass1,da1) <SanDisk LT0400MO C401> at scbus1 target 6 lun 0 (pass2,da2) <SanDisk LT0400MO C404> at scbus1 target 7 lun 0 (pass3,da3) <ATA ST8000VN0022-2EL SC61> at scbus1 target 8 lun 0 (pass4,da4) <ATA ST8000VN0022-2EL SC61> at scbus1 target 9 lun 0 (pass5,da5) <ATA ST8000VN0022-2EL SC61> at scbus1 target 10 lun 0 (pass6,da6) <ATA ST8000VN0022-2EL SC61> at scbus1 target 11 lun 0 (pass7,da7) <SAMSUNG SSD SM841N mSATA 128GB DXM45D0Q> at scbus2 target 0 lun 0 (pass8,ada0) <SAMSUNG SSD SM841N mSATA 128GB DXM45D0Q> at scbus3 target 0 lun 0 (pass9,ada1) <Raritan Virtual Disc 1 0.01> at scbus7 target 0 lun 0 (pass10,cd0) <Raritan Virtual Disc 2 0.01> at scbus8 target 0 lun 0 (pass11,da8) root@NAS1[/]#
The drives show up in /dev
Code:
root@NAS1[/]# ls /dev acpi cpuctl12 cpuctl33 da2 fido nfslock rdma_cm ttyva ada0 cpuctl13 cpuctl34 da3 full null reroot ttyvb ada0p1 cpuctl14 cpuctl35 da3p1 geom.ctl nvd0 snp ufssuspend ada0p2 cpuctl15 cpuctl36 da3p2 gptid nvd0p1 stderr ugen0.1 ada0p3 cpuctl16 cpuctl37 da4 hpet0 nvd0p2 stdin ugen0.2 ada1 cpuctl17 cpuctl38 da4p1 io nvme0 stdout ugen0.3 ada1p1 cpuctl18 cpuctl39 da4p2 ipmi0 nvme0ns1 sysmouse ugen1.1 ada1p2 cpuctl19 cpuctl4 da5 iscsi pass0 ttyu0 ugen2.1 ada1p3 cpuctl2 cpuctl5 da5p1 kbd1 pass1 ttyu0.init ugen3.1 apm cpuctl20 cpuctl6 da5p2 kbd2 pass10 ttyu0.lock ugen3.2 apmctl cpuctl21 cpuctl7 da6 kbd3 pass11 ttyu1 ugen3.3 audit cpuctl22 cpuctl8 da6p1 kbdmux0 pass2 ttyu1.init ugen3.4 auditpipe cpuctl23 cpuctl9 da6p2 klog pass3 ttyu1.lock uhid0 bpf cpuctl24 crypto da7 kmem pass4 ttyv0 ukbd0 bpf0 cpuctl25 ctty da7p1 led pass5 ttyv1 ukbd1 cam cpuctl26 cuau0 da7p2 mdctl pass6 ttyv2 ums0 cd0 cpuctl27 cuau0.init da8 megaraid_sas_ioctl_node pass7 ttyv3 urandom console cpuctl28 cuau0.lock devctl mem pass8 ttyv4 usb consolectl cpuctl29 cuau1 devctl2 mirror pass9 ttyv5 usbctl cpuctl0 cpuctl3 cuau1.init devstat mlx5ctl pci ttyv6 xpt0 cpuctl1 cpuctl30 cuau1.lock dtrace mps0 ptmx ttyv7 zero cpuctl10 cpuctl31 da0 dumpdev mps1 pts ttyv8 zfs cpuctl11 cpuctl32 da1 fd mrsas0 random ttyv9
diskinfo shows that only one of the disks are present.
Code:
root@NAS1[/]# diskinfo da0 diskinfo: da0: ioctl(DIOCGMEDIASIZE) failed, probably not a disk. root@NAS1[/]# diskinfo da1 diskinfo: da1: ioctl(DIOCGMEDIASIZE) failed, probably not a disk. root@NAS1[/]# diskinfo da2 diskinfo: da2: ioctl(DIOCGMEDIASIZE) failed, probably not a disk. root@NAS1[/]# diskinfo da3 da3 512 400088457216 781422768 4096 0 48641 255 63 root@NAS1[/]# diskinfo da4 da4 512 8001563222016 15628053168 4096 0 972801 255 63
geom disk list shows the drives having 0B.
Code:
root@NAS1[/]# geom disk list Geom name: nvd0 Providers: 1. Name: nvd0 Mediasize: 512110190592 (477G) Sectorsize: 512 Mode: r1w1e3 descr: SAMSUNG MZVLB512HAJQ-000L7 ident: S3TNNF0K423068 rotationrate: 0 fwsectors: 0 fwheads: 0 Geom name: cd0 Providers: 1. Name: cd0 Mediasize: 0 (0B) Sectorsize: 2048 Mode: r0w0e0 descr: Raritan Virtual Disc 1 ident: (null) rotationrate: unknown fwsectors: 0 fwheads: 0 Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 128035676160 (119G) Sectorsize: 512 Mode: r2w2e4 descr: SAMSUNG SSD SM841N mSATA 128GB lunid: 5002538500000000 ident: S1JXNYAF625690 rotationrate: 0 fwsectors: 63 fwheads: 16 Geom name: ada1 Providers: 1. Name: ada1 Mediasize: 128035676160 (119G) Sectorsize: 512 Mode: r2w2e4 descr: SAMSUNG SSD SM841N mSATA 128GB lunid: 5002538500000000 ident: S1JXNYAF523127 rotationrate: 0 fwsectors: 63 fwheads: 16 Geom name: da0 Providers: 1. Name: da0 Mediasize: 0 (0B) Sectorsize: 512 Mode: r0w0e0 descr: SanDisk LT0400MO lunid: 5001e82002897b60 ident: 42564448 rotationrate: 0 fwsectors: 0 fwheads: 0 Geom name: da1 Providers: 1. Name: da1 Mediasize: 0 (0B) Sectorsize: 512 Mode: r0w0e0 descr: SanDisk LT0400MO lunid: 5001e82002897bac ident: 42564524 rotationrate: 0 fwsectors: 0 fwheads: 0 Geom name: da2 Providers: 1. Name: da2 Mediasize: 0 (0B) Sectorsize: 512 Mode: r0w0e0 descr: SanDisk LT0400MO lunid: 5001e820028a9c14 ident: 42638356 rotationrate: 0 fwsectors: 0 fwheads: 0 Geom name: da3 Providers: 1. Name: da3 Mediasize: 400088457216 (373G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r0w0e0 descr: SanDisk LT0400MO lunid: 5001e820028b6f30 ident: 42692400 rotationrate: 0 fwsectors: 63 fwheads: 255 Geom name: da4 Providers: 1. Name: da4 Mediasize: 8001563222016 (7.3T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e5 descr: ATA ST8000VN0022-2EL lunid: 5000c500b2cf477d ident: ZA1BS9A1 rotationrate: 7200 fwsectors: 63 fwheads: 255 Geom name: da5 Providers: 1. Name: da5 Mediasize: 8001563222016 (7.3T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e5 descr: ATA ST8000VN0022-2EL lunid: 5000c500b2cfe681 ident: ZA1D96YE rotationrate: 7200 fwsectors: 63 fwheads: 255 Geom name: da6 Providers: 1. Name: da6 Mediasize: 8001563222016 (7.3T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e5 descr: ATA ST8000VN0022-2EL lunid: 5000c500b2d12f5d ident: ZA1D8PBC rotationrate: 7200 fwsectors: 63 fwheads: 255 Geom name: da7 Providers: 1. Name: da7 Mediasize: 8001563222016 (7.3T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e5 descr: ATA ST8000VN0022-2EL lunid: 5000c500b2d24069 ident: ZA1D9QYF rotationrate: 7200 fwsectors: 63 fwheads: 255 Geom name: da8 Providers: 1. Name: da8 Mediasize: 0 (0B) Sectorsize: 512 Mode: r0w0e0 descr: Raritan Virtual Disc 2 ident: 2DB31DABE15E8AE rotationrate: unknown fwsectors: 0 fwheads: 0
