System
I was running my server on nightly builds from early 21.06, but all of the sudden server crashed suddenly (still investigating root cause) few days ago. At that moment was on build nightly 21.08-MASTER-20210812-172919, after powering server back Z2 was gone.
To be honest was surprised, since it has 12 disks, so suppose to survive up to 2 disks loss in Z2, as well as there was 2 hot spares, so I was quite confident it will survive any crash... I have been mistaken..
I tried to bring Z2 pool online manually with -f. It came online, in degraded state, stating there is resilver-ing process going on, but looking on iotop, zpool iostat, was seating there for hours w/o any visible activity.
After a day, I lost patience
and considering I did not see any signs of disk IO, decided to try reboot. Did not help.
Tried to update to most recent nightly build 21.08-MASTER-20210830-23-2921, did not help either. After another round of reboots, not it has been for over 2.5 days trying to import ZFS pools and do not load at all.
Loaded ISO for 21.08 BETA.1, tried to boot and upgrade, which had failed.
This this I would assume I would have to say goodbye to 20 VMs state hosted on NFS share off that TrueNAS Scale
and RIP curated ISO and some apps installation for Windows VMs.
If anybody from iXSystems interested to collect logs and investigate for future improvements, I can grab some and share, before I wipe out system in next few days.
Here are some screenshots with errors:
Filled out - https://jira.ixsystems.com/browse/NAS-112174
Dell R730xd, CPU: 2 x Intel(R) Xeon(R) CPU E5-2650L v3 @ 1.80GHz
Memory: 128GB ECC, Dell HBA330 mini (IT mode)
Disks / Pools:
Mirror (boot)
2 x 250GB 12Gb/s SAS SSD (LENOVO HUSMR1625ASS20E)
RAID-Z2
12 x 1.9TB 12Gb/s SAS SSD (SAMSUNG MZILS1T9HCHP 003)
HOT SPARES
2 x 1.9TB 12Gb/s SAS SSD (SAMSUNG MZILS1T9HCHP 003)
SLOG cache
2 x 512GB NVMe SSD
I was running my server on nightly builds from early 21.06, but all of the sudden server crashed suddenly (still investigating root cause) few days ago. At that moment was on build nightly 21.08-MASTER-20210812-172919, after powering server back Z2 was gone.
To be honest was surprised, since it has 12 disks, so suppose to survive up to 2 disks loss in Z2, as well as there was 2 hot spares, so I was quite confident it will survive any crash... I have been mistaken..
I tried to bring Z2 pool online manually with -f. It came online, in degraded state, stating there is resilver-ing process going on, but looking on iotop, zpool iostat, was seating there for hours w/o any visible activity.
After a day, I lost patience
Tried to update to most recent nightly build 21.08-MASTER-20210830-23-2921, did not help either. After another round of reboots, not it has been for over 2.5 days trying to import ZFS pools and do not load at all.
Loaded ISO for 21.08 BETA.1, tried to boot and upgrade, which had failed.
This this I would assume I would have to say goodbye to 20 VMs state hosted on NFS share off that TrueNAS Scale
If anybody from iXSystems interested to collect logs and investigate for future improvements, I can grab some and share, before I wipe out system in next few days.
Here are some screenshots with errors:
Filled out - https://jira.ixsystems.com/browse/NAS-112174
Last edited: