z2 pool of 6 drives is resilvering suddenly, 4 drives are degraded? Help to undestand please!

Angrily4845

Cadet
Joined
Dec 13, 2022
Messages
5
I am in a bit of a (maybe) pickle and would appreciate some help, thoughts and just understanding what is happening.

So I have just shutdown my server and started it back, without any hardware changes.
The below is what is happening currently:
1698298837862.png


I do not understand concretely what is happening and about 40 mins of searching nets nothing close to what I am experiencing.
How are 4 drives suddenly degraded and 2 are resilvering if I read correctly?

Is anyone able to say with a degree of certainty if my pool will actually recover?
I checked SMART data on all these drives and it seems fine, there is another 2 drives in a striped pool off the same controller running just fine.

I am very confused at my current situation, would greatly appreciate if someone can help walk me through any troubleshooting or just explain what I have missed.

Your help and time is appreciated. Thanks!
 

Angrily4845

Cadet
Joined
Dec 13, 2022
Messages
5
Well I am at a loss, this happened, it seems its all ok now?!?

It stopped on its own like this:
1698302736774.png


I did the zpool clear at the end and now it looks like this, much better
1698302855689.png


I am actually not satisfied because I do not know what has actually happened to cause this. I have been having a rock stable time on TrueNAS for years, even doing drive replacements without issue.

Any help on identifying what gives, even tho the issue appears to be resolved?
Thanks!
 

somethingweird

Contributor
Joined
Jan 27, 2022
Messages
183
Wouldn't it be nice if we got hardware specs?
 

Angrily4845

Cadet
Joined
Dec 13, 2022
Messages
5
It could, but skipped my mind while trying to figure out what was going on

Hardware:
- Dell T630 chassis thats been modded to accept ATX and EATX motherboards
- The T630 HDD backplane modded to work with any motherboard/SAS controller
- Gigabyte MZ32-AR0 v1.0 motherboard
- AMD Epyc 7532 CPU
- 8x 32GB 2400MT Registered ECC memory
- Micron 9300 Max 3.84TB U.2 drive (VM storage as LVM thin currently while modding the last/3rd of these, otherwise CEPH is running on it for VM storage)
- Crucial MP510 960GB M.2 drive (unused this second)
- Dual Samsung 845DC PRO 400GB in a ZFS mirror running Proxmox as boot drives
- Mellanox CX3 dual 40GB NIC
- LSI SAS 9300-8i flashed with stock firmware in IT mode connected to the T630 backplane
- 5x Toshiba Enterprise 16TB HDD and 1x Toshiba Enterprise 20TB HDD in the above mentioned Z2 pool "big"
- 2x WD White 8TB HDD in a stripe in the same TrueNAS VM/same controller in pool "small"

Software:
- Proxmox 7.4 as base
- VM running TrueNAS CORE 13.0-U5.2 (assigned 8 CPU cores, 64GB memory with balooning off, q35 machine type and all VirtIO drivers for performance)
- The TrueNAS VM has the LSI SAS 9300-8i card passed through correctly to the VM

Thanks!
 
Top