Truenas Bootloop after drive removal

[Yous34]

I got notification that one of my drives was failing, my dedup drive that was a 8gb usb stick, i have 8 TB of total storage, so i removed it with the zpool command, but as i did so system crashed, and i can't boot again, i tried doing an "upgrade" install with the iso but nothing happened, did a clean install and imported the pool and the system went in boot loop again, if more information is needed i can provide, I'll attach a couple of pictures of the errors i got while booting

 

[sretalla]

Well, you gave no hardware information other than an 8GB USB stick for dedup and 8TB worth of pool (not even which pool layout).

What I can tell you is that without that USB stick working, you're probably looking at a lost pool.

It's probably too late for you to be getting this advice, but using a single USB stick as a pool integral device is pool suicide.

If you elect to continue using ZFS with what I would guess is low-end consumer hardware, you'll want to make better decisions about how you put it to work and understanding what will help and what won't.

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often since about 2022. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data pools. On occasion, people use USB drives, (flash...

www.truenas.com

It's not something that would be recommended at all, but the only "safe" thing you could really do with a USB disk is L2ARC... which would be a waste of time anyway in terms of performance and likely burn out in weeks or very few months, but at least it wouldn't take your pool down with it.

 

[Yous34]

Well, you gave no hardware information other than an 8GB USB stick for dedup and 8TB worth of pool (not even which pool layout).

What I can tell you is that without that USB stick working, you're probably looking at a lost pool.

It's probably too late for you to be getting this advice, but using a single USB stick as a pool integral device is pool suicide.

If you elect to continue using ZFS with what I would guess is low-end consumer hardware, you'll want to make better decisions about how you put it to work and understanding what will help and what won't.

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often since about 2022. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data pools. On occasion, people use USB drives, (flash...

www.truenas.com

It's not something that would be recommended at all, but the only "safe" thing you could really do with a USB disk is L2ARC... which would be a waste of time anyway in terms of performance and likely burn out in weeks or very few months, but at least it wouldn't take your pool down with it.

So Hardware specs:
Cpu i9 11900k
cooler: noctua L9x65 with a higher cfm fan
ram: 32 gb single stick corsair vengeance 3000mhz
mobo: Aorus ultra z590i
Storage:
2 x 2tb ssds (1 gen 4 1 gen 3)
4 x 1 tb (3 nvmes 1 sata)
1x 500gb sata ssd as boot drive

Running a pcie x16 to 4x nvme but using 3 becuase of pcie bifurcation
pool was a stripe, first error i know, but was already too late to reconfigure it

is there anyway to recover at least some of the data from the pool? even using a different os, because from what i can see the data it's still all there and i wasn't running and encrypted pool

 

[sretalla]

ZFS Deduplication

Provides general information on ZFS deduplication in TrueNAS, hardware recommendations, and useful deduplication CLI commands.

www.truenas.com

The DDT is a fundamental ZFS structure. It is treated as part of the pool’s metadata. If a pool (or any dataset in the pool) has ever contained deduplicated data, the pool will contain a DDT, and that DDT is as fundamental to the pool data as any of its other file system tables. Like any other metadata, DDT contents may temporarily be held in the ARC (RAM/memory cache) or L2ARC (disk cache) for speed and repeated use, but the DDT is not a disk cache. It is a fundamental part of the ZFS pool structure, how ZFS organises pool data on its disks. Therefore like any other pool data, if DDT data is lost, the pool is likely to become unreadable. So it is important it is stored on redundant devices.

 

[Yous34]

Update:
i ran ubuntu server from a usb stick and i can access the files, but i don't have storage to move them to, is my only option to buy storage to move x tb of storage equal to the ammount of data i have to move the old files to ad recreate the pool that way, or is there another way to directly repair the pool?

 

[HoneyBadger]

is my only option to buy storage to move x tb of storage equal to the ammount of data i have to move the old files to ad recreate the pool that way, or is there another way to directly repair the pool?

If you are actually able to mount your pool read-only and see the data, purchasing additional storage space and copying all files off is the only option I can recommend; I would not recommend any action that could cause writes to your existing disks.

 

[Yous34]

If you are actually able to mount your pool read-only and see the data, purchasing additional storage space and copying all files off is the only option I can recommend; I would not recommend any action that could cause writes to your existing disks.

will do, so what would be a good pool configuration, i'm done with stripe, i will have 4 1tb drives, 2 2tb, and either an 8tb, if it's better i can buy 2 4tb instead (the latter would be Hdd, rest is ssd)

 

[HoneyBadger]

will do, so what would be a good pool configuration, i'm done with stripe, i will have 4 1tb drives, 2 2tb, and either an 8tb, if it's better i can buy 2 4tb instead (the latter would be Hdd, rest is ssd)

I'm assuming that all of those initial devices are NVMe SSDs save for the one 1TB M.2 device called out as SATA.

It seems like you're after performance - but the mixed sizes means you'll have to adjust a little. I'd do the 2x2TB in a mirror (2TB usable) and the 4x1TB in RAIDZ1 (3TB usable) - use the additional 2x4TB HDD in a mirror setup (4TB usable) for your "bulk data" and possible to hold backup/snapshots of the more valuable data on the RAIDZ1.