Another pool degraded question.

[bar1]

Hi everyone,
Sorry I am sure it was asked many time but I can't find the answer.

I have a degraded pool, and I suspect this is the reason:
errors: Permanent errors have been detected in the following files:

<0x9572>:<0x927>
/Bar1-xxx/xxx/xxx/xx/vm-123-disk-0.raw

This file can be deleted however I can only mount my pool as read only....


How do I go about deleting this file and getting my pool up and running again?

 

[bar1]

By the way first prize will be to try and repair this file, but I do have a backup

 

[bar1]

Is this safe to do?

 

[Arwen]

You really need to learn both ZFS and TrueNAS, (based on your questions).

Please follow the forum rules, (up at the top of any forum page, in bold red). And the output of zpool status as loss of a file indicates that you probably have pool problems.

 

[bar1]

Thanks for the useful info.

 

[Arwen]

What I meant was that the forum rules request hardware configuration, version of TrueNAS and a description of your ZFS pool put in your post. (Or follow on post.)

I also suggested the output of the command zpool status but was not clear that you should put it in a post so we can look at it.

As for the reading, their are a TON of Resources, (and some sticky forum threads). Some forum members keep the suggested newer user ones in their signature. But, I have not yet done that, so I have to look them up. Here are some of the ones I mean;

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

ZFS Storage Pool Layout

This resource was originally created by user: @Davvo on the TrueNAS Community Forums Archive. https://www.truenas.com/community/resources/zfs-storage-pool-layout.201/download [1] This amazing document, created by iXsystems in February 2022 as a “White Paper”, cleanly explains how to qualify...

www.truenas.com

To answer your question, "Is this safe to do?", depends on what you want to accomplish. Let's stick to the pool corruption first.

 

[bar1]

I would like to get my pool up and running,
I am running truenas scale, without raid, I am willing to lose the corrupt file but prefer not to.
I do not want to loose the whole ZFS volume / the datasets on this hard drive, but to repair it and get it online.

I thought it was obvious....apologizes, let me clarify.
Is it safe to add the unassigned disk to the new pool?
Meaning safe as no data will be erased from the disk.

By the way, I can mount the disk just fine as read only.


root@truenas[/]# zpool status
pool: Bar1-8TB
state: ONLINE
scan: scrub repaired 0B in 17:32:29 with 0 errors on Sun Dec 18 17:32:46 2022
config:

NAME STATE READ WRITE CKSUM
Bar1-8TB ONLINE 0 0 0
b43e1073-1684-4339-bce3-302c3d9e2e8c ONLINE 0 0 0

errors: No known data errors

pool: Bar1-APPS
state: ONLINE
scan: scrub repaired 0B in 00:01:51 with 0 errors on Sun Dec 11 00:02:02 2022
config:

NAME STATE READ WRITE CKSUM
Bar1-APPS ONLINE 0 0 0
6e39535a-010b-46e8-a9b9-c847a829251e ONLINE 0 0 0

errors: No known data errors

pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:01:23 with 0 errors on Tue Jan 3 03:46:25 2023
config:

NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sdc3 ONLINE 0 0 0

errors: No known data errors
root@truenas[/]#


Currently not even showing the corrupted pool.
If I mount as readonly:

oot@truenas[/3]# zpool status -v
pool: Bar1-18TB
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
scan: scrub in progress since Sun Dec 18 00:00:09 2022
4.40T scanned at 0B/s, 4.40T issued at 0B/s, 7.96T total
0B repaired, 55.34% done, no estimated completion time
config:

NAME STATE READ WRITE CKSUM
Bar1-18TB DEGRADED 0 0 0
d9d82275-2a5f-4c78-a81e-499766926f4a DEGRADED 0 0 0 too many errors

errors: Permanent errors have been detected in the following files:

<0x9572>:<0x927>
/xxxxvm-123-disk-0.raw

pool: Bar1-8TB
state: ONLINE
scan: scrub repaired 0B in 17:32:29 with 0 errors on Sun Dec 18 17:32:46 2022
config:

NAME STATE READ WRITE CKSUM
Bar1-8TB ONLINE 0 0 0
b43e1073-1684-4339-bce3-302c3d9e2e8c ONLINE 0 0 0

errors: No known data errors

pool: Bar1-APPS
state: ONLINE
scan: scrub repaired 0B in 00:01:51 with 0 errors on Sun Dec 11 00:02:02 2022
config:

NAME STATE READ WRITE CKSUM
Bar1-APPS ONLINE 0 0 0
6e39535a-010b-46e8-a9b9-c847a829251e ONLINE 0 0 0

errors: No known data errors

pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:01:23 with 0 errors on Tue Jan 3 03:46:25 2023
config:

NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sdc3 ONLINE 0 0 0

errors: No known data errors

 

[bar1]

I am very scared to do something wrong and loose my whole drive,
I read some of the documentation, and been using ZFS for years now, but never had an issue.
I will apriciate an advice here please.

 

[Etorix]

errors: Permanent errors have been detected in the following files:

<0x9572>:<0x927>
/Bar1-xxx/xxx/xxx/xx/vm-123-disk-0.raw

The latter is a real file, and could be deleted or restored individually from backup.
The former is ZFS internal metadata and cannot be repaired.

Copy everything which has changed since the last backup, destroy the pool and restore the whole lot from backup.

 

[bar1]

what exactly will happen if I do this?

 

[Davvo]

Please put the output of any code you run within the appropriate BBCode as per forum rules.
As well as please list your full hardware specs as per forum rules.

You also don't look like someone who understands what ZFS is and how it works so please read the resources you were liked to.
That file can't be recovered anymore.

 

[bar1]

SYSTEM:
TrueNAS-SCALE-22.12.0
PROXMOX VM - 16GB RAM
4 Cores
18tb Hard disk passthrough
8GB Hard disk passthrough

I see some advise to delete the file, and run scrub, but I am struggling to mount the file system not in read only

 

[Davvo]

I hate being a nitpicker.

Forum Rules

Welcome to the TrueNAS Community! Updated 11/28/2023 First and foremost I want to thank you for joining the TrueNAS community! My name is JoshDW19 and I'm the Community Manager here at TrueNAS. The TrueNAS Community is a friendly place to discuss open source software, technology, and products...

www.truenas.com

Correct me if I'm wrong: you are running TrueNAS as a VM in proxmox with 16GB of non-ECC RAM, and you are using single drives with no redundancy for your pools.

 

[Whattteva]

This.... looks ugly.
If you're going to virtualize, you need to passthrough the entire controller, not just the hard disks.

 

[bar1]

I was struggling with the controller passthrough...did a bit of research and people said hard drive passthrough is fine :-(
I guess not.

That is correct @Davvo and I promise once i sort out my ZFS I will set up my system properly as a spolier (tried to figure it out, I understand it probably super easy but I am stressing about my pool at the moment).

I do have a backup just trying to avoid a long restore.

 

[bar1]

Setup: important data on the 18TB get replicated to the 8TB
and all the data get synced to encryped google drive (however didn't get it right just yet)

 

[Davvo]

You can try zpool import -f Bar1-18TB.
Hard drive passthrough is fine if the drives are directly attached to the motherboard iirc.

 

[Whattteva]

Well, it looks like at least you did something right (backups). Plenty of people come here weeping losing tens of TB of precious data on botched sub-optimal setups. Why anyone would do that on data they care about is beyond me.

More on-topic, I'm guessing some of your meta-data got bungled because you're not quite passing through the actual controller.
You still haven't really given us your HW specs. What you instead gave us is your VM specs... which is obviously not the same thing.
My guess that you had problem passing through your controller is due to the CPU not having VT-d support.

 

[bar1]

Code:

zpool import -f Bar1-18TB

didn't work...

Ok....external hard drive is connected, let me start a copy while in read only.

I am also not too sure what to do after....do I just wipe the drive and start over?

 

[Davvo]

didn't work...

Try zpool import -F Bar1-18TB.
Although I would just nuke everything and start from scratch with a proper system.