System crashing while replication starts or deleteing snapshot

piterek24

Cadet
Joined
Mar 19, 2022
Messages
2
Hi

My system have one error on the pool, here is status:

Code:
# zpool status -v
  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:06 with 0 errors on Sat Oct 29 03:45:06 2022
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          ada0p2    ONLINE       0     0     0

errors: No known data errors

  pool: pool1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 00:08:25 with 1 errors on Sun Oct 30 17:52:29 2022
config:

        NAME                                            STATE     READ WRITE CKSUM
        pool1                                           ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/5b20dfcb-a6f0-11ec-89e1-6c2b59f6c1f1  ONLINE       0     0     0
            gptid/5b221788-a6f0-11ec-89e1-6c2b59f6c1f1  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        pool1/ds1-wspolny@auto-2022-10-14_09-00:/SPOLKA/Administracja/DRUKI-WZORY-OGLOSZENIA/Ogloszenia wentylacja/Archiw.zip



My snapshot tasks: every hour during work days, begin 9:00, end 20:00, lifetime: 2 weeks, as on the screen:
1667150095759.png


The problem:

System is crashing every time I start replication task to remote TrueNAS or when I click "delete" in the GUI on the snapshot "auto-2022-10-14_09-00", which is mentioned in the error within the pool1.
I also tried to delete snapshot from CLI:
# zfs destroy -r pool1/ds1-wspolny@auto-2022-10-14_09-00
but immediately after I pressed enter, my system crashed.

# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
[...]
pool1/ds1-wspolny@auto-2022-10-14_09-00 3.43M - 83.9G -
[...]

Don't know how to deal with this, please advise.
Thanks!
Piotr

My system specs:

Code:
TrueNAS-12.0-U8.1

TrueNAS-12.0-U8.1
Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz

Memory: 16GB (2x8GB), non-ECC.

boot device: ada0
pool1: ada1 & ada2

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <SanDisk X600 2.5 7MM SATA 256GB X6113012> ACS-4 ATA SATA 3.x device
ada0: Serial Number ***********
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 244198MB (500118192 512 byte sectors)
ada1 at ahcich4 bus 0 scbus1 target 0 lun 0
ada1: <WDC WDS500G1R0A-68A4W0 411000WR> ACS-4 ATA SATA 3.x device
ada1: Serial Number ***********
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada1: Command Queueing enabled
ada1: 476940MB (976773168 512 byte sectors)
ada2 at ahcich5 bus 0 scbus2 target 0 lun 0
ada2: <WDC WDS500G1R0A-68A4W0 411000WR> ACS-4 ATA SATA 3.x device
ada2: Serial Number ***********
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada2: Command Queueing enabled
ada2: 476940MB (976773168 512 byte sectors)


# smartctl -a /dev/ada0
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     SanDisk X600 2.5 7MM SATA 256GB
Serial Number:    ***********
LU WWN Device Id: 5 001b44 4a900c558
Firmware Version: X6113012
User Capacity:    256,060,514,304 bytes [256 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 30 18:52:07 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


# smartctl -a /dev/ada1
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     WD Blue / Red / Green SSDs
Device Model:     WDC  WDS500G1R0A-68A4W0
Serial Number:    ***********
LU WWN Device Id: 5 001b44 8b76a1290
Firmware Version: 411000WR
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sun Oct 30 18:46:26 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

# smartctl -a /dev/ada2
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     WD Blue / Red / Green SSDs
Device Model:     WDC  WDS500G1R0A-68A4W0
Serial Number:    ***********
LU WWN Device Id: 5 001b44 8b76a1fdc
Firmware Version: 411000WR
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sun Oct 30 18:47:26 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
 

piterek24

Cadet
Joined
Mar 19, 2022
Messages
2
Hi

My system have one error on the pool, here is status:

Code:
# zpool status -v
  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:06 with 0 errors on Sat Oct 29 03:45:06 2022
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          ada0p2    ONLINE       0     0     0

errors: No known data errors

  pool: pool1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 00:08:25 with 1 errors on Sun Oct 30 17:52:29 2022
config:

        NAME                                            STATE     READ WRITE CKSUM
        pool1                                           ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/5b20dfcb-a6f0-11ec-89e1-6c2b59f6c1f1  ONLINE       0     0     0
            gptid/5b221788-a6f0-11ec-89e1-6c2b59f6c1f1  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        pool1/ds1-wspolny@auto-2022-10-14_09-00:/SPOLKA/Administracja/DRUKI-WZORY-OGLOSZENIA/Ogloszenia wentylacja/Archiw.zip



My snapshot tasks: every hour during work days, begin 9:00, end 20:00, lifetime: 2 weeks, as on the screen:
View attachment 59569

The problem:
System is crashing every time I start replication task to remote TrueNAS or when I click "delete" in the GUI on the snapshot "auto-2022-10-14_09-00", which is mentioned in the error within the pool1.
I also tried to delete snapshot from CLI:
# zfs destroy -r pool1/ds1-wspolny@auto-2022-10-14_09-00
but immediately after I pressed enter, my system crashed.

# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
[...]
pool1/ds1-wspolny@auto-2022-10-14_09-00 3.43M - 83.9G -
[...]

Don't know how to deal with this, please advise.
Thanks!
Piotr

My system specs:

Code:
TrueNAS-12.0-U8.1

TrueNAS-12.0-U8.1
Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz

Memory: 16GB (2x8GB), non-ECC.

boot device: ada0
pool1: ada1 & ada2

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <SanDisk X600 2.5 7MM SATA 256GB X6113012> ACS-4 ATA SATA 3.x device
ada0: Serial Number ***********
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 244198MB (500118192 512 byte sectors)
ada1 at ahcich4 bus 0 scbus1 target 0 lun 0
ada1: <WDC WDS500G1R0A-68A4W0 411000WR> ACS-4 ATA SATA 3.x device
ada1: Serial Number ***********
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada1: Command Queueing enabled
ada1: 476940MB (976773168 512 byte sectors)
ada2 at ahcich5 bus 0 scbus2 target 0 lun 0
ada2: <WDC WDS500G1R0A-68A4W0 411000WR> ACS-4 ATA SATA 3.x device
ada2: Serial Number ***********
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada2: Command Queueing enabled
ada2: 476940MB (976773168 512 byte sectors)


# smartctl -a /dev/ada0
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     SanDisk X600 2.5 7MM SATA 256GB
Serial Number:    ***********
LU WWN Device Id: 5 001b44 4a900c558
Firmware Version: X6113012
User Capacity:    256,060,514,304 bytes [256 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 30 18:52:07 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


# smartctl -a /dev/ada1
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     WD Blue / Red / Green SSDs
Device Model:     WDC  WDS500G1R0A-68A4W0
Serial Number:    ***********
LU WWN Device Id: 5 001b44 8b76a1290
Firmware Version: 411000WR
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sun Oct 30 18:46:26 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

# smartctl -a /dev/ada2
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     WD Blue / Red / Green SSDs
Device Model:     WDC  WDS500G1R0A-68A4W0
Serial Number:    ***********
LU WWN Device Id: 5 001b44 8b76a1fdc
Firmware Version: 411000WR
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sun Oct 30 18:47:26 2022 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

Forgot to mention - there is a core file zfs.core in /var/db/system/cores, it shows:
(gdb) core zfs.core
[New LWP 101212]
[New LWP 101489]
Core was generated by `zfs: sending pool1/ds1-wspolny@auto-2022-10-14_09-00 (100%: 6235401256/0)'.
Program terminated with signal SIGABRT, Aborted.
#0 0x0000000800873f3a in ?? ()
[Current thread is 1 (LWP 101212)]

and system crashed after dumping core.
 
Top