SOLVED Disk Replacement - failed on TrueNAS-12.0-U3.1

[cherup]

Im am on TrueNAS-12.0-U3.1 and using Freenas / Truenas quite a while.
I have replaced HDD several times without problem - but currently I am struggeling with the replacement.

The new HDD / dev/ada1 is seen as a disk in the GUI. When trying to to the replacement the following error appears:

Error: Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/middlewared/job.py", line 367, in run
await self.future
File "/usr/local/lib/python3.8/site-packages/middlewared/job.py", line 403, in __run_body
rv = await self.method(*([self] + args))
File "/usr/local/lib/python3.8/site-packages/middlewared/schema.py", line 973, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/replace_disk.py", line 93, in replace
enc_disks = await self.middleware.call(
File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1238, in call
return await self._call(
File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1195, in _call
return await methodobj(*prepared_call.args)
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/format_disks.py", line 56, in format_disks
await asyncio_map(format_disk, disks.items(), limit=16)
File "/usr/local/lib/python3.8/site-packages/middlewared/utils/asyncio_.py", line 16, in asyncio_map
return await asyncio.gather(*futures)
File "/usr/local/lib/python3.8/site-packages/middlewared/utils/asyncio_.py", line 13, in func
return await real_func(arg)
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/format_disks.py", line 29, in format_disk
await self.middleware.call(
File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1238, in call
return await self._call(
File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1206, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1110, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.8/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/disk_/format.py", line 25, in format
raise CallError(f'Failed to wipe disk {disk}: {job.error}')
middlewared.service_exception.CallError: [EFAULT] Failed to wipe disk ada1: [EFAULT] Command dd if=/dev/zero of=/dev/ada1 bs=1M count=32 failed (code 1):
dd: /dev/ada1: Input/output error
1+0 records in
0+0 records out
0 bytes transferred in 0.954561 secs (0 bytes/sec)

Fdisk reports the following:

root@freenas:~ # fdisk /dev/ada1
******* Working on device /dev/ada1 *******
parameters extracted from in-core disklabel are:
cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

fdisk: invalid fdisk partition table found
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 4294966977 (2097151 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 15/ head 15/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>

Gpart output:

root@freenas:~ # gpart status
Name Status Components
ada0p1 OK ada0
ada0p2 OK ada0
ada2p1 OK ada2
ada2p2 OK ada2
da0p1 OK da0
da0p2 OK da0
da2p1 OK da2
da2p2 OK da2

and zpool status:

pool: trinity
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: scrub repaired 0B in 06:56:50 with 0 errors on Sun May 16 06:56:50 2021
config:

NAME STATE READ WRITE CKSUM
trinity DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
gptid/197be059-6d27-11ea-9595-d050994450a4 ONLINE 0 0 0
gptid/80f80c06-9205-11eb-99ae-d050994450a4 OFFLINE 0 0 0
gptid/5e265f86-9a29-11e7-ae8b-d050994450a4 ONLINE 0 0 0

So it seems that no disk operationd / format etc. are working.
Any help is appreaciated.

BR,
Cherup

 

[cherup]

It is solved - was no truenas problem, but part of my server hardware was faulty what causes this problem. After moving to a new hardware the new HDD was added without problems....