Cant create Pool on TrueNas Scale

[gefidalgo]

i have recently installed TrueNas Scale on a new Server (i am already running it on another 2, although different hardware). Truenas does recognize the Disks but while attempting to create a pool (even a Single Disk Stripe) i am getting following error log:

Error: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 378, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 414, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1001, in nf
return await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 673, in do_create
formatted_disks = await self.middleware.call('pool.format_disks', job, disks)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1239, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1196, in _call
return await methodobj(*prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/format_disks.py", line 56, in format_disks
await asyncio_map(format_disk, disks.items(), limit=16)
File "/usr/lib/python3/dist-packages/middlewared/utils/asyncio_.py", line 16, in asyncio_map
return await asyncio.gather(*futures)
File "/usr/lib/python3/dist-packages/middlewared/utils/asyncio_.py", line 13, in func
return await real_func(arg)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/format_disks.py", line 32, in format_disk
devname = await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1239, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1207, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1111, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
File "/usr/lib/python3/dist-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/disk_/disk_info_linux.py", line 97, in gptid_from_part_type
raise CallError(f'Partition type {part_type} not found on {disk}')
middlewared.service_exception.CallError: [EFAULT] Partition type 6a898cc3-1dd2-11b2-99a6-080020736631 not found on sda

Server is a HS-1235T from extracted from an IBM XIV storage Gen3. SAS controller is an LSI 9201-8i (on IT mode from fabric)

I cant make sense on the error, it is well over my current understanding of TrueNAS.

 

[sretalla]

I think you should probably do some quick searching on ashift=9 and reformatting the disks.

 

[gefidalgo]

I continue digging into this, i swaped with another disk i had and there was no issue to create the pool (the problem presents itself with this specific hard drives)

# I deleted all partitions

Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): d
Partition number (1,2, default 2): 2

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

# checked

truenas# cat /proc/partitions
major minor #blocks name

8 0 1953514584 sda

#went back to create pool and rechecked partitions (at pool creation 2 partitions are created)

truenas# fdisk -l /dev/sda
Disk /dev/sda: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000NM0043 A4
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 32B4345D-D7A5-4776-94A6-6E1CA7D7E195

Device Start End Sectors Size Type
/dev/sda1 128 4194304 4194177 2G Linux swap
/dev/sda2 4194432 3907029134 3902834703 1.8T Solaris /usr & Apple ZFS

# note that "/dev/sda2" is of type "Solaris /usr & Apple ZFS" which is partition type / Alias < 67 Solaris /usr & Apple ZFS 6A898CC3-1DD2-11B2-99A6-080020736631>

Neverthesless, at pool creation it still states:

f'Partition type {part_type} not found on {disk}')
middlewared.service_exception.CallError: [EFAULT] Partition type 6a898cc3-1dd2-11b2-99a6-080020736631 not found on sda

# I dont know why, even if the partition is created why it cant be found later.

 

[gefidalgo]

I think you should probably do some quick searching on ashift=9 and reformatting the disks.

I can Confirm its a problem With TrueNAS scale. I decided to test on TrueNas Core and pool are creates correctly without errors using same Hardware.

 

[csj]

Hi @gefidalgo please see https://jira.ixsystems.com/browse/NAS-110846 for details on my findings

 

[gefidalgo]

Hi @gefidalgo please see https://jira.ixsystems.com/browse/NAS-110846 for details on my findings

Hi Caleb. I did follow your suggestion on wiping and reformatting but the end results keeps the same (cant create the pool)

also i Need to mention again that in FreeBSD TrueNAS core this works flawlessly from the GUI without bothering with formatitng and ceroing the drives to remove bits protection if present. (my results have being consistent, i have 9 blades servers, all with same drives, 8 of them now with TrueNAS core and 1 with TrueNAS Scale. Only not working on the TrueNAS Scale environment.

 

[longsim]

I face the same problem, on the Scale 22.02 RC1.1 ver.

 

[fw_crocodile]

Same problem here on TureNAS SCALE. I'm trying to create a vdev with 3 disks. The error come out most of the time for the first one, but not always.

The partitions seem correctly created on all disk. Could be only a too short timeout somewhere?

 

[fw_crocodile]

Same problem here on TureNAS SCALE. I'm trying to create a vdev with 3 disks. The error come out most of the time for the first one, but not always.

The partitions seem correctly created on all disk. Could be only a too short timeout somewhere?

Everything went fine after rebooting the NAS. I think something about updating the state of the disks itself. In fact in /dev, some (not all) newly created partitions where effectively missing before the reboot.

 

[ScaleyNas]

Everything went fine after rebooting the NAS. I think something about updating the state of the disks itself. In fact in /dev, some (not all) newly created partitions where effectively missing before the reboot.

I'm using, learning and installing SCALE for the first time today. I had this problem too. 2 brand new Seagate IronWolf 6TB drives and 2 I already owned. I tried to create a 3 disk ZF1 pool, failed with the EFAULT error. OK. Let me wipe the disks. Wiped all the disks and still got the "EFAULT error partition type not found..."
Having read your post, I rebooted and...tah-dah! was immediately able to create the pool.

 

[pixelwave]

Shutting down Truenas Scale > hooking up 3x WD Red 3.5 Disks > reboot > trying to setup ZFS pool (z1) with those three drives > same error!

Reboot System again > trying setup again > works!

Infact .. rebooting seems to solve that issue.

 

[avgJoe]

I had the same problem. I was able to create a pool with 4x1TB HDD with no issues on 4 nodes. All 4 nodes failed when trying to add the two SSD drives to a new pool. Rebooting all 4 nodes allowed me to add SSD without problem.
I have a Nutanix NX-3460-G4 (Haswell)

 

[truefriend-cz]

I have the same problem.

 

[pixelwave]

I have the same problem.

Did you try reboot?

 

[truefriend-cz]

Did you try reboot?

Yes. After restart it is fixed. The question is for how long.

 

[Adam Flagg]

I also saw this issue and after rebooting was able to create the initial pool and data vdev. I'm not sure why this was an issue however. My disks were previously (3 of 4) in a Synology formatted with BTRFS and one had previously been wiped already.

 

[Bl00dWolf]

Thx you guys.
Exactly same happened to me today. After reboot all is ok

 

[majorgear]

I ran into the same issue on my virtualized TrueNAS server. Rebooting the virtual machine didn't change anything, but rebooting the physical machine ( hypervisor ) resolved the issue. It would stop formatting at 15% and then throw the error. And it was a different drive every time.

 

[appliance]

Same here. Need to put it in the guide, reboot required before the main function of the appliance can be applied;)