Zpool import stuck on boot

[stand15]

There is about a 1 in 2 chance that when I boot the system fails to import the zpool. When it fails, the web UI will report the following message:
Reporting database used size 2.67 GiB is larger than 1.01 GiB.
Also, the jobs panel will show an import job which is stuck at 80.00%.

Looking around on the forum, I found a post that suggested that this issue has been resolved in the Bluefin Beta.

SOLVED - Reporting database used size 2.67 GiB is larger than 1.01 GiB.

I'm running TrueNAS-SCALE-22.02.3 in pve, I have a raidz2 main zpool consisting of 4 16T drives (passthrough sata controller into TrueNAS), and a 50g ssd zpool for testing. 10G RAM, host CPU at j4125, working in host mode. Today, after I started TrueNAS, I received warning "Reporting database...

www.truenas.com

However, I am running TrueNAS-SCALE-22.12.1, yet the same issue still seems to persist. Or am I wrong about this assumption?

Any suggestions on how I could go about solving this issue? I have attached the debug file below.

The system has the following specs:
CPU: Xeon E-2124g
Motherboard: Asrock C246 Wsi
Ram: 2 x 8GB DDR4 Ecc memory
2 x 120g SDD for boot-pool
4 x 8TB HDD in Raidz1 for storage.
For networking I am using the two onboard NICS

 

[Nodefield]

In our systems, I have seen really long boot zpool import times when there is ridiculously huge number of snapshots (thousands++).

Import does not fail like you described, but can take hours to go through.

 

[Ericloewe]

Like <10k or closer to 100k? A few thousand snapshots shouldn't be causing such a thing.

 

[stand15]

The pool holds only a few snapshots, so I can't imagine that being the issue.

 

[Arwen]

What is / are the model(s) of the 8TB hard drives?

How are they connected, SATA system board? PCIe card?

 

[stand15]

What is / are the model(s) of the 8TB hard drives?

WD80EMAZ, 2x WD80EZAZ, Ultrastar DC HC320.

How are they connected, SATA system board? PCIe card?

The 4 HDD’s are attached to the motherboard through the OCuLINK connector. One boot drive is attached via sata. The other boot drive (nvme) is mounted in the m.w2 slot.

Looking at the motherboard diagram all sata ports are provided by the motherboard chipset (intel C246).

 

[stand15]

I updated the system to the most current release (21.12.2). Still the same behavior. The boot process gets stuck at “A start job is running for Import ZFS pools. After the 15min 16S timer is reached, the boot process continues and the webui becomes available.

After that, I had a second look at the middleware log. Here I noticed that the application's dataset, in specific, got called out for not being able to import. So I delete the dataset, rebooted, and reinstalled the application's.

This seems to have solved all issues I was having. ZFS import now takes less than 10 seconds, compared to 15+ minutes before.

 

[Arwen]

Glad you found the cause.

 

[eugenefvdm]

> I had a second look at the middleware log

I'm having a similar problem. Where is the middleware log?