Fatal trap 12: arc_reclaim_thread using iSCSI zvol

[PnoT]

I've been trying to figure out why this kernel mode fault is happening [attached file] on two different FreeNAS machines that serve up a zvol over iSCSI to a Window 2016 DPM server. The zvol is formatted and used by Data Protection Manager as a backup volume and is primarily for backing up my VMs to.

The two systems are as follows:

X8SIL-F with 16GB of ECC RAM, Intel 520 10GB NIC, 5 x 3TB Enterprise Toshiba drives, and an LSI 9211-8i on the latest firmware. The X8 was once used as an iSCSI target for DPM with the same drives (a month ago) and ran flawless until recently.

X10SRL-F with 128GB ECC DDR4, Intel 520 10GB NIC, 24 x 8TB WD RED, 5 x 3TB Enterprise drives, LSI 9211-8i, LSI 9341-8i with the latest firmware.
The X10 is my primary media server and it has never crashed and has received a little over 80TB to it at 770MB/sec without a hiccup and runs 24/7.

Zvol configuration is 5 x 3TB RAIDZ1


I first read a lot of posts on this issue and most of time it's been hardware related in nature so a full run of Memtest86+ was done on both systems for over 24 hours without fail. I then tested the 5x3TB Toshiba drives and they've never had any bad sectors and short / long tests are clean. In order to rule out my X8 the zvol was created on the X10 system and after a short period of time (2 mins) the crash happened. The transfers to the zvol were upwards of 500MB/sec.

I've also tried FreeNAS 11 Stable but that resulted in the same crash. Any ideas why this could be happening on two different systems?

 

[dlavigne]

That's often the sign of a bad boot device. Did a fresh install on a different USB stick resolve the issue? Or, does the system boot fine and the error occurs once there is a load on the system?

 

[PnoT]

That's often the sign of a bad boot device. Did a fresh install on a different USB stick resolve the issue? Or, does the system boot fine and the error occurs once there is a load on the system?

I have the same exact issue on both systems. The X8 uses a Lexar USB 64GB and the X10 has a mirrored set of Intel 320 SSDs.

Is there anywhere else, log wise, I should be looking to pull more information?

 

[rs225]

Assuming both of these don't have a Skylake CPU, this sounds like it is worth opening a ticket.

 

[Ericloewe]

Yeah, definitely file a bug report if this is reproducible on both systems. Check the installation on both with the verify install button and make sure the boot pools are healthy after a scrub, first.

 

[Ericloewe]

By the way, what's the other version, besides 11-Stable (which I presume to be -U2 or -U3)?

 

[PnoT]

The X10 is E5-1620 v3 and the X8 is a X3430 so neither are Skylake.

I'm running FreeNAS-9.10-U6 and also tried FreeNAS-11.0-U2.

Bug submission: https://bugs.freenas.org/issues/25812

 

[kurtc]

Did you ever get a resolution to this? I believe I may be experiencing the same issue.

 

[PnoT]

I did not.. just moved onto another piece of hardware and a different solution.