TrueNAS Scale - System freeze

[friendlyguy]

So, i was able to write a TB with fio. I was never able to transfer that much via smb.
I think thats progress: currently thinking its probably the pci-e slot where the nic sits in.
I exchanged the nic for a test, but had the same freeze.
the command i used: "fio --rw=write --name=test --size=1TB"

 

[HoneyBadger]

Which NIC did you exchange it with - the same model?

Certain models of 10Gbps NICs are also notorious for running hot - I know you mentioned earlier on that you have a "wind tunnel" going on with the plastic air shroud, but there's an option for a second fan. Can you install that, and possibly share some pictures of the slot area with an eye towards the cooling?

Somewhat counter-intuitively, leaving the slot cover off in the gap between the HBA and NIC may actually make things worse - if there's an area where air can freely rush out, it might take the path of least resistance and not actually blow across the fins of the heatsink on your cards.

 

[friendlyguy]

@HoneyBadger: i exchanged the brocade(CNA1020) against an intel(x520) but that didnt help.

Currently the system is running and i cant make a picture but let me describe the airflow for you:
Direction is front to back.
By Design there are 3 Fans in the front of the chassis:
2 blow into an airshroud that covers cpu, ram and nortbridge.( +-)
The third fan is blowing at the pci slots.
Then there are 2 more fans at the back of the chassis: both are sucking air out of that shroud.
In addition to the 5 "stock" fans i put in another 80mm fan that sits a few cm in front of the hba and the nic that also blows to the back of the chassis.
Nevertheless: I`ll make a picture the next time i pull the server out of the rack.

I just let iperf3 run for an Hour:

that also didnt cause the system to freeze.

 

[Davvo]

I bring up again a clean install as a troubleshoot step, just to make sure you aren't bashing you head against something easily solvable.

 

[friendlyguy]

Morning Gents!
I`ve just replaced the board / cpu / ram against identical devices.
Throwing with data at it "as we speak", lets see if its related to any of those components i replaced.
BTW: Case is a Supermicro CSE-846TQ-r900, found a sticker on it.
Also took a bunch of pictures of its entrails: (sorry, couldnt figure out how to set the orientation of the pictures in this forum.)

 

[friendlyguy]

okay guys, i believe it was one of the following: cpu/ ram / board.
I`ve copied several TB of data onto the system and so far i didn't experience any freezes.