Networking LAN deadlocks

Status
Not open for further replies.

Zarovzky

Cadet
Joined
Apr 16, 2018
Messages
6
When I built the server it was around version 10.4 or so.
The stability was great, barely a restart a month.
I have the latest update of the system as per today: FreeNAS-11.1-U4
and I have to restart it daily because networking fails multiple times.

Every day since I've updated to the version 11.1 the network fails, mainly no ping, so nothing works.
What *solves* it is to reboot directly from the button.

I do have a good system : 64gb ECC DDR4, Ryzen 1600 overclocked to 3800ghz, Asrock X370 taichi mobo.
The system is used for NextCloud mainly but also for another applications I have which hammer MySQL badly.
My Apps make CPU rise at up to 20% which is mostly taken by MySQL.

Networking graph, the spikes are moments before lan stops working.
These spikes are not caused by any app (see attachments).

I've seen this problem lurking around in 11+ versions but no solution so far.

PD: ill post more information in the upcoming days.
 

Attachments

  • lan.png
    lan.png
    12.3 KB · Views: 396
  • lan2.png
    lan2.png
    13.1 KB · Views: 373

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Thanks for posting and welcome to the forum. I have a similar issue but thought it was a flakey NIC. I look forward to seeing what you find.

In my case the interface just stops responding. I can down and up the interface but it has no effect. I am able to reboot from the console but by then my VMs have all crashed. Mine also seems to be under load before it goes down.
 

Zarovzky

Cadet
Joined
Apr 16, 2018
Messages
6
Thanks for posting and welcome to the forum. I have a similar issue but thought it was a flakey NIC. I look forward to seeing what you find.

In my case the interface just stops responding. I can down and up the interface but it has no effect. I am able to reboot from the console but by then my VMs have all crashed. Mine also seems to be under load before it goes down.
Did you replace the NIC and had same results? Same spike with the new NIC?
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
I ended up swapping the drive to a different server and saw the same issue. I also see a lot of iSCSI timeouts but i'm not sure if that's related. Old server was em and new is bce. It happen less on the "new" server but still happen when loaded for more than a a few minutes.
 

Zarovzky

Cadet
Joined
Apr 16, 2018
Messages
6
I ended up swapping the drive to a different server and saw the same issue. I also see a lot of iSCSI timeouts but i'm not sure if that's related. Old server was em and new is bce. It happen less on the "new" server but still happen when loaded for more than a a few minutes.
I fail to understand why are networking deadlocks related to the drives... My main system is not on a failing drive, the jails are located on an SSD which is barely new. Neither the apps interact with the older drives.

Could you tell me which log file did you check for the iSCSI timeouts? Will check and add mine too.
 
Last edited:

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
If you enable "Show console messages in the footer" under System -> Advanced -> (about half way down) you will see the messages as they are logged to /var/log/messages.

I fail to understand why are networking deadlocks related to the drives...
Its not the drives. I think the iSCSI (a network service) timeouts are a symptom of the underlying issue.
 
Status
Not open for further replies.
Top