Server Crashes and Loops Strange Message

[Caretaker]

My server keeps 'crashing' (probably not the right word, all connections are lost or are unresponsive) and when I connect via IPMI too see whats up this is what I see scrolling endlessly.

Anyone have any idea what is going on? I'm thinking I just need to reinstall the FreeNAS OS and recover my conf and/or tank.

Any input would be wonderful, thanks in advance.

FreeNAS-9.10-STABLE-201606270534 (dd17351)
Hardware:
ASRock E3C224
Intel E3-1246 v3 @ 3.50GHz
16GB ECC RAM
RAIDZ 4TBx3 HGST (primary)
RAIDZ 3TBx3 Seagate (connected, but unused)

 

[SweetAndLow]

Read the forum rules and fix your post.

 

[joeschmuck]

I'm thinking I just need to reinstall the FreeNAS OS and recover my conf

That is what I'd do.

Read the forum rules and fix your post.

Agreed!

 

[SweetAndLow]

You have a good setup so the errors you are seeing are probably due to a bad USB stick or freenas install. Your config file is saved on your pool or you should back it up manually if you can still access the GUI.

 

[Caretaker]

the errors you are seeing are probably due to a bad USB stick or freenas install.

I can still access the GUI, this error only occurs occasionally. Typically when I am changing settings. I will replace the USB drive and reinstall. I'm scheduled to add in an additional NIC in a few days anyway.

Thank you for your advice. Sorry my original post was lacking information.

 

[SweetAndLow]

You probably don't need another nic, why do you think you want another?

 

[Caretaker]

LACP and three jails that can be a little throughput hungry. I've had some DHCP/MAC address/port forwarding issues where adjusting one jail will ruin another jails connection. As well as the gigabit port only reporting 100 Mb connection when all jails are running.

The easiest solution I had was just give every jail it's own port and team all of the remaining ports for local data transfers. 40$ well spent to me.

 

[SweetAndLow]

Your switch supports lacp? And you have have something else wrong if modifying one jail breaks another. Sounds like your network could use don't tlc instead of hacking your way around it.

 

[Caretaker]

Oh yeah, my switch is a gem, Cisco SG200-26

your network could use don't tlc instead of hacking your way around it.

I'm not catching what you're saying here. ^ If you're saying my network needs work I mean no disrespect but your assumption is wrong.

So it's a compound issue. One, making adjustments to my jails can cause the error talked about in my OP maybe 10% of the time. Two, I have nearly 4+ connections and spoofed MAC adds going in and out of one wire and most of the time FreeNAS doesn't seem to correctly apply my spoofed MACs anyways (another reason I was thinking its just best to wipe and restart).

So my switch and router have conflicting ARP/DHCP tables where one MAC add can have ~2-4 IPs assigned to it with particular ports assigned to each and something is getting lost along the way (most likely something with FreeNAS). I've had issue with one jail (plex) being very hit and miss on if it's port is being forwarded correctly or not. As well as -like I said before- how one jails connection can seem to 'over power' another.

Further I have subnets in my network so when it comes down to it, having more ports is helpful, not to mention inexpensive.

My FreeNAS is being asked to do a good bit of work, that's why I have relatively beefy hardware installed in it.

 

[SFoskett]

I think I might have the same issue with mine. I'm not entirely sure since I didn't capture the output of some crashes, but here's a screenshot of one from yesterday... Of course, the blank console bug makes tracking this down seriously difficult, too...

I noted crashes often when creating and reconfiguring networking for my jails. Maybe 2-4 crashes while doing this in the 11 days I've been using FreeNAS. So I'm not gun-shy to mess with my jails!

I'm not sure if this is specific to 9.10.1. Something similar definitely happened in 9.10. Fresh install on new server-grade hardware, ECC, etc. All BIOS is up to date. New 64GB SanDisk USB boot drive.

 

[Caretaker]

I'm not sure if this is specific to 9.10.1. Something similar definitely happened in 9.10. Fresh install on new server-grade hardware, ECC, etc. All BIOS is up to date. New 64GB SanDisk USB boot drive.

That sounds just like the same issue, it would happen to me as well when changing conf with jails networking. I did a clean install but because of some odd errors I was having with pkg with a fresh 9.10/9.10.1 I went back to my original download of 9.3.X (something) and then updated and that got everything working for me (along with using a 4port Gigabit NIC to divide traffic up).

IDK if doing my crazy install of an old v. and then updating will help you. If you want to try I just looked it up...

Code:

FreeNAS-9.3-STABLE-201512121950
http://iso.cdn.freenas.org/9.3.1/STABLE/201510290351/x64/

Also if you're running a lot of jails like I am, and want to go full overkill pick up a 4-Port Gb NIC (can link to a good one for 43$ shipped, two day) and tell each jail to use one of the ports on it. Right now for what I'm doing this is as been a really big help.

GL!