Regularly Losing Access to Shares & Web Interface

[wraith]

Hi All,

My system has very recently commenced exhibiting some odd behaviour. About once a day I will lose access to all the shared folders as well as well as access to the web interface. A power cycle sometimes resolves this issue but, sometimes, need to powercycle several times to regain access to the web interface and shared folders. IPMI appears to show the bootup is proceeding normally albeit with a bunch of temperature warnings [around 42º; I'm in Australia] at the very end. Pinging the IP address is not successful.

I can successfully ping the IPMI IP. The IP to the server is fixed via the DHCP client

I've noted that it does not appear to be registered on the router; Which I'm interpreting that it hasn't reached out for an IP... [??]

The LAN activity light on the chassis is active

I'm running WIndows 10 22H2 build 19045.2486 on the client.

Is there anyway I can get access to the log files without access to shared files or the web interface? Any suggestions on troubleshooting? Not sure how to export the IPMI bootup sequence

Thanks in advance

 

[Davvo]

IPMI and the WebUI have different IPs.
Go into the IPMI and use iKVM or similar to look at the output of the screen, once it's booted it should give you the Nas ip.

You probably want to use static IP.

 

[wraith]

Hi Davvo

Thanks for assisting.

IPMI and the WebUI have different IPs.
Go into the IPMI and use iKVM or similar to look at the output of the screen, once it's booted it should give you the Nas ip.

My DHCP is setup to always dish out 192.168.0.250 to the IPMI whilst the web interface is on 192.168.0.251. When I access IPMI, the screen size is not useful, nor can I determine or figure out of how to make it larger and scroll up the text to see what else has been put to screen

When you suggest to use a static IP, isn't that what is happening when I fix the IP address 192.168.0.251 to the MAC address? Or is what you're suggesting a different method? Do I need to access TrueNAS to do that? If so, may have an issue! :P

 

[wraith]

OK. After another powercycle I'm back into TrueNAS.

I understand the log files are at "/var/log" but there is a lot of files. Where is the best place to start looking/troubleshooting. Also, looking at some that end in .bz2, opening in nano provides uninterpretable text...

 

[Davvo]

When I access IPMI, the screen size is not useful, nor can I determine or figure out of how to make it larger and scroll up the text to see what else has been put to screen

You want to look into Remote Control > iKVM/HTML5.

Also, looking at some that end in .bz2, opening in nano provides uninterpretable text...

Those are compressed files.

bzip2(1)

www.freebsd.org

 

[wraith]

You want to look into Remote Control > iKVM/HTML5.

Hi @Davvo - My version of IPMI doesn't have the iKVM/HTML5 option. Either way, I changed the ADMIN password and it won't accept it again and currently locked out of IPMI.

 

[wraith]

I thought the issue had resolved itself until it started happening, again, today. A few reboots and all sorted. Until I briefly lost access to the server around 5:15pm [on 18-Feb-23]. I was able to get in and get a copy of all the log files. Seeking some help in interpreting.

What I can gather from trying to interpret myself:

• middlewared.log :- File does show ERRORS and DEBUG entries happening sometime after 4:45pm and before 5:15pm on 18-Feb-23. Seem to be mail related or something to do with python3.9? Could this be causing instability?
• console.log :- Nothing suspicious that my untrained eye can identify.
• daemon.log :- Nothing suspicious either
• log.smbd :- A few ERRORS stating that samba is already running... Potential issue?

Anything above could be an issue? Anything I"ve missed? Other logs?

TIA

 

[ChrisRJ]

Other logs?

Look at /var/log/messages

 

[wraith]

Look at /var/log/messages

Hi @ChrisRJ ,
Thanks for helping out.
I had tried "more /var/log/messages" but got the following around 5:15pm. Did't seem to as detailed/helpful as the .log files I found. It did show the network adaptor going down, up and then down again but no reason...?

 

[Davvo]

Hi @Davvo - My version of IPMI doesn't have the iKVM/HTML5 option. Either way, I changed the ADMIN password and it won't accept it again and currently locked out of IPMI.

You can update it. IIRC there might be a pin to short to reset IPMI password.

 

[Samuel Tai]

Try a different cable, and check it's clicking in on both ends. I've seen this sort of behavior before with bad cables.

 

[Jailer]

Hi @Davvo - My version of IPMI doesn't have the iKVM/HTML5 option. Either way, I changed the ADMIN password and it won't accept it again and currently locked out of IPMI.

From the system shell in TruNASNAS or putty run ipmicfg -fd or ipmitool raw 0x3c 0x40. I can't remember which one works but I've run one of them before to reset the BMC to factory defaults when I was locked out.

 

[wraith]

Try a different cable, and check it's clicking in on both ends. I've seen this sort of behavior before with bad cables.

Hi Samuel. Thanks for helping out. I tried pulling the network cable out and reseating (??) and this did tend to resolve the issue making me think the issue is a dodgy cable. I don't have any spare but I did notice that I forgot that I had two network cables linked into the server. I've set up link aggregation as a fail over until I grab another cable. Thanks again! I'll report back if things continue pear shaped.