Can't access GUI or SSH, cleared via reboot...Need help

[jobzombi]

The system has been up and running fine for over a month. But lately on occasion, I cannot access the webUI or SSH to it. The strange part is that the jails are still accessible, as I am able to access Plex without a problem. I'm seeing the following in messages:

Code:

Mar 15 19:37:56 freenas proftpd[22069]: 127.0.0.1 (192.168.1.14[192.168.1.14]) - FTP no transfer timeout, disconnected
Mar 15 19:40:01 freenas proftpd[22198]: 127.0.0.1 (192.168.1.14[192.168.1.14]) - Connection refused (MaxConnectionsPerHost 4)
Mar 15 19:40:02 freenas proftpd[22201]: 127.0.0.1 (192.168.1.13[192.168.1.13]) - Connection refused (MaxConnectionsPerHost 4)
Mar 15 19:42:04 freenas proftpd[22143]: 127.0.0.1 (192.168.1.13[192.168.1.13]) - FTP no transfer timeout, disconnected
Mar 15 19:43:42 freenas proftpd[22171]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 19:45:00 freenas proftpd[22193]: 127.0.0.1 (192.168.1.14[192.168.1.14]) - FTP no transfer timeout, disconnected
Mar 15 19:58:59 freenas proftpd[22386]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 20:11:02 freenas proftpd[22595]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 20:23:55 freenas proftpd[22759]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 20:37:16 freenas proftpd[22959]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 20:46:32 freenas proftpd[23068]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 21:00:08 freenas proftpd[23257]: 127.0.0.1 (192.168.1.13[192.168.1.13]) - FTP no transfer timeout, disconnected
Mar 15 21:03:33 freenas proftpd[23311]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 21:05:09 freenas proftpd[23361]: 127.0.0.1 (192.168.1.13[192.168.1.13]) - FTP no transfer timeout, disconnected
Mar 15 21:20:47 freenas proftpd[23554]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 21:37:47 freenas proftpd[23771]: 127.0.0.1 (192.168.1.14[192.168.1.14]) - FTP no transfer timeout, disconnected
Mar 15 21:38:14 freenas proftpd[23789]: 127.0.0.1 (192.168.1.13[192.168.1.13]) - FTP no transfer timeout, disconnected
Mar 15 21:53:12 freenas proftpd[23960]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:02:47 freenas proftpd[24085]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:10:27 freenas proftpd[24220]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:29:32 freenas proftpd[24477]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:34:53 freenas proftpd[24538]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:40:32 freenas proftpd[24620]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:46:41 freenas proftpd[24702]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 22:51:58 freenas proftpd[24772]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 23:05:24 freenas proftpd[24965]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 15 23:41:13 freenas proftpd[25428]: 127.0.0.1 (192.168.1.12[192.168.1.12]) - FTP no transfer timeout, disconnected
Mar 18 01:04:59 freenas syslog-ng[1477]: syslog-ng starting up; version='3.6.4'
Mar 18 01:04:59 freenas Copyright (c) 1992-2016 The FreeBSD Project.
Mar 18 01:04:59 freenas Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Mar 18 01:04:59 freenas		 The Regents of the University of California. All rights reserved.
Mar 18 01:04:59 freenas FreeBSD is a registered trademark of The FreeBSD Foundation.
Mar 18 01:04:59 freenas FreeBSD 10.3-STABLE #0 r295946+1805185(9.10.2-STABLE): Wed Jan 11 17:12:42 UTC 2017
Mar 18 01:04:59 freenas root@gauntlet:/freenas-9.10-releng/_BE/objs/freenas-9.10-releng/_BE/os/sys/FreeNAS.amd64 amd64
Mar 18 01:04:59 freenas FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
Mar 18 01:04:59 freenas CPU: Intel(R) Xeon(R) CPU		   E5649  @ 2.53GHz (2533.47-MHz K8-class CPU)
Mar 18 01:04:59 freenas Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
Mar 18 01:04:59 freenas Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Mar 18 01:04:59 freenas Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI>
Mar 18 01:04:59 freenas AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
Mar 18 01:04:59 freenas AMD Features2=0x1<LAHF>
Mar 18 01:04:59 freenas VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
Mar 18 01:04:59 freenas TSC: P-state invariant, performance statistics

It stopped writing to the messages log late at night the 15th. Since I could not reboot it via SSH or the WebUI, I did so via the server itself, a Proliant DL360 G7. As far as I could see, my router still saw the server and I could still ping the IP address from my network. SMB drive and FTP were also down. I will have to wait for it to happen again to attach a monitor and view what is on the console screen itself, since I forgot to do so last night. In the meantime any help I can get to either figure what this is or how I can get more debugging info out of the server, would be great.

 

[dlavigne]

Were you able to figure this out?

 

[jobzombi]

Were you able to figure this out?

No I have not....but since then it happened again and I managed to get a screen connected:



The top portion is what was there when the screen was connected...When I plugged in the keyboard I got the middle message...then I tried some commands, but none were taking.

Update:
da4 is my boot partition. I use a thumb-drive. It is a 64GB Sandisk Cruzer Fit. I have FreeNas reboot nightly. Could this be a cause? I have a script that deletes some older files on start-up and well, with a nightly reboot, runs nightly. I can't really say if I can notice some correlation as I had the reboot setup since the beginning and I can't say that I did or did not notice this behavior before that.

 

[wblock]

The USB stick is failing. Unless you have a USB hard drive, which is a bad idea.

 

[zoomzoom]

@jobzombi It's either what @wblock said, or you have power management enabled on your HDDs, which appears to not play well when used with certain motherboards/SATA controllers.

• For example, I have an AsRock C2750D4I and the Marvell & Intel SATA controllers will cause the same camstatus errors on any drive power management is enabled on. Once disabled and rebooted, the system will no longer throw camstatus errors and time out due to them.