FreeNAS randomly discommunicating

[lyle]

I've had this hardware setup for at least two years and it's been working flawlessly. Now, suddenly, I have a problem. At random my FreeNAS is totally unavailable. Can't access the Windows shares, can't login into the gui via IE, and pings are unsuccessful. The only solution has been a hard shutdown holding down the power button (I know! Right?). After it comes back up it again runs flawlessly. I see no error messages in the gui and no alarms are emailed to me. Bright green OK light in the top right corner. This is happening approx. every 48hrs. The ONLY change I've made is replacing my modem & router with a new modem/router from my isp. I can't see this having anything to do with the problem but it's a heck of coincidence (??).

Would really appreciate some guidance on where to start troubleshooting this highly undesirable situation. Newbie alert, so please write as if I'm 7.

many thanks!
Lyle

 

[IceBoosteR]

Hi,

do you see any message in "/var/log/messages" ?
Maybe you should update your new routers firmware. Its seems like the router forgets the IP of your FreeNAS Server. You should check out, if your local computers ARP cache is still having the MAC adress of the freenas server stored (Windows: CMD -> "arp -a")

The other thing you can try, is to put a monitor on your freenas system and a keyboard and try different thing, if you got into the problem. Are you able to set up a piung into the local network/internet and things like that. On that way you may figure out whats wrong. At this time, I would point on the router ;)

 

[Andreas Corneliussen]

I had the same issue with a broken boot USB stick which made the system unresponsive when a specific sector were accessed (or at least that is what I think is happening, it broke my desktop Linux system the same way when I tried to write an Ubuntu installer to it). Took some time for freenas to detect these problems and show it in the gui and log files.

 

[lyle]

Hi, Ice. Thanks for your reply.

Don't know how to access the /var/log/messages. I will research this.

I looked at the ARP cache and it is showing the IP address of the freenas server as dynamic, not static. So I think you are on to something. I just created a DHCP reservation on the router to assign the IP address to the freenas server. We'll see if that works, fingers crossed. Strange though, I rebooted the router and rebooted the freenas server, then I looked at the ARP cache and it is still showing the freenas IP address as dynamic. I double-checked my reservation on the router after the reboot and all looks good. Maybe I need to reset the ARP cache somehow?

Good idea to plug in a monitor and keyboard. Duh!! should have thought of that. Will try that next if the dhcp reservation hasn't cured things.

Many thanks!
Lyle

 

[Andreas Corneliussen]

You can access /var/log/messages by starting a terminal and writing the command
less /var/log/messages

 

[Kevin Horton]

You can access /var/log/messages by starting a terminal and writing the command
less /var/log/messages

less /var/log/messages will show one screen full of output from /var/log/messages.

Tap the space bar to advance to the next screen.

Pressing the "ctrl" and "b" keys together will backup one screen.

The "q" key will stop showing the output and return you to the terminal.

 

[SweetAndLow]

Setting a IP reservation is still considered dynamic. It will make it so your system gets the same IP though.

Hardware specs and freenas version need to be provided with every new thread. Post that info along with logs.

Sent from my Nexus 5X using Tapatalk

 

[lyle]

Thanks all for your help. Output from the /var/log/messages below. All Greek to me!

Sweet&Low: I did post my hardware specs and freenas version in my first and subsequent posts. The info is in my signature....

1. May 22 00:00:00 freenas newsyslog[13270]: logfile turned over due to size>100K
2. May 22 00:00:00 freenas syslog-ng[1572]: Configuration reload request received, reloading configuration;
3. May 22 00:31:40 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
4. May 22 00:31:40 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
5. May 22 01:31:47 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
6. May 22 01:31:47 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
7. May 22 02:31:54 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
8. May 22 02:31:54 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
9. May 22 03:32:01 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
10. May 22 03:32:01 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
11. May 22 04:32:08 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
12. May 22 04:32:08 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
13. May 22 05:32:15 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
14. May 22 05:32:15 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
15. May 22 06:32:22 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
16. May 22 06:32:22 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
17. May 22 07:32:29 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
18. May 22 07:32:29 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
19. May 22 08:32:37 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
20. May 22 08:32:37 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
21. May 22 09:32:44 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
22. May 22 09:32:44 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
23. May 22 10:32:52 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
24. May 22 10:32:52 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
25. May 22 11:32:58 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
26. May 22 11:32:58 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
27. May 22 12:33:05 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
28. May 22 12:33:05 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name
29. May 22 13:33:12 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
30. May 22 13:33:12 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name

 

[SweetAndLow]

I've had this hardware setup for at least two years and it's been working flawlessly. Now, suddenly, I have a problem. At random my Freenas is totally unavailable. Can't access the Windows shares, can't login into the gui via IE, and pings are unsuccessful. The only solution has been a hard shutdown holding down the power button (I know! Right?). After it comes back up it again runs flawlessly. I see no error messages in the gui and no alarms are emailed to me. Bright green OK light in the top right corner. This is happening approx. every 48hrs. The ONLY change I've made is replacing my modem & router with a new modem/router from my isp. I can't see this having anything to do with the problem but it's a heck of coincidence (??).

Would really appreciate some guidance on where to start troubleshooting this highly undesirable situation. Newbie alert, so please write as if I'm 7.

many thanks!
Lyle

Here is your first post, there is zero hardware specs and no freenas version. You have to post it in the comment. Signatures do not show up with some clients, specifically mobile views.

 

[lyle]

Gotchya!

Intel G2030 Dual-Core processor, 3GHz
Supermicro X9SCL-F motherboard
8GB Kingston DDR3 1333 ECC SDRAM
2x Seagate 2TB Barracuda Green hard drives (raid mirror)
2x Seagate 1TB Barracuda LP hard drive
FreeNAS 9.10.2

 

[IceBoosteR]

Hi,

ok normally the IP adress need to be dynamic, as SweetAnd Low said, otherwise your home network would not work.
As the IP is still in the arp cache and not removed (done after a timeout of 60 seconds per default), so the server is still available from the network somehow.
Maybe you should try, if the error does occur next time, that you try to SSH into your machine, ping it, check the arp, check the router and later check the machine itself....
Maybe the new router has a hidden botton checked, where some stupid features cause this problems. (So maybe check the router again^^)

Regards
Ice

Edit:
Is it always the same IP address you which you access FreeNAS? Or is this IP changing?
If so - go to your router setting, and mark the IP for FreeNAS as a static one.
My idea was, that fN get a new DHCP lease and you cannot access your shares, as they are mapped to a IP, and the GUI also (because you don't now the new FN IP ;) )

 

[IceBoosteR]

Aaand your messages in the the logfile is maybe related to this bug:
https://bugs.freenas.org/issues/21915
And can also be found in this thread:
https://forums.freenas.org/index.php?threads/dmidecode-alert.54629/

I hate to say this, but I think this can be ignored.

 

[lyle]

Thanks for all your help, Ice. So far I am at 3 days, 29 mins of uptime with my freenas, so I am thinking (hoping) that the DHCP reservationhas solved the problem.

Regarding your questions:

Is it always the same IP address you which you access FreeNAS? Or is this IP changing?

Well my goal is to have it NOT changing, that it is always locked into 192.168.0.250. This is the way I had it set up before I made my modem/router change. I thought that all that work was done in the setup of Freenas, but I forgot that there is the router side to consider.

If so - go to your router setting, and mark the IP for FreeNAS as a static one.

I have looked extensively through all my router menu options and simply cannot figure out how to assign a static IP to the Freenas. The closest I have come is this DHCP reservation and, to be honest, I don't what the difference is between a static IP address compared to a reserved IP address. But as I say, so far I have not had a recurrence of the problem since I made the reservation.

My idea was, that fN get a new DHCP lease and you cannot access your shares, as they are mapped to a IP, and the GUI also (because you don't now the new FN IP ;) )

I think you are absolutely correct in this diagnosis!! I will definitely SSH into the machine or hook up a video monitor and keyboard if the problem reoccurs to see what's going on.

Thanks again!

 

[SweetAndLow]

Thanks for all your help, Ice. So far I am at 3 days, 29 mins of uptime with my freenas, so I am thinking (hoping) that the DHCP reservationhas solved the problem.

Regarding your questions:

Is it always the same IP address you which you access FreeNAS? Or is this IP changing?

Well my goal is to have it NOT changing, that it is always locked into 192.168.0.250. This is the way I had it set up before I made my modem/router change. I thought that all that work was done in the setup of Freenas, but I forgot that there is the router side to consider.

If so - go to your router setting, and mark the IP for FreeNAS as a static one.

I have looked extensively through all my router menu options and simply cannot figure out how to assign a static IP to the Freenas. The closest I have come is this DHCP reservation and, to be honest, I don't what the difference is between a static IP address compared to a reserved IP address. But as I say, so far I have not had a recurrence of the problem since I made the reservation.

My idea was, that fN get a new DHCP lease and you cannot access your shares, as they are mapped to a IP, and the GUI also (because you don't now the new FN IP ;) )

I think you are absolutely correct in this diagnosis!! I will definitely SSH into the machine or hook up a video monitor and keyboard if the problem reoccurs to see what's going on.

Thanks again!

Static IP is configured on freenas. Your router has zero say in things when it comes to static IPS.

Sent from my Nexus 5X using Tapatalk

 

[IceBoosteR]

lyle, you are welcome.
Let us now of you face any issue here in the future.

Static IPs are configured on FreeNAS as SweetAndLow said, and IP reservation takes place on the router.
Maybe I use the wrong words here ...
More information about this in short here:
http://lifehacker.com/5822605/how-t...o-you-never-have-to-check-an-ip-address-again

 

[lyle]

Thanks, guys. 5 days of uptime and counting, so I think I'm in the clear.

Still confused tho. I have the static IP setup in Freenas, so according to S&L I shouldn't have needed to mess with the router. And the "arp -a" command is still showing the ip address is dynamic.

It seems that I need BOTH the static IP address set up in Freenas AND the reservation set up on the router. Good to know!

thanks again!

 

[Andreas Corneliussen]

Did the freenas have an entry in the router's DHCP client table? If so then your setup of the static IP is most likely wrong in freenas.
Another possibility, if you indeed have static IP setup in the freenas configuration, is that this IP were in the router's DHCP range such that the router might have given another device on the network the same IP as the freenas which then can cause all sorts of problems in the network.

 

[lyle]

Yes, it did show up in the client table. However, I didn't make any changes to freenas when I installed the new router, so you would think if it worked before in would work with the new router. ???

The static IP I setup in Freenas is 250 and the range of the DHCP is setup as 10-200. Definitely well outside the range.

6 days of uptime so far, so I think things are solved. If you had a link to the correct way to set up a static IP in Freenas, I would definitely be interested in reviewing it.

 

[Andreas Corneliussen]

I must say that I'm relatively new to freenas and I have set mine up to DHCP and assigned fixed IPs for the MAC adresses in the router. I have search the net a little, and for some reason everyone telling how to setup fixed ip in freenas is talking about using the command line:
http://gamblisfx.com/how-to-change-ip-address-on-freenas-server/

But all in all my answer above were more out of a general knowledge about networking as it were from specific knowledge about the freenas system. But as you told me that it were present in the DHCP table of the new router, it must have asked the DHCP server for an IP at some point. Generally routers do not list IPs of systems in the DHCP table which they have not given out on a DHCP request. So a correctly setup static IP (if it were set in the freenas system) should not show up in the DHCP table.

 

[Kevin Horton]

I must say that I'm relatively new to freenas and I have set mine up to DHCP and assigned fixed IPs for the MAC adresses in the router.

This is actually a pretty good setup for home use, as it tolerates the all too common case of a new router having a different default LAN ip range than the old one. If you have the misfortune to change the network IP range, and have the FreeNAS server set up on a static IP address, it will become unreachable on the network. You need the foresight to temporarily switch FreeNAS to DHCP before changing the router, but that lesson is normally learned the hard way.