Web Gui Dies in 9.1.1

[skl111]

I have installed Freenas 9.1.1
I installed 5 plugins, plex, dlna, sickbeard, sab+,plex
After a while the nginx dies and I find this error in the logs.

013/11/17 17:22:16 [error] 32693#0: *1 connect() to 127.0.0.1:9042 failed (61: Connection refused) while connecting to upstream, client: 192.168.2.103, server: localhost, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9042", host: "192.168.2.11"
2013/11/17 17:22:16 [error] 32693#0: *1 kevent() reported that connect() failed (61: Connection refused) while connecting to upstream, client: 192.168.2.103, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9042", host: "192.168.2.11"
2013/11/17 17:22:25 [error] 32693#0: *1 kevent() reported that connect() failed (61: Connection refused) while connecting to upstream, client: 192.168.2.103, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9042", host: "192.168.2.11"

I reboot and the system works for a while.
I should add I am actively using sickbeard+sab + when it dies.
The NAS page and all the plugin pages go down.

 

[dlavigne]

Which NIC driver (from ifconfig)? How much RAM and CPU?

 

[skl111]

Which NIC driver (from ifconfig)? How much RAM and CPU?

4gb ram
X3 b73 Amd CPU
Broad com dual port pci e bge nic.

 

[skl111]

4gb ram
X3 b73 Amd CPU
Broad com dual port pci e bge nic.

Code:

bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:e0:ed:08:9e:32
        inet6 fe80::2e0:edff:fe08:9e32%bge0 prefixlen 64 scopeid 0x1
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (none)
        status: no carrier
bge1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:e0:ed:08:9e:32
        inet6 fe80::2e0:edff:fe08:9e33%bge1 prefixlen 64 scopeid 0x2
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether b8:97:5a:03:bc:35
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (none)
        status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0xd
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:e0:ed:08:9e:32
        omitted
        inet6 fe80::2e0:edff:fe08:9e32%lagg0 prefixlen 64 scopeid 0xe
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: bge1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: bge0 flags=0<>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:7a:83:8c:f3:00
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair4a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 20 priority 128 path cost 2000
        member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 19 priority 128 path cost 2000
        member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 18 priority 128 path cost 2000
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 17 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 16 priority 128 path cost 2000
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 14 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:39:8b:00:10:0a
        inet6 fe80::39:8bff:fe00:100a%epair0a prefixlen 64 scopeid 0x10
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:63:6b:00:11:0a
        inet6 fe80::63:6bff:fe00:110a%epair1a prefixlen 64 scopeid 0x11
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:79:26:00:12:0a
        inet6 fe80::79:26ff:fe00:120a%epair2a prefixlen 64 scopeid 0x12
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair3a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:d0:75:00:13:0a
        inet6 fe80::d0:75ff:fe00:130a%epair3a prefixlen 64 scopeid 0x13
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair4a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:88:c4:00:14:0a
        inet6 fe80::88:c4ff:fe00:140a%epair4a prefixlen 64 scopeid 0x14
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active

One Note, I still have access to file shares

 

[skl111]

Well i did some looking around and found that this is the script that dies.

/usr/local/bin/python /usr/local/www/freenasUI/manage.py runfcgi method=threaded host=127.0.0.1 port=9042 pidfile=/var/run/django.pid

I start it backup and the webserver works again. And made this script in cron for 5 min intervals.

Code:

#!/bin/bash
 
# Check if gui is running
if ps wwax | grep "[f]reenasUI" > /dev/null
then
    echo "Running" > /dev/null
else
    echo "Stopped" > /dev/null
    /usr/local/bin/python /usr/local/www/freenasUI/manage.py runfcgi method=threaded host=127.0.0.1 port=9042 pidfile=/var/run/django.pid
fi

 

[Dusan]

Well i did some looking around and found that this is the script that dies.

/usr/local/bin/python /usr/local/www/freenasUI/manage.py runfcgi method=threaded host=127.0.0.1 port=9042 pidfile=/var/run/django.pid

I start it backup and the webserver works again. And made this script in cron for 5 min intervals.

Code:

#!/bin/bash
 
# Check if gui is running
if ps wwax | grep "[f]reenasUI" > /dev/null
then
    echo "Running" > /dev/null
else
    echo "Stopped" > /dev/null
    /usr/local/bin/python /usr/local/www/freenasUI/manage.py runfcgi method=threaded host=127.0.0.1 port=9042 pidfile=/var/run/django.pid
fi

You can replace the entire script by "service django start". It will check if the GUI is running and start it if it isn't.

 

[Cordel]

I'm seeing the same issue on a test machine I just setup.
Intel(R) Pentium(R) D CPU 2.80GHz
4GB ECC RAM

Everything is Vanilla, fresh install from CD on to a Micron USB flash card. I have set the admin password, and enabled the console on the serial port.
Set up a pair of drives in a ZFS stripe, Created a dataset for the jail/plugin services, and set the path and IP for the jails. No other changes.

For sanity, I set the gui, to the IP of the NIC.
I might get to view a few pages or change a setting before it disappears.

I have also tried reinstalling, burning a new CD with a fresh ISO.

Code:

/usr/local/bin/python -m pdb /usr/local/www/freenasUI/manage.py runfcgi method=threaded host=127.0.0.1 port=9042 pidfile=/var/run/django.pid
> /usr/local/www/freenasUI/manage.py(30)<module>()
-> import sys
(Pdb) c
The program exited via sys.exit(). Exit status:  0
> /usr/local/www/freenasUI/manage.py(30)<module>()
-> import sys

Code:

Request Method: GET
Request URL: http://192.168.120.6/system/info/?dojo.preventCache=1386449131667
Software Version: FreeNAS-9.1.1-RELEASE-x64 (a752d35)
Exception Type: CalledProcessError
Exception Value:
Command '['sysctl', '-n', 'hw.physmem']' returned non-zero exit status -11
Exception Location: /usr/local/lib/python2.7/subprocess.py in check_output, line 575
Server time: Sat, 7 Dec 2013 12:45:32 -0800
Traceback
 
 
 
Environment: Software Version: FreeNAS-9.1.1-RELEASE-x64 (a752d35) Request Method: GET Request URL: http://192.168.120.6/system/info/?dojo.preventCache=1386449131667 Traceback: File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response 109. response = middleware_method(request, callback, callback_args, callback_kwargs) File "/usr/local/www/freenasUI/../freenasUI/freeadmin/middleware.py" in process_view 165. return login_required(view_func)(request, *view_args, **view_kwargs) File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view 25. return view_func(request, *args, **kwargs) File "/usr/local/www/freenasUI/../freenasUI/system/views.py" in system_info 104. sysinfo = _system_info(request) File "/usr/local/www/freenasUI/../freenasUI/system/views.py" in _system_info 71. subprocess.check_output(['sysctl', '-n', 'hw.physmem']) File "/usr/local/lib/python2.7/subprocess.py" in check_output 575. raise CalledProcessError(retcode, cmd, output=output) Exception Type: CalledProcessError at /system/info/ Exception Value: Command '['sysctl', '-n', 'hw.physmem']' returned non-zero exit status -11
 
Request information
 
GET
 
VariableValue
dojo.preventCache u'1386449131667'
POST
 
No POST data
FILES
 
No FILES data
COOKIES
 
VariableValue
csrftoken 'A4guFw3E54O9MvIMt1Xv5PgJCtsTye5g'
sessionid 'dx2l2a17qk7ro1gmvw836ghzn2ndw7p5'
fntreeSaveStateCookie 'root%2Croot%2F90%2Croot%2F1%2Croot%2F11%2Croot%2F11%2F15%2Croot%2F11%2F32%2Croot%2F11%2F36'
META
 
VariableValue
wsgi.multiprocess False
HTTP_REFERER 'http://192.168.120.6/'
REDIRECT_STATUS '200'
SERVER_SOFTWARE 'nginx/1.4.1'
SCRIPT_NAME u''
REQUEST_METHOD 'GET'
PATH_INFO u'/system/info/'
SERVER_PROTOCOL 'HTTP/1.1'
QUERY_STRING 'dojo.preventCache=1386449131667'
CONTENT_LENGTH ''
HTTP_USER_AGENT 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:25.0) Gecko/20100101 Firefox/25.0'
HTTP_CONNECTION 'keep-alive'
HTTP_COOKIE 'csrftoken=A4guFw3E54O9MvIMt1Xv5PgJCtsTye5g; sessionid=dx2l2a17qk7ro1gmvw836ghzn2ndw7p5; fntreeSaveStateCookie=root%2Croot%2F90%2Croot%2F1%2Croot%2F11%2Croot%2F11%2F15%2Croot%2F11%2F32%2Croot%2F11%2F36'
SERVER_NAME 'localhost'
REMOTE_PORT '60544'
wsgi.url_scheme 'http'
SERVER_PORT '80'
SERVER_ADDR '192.168.120.6'
DOCUMENT_ROOT '/usr/local/etc/nginx/html'
HTTP_X_REQUESTED_WITH 'XMLHttpRequest'
DOCUMENT_URI '/system/info/'
wsgi.input <flup.server.fcgi_base.InputStream object at 0x80e01cd90>
HTTP_HOST '192.168.120.6'
wsgi.multithread True
HTTP_CONTENT_TYPE 'application/x-www-form-urlencoded'
REQUEST_URI '/system/info/?dojo.preventCache=1386449131667'
HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
wsgi.version (1, 0)
GATEWAY_INTERFACE 'CGI/1.1'
wsgi.run_once False
wsgi.errors <flup.server.fcgi_base.TeeOutputStream object at 0x80e01cd10>
REMOTE_ADDR '192.168.120.160'
HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.5'
CONTENT_TYPE 'application/x-www-form-urlencoded'
CSRF_COOKIE u'A4guFw3E54O9MvIMt1Xv5PgJCtsTye5g'
HTTP_ACCEPT_ENCODING 'gzip, deflate'

Code:

Traceback (most recent call last):
  File "/usr/local/www/freenasUI/manage.py", line 40, in <module>
    from django.core.management import execute_from_command_line
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 9, in <module>
    from django.core.management.base import BaseCommand, CommandError, handle_default_options
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 15, in <module>
    from django.utils.encoding import force_str
  File "/usr/local/lib/python2.7/site-packages/django/utils/encoding.py", line 5, in <module>
    from decimal import Decimal
SystemError: invalid comp_op: �

I don't know enough about python or dojango to go through it my self. It obviously starts with no errors.
But most often all I see is the red explanation point with "Sorry an error has occurred."
Hope this helps a little.
Edit to add more error detail.

 

[William Grzybowski]

Sounds like your usb flash drive is corrupted. This a fresh install in a new drive.

 

[Cordel]

I have done Two things this last round of reloading, and thus far appear to have rid myself of django diving off.
Prior to reloading I removed the ZFS Volumes from both drives and wiped them (They did have a Vol from 8.3 which was the last test I had done).
Then reloaded FN.

So I'm wondering why importing a ZFS Volume set might be an issue?
After four reloads, the drives were the only thing that "Aint Vanilla" so I decided I had to try.
I'll have to see later if I can replicate it, and try to pin it down, specially if it does have anything to do with importing a zfs set.