NFS will not work despite reinstall

Status
Not open for further replies.
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
NFS died on my FreeNAS 11.1-U6 two days ago and I haven't been able to get it working again since then despite doing a reinstall of 11.1-U4. The service won't start after boot and trying to start it over the Web UI only results in a hung UI. Sometimes it does eventually show as active in the UI, but it isn't.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
Take a look at the console for error messages and post them here. Also a run down of your config too.
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
j0hnby said:
Take a look at the console for error messages and post them here. Also a run down of your config too.
Click to expand...
Nothing happens at all in the console when trying to start/stop NFS. My config is really basic I just did a reinstall, added a cert and imported my existing volume.
 
rvassar

rvassar

Guru
Joined
May 2, 2018
Messages
972
Look in /var/log/messages and the other logs. There has to be something complaining somewhere. Until you find it, all we're going to be able to do is guess and speculate.
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
rvassar said:
Look in /var/log/messages and the other logs. There has to be something complaining somewhere. Until you find it, all we're going to be able to do is guess and speculate.
Click to expand...
I found a bunch of python errors in middlewared.log
Code:
[2018/09/06 14:10:00] (DEBUG) ServiceService._simplecmd():255 - Calling: stop(nfs)
[2018/09/06 14:10:12] (DEBUG) ServiceService._simplecmd():255 - Calling: stop(nfs)
[2018/09/06 14:11:00] (WARNING) application.call_method():174 - Exception while calling notifier.stop(*['nfs', None, True])
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
	result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 908, in call_method
	return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 886, in _call
	return await self.run_in_thread(methodobj, *args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 849, in run_in_thread
	return await self.run_in_thread_pool(self.__threadpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 845, in run_in_thread_pool
	await task
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
	result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 416, in call
	raise CallTimeout("Call timeout")
middlewared.client.client.CallTimeout: Call timeout
[2018/09/06 14:11:00] (DEBUG) middlewared.logger.CrashReporting.report():97 - Sending a crash report...
[2018/09/06 14:11:00] (DEBUG) raven.base.Client.send_remote():684 - Sending message of length 3793 to https://sentry.ixsystems.com/api/2/store/
[2018/09/06 14:11:12] (WARNING) application.call_method():174 - Exception while calling notifier.stop(*['nfs', None, True])
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
	result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 908, in call_method
	return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 886, in _call
	return await self.run_in_thread(methodobj, *args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 849, in run_in_thread
	return await self.run_in_thread_pool(self.__threadpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 845, in run_in_thread_pool
	await task
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
	result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 416, in call
	raise CallTimeout("Call timeout")
middlewared.client.client.CallTimeout: Call timeout
[2018/09/06 14:11:12] (DEBUG) middlewared.logger.CrashReporting.report():97 - Sending a crash report...
[2018/09/06 14:11:12] (DEBUG) raven.base.Client.send_remote():684 - Sending message of length 3459 to https://sentry.ixsystems.com/api/2/store/
[2018/09/06 14:16:44] (DEBUG) ServiceService._simplecmd():255 - Calling: stop(nfs)
[2018/09/06 14:17:44] (WARNING) application.call_method():174 - Exception while calling notifier.stop(*['nfs', None, True])
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
	result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 908, in call_method
	return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 886, in _call
	return await self.run_in_thread(methodobj, *args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 849, in run_in_thread
	return await self.run_in_thread_pool(self.__threadpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 128, in throw
	return self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 845, in run_in_thread_pool
	await task
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
	result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/www/freenasUI/middleware/notifier.py", line 216, in stop
	return c.call('service.stop', what, {'onetime': onetime}, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 416, in call
	raise CallTimeout("Call timeout")
middlewared.client.client.CallTimeout: Call timeout
[2018/09/06 14:17:44] (DEBUG) middlewared.logger.CrashReporting.report():97 - Sending a crash report...
[2018/09/06 14:17:44] (DEBUG) raven.base.Client.send_remote():684 - Sending message of length 3326 to https://sentry.ixsystems.com/api/2/store/
 
rvassar

rvassar

Guru
Joined
May 2, 2018
Messages
972
That looks like the GUI tools timing out on the stop/start. Anything from "nfsd", or ZFS?
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
rvassar said:
That looks like the GUI tools timing out on the stop/start. Anything from "nfsd", or ZFS?
Click to expand...
Nothing at all on "nfsd" and nothing suspicious about "ZFS" either. I found this in daemon.log.

Code:
ep  6 15:23:26 helium mDNSResponder: mDNSResponder (Engineering Build) (Mar 14 2018 03:28:04) stopping
Sep  6 15:23:26 helium mDNSResponder: mDNS_FinalExit failed to send goodbye for: 0000000801091580 01   26 _https._tcp.local. PTR helium._https._tcp.local.
Sep  6 15:23:26 helium mDNSResponder: mDNS_FinalExit failed to send goodbye for: 0000000801092980 01   35 _middleware-ssl._tcp.local. PTR helium._middleware-ssl._tcp.local.
Sep  6 15:23:26 helium mDNSResponder: mDNSResponder (Engineering Build) (Mar 14 2018 03:28:04) starting
Sep  6 15:23:26 helium mDNSResponder: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
Sep  6 15:23:26 helium mDNSResponder: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
Sep  6 15:23:26 helium mDNSResponder: CheckNATMappings: Failed to allocate port 5350 UDP multicast socket for PCP & NAT-PMP announcements
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
Microkernel said:
Nothing happens at all in the console when trying to start/stop NFS. My config is really basic I just did a reinstall, added a cert and imported my existing volume.
Click to expand...

Did you specify any exports in the NFS config? You haven't mentioned that so just checking with simple things.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
Not port - exports. Configure what you want to share via the NFS service (Find it under Sharing, Add UNIX (NFS) Share)
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
j0hnby said:
Not port - exports. Configure what you want to share via the NFS service (Find it under Sharing, Add UNIX (NFS) Share)
Click to expand...
Sorry, I misread your post. I do have shares setup. Everything worked fine for a week when NFS just started to fail out of nowhere.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
Ah ok. Didn't think it would be that easy ;) haha.

If you turn on the console output to web option on the system, do you see anything for NFS in there if you try to start/stop the service?
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
j0hnby said:
Ah ok. Didn't think it would be that easy ;) haha.

If you turn on the console output to web option on the system, do you see anything for NFS in there if you try to start/stop the service?
Click to expand...
I already have the console displayed, but just like in the logs, nothing happens at all.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
That's very strange then - failure to start, but no errors. When you did the fresh install, you didn't restore a backup did you? Plus you are using the "old" ui right?
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
j0hnby said:
That's very strange then - failure to start, but no errors. When you did the fresh install, you didn't restore a backup did you? Plus you are using the "old" ui right?
Click to expand...
Yep I am using the old UI and I did a complete reinstall although I didn't wipe the drive before I did. I'll try doing that now and if that doesn't work I'll be looking for an alternative to FreeNAS.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
That is REALLY odd then. Happy for a teamview later if you want - I am curious to what's going on.
 
M

Microkernel

Dabbler
Joined
Sep 5, 2018
Messages
11
j0hnby said:
That is REALLY odd then. Happy for a teamview later if you want - I am curious to what's going on.
Click to expand...
Not really a big fan of teamviewer, but we can sort something out later if this reinstall doesn't help either if you want.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
It's up to you matey if you want the assistance - let me know either way.
 
j0hnby

j0hnby

Explorer
Joined
Apr 22, 2017
Messages
68
Cool stuff - if you don't like TV, feel free to suggest an alternate. I am at work at the moment but should be home/online by about 18:30 BST
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "NFS will not work despite reinstall"

Similar threads

F
System hung on reboot
Replies
3
Views
3K
dlavigne
D
HomebrewDotNet
Nfs crashes after running for a while and other stability issues
Replies
6
Views
2K
HomebrewDotNet
HomebrewDotNet
A
Web GUI missing - middleware not found
Replies
0
Views
715
avitus27
A
J
  • Locked
sabnzbd won't start - "CoreFoundation.h" file not found
Replies
4
Views
1K
JohnDoe123
J
cunningorb
  • Locked
Install Nextcloud 12 on FreeNAS 11 - all guides not working
2
Replies
21
Views
12K
Kokodreams
K
Top