Upgrading from 11.0-U4 to 11.1-RELEASE caused many issues!

Status
Not open for further replies.

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
Hi,

My FreeNAS server was working just fine until I did the following

1) Upgraded the FreeNAS (from 11.0-U4 to 11.1-RELEASE).
2) Upgraded the ECC-RAM (from 128GB to 256GB).
3) Enabled System->Advanced->Enable autotune.

Then, the FreeNAS started to throw some errors. Like:

First
when I reboot, and try to unlock the encrypted ZFS pool, it gives me this error:

in web-UI:
Code:
Environment:

Software Version: FreeNAS-11.1-RELEASE (dc7d195f4)
Request Method: POST
Request URL: http://storage.hom/storage/volume/4/unlock/?X-Progress-ID=685c823b-66d8-47d7-844a-d894f25bd1eb


Traceback:
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  42.			 response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _legacy_get_response
  249.			 response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  178.			 response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
  162.		 return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.				 return view_func(request, *args, **kwargs)
File "./freenasUI/storage/views.py" in volume_unlock
  1005.			 form.done(volume=volume)
File "./freenasUI/storage/forms.py" in done
  2797.			 _notifier.restart(svc)
File "./freenasUI/middleware/notifier.py" in restart
  223.			 return c.call('service.restart', what, {'onetime': onetime}, **kwargs)
File "./freenasUI/middleware/notifier.py" in restart
  223.			 return c.call('service.restart', what, {'onetime': onetime}, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py" in call
  389.			 raise CallTimeout("Call timeout")

Exception Type: CallTimeout at /storage/volume/4/unlock/
Exception Value: Call timeout


in console:
4lVTm

screenshot-037.png

But the pool was unlocked.


Second thing,
the emby plugin just stopped working and doesn't want to start complaining about mono-sgen. even with a newly installed emby plugin.. the error:

Code:
kernel: pid 33991 (mono-sgen), uid 989: exited on signal 6





Third thing, the transfer rate is really slow,
from FreeNAS to client: 70~80MB
from client to FreeNAS: 130~150MB
Even when I re-transfer it again.. seems the cache is not working.
ps: my connection is 10G Base-T (with MTU 9000 on FreeNAS, switch, and clients)


And
My swap was 40GB.. but when I've upgraded the RAM it becomes only 10GB !!


And
when I try to boot the 11.0-U4, it doesn't boot and gave me this error:
y4J3J

screenshot-035.png



Even when I disable the autotune, and delete all generated entries in System->Tunables table:
7YFrf

screenshot-036.png



This is my FreeNAS boot process:
 
Last edited:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
My first comment would be to take the system back to the original state, meaning remove the RAM you installed and restore your previous boot environment. Hopefully you saved a configuration of your 11.0-U4 setup, then you just need to restore it if your roll back failed.

Once you are able to get your system back to normal, then upgrade your RAM and test it, since you didn't mention that you tested it. If everything works fine then upgrade to 11.1 if you must. Personally I'd wait for 11.1-U1 which is due shortly.
 
Last edited by a moderator:

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
Wow, that's a lot of changes to make (it sounds like all at once)
to a server's configuration!

I second the advice from joeschmuck!!!

AND will add:

If you manage to get the box back to where it was when you started all this:
  • stay away from auto tune!!!
 

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
My first comment would be to take the system back to the original state, meaning remove the RAM you installed and restore your previous boot environment. Hopefully you saved a configuration of your 11.0-U4 setup, then you just need to restore it if your roll back failed.

Once you are able to get your system back to normal, then upgrade your RAM and test it, since you didn't mention that you tested it. If everything works fine then upgrade to 11.1 if you must. Personnaly I'd wait for 11.1-U1 which is due shortly.

I did exactly what you said, removed the newly installed RAM, restoring my 11.0-U4 config, reboot
Now both 11.1-RELEASE and 11.0-U4 don't want to boot anymore.
I'm stuck in the boot window:
screenshot-038.png


any suggestions?


stay away from auto tune!!!
I have learned my lesson :(
 

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
Ok, I kept on trying to boot it by rebooting and playing with boot-menu. till it was able to boot 11.1-RELEASE (seemed to have the 11.0-U4 config, but the title was 11.1-RELEASE)

I even tried:
Code:
set vfs.zfs.recover=1

in the kernel window

after I managed to login into web-UI, I've activated the 11.0-U4.
rebooted
everything was fine.
the swap was back to 40gb
emby worked just fine

but, I faced new issues:
1) the reports took a while till it is able to show the statistics (not a big deal)
2) the temperature of the all HDDs is really high 49~57C

any idea??
 
Last edited:

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
but, I faced new issues:
1) the reports took a while till it is able to show the statistics (not a big deal)
2) the temperature of the all HDDs is really high 49~57C

any idea??
If you are using the new UI, I would suspect the 11.0-U4 version (of the new UI)
is maybe not reporting the temps correctly. You could double check the temps by checking with SMART via SSH.
 

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
If you are using the new UI, I would suspect the 11.0-U4 version (of the new UI)
is maybe not reporting the temps correctly. You could double check the temps by checking with SMART via SSH.

The temps calmed down.. but they're still high in 40-ish
screenshot-039.png


I rebooted... the 11.1-RELEASE config was restored.. the UI was changed to 11.1-RELEASE
and the emby plugin won't start again
the swap became 10GB again..


I'm trying now to activate the 11.0-U4 first, then restore its config
 

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
same thing :mad:

I wonder, if I re-install FreeNAS.. would I be able to import my zpools easily? Will it automatically recognize the disks UUIDs and know which belongs to which zpool? Or does rely it on the numbers in /dev/daX?
 
Last edited by a moderator:

Nick2253

Wizard
Joined
Apr 21, 2014
Messages
1,633
ZFS does not use the /dev numbers. It will automatically recognize the drives, and should automatically import the pool. If you reinstall, I would suggest the following: backup your 11.0U4 config, wipe your boot SSDs, install 11.0U4, and import your backed up config.
 
Last edited by a moderator:

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
I would suggest the following: backup your 11.0U4 config, wipe your boot SSDs, install 11.0U4, and import your backed up config.
I completely concur with this action. And stay away from 11.1 for now, just get your system back to normal operation and leave it there for a few days to ensure you have no other side effects going on. Then I'd wait for 11.1-U1 which is due out in a few days before attempting an upgrade to it.

Also, if you have another SSD you could use to boot from, just remove the original pair and use the single SSD until you have things working again. This action would preserve your original drives just in case you need them for some odd recovery reason.
 
Last edited by a moderator:

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
Also, if you have another SSD you could use to boot from, just remove the original pair and use the single SSD until you have things working again. This action would preserve your original drives just in case you need them for some odd recovery reason.
It is exactly what I am doing right now.. :)

Thanks
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
the swap became 10GB again..
The rest isn't, but this is entirely expected behavior. FreeNAS 11.1 is mirroring swap, and only implementing a certain number (five, I believe) of mirrored pairs. Five mirrors of 2 GB each would be 10 GB of swap. This shouldn't be causing your problems either.
 

DeaDSouL

Dabbler
Joined
Dec 21, 2015
Messages
34
The rest isn't, but this is entirely expected behavior. FreeNAS 11.1 is mirroring swap, and only implementing a certain number (five, I believe) of mirrored pairs. Five mirrors of 2 GB each would be 10 GB of swap. This shouldn't be causing your problems either.
That is what it looked like from a fresh install :)


And stay away from 11.1 for now,
You were right.. it seems FreeNAS 11.1-RELEASE is buggy.


Thanks guys :)
 
Status
Not open for further replies.
Top