11.2 semi-automatic replication connection timeout

[David Steinberger]

I've been using replication tasks for some time, but thus far always used the manual setup, which was/is working flawless.
It's vacation time and I decided to try out the semi-automated replication setup but can't get it to work. Upon save I always get: Failed to connect to remote host ws://<<< FREENAS IP >>>:80/websocket with following exception timed out.

In debug.log I see a stack-trace w/ pretty much the same information:

Code:

Dec 26 13:48:50 nas uwsgi: [api.utils:12] Traceback (most recent call last):
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/api/utils.py", line 415, in save
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     form.save()
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/middleware/form.py", line 55, in save
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     result = self.__save()
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/middleware/form.py", line 114, in __save
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     raise
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/middleware/form.py", line 109, in __save
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     return c.call(f"{self.middleware_plugin}.{self._middleware_action}", *args, **kwargs)
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 453, in call
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     raise ValidationErrors(c.extra)
Dec 26 13:48:50 nas uwsgi: [api.utils:12] middlewared.client.client.ValidationErrors: [EINVAL] replication_create.remote_token: Failed to connect to remote host ws://<<< FREENAS IP >>>:80/websocket with following exception timed out
Dec 26 13:48:50 nas uwsgi: [api.utils:12]
Dec 26 13:48:50 nas uwsgi: [api.utils:12] During handling of the above exception, another exception occurred:
Dec 26 13:48:50 nas uwsgi: [api.utils:12]
Dec 26 13:48:50 nas uwsgi: [api.utils:12] Traceback (most recent call last):
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/api/utils.py", line 251, in dispatch
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     request_type, request, *args, **kwargs
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "/usr/local/lib/python3.6/site-packages/tastypie/resources.py", line 482, in dispatch
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     response = method(request, **kwargs)
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "/usr/local/lib/python3.6/site-packages/tastypie/resources.py", line 1384, in post_list
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     updated_bundle = self.obj_create(bundle, **self.remove_api_resource_names(kwargs))
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "/usr/local/lib/python3.6/site-packages/tastypie/resources.py", line 2175, in obj_create
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     return self.save(bundle)
Dec 26 13:48:50 nas uwsgi: [api.utils:12]   File "./freenasUI/api/utils.py", line 421, in save
Dec 26 13:48:50 nas uwsgi: [api.utils:12]     response_class=http.HttpConflict,
Dec 26 13:48:50 nas uwsgi: [api.utils:12] tastypie.exceptions.ImmediateHttpResponse
Dec 26 13:49:00 nas /autosnap.py: [ws4py:360] Closing message received (1000) 'b'''
Dec 26 13:49:01 nas /autorepl.py: [ws4py:360] Closing message received (1000) 'b'''
Dec 26 13:49:01 nas /autorepl.py: [tools.autorepl:221] Autosnap replication started
Dec 26 13:49:01 nas /autorepl.py: [tools.autorepl:222] temp log file: /tmp/repl-44754
Dec 26 13:49:01 nas /autorepl.py: [tools.autorepl:291] Checking dataset tank/storage/nextcloud
Dec 26 13:49:02 nas /autorepl.py: [tools.autorepl:625] Autosnap replication finished

Does anyone know how I can troubleshoot this?

 

[DD4711]

Same here. I'm too trying to get replication running from pool one to pool two on the same FreeNAS. Hopefully someone can help :)

 

[David Steinberger]

Just to be clear, I was not talking about setting up replication in general, going with manual works beautifully. What's not working for me is the semi-automated setup. @DD4711 just go manual for now and follow the troubleshooting section in the manual.

 

[Bonnie Follweiler]

I tried to replicate your issue and was unable to. My Replication Task set up by semi-Automatic is working. Just a few questions
What was the System version of the sending and receiving system?
What browser were you using?
Did you try it on the new GUI or the Legacy UI?
What is the path of the remote dataset you are using? For instance I used reptank/reptank
After entering the Temporary Auth Token did you click the SSH Key Scan/Scan SSH Key button before clicking the Save button?

 

[David Steinberger]

Hi Bonnie,

What was the System version of the sending and receiving system?

FreeNAS-11.2-RELEASE-U1; Initially I did it w/ 11.2 but patched it to U1 last night. I did try it again with U1 now. I am sending it from one pool to another that I created for testing purpose (named test); Both pools are encrypted.

What browser were you using?

Google Chrome Version 71.0.3578.98

Did you try it on the new GUI or the Legacy UI?

new GUI

What is the path of the remote dataset you are using?

test; I also tried test/test just to see if that changes anything

After entering the Temporary Auth Token did you click the SSH Key Scan/Scan SSH Key button before clicking the Save button?

Yes

The POST request that's fired upon save to http://192.168.1.254/api/v1.0/storage/replication/ errors out with a 409 and the response Failed to connect to remote host ws://192.168.1.254:80/websocket with following exception timed out. That error message is then displayed in the UI.

 

[guitarmassacre]

Dave, did you ever figure it out? I'm getting the same error.
"Failed to connect to remote host ws://192.168.1.2:80/websocket with following exception timed out"

My local and remote host is the same machine, just trying to replicate one pool to another. Not sure why the connection is timing out.

 

[Matt_G]

I am getting that same error myself trying to replicate between two pools on the same box using semi-automatic setup.
Had to use manual.
Also, using localhost as the remote host name did not work. Wouldn't accept the SSH key.
Had to use the IP address.
Why it has to use SSH when the data never leaves the system is beyond me.
The developers really need to look at pool to pool replication on the same server.
It needs some work/love.