Freenas to Synology, have beat it to death!

[Dirk]

Got a two disk Synology to backup the Freenass 11.3 media servers media.

Initially set up with a 4TB drive, installation of DSM 6.x (the latest) was fine, and a little thrashing around and Freenas transferred about a TB before drive went disks up with bad blocks.

So, start again with a WD datacenter 8tb drive which took two day to do a parity check. Everything stayed the same on the Freenas box.

Freenas side.........

An " rsync" user with same user name and password on Synology.

Synology side........

rsync enabled which created NetBackup share. User rsync added to share with full permission. ssh also enabled.

Freenas rsync task.......

Path to directory is pointing to the media directory on Freenas
User "rsync"
IP "Yada.yada.yada.yada"
Rsync mode "module"
Remote module "NetBackup"
Direction "push"

Hitting save results in a failed notification and suggests checking log. the only thing in the log downloaded is

"rsync: failed to connect to 192.168.0.30 (192.168.0.30): Connection refused (61)
rsync error: error in socket IO (code 10) at clientserver.c(127) [sender=3.1.3]"

From a shell on freenas I can ssh into the synology with rsync@192.168.0.30 and provide the password.

Since it already worked before, I'm at a loss as to what I'm missing.

Thanks

 

[Spearfoot]

Got a two disk Synology to backup the Freenass 11.3 media servers media.

Initially set up with a 4TB drive, installation of DSM 6.x (the latest) was fine, and a little thrashing around and Freenas transferred about a TB before drive went disks up with bad blocks.

So, start again with a WD datacenter 8tb drive which took two day to do a parity check. Everything stayed the same on the Freenas box.

Freenas side.........

An " rsync" user with same user name and password on Synology.

Synology side........

rsync enabled which created NetBackup share. User rsync added to share with full permission. ssh also enabled.

Freenas rsync task.......

Path to directory is pointing to the media directory on Freenas
User "rsync"
IP "Yada.yada.yada.yada"
Rsync mode "module"
Remote module "NetBackup"
Direction "push"

Hitting save results in a failed notification and suggests checking log. the only thing in the log downloaded is

"rsync: failed to connect to 192.168.0.30 (192.168.0.30): Connection refused (61)
rsync error: error in socket IO (code 10) at clientserver.c(127) [sender=3.1.3]"

From a shell on freenas I can ssh into the synology with rsync@192.168.0.30 and provide the password.

Since it already worked before, I'm at a loss as to what I'm missing.

Thanks

Try setting up ssh public key pair authentication between your FreeNAS and Synology servers. Your goal is to be able to start an ssh session without having to enter a password.

 

[Dirk]

Well after many more hours I did determine that it was indeed asking for a password by runing rsync from the command line. This suggests that the rsync task setup is lacking in Freenas tasks and can't work in module mode.

However It seems there is another problem in that trying to get a backup run from the command line also has a problem. It runs for a bit of time on the 4.5TB and just stops. Is there a log file somewhere to look at? This has consumed a good 20 plus hours of time so far just trying to get a backup. Very frustrating.
Thanks

 

[Spearfoot]

Well after many more hours I did determine that it was indeed asking for a password by runing rsync from the command line. This suggests that the rsync task setup is lacking in Freenas tasks and can't work in module mode.

However It seems there is another problem in that trying to get a backup run from the command line also has a problem. It runs for a bit of time on the 4.5TB and just stops. Is there a log file somewhere to look at? This has consumed a good 20 plus hours of time so far just trying to get a backup. Very frustrating.
Thanks

See this thread of mine from several years ago. It discusses some of the idiosyncracies of rsync, the main one being that it does not support Windows permissions. I also posted a script there that works with Synology and other UNIX-style operating systems.

Impaired rsync permissions support for Windows datasets

Rsync fails when copying Windows datasets with the '-p' ('Preserve Permissions') option. And there are no plans to implement this option in the future, because "rsync targets primarily unix-like systems." Frankly, I don't blame the rsync team one bit for avoiding the Windows nightmare. :smile...

www.ixsystems.com

 

[Dirk]

Thanks for the input spear. I'm not sure why the reference of Windows permissions though as the rsync is starting on Freenas 11.3 at one end and trying to push to a Synology with latest DSM on the other.

Current error on Freenas when I try to save the task is below.

"Password: @Error: account system disabled
rsync error: error starting client-server protocol (code 5) at main.c(1671) [sender=3.1.3]"

I've been researching that for several hours.
Thanks

 

[Spearfoot]

I'm not sure why the reference of Windows permissions though as the rsync is starting on Freenas 11.3 at one end and trying to push to a Synology with latest DSM on the other.

Because you might have Windows/SMB shares on either end of that connection.

Anyway, the rsync script available on that thread works for me and many other people, regardless of whether Windows permissions is an issue.

Do you have 'Enable network backup service' turned on under DSM's Control Panel->Backup Services on your Synology?

 

[Dirk]

Thanks for hanging in there Spear I understand the part on the possibility of SMB though Windows may not be involved.

Enable network backup service is one of the rabbit holes I went down early in this mess, only to find it does not exist anymore in DSM 6

 

[Newfoundland.Republic]

Hmmmm.... Mine is working fine. But... It has been some time since I set it up so I can't remember what I did. I do remember that it was a pain in the a$$ to get working.

Related, somewhat, question: Where is the actual script saved when you set up the rsync task? You could run it from the command line and see if it works.

 

[Dirk]

Not sure if Freenas even saves a script, it could just be passing what is in the GUI setup on the fly. If I execute an rsync from the command line it works but then dies at about a TB transferred, but that's another matter. The fact it will run from the command without wanting a password was part of the process. Why it spits up that latest error is beyond me when saving the GUI configuration (it test the configuration on exit).

 

[Dirk]

Well, finally got a stable SSH key setup with the help of a simple message at the end of this thread to insure all keys were the same (they were repeatedly not which may have come from the first challenge to allow the IP on a permanent basis). I also added the same key to the SSH part of the rsync users account on Freenas.

https://www.ixsystems.com/community...-that-does-not-show-up-on-on-any-sites.74523/

The same secondary problem still exists in that after the rsync is run "rsync -av --progress /mnt/alldrives/series rsync@192.168.0.nn:NetBackup"
It fails after running a couple hours. Again i'm not sure where to look for log info. The only thing I do find the next morning is that Freenas has logged out the root login which leaves the shell after I log back in at a fresh screen instead of the --progress shown before.

In any event, the issue of the GUI rsync settup still exists and with simply the user, dest.IP, module, and source defined it errors out with

"Password: @Error: auth failed on module NetBackup
rsync error: error starting client-server protocol (code 5) at main.c(1671) [sender=3.1.3]"

This same error happens if using SSH mode in the task setup.
Thanks

 

[Spearfoot]

Well, finally got a stable SSH key setup with the help of a simple message at the end of this thread to insure all keys were the same (they were repeatedly not which may have come from the first challenge to allow the IP on a permanent basis). I also added the same key to the SSH part of the rsync users account on Freenas.

https://www.ixsystems.com/community...-that-does-not-show-up-on-on-any-sites.74523/

The same secondary problem still exists in that after the rsync is run "rsync -av --progress /mnt/alldrives/series rsync@192.168.0.nn:NetBackup"
It fails after running a couple hours. Again i'm not sure where to look for log info. The only thing I do find the next morning is that Freenas has logged out the root login which leaves the shell after I log back in at a fresh screen instead of the --progress shown before.

In any event, the issue of the GUI rsync settup still exists and with simply the user, dest.IP, module, and source defined it errors out with

"Password: @Error: auth failed on module NetBackup
rsync error: error starting client-server protocol (code 5) at main.c(1671) [sender=3.1.3]"

This same error happens if using SSH mode in the task setup.
Thanks

Try adding a log file specification to your rsync command -- perhaps this will let you find out what the problem is. Example:

Code:

rsync -av --progress --log-file=/mnt/tank/admin/rsyncstatus.log /mnt/alldrives/series rsync@192.168.0.nn:NetBackup 

EDIT: Also -- check /var/log/daemon.log for rsync messages.

 

[Dirk]

Thanks spear, I actually found that minutes ago after posting and will fire off another CLI run now. Odd though that the task in both module mode and SSH mode both will not save without an error. It would be interesting to be able to see what the task is "building" and compare to CLI line that is working.

Will report back later.

 

[Dirk]

This is unbelievable!!! Using the same rsync command with the addition of the log file perimeter now fails requesting the password when it worked last night! The only thing between then and now is reading these messages after finding Freenas logged out this morning, the rsync CLI initiated task failed, and me playing with the rsync task settings in Freenas getting it to actually go to SSH mode without complaining about SSH authorization.

The ONLY thing I can think of is working with the rsync task in the GUI is messing with the keys. I've done nothing, and as you see from my message above, this is not the first or third time this has happened.

I'm going to check the keys.

 

[Dirk]

It's a good thing to be unemployed due to a virus I wouldn't be able to afford the time!

1. Fire up the shell
2. su rsync
3. rsync -av --progress --log-file=/mnt/AllDrives/homes/rsync/.ssh/rsyncrun.log /mnt/AllDrives/Series rsync@192.168.0.nn:NetBackup
4. Number three is where the pub key and authorized_keys file is (/mnt/AllDrives/homes/rsync/.ssh/). Those two and the key file on the Synology as well as the user account on Freenas's SSH setting match.
5. Authenticity of host (ip of synology) cannot be established. No matching key fingerprint found in dns. Are you sure you want to continue connecting? "yes"
6: known_hosts file created next to the pub key file with an entirely different key.

Rinse lather repeat for about 20 more hours now.

 

[Spearfoot]

It's a good thing to be unemployed due to a virus I wouldn't be able to afford the time!

1. Fire up the shell
2. su rsync
3. rsync -av --progress --log-file=/mnt/AllDrives/homes/rsync/.ssh/rsyncrun.log /mnt/AllDrives/Series rsync@192.168.0.nn:NetBackup
4. Number three is where the pub key and authorized_keys file is (/mnt/AllDrives/homes/rsync/.ssh/). Those two and the key file on the Synology as well as the user account on Freenas's SSH setting match.
5. Authenticity of host (ip of synology) cannot be established. No matching key fingerprint found in dns. Are you sure you want to continue connecting? "yes"
6: known_hosts file created next to the pub key file with an entirely different key.

Rinse lather repeat for about 20 more hours now.

What version of FreeNAS are you using? 11.3-U?

The known_hosts file is, under normal circumstances, only updated when you log onto a machine for the first time. The SSH system will add the host key to it, and this key is different from your user key.

 

[Dirk]

What version of FreeNAS are you using? 11.3-U?

The known_hosts file is, under normal circumstances, only updated when you log onto a machine for the first time. The SSH system will add the host key to it, and this key is different from your user key.

Yup 11.3-U3.2 If they are both different keys, why have two, and why with the pub key in the same place. I'm missing some SSH something here.

If this is normal to create the known_hosts file then I'm even more at a loss. The pub key, newly created known_hosts file, and the authorized keys are there on Freenas. The .PUB key, and authorized_keys file are on the Synology and all keys match.

Thanks for hanging in there with it Spear!!!

 

[Spearfoot]

Yup 11.3-U3.2 If they are both different keys, why have two, and why with the pub key in the same place. I'm missing some SSH something here.

If this is normal to create the known_hosts file then I'm even more at a loss. The pub key, newly created known_hosts file, and the authorized keys are there on Freenas. The .PUB key, and authorized_keys file are on the Synology and all keys match.

Thanks for hanging in there with it Spear!!!

Servers aren't users -- so they both have their own keys.

In your home directory there should be a folder named .ssh. In that directory there should be (at least) 4 files. Here are mine, for user 'root' on one of my FreeNAS systems:

Code:

root@bandit:~ # cd .ssh
root@bandit:~/.ssh # ll
total 11
-rw-------  1 root  wheel  3505 Jun 19 00:49 authorized_keys
-rw-------  1 root  wheel  3243 Apr 19  2017 id_rsa
-rw-r--r--  1 root  wheel   751 Apr 19  2017 id_rsa.pub
-rw-r--r--  1 root  wheel  1502 Jun 19 00:48 known_hosts

authorized_keys contains keys for the root user account on other systems
id_rsa is your private key
id_rsa.pub is your public key -- this will be copied to authorized_keys on other systems you access using SSH
known_hosts contains host keys for the hosts you access using SSH

When user 'root' logs on to this server from another server, SSH compares the remote user's id_rsa.pub key with keys stored in the local 'root' user's authorized_keys file. If they match, it allows an SSH connection without requiring a password. The same thing happens, only reversed, when local user 'root' connects to another server using SSH.

SSH also checks the remote server's host key against those stored in known_hosts. That's why there are two keys. The first time you log onto a remote system with SSH, it won't find the remote host's key in known_hosts and will ask if you want to add it and allow the connection.

This is all for the sake of security.

Anyway, I noticed you pointed your rsync log to a file in the .ssh directory -- this isn't a good idea. That directory is reserved for use by the SSH service -- which is critically important -- and we don't want to inadvertently dork it up...

In light of all this... are you able to log on to the remote system -- a Synology system, right? -- using SSH? Something like this: ssh rsync@192.168.1.nn should log you on to the remote system.

Also... I see you've created a user named 'rsync' -- you need to be logged on to your FreeNAS system as 'rsync' if you're going to run SSH or rsync commands using that account. Again, because the keys need to match up with what is set up on the remote (Synology) system.

And lastly -- I swear! -- just FYI, you don't need to worry about hiding your Class 'C' IP address ('192.168.1.nn'). No one can reach in and hack your system using that IP as these aren't visible on, or reachable from, the internet. They are reserved by the IANA for local LAN use (along with a few others) and everyone in the world uses the same IP subnets for that purpose.

 

[Dirk]

Thanks Spear. An FYI I started in the really 80's in computer science and went sideways into aeronautical engineering. I was writing Fortran code on a Vax running VMS in an editor called VI . Funny how many V's there were back then! My first computer a 286 with floppy, 500MB hard drive you would not have wanted to drop on your toes, and much to my girlfriends chagrin, I spent the extra $500 on the optional 2 megs memory and the huge 14 inch monitor. Total cost over 2K in mid 80's money.
There was no software for a PC, so I wound up converting Fortran code that was in the public domain from NASA to GW Basic. My first software was a program that modified 65 series airfoils for better lift. It would show the before and after airfoils by mathematically pixel mapping the screen, and print a full size template on a pin printer also by determining where to fire X and Y on the printer. Took and hour
I wound up in IT dept at the local community college in the late 90's as field help desk part time in the morning, and part time at a couple of casinos in my area which shortly resulted in being IT manager. Then in 2000 the .com bubble bust and network manager jobs were $16 an hour.

Were was this all going many may ask! Jut wanted let you know I'm a fossil, but not a total noob. You patience is appreciated and it's frustrating to spend this much time on things.

So, back to our station.... Freenas

I got the caveat on the log file location and will move it

The rsync user is in the administrators group and permissions on that .ssh directory seem to be good. Keys on local and remote authorized_keys match. First time in without known_hosts file results in what you would expect and have described above. The Freenas shell of course starts as root, and I su rsync before trying to run the rsync from command line. It requests a password. As a simple test, trying to SSH into the Synology under the same conditions also results in a password request.
/volume1/homes/rsync/.ssh on the Synology have keys that match though no known_hosts file is generated if it should be.

I know a private network IP isn't reachable, after 25 years reading forum messages it just seemed etiquette I really appreciate the effort in running down the SSH files, I was curious why the duplication of key info. So if rsync went to another site would it be appended with another known hosts entry? and in actuality any user using SSH is going to have three files then (aside side from the private key)

Best regards

 

[Spearfoot]

Thanks Spear. An FYI I started in the really 80's in computer science and went sideways into aeronautical engineering. I was writing Fortran code on a Vax running VMS in an editor called VI . Funny how many V's there were back then! My first computer a 286 with floppy, 500MB hard drive you would not have wanted to drop on your toes, and much to my girlfriends chagrin, I spent the extra $500 on the optional 2 megs memory and the huge 14 inch monitor. Total cost over 2K in mid 80's money.
There was no software for a PC, so I wound up converting Fortran code that was in the public domain from NASA to GW Basic. My first software was a program that modified 65 series airfoils for better lift. It would show the before and after airfoils by mathematically pixel mapping the screen, and print a full size template on a pin printer also by determining where to fire X and Y on the printer. Took and hour
I wound up in IT dept at the local community college in the late 90's as field help desk part time in the morning, and part time at a couple of casinos in my area which shortly resulted in being IT manager. Then in 2000 the .com bubble bust and network manager jobs were $16 an hour.

Were was this all going many may ask! Jut wanted let you know I'm a fossil, but not a total noob. You patience is appreciated and it's frustrating to spend this much time on things.

So, back to our station.... Freenas

View attachment 40732

I got the caveat on the log file location and will move it

The rsync user is in the administrators group and permissions on that .ssh directory seem to be good. Keys on local and remote authorized_keys match. First time in without known_hosts file results in what you would expect and have described above. The Freenas shell of course starts as root, and I su rsync before trying to run the rsync from command line. It requests a password. As a simple test, trying to SSH into the Synology under the same conditions also results in a password request.
/volume1/homes/rsync/.ssh on the Synology have keys that match though no known_hosts file is generated if it should be.

I know a private network IP isn't reachable, after 25 years reading forum messages it just seemed etiquette I really appreciate the effort in running down the SSH files, I was curious why the duplication of key info. So if rsync went to another site would it be appended with another known hosts entry? and in actuality any user using SSH is going to have three files then (aside side from the private key)

Best regards

You may be as old as I am! I started working on computers in the early 80's, too. Electrical engineering degree, but never worked in the field -- always been a software developer / software engineer / software architect. I work on flight simulators for the USAF now. First job was at a water resources engineering firm; FORTRAN hydraulics & hydrology models we'd upload and run on Crays at Boeing Computer Services using our trusty 1200 Baud Hayes modem; wrote my own Turbo Pascal program to run a digitizing tablet for map work; then we got a Harris minicomputer with a 3-byte word size! Try getting FORTRAN code running on that to match the output of 4-byte systems! Yikes. Those were the days!

I didn't mean to impugn your knowledge or experience -- on forums you never know if your audience is an Einstein-level genius or a less-gifted Jukes family member. And you run into both, as I'm sure you know!

Anyway, back to rsync... I'm hampered by the fact that I no longer have a Synology to work with; gave mine to one of my daughters. And it's been several years, so I don't recall the idiosyncracies of DSM. Dredging my memory: it seems like user 'admin' is the equivalent of or a synonym for the 'root' account and I'm certain you have to configure the system to accept rsync connections, including setting permissions.

From old notes I see that pathing was like this on my Synology (named 'bertrand'):

Code:

rsync-invoke.sh /mnt/tank/devtools/ root@bertrand:/volume1/devtools /mnt/tank/sysadmin/log/push-to-bertrand.log

That's about all I've got for the Synology side. I know it can be done; but you're on your own.

If you want to su as another user and get their environment -- which includes the SSH setup -- you need to use the '-' option, like this:

Code:

su - rsync

Running that will plop you into the 'rsync' user's home directory, complete with their shell environment setup. There should be a .ssh directory there, with the SSH files described earlier. And these contain the settings that must match the 'rsync' user account you've set up on the Synology.

It would be easier just to use PuTTY or MobaXterm (or whatever your favorite terminal program may be) to log on to your FreeNAS system as 'rsync' -- you don't have to always log on to shell sessions as 'root'.

Also, check out my Git repository, 'Scripts for using rsync' -- they might be handy for you later, after you've successfully beaten everything into shape:

GitHub - Spearfoot/scripts-for-using-rsync: Shell scripts for using rsync

Shell scripts for using rsync. Contribute to Spearfoot/scripts-for-using-rsync development by creating an account on GitHub.

github.com

Good luck!

 

[Dirk]

Wow, we have seen the world 30 years ago. I designed a single seat aircraft (front half of fuselage is still in the attic) and was going to use it as a thesis, but never got that far. A person at NASA Ames (Ray Hicks then working on the aerospace plane project) befriended me and had their computers spit out an airfoil specific to it. Visited once and have a pic of the wife standing in front of the worlds largest wind tunnel. You cant see her. Gave it up when the marriage succumbed to AIDS (Aircraft Induced Divorce Syndrome).

Been a pilot since late 70's, ever need someone to test the simulator shoot me a note!

No worries on the communication, I've taken no offense at anything you've said. I threw that out there for exactly the reasons you cited. You don't know the level of experience and you've been so generous with your time I didn't want you to think you had to lay out every little thing.

I'm going to agree that it's got to be on the Synology end, the rsync user has a home directory with admin level access and it has the ,pub key with authorized_keys in it.s .ssh directory (they all match) The fact that both the rsync and doing an SSH login both want a password suggests the Synology end isn't being seen correctly. I'll work on it from that end. You are correct, the Synology uses and admin account. Trying to do anything usually requires SSH in on that account and then su -i If I remember correctly to bee root.

Thanks, I'll figure it out and put another post here to close the issue and then start a thread outlining how to do it.

Be safe!