Rsync Tasks

16 minute read.

Last Modified 2022-05-31 12:01 EDT

You often need to copy data to another system for backup or when migrating to a new system. A fast and secure way of doing this is by using rsync. These instructions assume that both sides of the rsync task, host and remote, use a TrueNAS systems.

Basic Requirements

Rysnc requires a dataset with the needed data on either the host or remote system. Rsync provides the ability to either push or pull data. When using the Rsync Tasks function to push, data is copied from a host system to a remote system. When using the Rsync Tasks function to pull, data is pulled from a remote system and put on the host system.

The remote system must have the rsync service activated. Additional requirements are listed further down for either rsync module and SSH tasks.

Creating an Rsync Task

Tutorial Video

(Video URL: https://www.truenas.com/docs/files/scaleangelfishrsync.mp4)

Go to Data Protection > Rsync Tasks and click Add. The Add Rsync Task configuration screen displays.

DataProtectionAddRsyncTask

The Rsync Mode field has two primary rsync modes: Module and SSH. Each mode has different requirements. See the related tabs below for each rsync mode.

Module

Module Requirements

Before you create an rsync task on the host system, you must create a module on the remote system.

When TrueNAS is the remote system, create a module. Go to System > Services and click for the Rsync service. See the specific configuration instructions in the Rsync Service section of this article.

Rsync Module Process

Log in to the host system interface, go to Data Protection > Rsync Tasks, and click Add. The Add Rsync Task configuration panel displays. Use the scroll bar displayed at the right to access all configuration fields.

DataProtectionAddRsyncTask

Select the source dataset to use with the rsync task and a user account to run the rsync task. Choose a direction for the rsync task.

Select a schedule for the rsync task. If you need a custom schedule, select Custom.

Advanced Scheduler

Choosing a Presets option populatess in the rest of the fields. To customize a schedule, enter crontab values for the Minutes/Hours/Days.

These fields accept standard cron values. The simplest option is to enter a single number in the field. The task runs when the time value matches that number. For example, entering 10 means that the job runs when the time is ten minutes past the hour.

An asterisk (*) means match all values.

You can set specific time ranges by entering hyphenated number values. For example, entering 30-35 in the Minutes field sets the task to run at minutes 30, 31, 32, 33, 34, and 35.

You can also enter lists of values. Enter individual values separated by a comma (,). For example, entering 1,14 in the Hours field means the task runs at 1:00 AM (0100) and 2:00 PM (1400).

A slash (/) designates a step value. For example, entering * in Days runs the task every day of the month. Entering */2 runs it every other day.

Combining the above examples creates a schedule running a task each minute from 1:30-1:35 AM and 2:30-2:35 PM every other day.

TrueNAS has an option to select which Months the task runs. Leaving each month unset is the same as selecting every month.

The Days of Week schedules the task to run on specific days in addition to any listed days. For example, entering 1 in Days and setting Wed for Days of Week creates a schedule that starts a task on the first day of the month and every Wednesday of the month.

The Schedule Preview dipslays when the current settings mean the task runs.

Examples of CRON syntax

Syntax	Meaning	Examples
*	Every item.	* (minutes) = every minute of the hour. * (days) = every day.
*/N	Every N^th item.	/15 (minutes) = every 15th minute of the hour. /3 (days) = every 3rd day. */3 (months) = every 3rd month.
Comma and hyphen/dash	Each stated item (comma) Each item in a range (hyphen/dash).	1,31 (minutes) = on the 1st and 31st minute of the hour. 1-3,31 (minutes) = on the 1st to 3rd minutes inclusive, and the 31st minute, of the hour. mon-fri (days) = every Monday to Friday inclusive (every weekday). mar,jun,sep,dec (months) = every March, June, September, December.

You can specify days of the month or days of the week.

TrueNAS lets users create flexible schedules using the available options. The table below has some examples:

Desired schedule	Values to enter
3 times a day (at midnight, 08:00 and 16:00)	months=; days=; hours=0/8 or 0,8,16; minutes=0 (Meaning: every day of every month, when hours=0/8/16 and minutes=0)
Every Monday/Wednesday/Friday, at 8.30 pm	months=*; days=mon,wed,fri; hours=20; minutes=30
1st and 15th day of the month, during October to June, at 00:01 am	months=oct-dec,jan-jun; days=1,15; hours=0; minutes=1
Every 15 minutes during the working week, which is 8am - 7pm (08:00 - 19:00) Monday to Friday	Note that this requires two tasks to achieve: (1) months=; days=mon-fri; hours=8-18; minutes=/15 (2) months=*; days=mon-fri; hours=19; minutes=0 We need the second scheduled item, to execute at 19:00, otherwise we would stop at 18:45. Another workaround would be to stop at 18:45 or 19:45 rather than 19:00.

Next, enter the Remote Host IP address or hostname. Use the format username@remote_host when the username differs from the host entered into the Remote Host field. Select Module in the Rsync Mode drop-down. Type the Remote Module Name exactly as it appears on the remote system.

Configure the remaining options according to your specific needs.

Options

Source

Name	Description
Path	Browse to the path to be copied. FreeBSD file path limits apply. Other operating systems can have different limits which might affect how they can be used as sources or destinations.
User	Select the user to run the rsync task. The user selected must have permissions to write to the specified directory on the remote host.
Direction	Direct the flow of data to the remote host. During a push, the dataset transfers to the remote module. During a pull, the dataset stores files from the remote system.
Description	Enter a description of the rsync task.

Schedule

Name	Description
Schedule	Select a schedule preset or choose Custom to open the advanced scheduler.
Recursive	Set to include all subdirectories of the specified directory. When unset, only the specified directory is included.

Remote

Name	Description
Remote Host	Enter the IP address or hostname of the remote system that will store the copy. Use the format username@remote_host if the username differs on the remote host.
Rsync Mode	Choose to either use a custom-defined remote module of the rsync server or to use an SSH configuration for the rsync task.

More Options

Name	Description
Times	Set to preserve modification times of files.
Compress	Set to reduce the size of data to transmit. Recommended for slow connections.
Archive	When set, rsync is run recursively, preserving symlinks, permissions, modification times, group, and special files. When run as root, owner, device files, and special files are also preserved. Equivalent to passing the flags `-rlptgoD` to rsync.
Delete	Delete files in the destination directory that do not exist in the source directory.
Quiet	Set to suppress informational messages from the remote server.
Preserve Permissions	Set to preserve original file permissions. This is useful when the user is set to root.
Preserve Extended Attributes	Extended attributes are preserved, but must be supported by both systems.
Delay Updates	Set to save the temporary file from each updated file to a holding directory until the end of the transfer when all transferred files are renamed into place.
Auxiliary Parameters	Additional rsync(1) options to include. Separate entries by pressing Enter. Note: The “” character must be escaped with a backslash (\.txt) or used inside single quotes ('*.txt').
Enabled	Enable this rsync task. Unset to disable this rsync task without deleting it.

The Module mode adds the Remote Module Name field to the Remote section. You must define at least one module in rsyncd.conf(5) of the rsync server or in the rsync modules of another system.

If the Enable checkbox is not selected it disables the task schedule, but you can still save the rsync task and run it manually.

SSH

SSH Requirements

The remote system must have SSH enabled. To enable SSH in TrueNAS, go to System > Services and toggle SSH on.

The host system needs an established SSH connection to the remote for the rsync task. To create the connection, go to Credentials > Backup Credentials > SSH Connections and click Add. Populate the SSH Connections configuration panel fields as follows: Select Semi-automatic for the Setup Method and set Private Key to Generate New.

Can this be set up in a command line instead?

To use a command line, go to the Shell on the host system. a, enter su - {USERNAME}, where {USERNAME} is the TrueNAS user account that runs the rsync task. Enter ssh-keygen -t rsa to create the key pair. When prompted for a password, press Enter without setting a password (a password breaks the automated task). Here is an example of running the command:

truenas# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification is saved in /root/.ssh/id_rsa.
Your public key is saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:NZMgbuPvTHeEqi3SA/U5wW8un6AWrx8ZsRQdbJJHmR4 tester@truenas.local
The key's randomart image is:
+---[RSA 2048]----+
|      . o=o+     |
|     . .ooE.     |
|      +.o==.     |
|     o.oo+.+     |
|     ...S+. .    |
|    . ..++o.     |
|     o oB+. .    |
|    . =Bo+.o     |
|     o+==oo      |
+----[SHA256]-----+

The default public key location is ~/.ssh/id_rsa.pub. Enter cat ~/.ssh/id_rsa.pub to see the key and copy the file contents. Copy it to the corresponding user account on the remote system in Credentials > Users. By default, SCALE only displays the root user and prompts you to display hidden users. Follow the directions to locate the sshd user account. Click on the sshd user and then on Edit. Paste the key in SSH Public Key.

Next, copy the host key from the remote system to the host system user’s .ssh/known_hosts directory, using ssh-keyscan. On the host system, open the Shell and enter ssh-keyscan -t rsa {remoteIPaddress} >> {userknown_hostsDir} where {remoteIPaddress} is the remote system IP address and {userknown_hostsDir} is the known_hosts directory on the host system. Example: ssh-keyscan -t rsa 192.168.2.6 >> /root/.ssh/known_hosts.

SSH Mode Process

Go to Data Protection > Rsync Tasks and click Add.

DataProtectionAddRsyncTaskSshMode

Configure the rsyn task SSH settings by selecting SSH on the Rsync Mode drop-down. Type the Port number and Remote Path.

Next, define the Source dataset to use for the rsync task and select a User account. The User field entry must be identical to the SSH Connection Username.

Choose a Direction for the rsync task as either Push or Pull and then define the task Schedule. If you need a custom schedule, select Custom.

Advanced Scheduler

Choosing a Presets option populatess in the rest of the fields. To customize a schedule, enter crontab values for the Minutes/Hours/Days.

An asterisk (*) means match all values.

You can set specific time ranges by entering hyphenated number values. For example, entering 30-35 in the Minutes field sets the task to run at minutes 30, 31, 32, 33, 34, and 35.

You can also enter lists of values. Enter individual values separated by a comma (,). For example, entering 1,14 in the Hours field means the task runs at 1:00 AM (0100) and 2:00 PM (1400).

A slash (/) designates a step value. For example, entering * in Days runs the task every day of the month. Entering */2 runs it every other day.

Combining the above examples creates a schedule running a task each minute from 1:30-1:35 AM and 2:30-2:35 PM every other day.

TrueNAS has an option to select which Months the task runs. Leaving each month unset is the same as selecting every month.

The Schedule Preview dipslays when the current settings mean the task runs.

Examples of CRON syntax

Syntax	Meaning	Examples
*	Every item.	* (minutes) = every minute of the hour. * (days) = every day.
*/N	Every N^th item.	/15 (minutes) = every 15th minute of the hour. /3 (days) = every 3rd day. */3 (months) = every 3rd month.
Comma and hyphen/dash	Each stated item (comma) Each item in a range (hyphen/dash).	1,31 (minutes) = on the 1st and 31st minute of the hour. 1-3,31 (minutes) = on the 1st to 3rd minutes inclusive, and the 31st minute, of the hour. mon-fri (days) = every Monday to Friday inclusive (every weekday). mar,jun,sep,dec (months) = every March, June, September, December.

You can specify days of the month or days of the week.

TrueNAS lets users create flexible schedules using the available options. The table below has some examples:

Desired schedule	Values to enter
3 times a day (at midnight, 08:00 and 16:00)	months=; days=; hours=0/8 or 0,8,16; minutes=0 (Meaning: every day of every month, when hours=0/8/16 and minutes=0)
Every Monday/Wednesday/Friday, at 8.30 pm	months=*; days=mon,wed,fri; hours=20; minutes=30
1st and 15th day of the month, during October to June, at 00:01 am	months=oct-dec,jan-jun; days=1,15; hours=0; minutes=1
Every 15 minutes during the working week, which is 8am - 7pm (08:00 - 19:00) Monday to Friday	Note that this requires two tasks to achieve: (1) months=; days=mon-fri; hours=8-18; minutes=/15 (2) months=*; days=mon-fri; hours=19; minutes=0 We need the second scheduled item, to execute at 19:00, otherwise we would stop at 18:45. Another workaround would be to stop at 18:45 or 19:45 rather than 19:00.

Next, enter the Remote Host IP address or hostname. Use the format username@remote_host if the username differs on the remote host. Configure the remaining options according to your specific needs.

Options

Source

Name	Description
Path	Browse to the path to be copied. FreeBSD file path limits apply. Other operating systems can have different limits which might affect how they can be used as sources or destinations.
User	Select the user to run the rsync task. The user selected must have permissions to write to the specified directory on the remote host.
Direction	Direct the flow of data to the remote host. During a push, the dataset transfers to the remote module. During a pull, the dataset stores files from the remote system.
Description	Enter a description of the rsync task.

Schedule

Name	Description
Schedule	Select a schedule preset or choose Custom to open the advanced scheduler.
Recursive	Set to include all subdirectories of the specified directory. When unset, only the specified directory is included.

Remote

Name	Description
Remote Host	Enter the IP address or hostname of the remote system that will store the copy. Use the format username@remote_host if the username differs on the remote host.
Rsync Mode	Choose to either use a custom-defined remote module of the rsync server or to use an SSH configuration for the rsync task.

More Options

Name	Description
Times	Set to preserve modification times of files.
Compress	Set to reduce the size of data to transmit. Recommended for slow connections.
Archive	When set, rsync is run recursively, preserving symlinks, permissions, modification times, group, and special files. When run as root, owner, device files, and special files are also preserved. Equivalent to passing the flags `-rlptgoD` to rsync.
Delete	Delete files in the destination directory that do not exist in the source directory.
Quiet	Set to suppress informational messages from the remote server.
Preserve Permissions	Set to preserve original file permissions. This is useful when the user is set to root.
Preserve Extended Attributes	Extended attributes are preserved, but must be supported by both systems.
Delay Updates	Set to save the temporary file from each updated file to a holding directory until the end of the transfer when all transferred files are renamed into place.
Auxiliary Parameters	Additional rsync(1) options to include. Separate entries by pressing Enter. Note: The “” character must be escaped with a backslash (\.txt) or used inside single quotes ('*.txt').
Enabled	Enable this rsync task. Unset to disable this rsync task without deleting it.

Additional options for the SSH Rsync Mode:

Remote SSH Port : Enter the SSH port number of the remote system. By default, 22 is reserved in TrueNAS.
Remote Path : Browse to the existing path on the remote host to sync with. Maximum path length is 255 characters.
Validate Remote Path : Select checkbox to automatically create the defined Remote Path when it does not exist.

If the Enabled checkbox is not selected it disables the task schedule without deleting the configuration. You can still run the rsync task by going to Data Protection > Rsync Tasks and clicking , then the Run Now icon.

Rsync Service and Modules

The rsync task does not work unless the related system service is turned on. To turn the rsync service on, go to System > Services and toggle the Rsync on. To activate the service whenever TrueNAS boots, select the Start Automatically checkbox.

SystemServicesRsyncOn

Click the to configure the service on the Services > RSYNC > Rsync screen. There are two tabs for rsync configuration: basic Configure options and Rsync Module creation and management.

Configure

SystemServicesEditRsyncConfigureTab

Name	Description
TCP Port	rsyncd listens on this port.
Auxiliary Parameters	Enter any additional parameters from rsyncd.conf(5).

Use the default settings unless a specific change is required. Remember to click Save after changing any settings.

Rsync Module

All created modules are listed on the Rsync Module tab.

SystemServicesRsyncRsyncModuleTab

If an rsync module is not listed, click Add RSYNC Modules or Add at the top right of the screen to add one. The SERVICE > RSYNC > RSYNC MODULE Add screen displays.

AddRsyncModuleScreenTop

Scroll down to see all configuration fields.

AddRsyncModuleScreenBottom

General

Name	Description
Name	Module name that matches the name requested by the rsync client.
Path	Browse to the pool or dataset to store received data.
Comment	Describe this module.
Enabled	Activate this module for use with Rsync. Unset this field to deactivate the module without completely removing it.

Access

Name	Description
Access Mode	Choose permissions for this rsync module.
Max Connections	Maximum number of connections to this module. 0 is unlimited.
User	TrueNAS user account that runs the rsync command during file transfers to and from this module.
Group	TrueNAS group account that runs the rsync command during file transfers to and from this module.
Hosts Allow	From rsyncd.conf(5). A list of patterns to match with the hostname and IP address of a connecting client. The connection is rejected if no patterns match. Separate entries by pressing `Enter`.
Hosts Deny	From rsyncd.conf(5). A list of patterns to match with the hostname and IP address of a connecting client. The connection is rejected when the patterns match. Separate entries by pressing `Enter`.

Other Options

Name	Description
Auxiliary Parameters	Enter any additional parameters from rsyncd.conf(5).

When a Hosts Allow list is defined, only the IPs and hostnames on the list are able to connect to the module.

To EDIT or DELETE a module, go to the Rsync Modules list and click for an entry.