Get a Quote   (408) 943-4100               TrueNAS Discord      VendOp_Icon_15x15px   Commercial Support Toggle between Light and Dark mode

Rsync Tasks

  16 minute read.

Last Modified 2022-05-31 12:01 EDT

You often need to copy data to another system for backup or when migrating to a new system. A fast and secure way of doing this is by using rsync. These instructions assume that both sides of the rsync task, host and remote, use a TrueNAS systems.

Basic Requirements

Rysnc requires a dataset with the needed data on either the host or remote system. Rsync provides the ability to either push or pull data. When using the Rsync Tasks function to push, data is copied from a host system to a remote system. When using the Rsync Tasks function to pull, data is pulled from a remote system and put on the host system.

The remote system must have the rsync service activated. Additional requirements are listed further down for either rsync module and SSH tasks.

Creating an Rsync Task

(Video URL: https://www.truenas.com/docs/files/scaleangelfishrsync.mp4)

Go to Data Protection > Rsync Tasks and click Add. The Add Rsync Task configuration screen displays.

DataProtectionAddRsyncTask

The Rsync Mode field has two primary rsync modes: Module and SSH. Each mode has different requirements. See the related tabs below for each rsync mode.

Module Requirements

Before you create an rsync task on the host system, you must create a module on the remote system.

When TrueNAS is the remote system, create a module. Go to System > Services and click for the Rsync service. See the specific configuration instructions in the Rsync Service section of this article.

Rsync Module Process

Log in to the host system interface, go to Data Protection > Rsync Tasks, and click Add. The Add Rsync Task configuration panel displays. Use the scroll bar displayed at the right to access all configuration fields.

DataProtectionAddRsyncTask

Select the source dataset to use with the rsync task and a user account to run the rsync task. Choose a direction for the rsync task.

Select a schedule for the rsync task. If you need a custom schedule, select Custom.

AdvancedSchedulerSCALE

Choosing a Presets option populatess in the rest of the fields. To customize a schedule, enter crontab values for the Minutes/Hours/Days.

These fields accept standard cron values. The simplest option is to enter a single number in the field. The task runs when the time value matches that number. For example, entering 10 means that the job runs when the time is ten minutes past the hour.

An asterisk (*) means match all values.

You can set specific time ranges by entering hyphenated number values. For example, entering 30-35 in the Minutes field sets the task to run at minutes 30, 31, 32, 33, 34, and 35.

You can also enter lists of values. Enter individual values separated by a comma (,). For example, entering 1,14 in the Hours field means the task runs at 1:00 AM (0100) and 2:00 PM (1400).

A slash (/) designates a step value. For example, entering * in Days runs the task every day of the month. Entering */2 runs it every other day.

Combining the above examples creates a schedule running a task each minute from 1:30-1:35 AM and 2:30-2:35 PM every other day.

TrueNAS has an option to select which Months the task runs. Leaving each month unset is the same as selecting every month.

The Days of Week schedules the task to run on specific days in addition to any listed days. For example, entering 1 in Days and setting Wed for Days of Week creates a schedule that starts a task on the first day of the month and every Wednesday of the month.

The Schedule Preview dipslays when the current settings mean the task runs.

Examples of CRON syntax

SyntaxMeaningExamples
*Every item.* (minutes) = every minute of the hour.
* (days) = every day.
*/NEvery Nth item.*/15 (minutes) = every 15th minute of the hour.
*/3 (days) = every 3rd day.
*/3 (months) = every 3rd month.
Comma and hyphen/dashEach stated item (comma)
Each item in a range (hyphen/dash).
1,31 (minutes) = on the 1st and 31st minute of the hour.
1-3,31 (minutes) = on the 1st to 3rd minutes inclusive, and the 31st minute, of the hour.
mon-fri (days) = every Monday to Friday inclusive (every weekday).
mar,jun,sep,dec (months) = every March, June, September, December.

You can specify days of the month or days of the week.

TrueNAS lets users create flexible schedules using the available options. The table below has some examples:

Desired scheduleValues to enter
3 times a day (at midnight, 08:00 and 16:00)months=*; days=*; hours=0/8 or 0,8,16; minutes=0
(Meaning: every day of every month, when hours=0/8/16 and minutes=0)
Every Monday/Wednesday/Friday, at 8.30 pmmonths=*; days=mon,wed,fri; hours=20; minutes=30
1st and 15th day of the month, during October to June, at 00:01 ammonths=oct-dec,jan-jun; days=1,15; hours=0; minutes=1
Every 15 minutes during the working week, which is 8am - 7pm (08:00 - 19:00) Monday to FridayNote that this requires two tasks to achieve:
(1) months=*; days=mon-fri; hours=8-18; minutes=*/15
(2) months=*; days=mon-fri; hours=19; minutes=0
We need the second scheduled item, to execute at 19:00, otherwise we would stop at 18:45. Another workaround would be to stop at 18:45 or 19:45 rather than 19:00.

Next, enter the Remote Host IP address or hostname. Use the format username@remote_host when the username differs from the host entered into the Remote Host field. Select Module in the Rsync Mode drop-down. Type the Remote Module Name exactly as it appears on the remote system.

Configure the remaining options according to your specific needs.

Source

NameDescription
PathBrowse to the path to be copied. FreeBSD file path limits apply. Other operating systems can have different limits which might affect how they can be used as sources or destinations.
UserSelect the user to run the rsync task. The user selected must have permissions to write to the specified directory on the remote host.
DirectionDirect the flow of data to the remote host. During a push, the dataset transfers to the remote module. During a pull, the dataset stores files from the remote system.
DescriptionEnter a description of the rsync task.

Schedule

NameDescription
ScheduleSelect a schedule preset or choose Custom to open the advanced scheduler.
RecursiveSet to include all subdirectories of the specified directory. When unset, only the specified directory is included.

Remote

NameDescription
Remote HostEnter the IP address or hostname of the remote system that will store the copy. Use the format username@remote_host if the username differs on the remote host.
Rsync ModeChoose to either use a custom-defined remote module of the rsync server or to use an SSH configuration for the rsync task.

More Options

NameDescription
TimesSet to preserve modification times of files.
CompressSet to reduce the size of data to transmit. Recommended for slow connections.
ArchiveWhen set, rsync is run recursively, preserving symlinks, permissions, modification times, group, and special files. When run as root, owner, device files, and special files are also preserved. Equivalent to passing the flags -rlptgoD to rsync.
DeleteDelete files in the destination directory that do not exist in the source directory.
QuietSet to suppress informational messages from the remote server.
Preserve PermissionsSet to preserve original file permissions. This is useful when the user is set to root.
Preserve Extended AttributesExtended attributes are preserved, but must be supported by both systems.
Delay UpdatesSet to save the temporary file from each updated file to a holding directory until the end of the transfer when all transferred files are renamed into place.
Auxiliary ParametersAdditional rsync(1) options to include. Separate entries by pressing Enter. Note: The “” character must be escaped with a backslash (\.txt) or used inside single quotes ('*.txt').
EnabledEnable this rsync task. Unset to disable this rsync task without deleting it.

The Module mode adds the Remote Module Name field to the Remote section. You must define at least one module in rsyncd.conf(5) of the rsync server or in the rsync modules of another system.

If the Enable checkbox is not selected it disables the task schedule, but you can still save the rsync task and run it manually.

SSH Requirements

The remote system must have SSH enabled. To enable SSH in TrueNAS, go to System > Services and toggle SSH on.

The host system needs an established SSH connection to the remote for the rsync task. To create the connection, go to Credentials > Backup Credentials > SSH Connections and click Add. Populate the SSH Connections configuration panel fields as follows: Select Semi-automatic for the Setup Method and set Private Key to Generate New.

To use a command line, go to the Shell on the host system. a, enter su - {USERNAME}, where {USERNAME} is the TrueNAS user account that runs the rsync task. Enter ssh-keygen -t rsa to create the key pair. When prompted for a password, press Enter without setting a password (a password breaks the automated task). Here is an example of running the command:

truenas# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification is saved in /root/.ssh/id_rsa.
Your public key is saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:NZMgbuPvTHeEqi3SA/U5wW8un6AWrx8ZsRQdbJJHmR4 tester@truenas.local
The key's randomart image is:
+---[RSA 2048]----+
|      . o=o+     |
|     . .ooE.     |
|      +.o==.     |
|     o.oo+.+     |
|     ...S+. .    |
|    . ..++o.     |
|     o oB+. .    |
|    . =Bo+.o     |
|     o+==oo      |
+----[SHA256]-----+

The default public key location is ~/.ssh/id_rsa.pub. Enter cat ~/.ssh/id_rsa.pub to see the key and copy the file contents. Copy it to the corresponding user account on the remote system in Credentials > Users. By default, SCALE only displays the root user and prompts you to display hidden users. Follow the directions to locate the sshd user account. Click on the sshd user and then on Edit. Paste the key in SSH Public Key.

Next, copy the host key from the remote system to the host system user’s .ssh/known_hosts directory, using ssh-keyscan. On the host system, open the Shell and enter ssh-keyscan -t rsa {remoteIPaddress} >> {userknown_hostsDir} where {remoteIPaddress} is the remote system IP address and {userknown_hostsDir} is the known_hosts directory on the host system. Example: ssh-keyscan -t rsa 192.168.2.6 >> /root/.ssh/known_hosts.

SSH Mode Process

Go to Data Protection > Rsync Tasks and click Add.

DataProtectionAddRsyncTaskSshMode

Configure the rsyn task SSH settings by selecting SSH on the Rsync Mode drop-down. Type the Port number and Remote Path.

Next, define the Source dataset to use for the rsync task and select a User account. The User field entry must be identical to the SSH Connection Username.

Choose a Direction for the rsync task as either Push or Pull and then define the task Schedule. If you need a custom schedule, select Custom.

AdvancedSchedulerSCALE

Choosing a Presets option populatess in the rest of the fields. To customize a schedule, enter crontab values for the Minutes/Hours/Days.

These fields accept standard cron values. The simplest option is to enter a single number in the field. The task runs when the time value matches that number. For example, entering 10 means that the job runs when the time is ten minutes past the hour.

An asterisk (*) means match all values.

You can set specific time ranges by entering hyphenated number values. For example, entering 30-35 in the Minutes field sets the task to run at minutes 30, 31, 32, 33, 34, and 35.

You can also enter lists of values. Enter individual values separated by a comma (,). For example, entering 1,14 in the Hours field means the task runs at 1:00 AM (0100) and 2:00 PM (1400).

A slash (/) designates a step value. For example, entering * in Days runs the task every day of the month. Entering */2 runs it every other day.

Combining the above examples creates a schedule running a task each minute from 1:30-1:35 AM and 2:30-2:35 PM every other day.

TrueNAS has an option to select which Months the task runs. Leaving each month unset is the same as selecting every month.

The Days of Week schedules the task to run on specific days in addition to any listed days. For example, entering 1 in Days and setting Wed for Days of Week creates a schedule that starts a task on the first day of the month and every Wednesday of the month.

The Schedule Preview dipslays when the current settings mean the task runs.

Examples of CRON syntax

SyntaxMeaningExamples
*Every item.* (minutes) = every minute of the hour.
* (days) = every day.
*/NEvery Nth item.*/15 (minutes) = every 15th minute of the hour.
*/3 (days) = every 3rd day.
*/3 (months) = every 3rd month.
Comma and hyphen/dashEach stated item (comma)
Each item in a range (hyphen/dash).
1,31 (minutes) = on the 1st and 31st minute of the hour.
1-3,31 (minutes) = on the 1st to 3rd minutes inclusive, and the 31st minute, of the hour.
mon-fri (days) = every Monday to Friday inclusive (every weekday).
mar,jun,sep,dec (months) = every March, June, September, December.

You can specify days of the month or days of the week.

TrueNAS lets users create flexible schedules using the available options. The table below has some examples:

Desired scheduleValues to enter
3 times a day (at midnight, 08:00 and 16:00)months=*; days=*; hours=0/8 or 0,8,16; minutes=0
(Meaning: every day of every month, when hours=0/8/16 and minutes=0)
Every Monday/Wednesday/Friday, at 8.30 pmmonths=*; days=mon,wed,fri; hours=20; minutes=30
1st and 15th day of the month, during October to June, at 00:01 ammonths=oct-dec,jan-jun; days=1,15; hours=0; minutes=1
Every 15 minutes during the working week, which is 8am - 7pm (08:00 - 19:00) Monday to FridayNote that this requires two tasks to achieve:
(1) months=*; days=mon-fri; hours=8-18; minutes=*/15
(2) months=*; days=mon-fri; hours=19; minutes=0
We need the second scheduled item, to execute at 19:00, otherwise we would stop at 18:45. Another workaround would be to stop at 18:45 or 19:45 rather than 19:00.

Next, enter the Remote Host IP address or hostname. Use the format username@remote_host if the username differs on the remote host. Configure the remaining options according to your specific needs.

Source

NameDescription
PathBrowse to the path to be copied. FreeBSD file path limits apply. Other operating systems can have different limits which might affect how they can be used as sources or destinations.
UserSelect the user to run the rsync task. The user selected must have permissions to write to the specified directory on the remote host.
DirectionDirect the flow of data to the remote host. During a push, the dataset transfers to the remote module. During a pull, the dataset stores files from the remote system.
DescriptionEnter a description of the rsync task.

Schedule

NameDescription
ScheduleSelect a schedule preset or choose Custom to open the advanced scheduler.
RecursiveSet to include all subdirectories of the specified directory. When unset, only the specified directory is included.

Remote

NameDescription
Remote HostEnter the IP address or hostname of the remote system that will store the copy. Use the format username@remote_host if the username differs on the remote host.
Rsync ModeChoose to either use a custom-defined remote module of the rsync server or to use an SSH configuration for the rsync task.

More Options

NameDescription
TimesSet to preserve modification times of files.
CompressSet to reduce the size of data to transmit. Recommended for slow connections.
ArchiveWhen set, rsync is run recursively, preserving symlinks, permissions, modification times, group, and special files. When run as root, owner, device files, and special files are also preserved. Equivalent to passing the flags -rlptgoD to rsync.
DeleteDelete files in the destination directory that do not exist in the source directory.
QuietSet to suppress informational messages from the remote server.
Preserve PermissionsSet to preserve original file permissions. This is useful when the user is set to root.
Preserve Extended AttributesExtended attributes are preserved, but must be supported by both systems.
Delay UpdatesSet to save the temporary file from each updated file to a holding directory until the end of the transfer when all transferred files are renamed into place.
Auxiliary ParametersAdditional rsync(1) options to include. Separate entries by pressing Enter. Note: The “” character must be escaped with a backslash (\.txt) or used inside single quotes ('*.txt').
EnabledEnable this rsync task. Unset to disable this rsync task without deleting it.

Additional options for the SSH Rsync Mode:

  • Remote SSH Port : Enter the SSH port number of the remote system. By default, 22 is reserved in TrueNAS.
  • Remote Path : Browse to the existing path on the remote host to sync with. Maximum path length is 255 characters.
  • Validate Remote Path : Select checkbox to automatically create the defined Remote Path when it does not exist.

If the Enabled checkbox is not selected it disables the task schedule without deleting the configuration. You can still run the rsync task by going to Data Protection > Rsync Tasks and clicking , then the Run Now icon.

Rsync Service and Modules

The rsync task does not work unless the related system service is turned on. To turn the rsync service on, go to System > Services and toggle the Rsync on. To activate the service whenever TrueNAS boots, select the Start Automatically checkbox.

SystemServicesRsyncOn

Click the to configure the service on the Services > RSYNC > Rsync screen. There are two tabs for rsync configuration: basic Configure options and Rsync Module creation and management.

SystemServicesEditRsyncConfigureTab

NameDescription
TCP Portrsyncd listens on this port.
Auxiliary ParametersEnter any additional parameters from rsyncd.conf(5).

Use the default settings unless a specific change is required. Remember to click Save after changing any settings.

All created modules are listed on the Rsync Module tab.

SystemServicesRsyncRsyncModuleTab

If an rsync module is not listed, click Add RSYNC Modules or Add at the top right of the screen to add one. The SERVICE > RSYNC > RSYNC MODULE Add screen displays.

AddRsyncModuleScreenTop

Scroll down to see all configuration fields.

AddRsyncModuleScreenBottom

General

NameDescription
NameModule name that matches the name requested by the rsync client.
PathBrowse to the pool or dataset to store received data.
CommentDescribe this module.
EnabledActivate this module for use with Rsync. Unset this field to deactivate the module without completely removing it.

Access

NameDescription
Access ModeChoose permissions for this rsync module.
Max ConnectionsMaximum number of connections to this module. 0 is unlimited.
UserTrueNAS user account that runs the rsync command during file transfers to and from this module.
GroupTrueNAS group account that runs the rsync command during file transfers to and from this module.
Hosts AllowFrom rsyncd.conf(5). A list of patterns to match with the hostname and IP address of a connecting client. The connection is rejected if no patterns match. Separate entries by pressing Enter.
Hosts DenyFrom rsyncd.conf(5). A list of patterns to match with the hostname and IP address of a connecting client. The connection is rejected when the patterns match. Separate entries by pressing Enter.

Other Options

NameDescription
Auxiliary ParametersEnter any additional parameters from rsyncd.conf(5).
When a Hosts Allow list is defined, only the IPs and hostnames on the list are able to connect to the module.

To EDIT or DELETE a module, go to the Rsync Modules list and click for an entry.