Time machine quota not working, anymore?

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
I just noticed that on my wife's Macbook the "Backup" share is shown with over 4 TB of available space. Which is the remaining free space of my zpool. This was not the case some time earlier, but I cannot tell when that changed. I use an SMB auxiliary parameter to limit the TM volume size. Seems like that does not have any effect, anymore. TN 12.0-U4.1.

Bildschirmfoto 2021-07-15 um 13.41.17.png
Bildschirmfoto 2021-07-17 um 14.29.56.png
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
If you use the default "multi-user timemachine", then a data set will be created for each user that logs in to that time machine SMB share.

I created specific users for each device.

And then TN will create a dataset for each user under the TM dataset.

I was then able to set individual quotas for each device's dataset.

Seems to work well.

I left the "legacy" timemachine for AFP only backups to older devices.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
I have only one dataset with guest access for two Macs and I would prefer to keep it that way. The auxliary parameter I cited used to work.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776

John45622

Contributor
Joined
Dec 2, 2020
Messages
105
Same here. I also have the problem that TN will not let me delete a 1TB TM sparse bundle from my share. When I do that the mac says deleting for 30minutes and comes back with a "could not complete because an object "bands" is in use. The DMG is not mounted and TM is turned off.
Now I have a 1TB old TM backup sitting on my TN server that I can't delete... :frown:
 

dak180

Patron
Joined
Nov 22, 2017
Messages
310

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
*ping* :smile:

Code:
root@freenas[~]# testparm -s
[...]
[Backup]
    ea support = No
    guest ok = Yes
    mangled names = no
    path = /mnt/hdd/share/timemachine
    read only = No
    vfs objects = catia fruit streams_xattr noacl aio_fbsd
    fruit:time machine max size = 1024G
    fruit:locking = none
    fruit:time machine = yes
    fruit:resource = stream
    fruit:metadata = stream
    fruit:encoding = native
    nfs4:chown = true
[...]
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Code:
static uint64_t fruit_disk_free(vfs_handle_struct *handle,
                                const struct smb_filename *smb_fname,
                                uint64_t *_bsize,
                                uint64_t *_dfree,
                                uint64_t *_dsize)
{
        struct fruit_config_data *config = NULL;
        struct fruit_disk_free_state state = {0};
        struct smb_Dir *dir_hnd = NULL;
        const char *dname = NULL;
        char *talloced = NULL;
        long offset = 0;
        uint64_t dfree;
        uint64_t dsize;
        bool ok;

        SMB_VFS_HANDLE_GET_DATA(handle, config,
                                struct fruit_config_data,
                                return UINT64_MAX);

        if (!config->time_machine ||
            config->time_machine_max_size == 0)
        {
                return SMB_VFS_NEXT_DISK_FREE(handle,
                                              smb_fname,
                                              _bsize,
                                              _dfree,
                                              _dsize);
        }

        dir_hnd = OpenDir(talloc_tos(), handle->conn, smb_fname, NULL, 0);
        if (dir_hnd == NULL) {
                return UINT64_MAX;
        }

        while ((dname = ReadDirName(dir_hnd, &offset, NULL, &talloced))
               != NULL)
        {
                ok = fruit_tmsize_do_dirent(handle, &state, dname);
                if (!ok) {
                        TALLOC_FREE(talloced);
                        TALLOC_FREE(dir_hnd);
                        return UINT64_MAX;
                }
                TALLOC_FREE(talloced);
        }

        TALLOC_FREE(dir_hnd);

        dsize = config->time_machine_max_size / 512;
        dfree = dsize - (state.total_size / 512);
        if (dfree > dsize) {
                dfree = 0;
        }

        *_bsize = 512;
        *_dsize = dsize;
        *_dfree = dfree;
        return dfree / 2;
}


Code:
static bool fruit_tmsize_do_dirent(vfs_handle_struct *handle,
                                   struct fruit_disk_free_state *state,
                                   const char *name)
{
        bool ok;
        char *p = NULL;
        size_t sparsebundle_strlen = strlen("sparsebundle");
        size_t bandsize = 0;
        size_t nbands;
        off_t tm_size;

        p = strstr(name, "sparsebundle");
        if (p == NULL) {
                return true;
        }

        if (p[sparsebundle_strlen] != '\0') {
                return true;
        }

        DBG_DEBUG("Processing sparsebundle [%s]\n", name);

        ok = fruit_get_bandsize(handle, name, &bandsize);
        if (!ok) {
                /*
                 * Beware of race conditions: this may be an uninitialized
                 * Info.plist that a client is just creating. We don't want let
                 * this to trigger complete failure.
                 */
                DBG_ERR("Processing sparsebundle [%s] failed\n", name);
                return true;
        }

        ok = fruit_get_num_bands(handle, name, &nbands);
        if (!ok) {
                /*
                 * Beware of race conditions: this may be a backup sparsebundle
                 * in an early stage lacking a bands subdirectory. We don't want
                 * let this to trigger complete failure.
                 */
                DBG_ERR("Processing sparsebundle [%s] failed\n", name);
                return true;
        }

        /*
         * Arithmetic on 32-bit systems may cause overflow, depending on
         * size_t precision. First we check its unlikely, then we
         * force the precision into target off_t, then we check that
         * the total did not overflow either.
         */
        if (bandsize > SIZE_MAX/nbands) {
                DBG_ERR("tmsize potential overflow: bandsize [%zu] nbands [%zu]\n",
                        bandsize, nbands);
                return false;
        }
        tm_size = (off_t)bandsize * (off_t)nbands;

        if (state->total_size + tm_size < state->total_size) {
                DBG_ERR("tm total size overflow: bandsize [%zu] nbands [%zu]\n",
                        bandsize, nbands);
                return false;
        }

        state->total_size += tm_size;


        DBG_DEBUG("[%s] tm_size [%jd] total_size [%jd]\n",
                  name, (intmax_t)tm_size, (intmax_t)state->total_size);

        return true;
}


So we can get more info about what is going on by increasing logging of vfs_fruit to DEBUG (10). vfs_fruit has its own logging class so you should be able to set an auxiliary parameter for Services->SMB log level = 1 fruit:10@/var/log/samba4/log.smbd.fruit. This will increase logging verbosity for VFS module and write it to a separate file. See if you have any errors being reported by the module (specifically from the above function).
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
@anodos Sorry for not replying sooner. Work got in the way. Damnit :wink:

I'll send you a log via private message. Thanks!
 

TheNetStriker

Dabbler
Joined
Jan 27, 2022
Messages
11
I had a similar problem on TrueNAS-SCALE-22.02-RC.2. After I added the Time Machine share on my 5 TB dataset my Mac only showed 1.1 TB disk space. After a while I found out that the "Multi-user time machine" preset set's the property zfs_core:base_user_quota to 1T. So I simply added the line zfs_core:base_user_quota=5T in Auxiliary Parameters and after that the whole 5 TB where available to my mac.

But I'm still not sure if this is the correct way to setup Time Machine in Truenas Scale. These are the steps how I did setup my Time Machine:

1) I added a "TimeMachine" Dataset with all default settings (Share type generic, no quotas on the Dataset) to my pool.
2) I added a samba share on the "TimeMachine" Dataset with the "Multi-user time machine" preset and zfs_core:base_user_quota=5T parameter.
3) I configured the Time Machine on both of my mac's using the same user using the volume that was automatically detected in the Time Machine settings. (e.g. truenashost.local)

Every other way that I tried to setup the Time Machine did not work or had problems later on.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
The fruit:time machine max size parameter is working for me again with 12.0-U7. Let's hope that stays with U8.
 

idyrizen

Cadet
Joined
Mar 21, 2022
Messages
4
Hallo! I am running 22.02.0 and have the exact problem described here.

Neither fruit:time machine max size (which was set to 0 by default) nor zfs_core:base_user_quota set to 5T managed to increase the avaliable storage on my time machine share. I followed the exact setup of @TheNetStriker mentioned above. It is still capped at 1TB. Any advice?

(Other info: Backups to the TM storage pool work as expected, I have restarte both SMBD service and truenas, disconnected and relogged several times from the samba share using my mac)

Testparm:


Code:
[TimeMachine]
        comment = Time Machine Backups
        ea support = No
        kernel share modes = No
        path = /mnt/XX/TimeMachine/%U
        posix locking = No
        read only = No
        smbd max xattr size = 2097152
        vfs objects = tmprotect fruit streams_xattr shadow_copy_zfs nfs4acl_xattr zfs_core io_uring
        zfs_core:base_user_quota = 5T
        zfs_core:zfs_auto_create = true
        tn:vuid = XX
        fruit:time machine max size = 0
        fruit:time machine = True
        fruit:resource = stream
        fruit:metadata = stream
        nfs4:chown = True
        nfs4acl_xattr:encoding = xdr
        nfs4acl_xattr:xattr_name = system.nfs4_acl_xdr
        nfs4acl_xattr:validate_mode = False
        nfs4acl_xattr:nfs4_id_numeric = True
        tn:home = False
        tn:path_suffix = %U
        tn:purpose = ENHANCED_TIMEMACHINE
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Hallo! I am running 22.02.0 and have the exact problem described here.

Neither fruit:time machine max size (which was set to 0 by default) nor zfs_core:base_user_quota set to 5T managed to increase the avaliable storage on my time machine share. I followed the exact setup of @TheNetStriker mentioned above. It is still capped at 1TB. Any advice?

(Other info: Backups to the TM storage pool work as expected, I have restarte both SMBD service and truenas, disconnected and relogged several times from the samba share using my mac)

Testparm:


Code:
[TimeMachine]
        comment = Time Machine Backups
        ea support = No
        kernel share modes = No
        path = /mnt/XX/TimeMachine/%U
        posix locking = No
        read only = No
        smbd max xattr size = 2097152
        vfs objects = tmprotect fruit streams_xattr shadow_copy_zfs nfs4acl_xattr zfs_core io_uring
        zfs_core:base_user_quota = 5T
        zfs_core:zfs_auto_create = true
        tn:vuid = XX
        fruit:time machine max size = 0
        fruit:time machine = True
        fruit:resource = stream
        fruit:metadata = stream
        nfs4:chown = True
        nfs4acl_xattr:encoding = xdr
        nfs4acl_xattr:xattr_name = system.nfs4_acl_xdr
        nfs4acl_xattr:validate_mode = False
        nfs4acl_xattr:nfs4_id_numeric = True
        tn:home = False
        tn:path_suffix = %U
        tn:purpose = ENHANCED_TIMEMACHINE
If a quota has already been written, you can edit the user quota through the webui for the ZFS dataset in question.
 
Top