Vcenter Backup via SCP to Truenas fails on chmod command

[zenon1823]

I don't know if this is a VMware issue or a TrueNAS permission issue so i thought id ask for some thoughts. Cause I'm running out.

I'm trying to run a backup from my VCSA to TrueNAS Scale 22.12.0
The dataset's owner and group are both root
I'm running the job as root over SCP

The backup runs for a few mins and transfers about 600MB (i see the files being created in the Vcenter folder) then the backup fails with the following error and then it deletes all the files.

I don't know why its trying to make them read-only but is there a TrueNASpermission reason why it wouldn't be able to chmod those files. I figure if its running as root and root is the dataset owner is should be able to do whatever it wants no?

(PS root is not the intended user in practice, I'm just trying to get it working before moving to a least privileged model)

Code:

2023-02-17T01:49:58.25 [MainProcess:PID-39241] ERROR: BackupManager encountered an exception: Failed to make the content of dir /mnt/Primary_Tank/Backup_Local/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-014622_/ read-only at 192.168.110.150; Err:chmod: changing permissions of '/mnt/Primary_Tank/Backup_Local/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-014622_/backup-metadata.json': Operation not permitted
chmod: changing permissions of '/mnt/Primary_Tank/Backup_Local/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-014622_/config_files.tar.gz': Operation not permitted
chmod: changing permissions of '/mnt/Primary_Tank/Backup_Local/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-014622_/database_full_backup.tar.gz': Operation not permitted
chmod: changing permissions of '/mnt/Primary_Tank/Backup_Local/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-014622_/full_wal_backup_meta.tar.gz': Operation not permitted
...

 

[Whattteva]

I don't think it has anything to do with SCP. SCP is just a copy program and can't execute things like chmod. It looks like you are mounting the dataset on the vcenter machine and then trying to do chmod there? How are you mounting it? TrueNAS NFS exports, by default, disables root mounting unless you set maproot option.

 

[zenon1823]

I am far from an SCP guru but I thought during an SCP session you could issue commands like chmod chown when changing permissings and owners? I'm thinking back to when i used to use winscp as a web host years ago. As far as I know its not mounting it on vcenter maching the configuration is just a regular scp like connection. see below:

I've since tried some other options like backing up to a SMB/CIFS share and it fails at the same point with a similar error about failing to make the files read-only.

Code:

2023-02-17T05:28:07.438 [MainProcess:PID-63137] ERROR: BackupManager encountered an exception: Err to set /storage/remote/backup/cifs/192.168.110.150/7kA69tbt/7WFxP8as/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-052418_ read-only; rc:1, stdout:
stderr:chmod: changing permissions of '/storage/remote/backup/cifs/192.168.110.150/7kA69tbt/7WFxP8as/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-052418_': Permission denied
chmod: changing permissions of '/storage/remote/backup/cifs/192.168.110.150/7kA69tbt/7WFxP8as/vCenter/sn_vcsa.home.snet.com/M_6.7.0.45000_20230217-052418_/backup-metadata.json': Permission denied

So maybe this is a VMware issue and not a TrueNAS permission issue... still looking/reading/working on both sides of it.

 

[Whattteva]

I am far from an SCP guru but I thought during an SCP session you could issue commands like chmod chown when changing permissings and owners?

No, SCP only copies stuff from A to B. It is different from SFTP, which includes an interactive session in addition to the file transfer session.

I'm thinking back to when i used to use winscp as a web host years ago. As far as I know its not mounting it on vcenter maching the configuration is just a regular scp like connection. see below: View attachment 63709

I'm not familiar with vCenter so someone else will have to step in here, but from the logs, it appears to me that it appears to be doing some sort of mounting of your TrueNAS dataset within its local filesystem unless it is actually executing chmod on /mnt/Primary_Tank/Backup_Local through ssh on your TrueNAS machine.

So maybe this is a VMware issue and not a TrueNAS permission issue... still looking/reading/working on both sides of it.

Will need someone more familiar with VMWare software as I don't use it. BTW, which permission model did you select when creating the backup dataset? UNIX permissions or SMB ACL's?

 

[zenon1823]

Will need someone more familiar with VMWare software as I don't use it. BTW, which permission model did you select when creating the backup dataset? UNIX permissions or SMB ACL's?

The dataset is configured as:
ACL Type : SMB/NFSv4
ACL Mode : Restricted

The ShareACL is set as Everyone Allowed Full
Access is set via the Filesystem ACLs
Root is owner with full access
Root is Group with modify
User1 with Full access

Root was used for SCP test and User1 was used for SMB test.

 

[Whattteva]

The dataset is configured as:
ACL Type : SMB/NFSv4
ACL Mode : Restricted

The ShareACL is set as Everyone Allowed Full
Access is set via the Filesystem ACLs
Root is owner with full access
Root is Group with modify
User1 with Full access

Root was used for SCP test and User1 was used for SMB test.

Well then, I think this may be your problem.
SMB ACL's do NOT work with chmod as far as I remember (sorry it's been a while since I used it). That's for UNIX permissions.

I believe, you have to use setfacl