File copy strips DOS hidden attribute?

[jhk62]

I am a TrueNAS newbie and run into an issue when comparing copied directories in a smb share. After setting up an unkown piece of software, I don't trust it and try to verify results to gain confidence. That's a result of working in the IT since a long time

I was alarmed when the copied directory tree showed more files than the original. This happens because hidden files in the original location are exposed in the copied location. The hidden attribute is stripped during the copy.

I researched for hours in the internet and tried several things but I don't get this fixed. And I want a 1:1 copy. Hidden files must remain hidden when copied. Cross checked this with my Win10 local filesystem and with a Synology NAS share. In both cases the hidden attribute is preserved.

After running into this issue, I shut down the prod environment and set up a test environment and verified that it behaves exactly the same. I know that the specs of this lab installation don't meet the requirements but as it behaves the same and performance/stability is of no concern, it should be ok for troubleshooting.

My user jh is a member of the samba group that I added to the dataset ACL. The same user is used on the Win10 host to mount the share.

Even a direct copy as root in the TrueNAS shell strips this attribute when

root@truenas-test[/mnt/t/w]# cp 1.jpg 2.jpg

But, interestingly, when doing this, the attribute is preserved

root@truenas-test[/mnt/t/w]# cp -p 1.jpg 3.jpg

DOS shell shows

Code:

X:\>attrib
A   H                X:\1.jpg
A                    X:\2.jpg
A   H                X:\3.jpg

Do you have a clue where to look?

Show : prod system

TrueNAS-12.0-U2.1
Supermicro X11SCH-F
i5-9400F
32GB RAM
8 x 3TB WD Red as RaidZ2

Show : lab system

TrueNAS-12.0-U2
Asrock Beebox
Virtualbox, 4GB memory for VM
I know, lousy but for testing...

Code:

root@truenas-test[/etc/local]# testparm -s
Load smb config files from /usr/local/etc/smb4.conf
Loaded services file OK.
Server role: ROLE_STANDALONE

# Global parameters
[global]
        aio max threads = 2
        bind interfaces only = Yes
        disable spoolss = Yes
        dns proxy = No
        enable web service discovery = Yes
        kernel change notify = No
        load printers = No
        logging = file
        max log size = 5120
        netbios name = TRUENAS
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        registry shares = Yes
        restrict anonymous = 2
        server role = standalone server
        server string = TrueNAS Server
        unix extensions = No
        username map = /usr/local/etc/smbusername.map
        username map cache time = 60
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        directory name cache size = 0
        dos filemode = Yes


[w]
        ea support = No
        kernel share modes = No
        path = /mnt/t/w
        posix locking = No
        read only = No
        vfs objects = streams_xattr shadow_copy_zfs ixnas aio_fbsd
        nfs4:chown = true
root@truenas-test[/etc/local]#

Code:

root@truenas-test[/etc/local]# getfacl /mnt/t/w
# file: /mnt/t/w
# owner: root
# group: wheel
            owner@:rwxpDdaARWcCos:fd-----:allow
            group@:rwxpDdaARWcCos:fd-----:allow
       group:samba:rwxpDdaARWc--s:fd-----:allow
         everyone@:--------------:fd-----:allow
root@truenas-test[/etc/local]#

 

[sretalla]

But, interestingly, when doing this, the attribute is preserved

root@truenas-test[/mnt/t/w]# cp -p 1.jpg 3.jpg

Have you looked at the manual? man cp

Code:

     -p    Cause cp to preserve the following attributes of each source file in the copy: modification time, access time, file flags, file mode, user ID, and group ID, as allowed by permissions.  Access Control
           Lists (ACLs) and Extended Attributes (EAs), including resource forks, will also be preserved.

           If the user ID and group ID cannot be preserved, no error message is displayed and the exit value is not altered.

           If the source file has its set-user-ID bit on and the user ID cannot be preserved, the set-user-ID bit is not preserved in the copy's permissions.  If the source file has its set-group-ID bit on and
           the group ID cannot be preserved, the set-group-ID bit is not preserved in the copy's permissions.  If the source file has both its set-user-ID and set-group-ID bits on, and either the user ID or group
           ID cannot be preserved, neither the set-user-ID nor set-group-ID bits are preserved in the copy's permissions.

You can also get a similar result using rsync with the -a switch (archive option, which preserves all attributes).

 

[anodos]

Hidden is stored in a file flag. You can view with "ls -lo".

Code:

root@homenas[/mnt/tank]# rm file2 file3 file4
root@homenas[/mnt/tank]# ls -lo
total 1
-rw-r--r--  1 root  wheel     uarch,hidden 0 Mar  1 02:15 file
drwxrws---  4 root  SMBGROUP  uarch        4 Jan 11 13:42 testds
root@homenas[/mnt/tank]# cp -p file file2
root@homenas[/mnt/tank]# ls -lo
total 2
-rw-r--r--  1 root  wheel     uarch,hidden 0 Mar  1 02:15 file
-rw-r--r--  1 root  wheel     uarch,hidden 0 Mar  1 02:15 file2
drwxrws---  4 root  SMBGROUP  uarch        4 Jan 11 13:42 testds
root@homenas[/mnt/tank]# 

 

[anodos]

When viewed via SMB protocol:

Code:

root@homenas[/mnt/dozer/SMB]# touch testfile
root@homenas[/mnt/dozer/SMB]# chflags uhidden testfile
root@homenas[/mnt/dozer/SMB]# 
root@homenas[/mnt/dozer/SMB]# cp -p testfile testfile2
root@homenas[/mnt/dozer/SMB]# 

Code:

root@homenas[/mnt/dozer/SMB]# smbclient //127.0.0.1/SMB -U smbuser
Enter WORKGROUP\smbuser's password: 
Try "help" to get a list of possible commands.
smb: \> allinfo testfile
altname: testfile
create_time:    Mon Mar  1 02:21:22 2021 PST
access_time:    Mon Mar  1 02:21:22 2021 PST
write_time:     Mon Mar  1 02:21:22 2021 PST
change_time:    Mon Mar  1 02:21:22 2021 PST
attributes: HA (22)
stream: [::$DATA], 0 bytes
smb: \> allinfo testfile2
altname: TJ9RF0~N
create_time:    Mon Mar  1 02:21:22 2021 PST
access_time:    Mon Mar  1 02:21:22 2021 PST
write_time:     Mon Mar  1 02:21:22 2021 PST
change_time:    Mon Mar  1 02:21:22 2021 PST
attributes: HA (22)
stream: [::$DATA], 0 bytes

Both have hidden (H).

 

[jhk62]

Yes, thus I tried "cp -p" to check if the Windows hidden attribute is there somewhere and can be copied at all. I will try rsync, maybe this is a temporary workaround. But I'm looking for a general solution for this issue.

Here are the steps that are reproducable in my environment.

Example within a smb share that is mounted from my TrueNAS server:

Win10 client, using the Windows Explorer
set a file's hidden attribute
move that file to a folder
check the file -> hidden attribute is there, as expected
copy that folder, duplicate or another location within the share
check the file -> hidden attribute is removed, this is not expected
copy the folder from the share to the Win10 native filesystem
check the file -> hidden attribute is there, as expected

Somehow the hidden attribute is removed when the file is copied within the smb share that is mounted from the TrueNAS server. When I do this within the native Windows filesystem, the copied file has the hidden attribute set. This is also the case if I do this within a smb share that is mounted from my Synology NAS.

Synology and TrueNAS both use smb4.

Code:

root@DS118:~# smbd --version
Version 4.4.16
Synology Build 25426, Dec 10 2020 18:42:48

Code:

root@freenas11[/mnt/t/w/2]# smbd --version
Version 4.10.18

As it works with the Synology share and both use smb4, I think that it's a question of how to configure the samba part.

It's not so unlikely that the reason for this issue sits in a distance of 80 cm from the monitor as these are my first steps using TrueNAS So, how can TrueNAS be setup in a way that such Windows attributes like hidden are preserved when files are copied e.g. using the Windows Explorer?

 

[GuaySam1]

I am a TrueNAS newbie and run into an issue when comparing copied directories in a smb share. After setting up an unkown piece of software, I don't trust it and try to verify results to gain confidence. That's a result of working in the IT since a long time

I was alarmed when the copied directory tree showed more files than the original. This happens because hidden files in the original location are exposed in the copied location. The hidden attribute is stripped during the copy.

I researched for hours in the internet and tried several things but I don't get this fixed. And I want a 1:1 copy. Hidden files must remain hidden when copied. Cross checked this with my Win10 local filesystem and with a Synology NAS share. In both cases the hidden attribute is preserved.

After running into this issue, I shut down the prod environment and set up a test environment and verified that it behaves exactly the same. I know that the specs of this lab installation don't meet the requirements but as it behaves the same and performance/stability is of no concern, it should be ok for troubleshooting.

My user jh is a member of the samba group that I added to the dataset ACL. The same user is used on the Win10 host to mount the share.

Even a direct copy as root in the TrueNAS shell strips this attribute when

root@truenas-test[/mnt/t/w]# cp 1.jpg 2.jpg

But, interestingly, when doing this, the attribute is preserved

root@truenas-test[/mnt/t/w]# cp -p 1.jpg 3.jpg

DOS shell shows

Code:

X:\>attrib
A   H                X:\1.jpg
A                    X:\2.jpg
A   H                X:\3.jpg

Do you have a clue where to look?

Show : prod system

TrueNAS-12.0-U2.1
Supermicro X11SCH-F
i5-9400F
32GB RAM
8 x 3TB WD Red as RaidZ2

Show : lab system

TrueNAS-12.0-U2
Asrock Beebox
Virtualbox, 4GB memory for VM
I know, lousy but for testing...

Code:

root@truenas-test[/etc/local]# testparm -s
Load smb config files from /usr/local/etc/smb4.conf
Loaded services file OK.
Server role: ROLE_STANDALONE

# Global parameters
[global]
        aio max threads = 2
        bind interfaces only = Yes
        disable spoolss = Yes
        dns proxy = No
        enable web service discovery = Yes
        kernel change notify = No
        load printers = No
        logging = file
        max log size = 5120
        netbios name = TRUENAS
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        registry shares = Yes
        restrict anonymous = 2
        server role = standalone server
        server string = TrueNAS Server
        unix extensions = No
        username map = /usr/local/etc/smbusername.map
        username map cache time = 60
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        directory name cache size = 0
        dos filemode = Yes
        Walgreenslistens


[w]
        ea support = No
        kernel share modes = No
        path = /mnt/t/w
        posix locking = No
        read only = No
        vfs objects = streams_xattr shadow_copy_zfs ixnas aio_fbsd
        nfs4:chown = true
root@truenas-test[/etc/local]#

Code:

root@truenas-test[/etc/local]# getfacl /mnt/t/w
# file: /mnt/t/w
# owner: root
# group: wheel
            owner@:rwxpDdaARWcCos:fd-----:allow
            group@:rwxpDdaARWcCos:fd-----:allow
       group:samba:rwxpDdaARWc--s:fd-----:allow
         everyone@:--------------:fd-----:allow
root@truenas-test[/etc/local]#

Thanks for providing the info, I will surely look into it!