Questions about SMB "Alternate Data Streams" (don't care for them, don't need them)

[winnielinnie]

If no Mac computers will ever communicate with my TrueNAS server, is there any risk or harm in completely disabling "Enable Alternate Data Streams" for all of my SMB shares?

They're incompatible with Linux, and I have no need for them in Windows. (Unless there's something "magical" about them in which others found a practical benefit?)



Questions that come to mind:

• Is there any risk if I download a file from the internet directly to an SMB share while using Windows, and for whatever reason it wants to save an ADS with the file, will it error out if the SMB share does not have "Enable Alternate Data Streams" allowed? Or will it safely ignore it and save the file as if there was no ADS involved?

• By disabling support for ADS (in the SMB Share's options), will all currently existing Alternate Data Streams be immediately destroyed?

• Where are the ADS currently stored on the share/server, if some currently exist at the moment? Do they reside directly within the relevant ZFS dataset that is being shared via SMB? (If that question even makes sense? I know with NTFS they're part of the filesystem, analogous to xattrs in a *nix filesystem, such as Ext4 and XFS).

• What are real-world benefits of Alternate Data Streams in a non-Mac environment? Phrased differently: what "added benefits" are there to leveraging ADS?

• Is their risk for errors or data corruption if using an iPhone or iPad to access the SMB shares without support for Alternate Data Streams? Or does the risk only apply to macOS specifically? [1] (see forum thread linked below)

[1] https://www.truenas.com/community/threads/mov-files-corrupted-after-moving-to-nas-drive.90849/

 

[anodos]

If no Mac computers will ever communicate with my TrueNAS server, is there any risk or harm in completely disabling "Enable Alternate Data Streams" for all of my SMB shares?

SMB share will no longer advertise stream support and clients will alter behavior accordingly. Any streams that have been written will no longer be accessible by SMB clients

They're incompatible with Linux, and I have no need for them in Windows. (Unless there's something "magical" about them in which others found a practical benefit?)

How are they incompatible with Linux? I'm skeptical of this assertion since they're also a Windows SMB feature. I also have tests in our CI that run on Unix clients specifically to validate streams behavior. C.F.:

Code:

@pytest.mark.dependency(name="STREAM_WRITTEN_SMB2")
def test_061_create_and_write_stream_smb2(request):
    """
    Create our initial stream and write to it over SMB2/3 protocol.
    Start with offset 0.
    """
    depends(request, ["STREAM_TESTFILE_CREATED"])
    c = SMB()
    c.connect(host=ip, share=SMB_NAME, username=SMB_USER, password=SMB_PWD, smb1=False)
    fd = c.create_file("streamstestfile:smb2_stream", "w")
    c.write(fd, b'test1', 0)
    c.close(fd)

    fd2 = c.create_file("streamstestfile:smb2_stream", "w")
    contents = c.read(fd2, 0, 5)
    c.close(fd2)
    c.disconnect()
    assert(contents.decode() == "test1")

IIRC there was some discussion of writing Linux xattrs in ADS through WSL on Windows. Might be something worth verifying.

Questions that come to mind:

• Is there any risk if I download a file from the internet directly to an SMB share while using Windows, and for whatever reason it wants to save an ADS with the file, will it error out if the SMB share does not have "Enable Alternate Data Streams" allowed? Or will it safely ignore it and save the file as if there was no ADS involved?

Probably just ignore it. See comment about about clients and how feature support works in SMB.

• By disabling support for ADS (in the SMB Share's options), will all currently existing Alternate Data Streams be immediately destroyed?

No, but they won't be preserved for instance in a server-side copy through SMB.

• Where are the ADS currently stored on the share/server, if some currently exist at the moment? Do they reside directly within the relevant ZFS dataset that is being shared via SMB? (If that question even makes sense? I know with NTFS they're part of the filesystem, analogous to xattrs in a *nix filesystem, such as Ext4 and XFS).

They're stored in xattrs. They're not xattrs in NTFS. They're alternate data streams. NTFS has xattrs that are a different thing (limited to 64 KiB in aggregate size IIRC).

• What are real-world benefits of Alternate Data Streams in a non-Mac environment? Phrased differently: what "added benefits" are there to leveraging ADS?

Behavior will be application-specific. You'd have to see if anything in your environment requires them.

• Is their risk for errors or data corruption if using an iPhone or iPad to access the SMB shares without support for Alternate Data Streams? Or does the risk only apply to macOS specifically?

Disabling doesn't lead to corruption. Haven't looked at forums thread. If SMB share lacks ADS support, then MacOS / iOS clients have to write the metadata in some different way. This usually takes place through special ._ files.

I may need to review vfs_fruit settings (if time machine / AAPL extensions are enabled) in case ADS support is disabled since we normally pass fruit metadata / resource writes through to the stream module (won't do this over the weekend).

 

[winnielinnie]

How are they incompatible with Linux? I'm skeptical of this assertion since they're also a Windows SMB feature.

Poorly worded. I'm implying that Linux clients will not generate them (as far as I'm aware), and for Windows clients/software that do, I have yet to see a personal need, myself. (What I did read about them leads to discussions about "security concerns" and "nearly useless features" that most people never take advantage of.) Hence, either way (Linux or Windows) they seem pointless to support on my SMB shares. And I have no macOS computers in which "forks" would come into play. (My only Apple devices are iPhone / iPad, i.e, iOS.)

If there's no definitive reason for someone to purposefully make use of ADS, then it's fine and safe to disable this feature (even for the sake of intentionally blocking ADS from downloaded files off the internet)?

I noticed it's enabled by default for newly created SMB shares.

 

[anodos]

They're enabled by default for maximum compatibility in mixed-OS environments. There should be in principal be no issue with disabling them at home. On Core you can view streams through shell by running command "lsextattr user *" in an SMB directory. Streams will contain a prefix "DosStream." This should give you a rough idea of impact of disabling.

 

[winnielinnie]

There should be in principal be no issue with disabling them at home. On Core you can view streams through shell by running command "lsextattr user *"

The only xattr I could find are "xdg.origin.url", which shows the URL from where I downloaded the file. (I don't care for this, nor do need it.)

But the above was obviously generated under Linux before setting up the SMB shares (hence the "xgd", when I was using the Ext4 filesystem directly, since SMB2/3 doesn't support xattr).

So for all intents and purposes, I've never used "Alternate Data Streams" while under Windows accessing the SMB shares.

So enabling "Alternate Data Streams" doesn't allow xattr under Linux anyways, right? (Assuming SMB2/3). And so far I have no evidence of having used (let alone require) ADS from Windows clients using the SMB shares.

Basically, there's no point in supporting it in my case, unless iOS devices make use of them (for which I have yet to notice.)

Naturally, out of curiosity, this raises another question. If xattr can essentially be saved to a *nix filesystem through an SMB share (with ADS enabled), isn't it technically possible for the SMB developers to also support outright xattr as an additional feature that can be toggled?

What you're saying is that with ADS support enabled for a share, such a stream (ADS) gets saved into an xattr, correct? (For example, lsexattr will show xattr with "DosStream" prefixes.) What is blocking SMB2/3 from allowing the same thing, yet from a Linux client in which the download wants to create an xattr, such as "xdg.origin.url"?

If I had downloaded those files with Linux through an SMB share, the associated xattr of "xdg.origin.url" would never exist, even with ADS enabled. However, if I download those same files with Windows through the same SMB share, they absolutely can have an xattr of "DosStream" associated with them? This means an xattr is being created through an SMB share, so I'm not sure what the reasoning is that they removed xattr from SMB2/3 (available with SMB1), while at the same time it's still supported when an ADS is detected.

• Linux/Unix client? SMB2/3 will not allow xattr through a share.
• Windows/macOS client? SMB2/3 will allow xattr through a share (but as an ADS/fork that gets converted into an xattr.)

Am I understanding this correctly?

 

[anodos]

The only xattr I could find are "xdg.origin.url", which shows the URL from where I downloaded the file. (I don't care for this, nor do need it.)

But the above was obviously generated under Linux before setting up the SMB shares (hence the "xgd", when I was using the Ext4 filesystem directly, since SMB2/3 doesn't support xattr).

So for all intents and purposes, I've never used "Alternate Data Streams" while under Windows accessing the SMB shares.

So enabling "Alternate Data Streams" doesn't allow xattr under Linux anyways, right? (Assuming SMB2/3). And so far I have no evidence of having used (let alone require) ADS from Windows clients using the SMB shares.

I don't understand your question.

Naturally, out of curiosity, this raises another question. If xattr can essentially be saved to a *nix filesystem through an SMB share (with ADS enabled), isn't it technically possible for the SMB developers to also support outright xattr as an additional feature that can be toggled?

What's the difference between "outright xattr" and an alternate data stream?

What you're saying is that with ADS support enabled for a share, such a stream (ADS) gets saved into an xattr, correct? (For example, lsexattr will show xattr with "DosStream" prefixes.) What is blocking SMB2/3 from allowing the same thing, yet from a Linux client in which the download wants to create an xattr, such as "xdg.origin.url"?

Yes, the ADS gets stored as either an SA-based xattr (depending on ZFS configuration) or in the internal attr dir in ZFS. Exact behavior of ADS writes depends on how the SMB client is implemented.

If I had downloaded those files with Linux through an SMB share, the associated xattr of "xdg.origin.url" would never exist, even with ADS enabled.

We disable OS/2 EA support by default because it has a _significant_ negative impact on directory listing performance. You can choose to enable if you wish by turning on support for those EAs. Every xattr in the user namespace that doesn't have the special dosstream prefix is presented as an OS/2 EA.

However, if I download those same files with Windows through the same SMB share, they absolutely can have an xattr of "DosStream" associated with them? Either way, an xattr is being created through an SMB share, so I'm not sure what the reasoning is that they removed xattr from SMB2/3 (available with SMB1), while at the same time it's still supported when an ADS is detected.

Am I understanding this correctly?

No one removed anything related to xattrs.

 

[winnielinnie]

No one removed anything related to xattrs.

This is a pie-in-face moment for me. Whoops.

I had read a RedHat document about SMB1/2/3, and how Unix Extensions support ("nounix" vs "unix") were removed with SMB2/3. My mind remembered it as "xattr" support.

So that makes everything moot. Apologies.

However, I do have a better grasp for ADS, and can comfortably say I don't need them, nor have I ever made use of them previously. So thank you!

 

[winnielinnie]

We disable OS/2 EA support by default because it has a _significant_ negative impact on directory listing performance. You can choose to enable if you wish by turning on support for those EAs. Every xattr in the user namespace that doesn't have the special dosstream prefix is presented as an OS/2 EA.

On this note, you mean that TrueNAS disables it by default per share? I notice there's no default global option set, yet there is a default "no" set for each and every share that is created (and can only be overridden/undone with a per-share auxiliary parameter.)

From what I understand, it's enabled as a global default (upstream Samba), and TrueNAS does not touch this default; however, TrueNAS will explicitly disable it with each and every SMB share that the user creates.

While we're on this tangential topic: If it's disabled by default due to performance reasons, why not disable it in the global parameters? If someone wishes to enable it on a specific share, they can then explicitly enable it themselves (auxiliary parameter), right? Seems like this would be a cleaner method, since it would yield fewer explicit parameters by default for every single SMB share, while only needing a single entry for the global configuration.