How to set up interfaces for SMB3.0 Multichannel

[Windows7ge]

I'm aware SMB Multichannel on FreeNAS is still considered experimental but I want to have a go at running it or attempting to run it at least.

Google for the past 2 days has been useless in the fact that every forum and article talking about SMB Multichannel on FreeNAS or quite frankly SMB Multichannel in general has talked about how it works and how it performs then they pretty much skip over how to configure it. I know SMB auto-configures itself at least according to my research but I expect that you still have to setup the network interfaces on both machines which includes how to set up the 4+ IP addresses between all the NIC's and no website is saying how to do that.

On Windows side it didn't care that I gave one interface an IP of 192.168.0.251/24 and its other 192.168.0.250/24 so long as I exempt the default gateway on the second interface.
Now FreeNAS didn't want to play nice with 192.168.0.253/24 and another with 192.168.0.249/24.

I'm assuming it won't let me do this because since all links are plugged into a switch it would cause an infinite loop of network traffic, broadcasts probably.

So, how do I configure the interfaces? I could place them on separate subnets, I assume since the switch I'm using is only layer 2 it shouldn't care that it's forwarding packets from multiple networks.
One article mentioned only setting up one interface IP on both machines and somehow the other will auto-negotiate and the two will be bonded under one IP. I'm not sure.

All insight is appreciated.

 

[dlavigne]

Were you able to find a howto?

 

[Windows7ge]

Were you able to find a howto?

Any how to's that I found did not discuss how to distribute the interface IP addresses. They just kind of went, plug it in, this is how it worked for us, these are the issues we had or you might have. No web articles, forum posts, youtube videos, or other material explained exactly how to set it up other than one poster on a forum stating he gave each interface an address on the same network but he was having issues and another briefly mentioning link aggregation operating at layer 2 with an IP address assigned to only one interface then SMB handling the distribution across the other interfaces but it was not definitive and I did not get that method working either.

 

[Ericloewe]

What do the Samba folks say you need for this? Maybe we can help from there.

 

[Windows7ge]

What do the Samba folks say you need for this? Maybe we can help from there.

Samba 4.6.4 or later, the option of SMB3.0_11 (or which ever other corresponds to your OS), a multiport NIC or NICs that support RSS (Receive Side Scaling) or RDMA (Remote Device Memory Access) <- This is apparently a feature of very enterprise grade NIC's and is only really necessary if you need extremely low latency so I'm sticking to RSS. Just about any network switch and that's about it.

For testing purposes I'm using the built-in NICs on the server: ASRock EP2C602-4L/D16
For the client I'm using the built in NICs on a: ASRock C2750D4I
For right now the switch is: TP-LINK TL-SG1016D

If I can get this to work I plan to aggregate two 10Gbit ports on the server to a 10Gbit switch to my host for a maximum of 20Gbit from server to client.

 

[Ericloewe]

What about the network configuration? I'm guessing the additional NICs need an IP address in the same subnet, but I'm wondering if Samba needs to be informed about the available NICs and their configuration, somehow.

 

[Windows7ge]

What about the network configuration? I'm guessing the additional NICs need an IP address in the same subnet, but I'm wondering if Samba needs to be informed about the available NICs and their configuration, somehow.

That's my biggest question right now which google doesn't seem to have an answer for. What configuration is necessary on the ports? Do they need to all be on the same network? Do they need to be assigned in pairs to different networks (10.0.0.0/30, 10.0.0.4/30)? Does there only need to be one pair assigned IPs then SMB use layer 2 protocols to virtually aggregate the two? I don't know, this is a completely new technology to me even though it has been around for a number of years years now according to my research.

Apparently this was a protocol developed by Microsoft for Windows but similar driver services were developed for Linux & MacOS so they could all use file sharing over the network under the same protocol. So Multichannel supposedly can work with SAMBA on Linux as of 4.6.4 (I think my version is 4.7.3) but it is still considered experimental.

If my FreeNAS version matters it is 11.0-U3

 

[Windows7ge]

but I'm wondering if Samba needs to be informed about the available NICs and their configuration, somehow.

One forum I located mentioned having issues until they added both routes to the routing table but this was on a Windows Server OS. Also I forgot to mention, it would appear FreeNAS will not allow you to assign two interfaces addresses on the same network. I think this is due to broadcasts. It could cause an infinite loop of them.

 

[Ericloewe]

Maybe Microsoft has some useful documents for this, now that I think about it...

 

[Windows7ge]

Maybe Microsoft has some useful documents for this, now that I think about it...

Considered that. Going to their website they discuss the technology, how it works, and the hardware needed but again don't say how to actually configure it. At least of the documentation I was able to locate on the site. They may have other pages that I could not find though.

 

[Johnnie Black]

but again don't say how to actually configure it.

In Windows there's nothing to configure, you just connect multiple NICs to the same switch/router, each will get a different IP and you're good to go.

 

[Johnnie Black]

Forgot to mention, I never tried on FreeNAS but did spend some time trying to get it working on Linux without much success, eventually upgraded to 10GbE.

Multichannel on Samba is disabled by default, you need to add to smb.conf:

Code:

server multi channel support = yes

Also asynchronous I/O needs to be enable, never checked if it is by default on FreeNAS.

 

[Windows7ge]

Forgot to mention, I never tried on FreeNAS but did spend some time trying to get it working on Linux without much success, eventually upgraded to 10GbE.

Multichannel on Samba is disabled by default, you need to add to smb.conf:

Code:

server multi channel support = yes

Also asynchronous I/O needs to be enable, never checked if it is by default on FreeNAS.

I finally found one article that although their configuration is far different from mine did mention all interfaces being on the same network. This is potentially problematic as FreeNAS doesn't want to let me have two interfaces on the same network. I'll toy around with it though.

smb.conf, sounds like a good place to start. However I haven't the slightest idea what directory it's in.
A quick lookup says the enabled status of asynchronous I/O is located in the smb.config file so if it's not already enabled I can enable it when I add the statement you mentioned.
A little more research says editing the files directly isn't a good idea because the file may be overwritten upon system restart or system update. These edits should be added to the Auxiliary Parameters option in the Advanced Menu. Now my only question is do I add these strings to the SMB service or do I add them to the Sharing -> Windows (SMB). I suppose checking the file directly to see what statements it currently has is my best option before adding anything. One forum post says it's located in /usr/local/etc/smb4.conf

Also for my configuration when I do finally aggregate two 10Gbit interfaces there will be no router. Each interface will receive a static address. There will be no gateway or network bridging.

 

[Johnnie Black]

This is potentially problematic as FreeNAS doesn't want to let me have two interfaces on the same network.

This might be a problem.

Also for my configuration when I do finally aggregate two 10Gbit interfaces there will be no router. Each interface will receive a static address. There will be no gateway or network bridging.

This is OK.

 

[Windows7ge]

This might be a problem.

I wonder if it's possible that because my current testing methodology includes me utilizing the interface used for the WebUI/SSH/SFTP services and is assigned the default gateway that if I use a network that isn't the same as the routers DHCP service (instead of 192.168.0.X use 172.16.0.X or 10.0.0.X) and use two dedicated interfaces (the motherboard has four so I can test this) and don't specify the gateway then perhaps it will let me. Otherwise I may have to configure some form of link aggregation to make it so FreeNAS will use both interfaces because as I see it now Multichannel needs layer 3 addresses it won't work using layer 2 like when aggregating ports on a switch.

 

[Windows7ge]

Forgot to mention, I never tried on FreeNAS but did spend some time trying to get it working on Linux without much success, eventually upgraded to 10GbE.

Multichannel on Samba is disabled by default, you need to add to smb.conf:

Code:

server multi channel support = yes

Also asynchronous I/O needs to be enable, never checked if it is by default on FreeNAS.

I SSHed into the server and opened the /usr/local/etc/smb4.conf file. It has two sections. A section for [global] and a section for one of my CIFS shares that I named [Archive]. There is neither a string for server multichannel support nor asynchronous I/O. You've shown me how to type in the first one. Would you happen to know how to check of asynchronous I/O is enabled or not then how to type it out to enable it?

 

[Johnnie Black]

Would you happen to know how to check of asynchronous I/O is enabled or not then how to type it out to enable it?

If not present you need to add to the global section:

Code:

aio read size = 1
aio write size = 1

Any value other than 0 will enable it, if you get it working you can then test if larger values improve performance.

 

[Windows7ge]

If not present you need to add to the global section:

Code:

aio read size = 1
aio write size = 1

Any value other than 0 will enable it, if you get it working you can then test if larger values improve performance.

Thank you. Also since the current test does include a router I set it to DHCP on the second interface and FreeNAS got an address on its own. Both interfaces now have an address on the same network. This will still prove to be problematic as I'll want to set the addresses statically in the future and it won't let me at least from the WebUI.

 

[Windows7ge]

if you get it working you can then test if larger values improve performance.

OK. At this moment it is partially working. I now see activity on both interfaces on windows side and there was a very brief blitz above 1Gbit but now all it's doing is kind of load balancing. On FreeNAS's side one of the interfaces is doing all of the work with very minute bits of activity on the other interface. It looks like Windows is bonding the interfaces but FreeNAS is using one cable to transmit information and the other to receive. Probably control packets. (One interface says high activity of TX data and the other a low quantity of RX data) Any ideas?

 

[Johnnie Black]

Any ideas?

Not me, like I said never had much success with it outside Windows, and lost interest once I upgraded to 10GbE.