Updated 2020-11-16 - improve description of Windows Explorer columns
NOTE: For brevity only, I've written "TrueNAS" throughout, as this is the unified platform title for v.12 (2020) onward. But everything works perfectly and exactly the same on "FreeNAS" older versions, as well. At worst, watch out for python compatibility, is all, I guess, but nobody's ever reported that as an issue yet.....
--------------------------------------------------------
We do live in interesting times. Well, those of us who love the "network neighbourhood" section of Windows File Explorer do, anyhow.
Just think, the magic of all your file shares and networked computers, auto-magically shown in 'My Computer' / 'Windows Explorer', to click and use at will.
What's that? Having problems with it? Oh dear...
As a bonus, we'll also try to clarify some of the background of what's really going on behind the scenes.
Let's start.....
This is the problem. Or problems:
In the early bronze age of computing, technologists decided to put wires between boxes, creating the first network. Some wise guy had the idea, "lets make them able to look at each other for printers and file shares." And some other wise guy figured "They could constantly shout around the LAN at anything that listens "I'M HERE AND I HAVE A FILE SHARE!"" That's a really good way to do it." Someone else figured that constant arguments and dialogs over elections would be a good addition - all in plain text, because computers didn't need security back then. Maybe call the leader something like a "master browser", and have it constantly challenged by other devices? Some unknown manager who'd been promoted too far, decided that was great thinking - noisy subnets are what we need, and whoever wants to file share across two subnets anyway? And unreliability spawns more tech jobs! And so was born..... NetBIOS!
Unfortunately that golden age came to an end, and with it, a new generation of home/small office networking arose. These people despised broadcasting, and decided that by far the best answer was to create a new system which separated discovery of shares and printers, from actual opening and exploring of those shares, and use a server for the discovery part, which was the really noisy and unpleasant bit. They called this idea, "WINS", because anyone who bought into it, was a WIN for their corporation, MumbleSoft. They kinda overlooked that users of file shares included homes and small offices that didn't have technical wizardsa able to make it all work, or able to afford Windows Server. ("I just want it to work" is a pathetic rallying cry, anyhow - real techies like to dig into the registry!)
Meanwhile SMB grew. It got versions SMB2, then SMB3, it grew encryption and signing. And all along, the guys at MumbleSoft figured this broadcasting shouting mess was perfect, and the few who didn't like it would pay for, or work out how to use, WINS. Or cope "somehow". But while SMB was regularly updated to make the actual sharing part more efficient and safer, they never once really bothered to update the 1800's design "discovery" part of things, that Windows Explorer used to find shares in the first place.
Meaning that even though SMB1 isn't actually needed for discovery or sharing (NetBIOS discovery is a completely different protocol and SMB2+3 have separate services), the NetBIOS discovery services are * still * internally dependent on that service as at 2019; they won't work unless the SMB1 service (and some others) are also enabled and fully running.
Then along came Wannacry. Suddenly SMB1's days were more numbered than nitroglycerine at a welding factory. (No, I don't know where that analogy came from either).
By 2016, SMB1 was being actively discouraged, and by 2018/9 Windows 10 was rolling out forced changes that would autonomously disable SMB1 if it wasn't being used. Which left some with a really nasty situation:
Which is a mess. Which we're going to fix.
First, a technical look at what's gone on
For Network Neighbourhood to work, and computers and printers to show up, Windows Explorer has to discover these other network services before it can expose them to the user. There are plenty of discovery protocols - DNS is one, AirPlay/mDNS/Bonjour/Rendezvous/dns-sd are widely used in the Apple world. NetBIOS can function as one of those.
Sorting out NetBIOS is horribly difficult and limited, because it's inextrixably tangled up with the SMB1 service internally (That's in addition to its own main services: "COMPUTER BROWSER SERVICE" and "WORKSTATION SERVICE"), as well as the Winsock networking stack. The code is buggy and unreliable, the approaches are utterly insecure, exploitable, and - by current standards - close to medieval. Overall the discovery aspect of File Shares is the main/only remaining major use for NetBIOS these days. When your network adapter or "ipconfig" says "NetBIOS over TCP/IP", that's what it's referring to. The remaining NetBIOS code could have been separated out, brought into the 1900s, updated or replaced, and the dodgy SMB1 code that doesn't actually get used in NetBIOS could have been stripped out and discarded at any of the many times when file shares themselves were gaining reliability and security updates as part of SMB2/SMB3, or anywhere in between, but never was.
Interestingly, the actual sharing of files is fine - that uses SMB2 or SMB3. But the end result is that the initial discovery and listing by Windows Explorer of any other file-sharing devices on the network, won't work, if SMB1 is disabled. Microsoft just left that section of code dependant on the SMB1 service running - and even back then, in hindsight, it was a horrible decision. Their own blogs beg people to stop using SMB1 these days.
NetBIOS and SMB1 are both genuine problems - the first is unreliable, laggy, and finnicky to the point of fragility for far too many people. It's responsible for computers that never show up or take ages to do so. The second is unsecured, unauthenticated, and trivially exploitable (once the firewall is breached). But how to move away from them?
WINS was one solution, for the broadcasting issues anyhow. WINS still used NetBIOS but it moved from shouting around the LAN to a single server that registers devices for anyone looking them up. Any device registers to it, any client queries it for resources.
But even WINS piggybacks on pretty much the same technology. It's an enterprise solution, so Microsoft expect you to run a server to use it - and pay for the server OS (good luck home users!). It's also outdated and badly prone to dropping the ball (because of clients using the old discovery stack or otherwise) and insecurity. Finally, even if you move off Windows, and get a shiney new NAS and let Samba do the WINS serving, you * still * get hammered way easily and it's * still * fragile and insecure, because even if Samba is cleanly coded and gets it right, the PCs it's trying to link with are using an insecure, exploitable, buggy, opaque, deprecated train wreck at their ends of it. Basically, it's still almost as big a problem.
Can we do File Shares without WINS as well as NetBIOS and SMB1? And if we ditch all of these, can we get Network Neighbourhood to work without them, and without leaving SMB1 installed/enabled?
It turns out that we can. And their replacement is far more snappy and reliable than these functions ever were.
Built into Windows since at least around 2005, are two features, one of which I'll cover here, the other I'll mention for anyone who might find it helpful.
The first is a protocol called "WS-Discovery" (WSD). It's a little-known replacement discovery protocol built into Windows, since Windows Vista. You can disable all three of SMB1, NetBIOS and WINS, and WSD will take over all the discovery they did, and ensure your shares turn up in Explorer as normal - and do a far better job of it as well. Although WSD is not widely known, it has a huge advantage - it's already built into Windows Explorer, and daemons are available for FreeBSD, Linux and other systems. (mDNS might be more widely used, but unless you're on Win10, it doesn't plug into Windows Explorer, like most other discovery methods.)
One problem - WSD isn't built into Samba, so non-Windows shares offering SMB/CIFS sharing, may not be discovered. Solution - a small open source scripted daemon that provides WSD for BSD and Linux systems. (And is included in TrueNAS 12+). Run that, and now your non-Windows shares can join the party too. It's written in Python3, so it's highly cross-platform-able. I'm using it here and turned off everything else and for the first time ever - I feel confident that Network Neighbourhood is indeed, "Just Working" (TM).
The second is something called Shell Extensions, which are plugins that can add items to Windows Explorer. I mention these because perhaps, some day, someone will use this to write a shell extension that manages file shares and network neighbourhood even better. Perhaps.
Last I should mention that Win10 is apparently getting Bonjour/mDNS/dns-sd added, so maybe that'll help some people, in some situations. But not all. There will be plenty of Windows users who still just need good old Network neighbourhood to work as it should.
How to do it
Some concrete steps. As always, take a backup first, or at least, be sure you remember what you did so you can undo it if required. I've given both registry entries and CLI commands - use whichever you prefer
And that's about it! It took me ages to find this out and work out how to make it work, and a lot of that was spent believing that I had to do it somehow via NetBIOS or WINS. Maybe you're get there quicker than I did.
Enjoy your SMB1+NetBIOS + WINS free life!
2019-06-15: Updated to add link to daemonise scripting on forum, as a workaround for now
2019-11-05: Ian Carson has also written a "How to" resource on this, that's worth reading too. ("How to get your FreeNAS Shares to appear in Windows Network Neighborhood")
2020-08-28: Updated to reflect changes in TrueNAS 12+.
NOTE: For brevity only, I've written "TrueNAS" throughout, as this is the unified platform title for v.12 (2020) onward. But everything works perfectly and exactly the same on "FreeNAS" older versions, as well. At worst, watch out for python compatibility, is all, I guess, but nobody's ever reported that as an issue yet.....
--------------------------------------------------------
We do live in interesting times. Well, those of us who love the "network neighbourhood" section of Windows File Explorer do, anyhow.
Just think, the magic of all your file shares and networked computers, auto-magically shown in 'My Computer' / 'Windows Explorer', to click and use at will.
What's that? Having problems with it? Oh dear...
- Normal Network Neighbourhood discovery using NetBIOS is often incredibly fussy, delicate, laggy, and arbitrary. Sometimes a device shows up, sometimes it doesn't. Sometimes it just takes time and magic. Or about 3 hours for discovery to kick in. There's no way to force a clean scan, since the issue could be in any number of hidden places in the OS, the NIC driver stack, Winsock, or .. whatever. Forum threads regularly seem to leave off with the problem unresolved. Troubleshooting guides and intense Googling rarely seem to actually solve the issues; the diagnostic commands like NET VIEW themselves fall over with various errors, and help pages routinely tell people to use ancient tools that went out with Windows 2003 and aren't compatible any more even if found. The networker's equivalent of "tried that already" and "Not a clue, sorry" is pervasive . And there's always that one computer that *never* shows up... and its friend who seems to be in a workgroup of its own.
- Maybe you don't like broadcast packets echoing round the LAN. Maybe you're using WINS. WINS is kinda better/okay, but kinda still got too many issues.
- Oh, and SMB1 - the basic sharing protocol they use for the discovery phase of Network Neighbourhood? You really ought to turn it off. Wannacry and all that. As insecure as Yahoo's user database. Even Windows 10 tries to kill it if it can. It's unencrypted, insecure, hackable... but... oh dear. If you turn it off, Network Neighbourhood automatic discovery will stop working too! Or AD. Or other things.
As a bonus, we'll also try to clarify some of the background of what's really going on behind the scenes.
Let's start.....
This is the problem. Or problems:
In the early bronze age of computing, technologists decided to put wires between boxes, creating the first network. Some wise guy had the idea, "lets make them able to look at each other for printers and file shares." And some other wise guy figured "They could constantly shout around the LAN at anything that listens "I'M HERE AND I HAVE A FILE SHARE!"" That's a really good way to do it." Someone else figured that constant arguments and dialogs over elections would be a good addition - all in plain text, because computers didn't need security back then. Maybe call the leader something like a "master browser", and have it constantly challenged by other devices? Some unknown manager who'd been promoted too far, decided that was great thinking - noisy subnets are what we need, and whoever wants to file share across two subnets anyway? And unreliability spawns more tech jobs! And so was born..... NetBIOS!
Unfortunately that golden age came to an end, and with it, a new generation of home/small office networking arose. These people despised broadcasting, and decided that by far the best answer was to create a new system which separated discovery of shares and printers, from actual opening and exploring of those shares, and use a server for the discovery part, which was the really noisy and unpleasant bit. They called this idea, "WINS", because anyone who bought into it, was a WIN for their corporation, MumbleSoft. They kinda overlooked that users of file shares included homes and small offices that didn't have technical wizardsa able to make it all work, or able to afford Windows Server. ("I just want it to work" is a pathetic rallying cry, anyhow - real techies like to dig into the registry!)
Meanwhile SMB grew. It got versions SMB2, then SMB3, it grew encryption and signing. And all along, the guys at MumbleSoft figured this broadcasting shouting mess was perfect, and the few who didn't like it would pay for, or work out how to use, WINS. Or cope "somehow". But while SMB was regularly updated to make the actual sharing part more efficient and safer, they never once really bothered to update the 1800's design "discovery" part of things, that Windows Explorer used to find shares in the first place.
Meaning that even though SMB1 isn't actually needed for discovery or sharing (NetBIOS discovery is a completely different protocol and SMB2+3 have separate services), the NetBIOS discovery services are * still * internally dependent on that service as at 2019; they won't work unless the SMB1 service (and some others) are also enabled and fully running.
Then along came Wannacry. Suddenly SMB1's days were more numbered than nitroglycerine at a welding factory. (No, I don't know where that analogy came from either).
By 2016, SMB1 was being actively discouraged, and by 2018/9 Windows 10 was rolling out forced changes that would autonomously disable SMB1 if it wasn't being used. Which left some with a really nasty situation:
- If you used or wanted Network Neighbourhood to "just work", you could choose between enabling SMB1 and getting hacked or ransomed, or disabling SMB1 and havinbg Neighhbourhood not work at all.
- Or, as Microsoft (as they now were) blithly put it, in a footnote, "SMB1 is really bad, and you can just map drives anyway, which is so much easier, so why worry" - ignoring the fact that the downsides to having to map any drive to file share with it, are exactly why people like automatic discovery in the first place.
Which is a mess. Which we're going to fix.
First, a technical look at what's gone on
For Network Neighbourhood to work, and computers and printers to show up, Windows Explorer has to discover these other network services before it can expose them to the user. There are plenty of discovery protocols - DNS is one, AirPlay/mDNS/Bonjour/Rendezvous/dns-sd are widely used in the Apple world. NetBIOS can function as one of those.
Sorting out NetBIOS is horribly difficult and limited, because it's inextrixably tangled up with the SMB1 service internally (That's in addition to its own main services: "COMPUTER BROWSER SERVICE" and "WORKSTATION SERVICE"), as well as the Winsock networking stack. The code is buggy and unreliable, the approaches are utterly insecure, exploitable, and - by current standards - close to medieval. Overall the discovery aspect of File Shares is the main/only remaining major use for NetBIOS these days. When your network adapter or "ipconfig" says "NetBIOS over TCP/IP", that's what it's referring to. The remaining NetBIOS code could have been separated out, brought into the 1900s, updated or replaced, and the dodgy SMB1 code that doesn't actually get used in NetBIOS could have been stripped out and discarded at any of the many times when file shares themselves were gaining reliability and security updates as part of SMB2/SMB3, or anywhere in between, but never was.
Interestingly, the actual sharing of files is fine - that uses SMB2 or SMB3. But the end result is that the initial discovery and listing by Windows Explorer of any other file-sharing devices on the network, won't work, if SMB1 is disabled. Microsoft just left that section of code dependant on the SMB1 service running - and even back then, in hindsight, it was a horrible decision. Their own blogs beg people to stop using SMB1 these days.
NetBIOS and SMB1 are both genuine problems - the first is unreliable, laggy, and finnicky to the point of fragility for far too many people. It's responsible for computers that never show up or take ages to do so. The second is unsecured, unauthenticated, and trivially exploitable (once the firewall is breached). But how to move away from them?
WINS was one solution, for the broadcasting issues anyhow. WINS still used NetBIOS but it moved from shouting around the LAN to a single server that registers devices for anyone looking them up. Any device registers to it, any client queries it for resources.
But even WINS piggybacks on pretty much the same technology. It's an enterprise solution, so Microsoft expect you to run a server to use it - and pay for the server OS (good luck home users!). It's also outdated and badly prone to dropping the ball (because of clients using the old discovery stack or otherwise) and insecurity. Finally, even if you move off Windows, and get a shiney new NAS and let Samba do the WINS serving, you * still * get hammered way easily and it's * still * fragile and insecure, because even if Samba is cleanly coded and gets it right, the PCs it's trying to link with are using an insecure, exploitable, buggy, opaque, deprecated train wreck at their ends of it. Basically, it's still almost as big a problem.
Can we do File Shares without WINS as well as NetBIOS and SMB1? And if we ditch all of these, can we get Network Neighbourhood to work without them, and without leaving SMB1 installed/enabled?
It turns out that we can. And their replacement is far more snappy and reliable than these functions ever were.
Built into Windows since at least around 2005, are two features, one of which I'll cover here, the other I'll mention for anyone who might find it helpful.
The first is a protocol called "WS-Discovery" (WSD). It's a little-known replacement discovery protocol built into Windows, since Windows Vista. You can disable all three of SMB1, NetBIOS and WINS, and WSD will take over all the discovery they did, and ensure your shares turn up in Explorer as normal - and do a far better job of it as well. Although WSD is not widely known, it has a huge advantage - it's already built into Windows Explorer, and daemons are available for FreeBSD, Linux and other systems. (mDNS might be more widely used, but unless you're on Win10, it doesn't plug into Windows Explorer, like most other discovery methods.)
One problem - WSD isn't built into Samba, so non-Windows shares offering SMB/CIFS sharing, may not be discovered. Solution - a small open source scripted daemon that provides WSD for BSD and Linux systems. (And is included in TrueNAS 12+). Run that, and now your non-Windows shares can join the party too. It's written in Python3, so it's highly cross-platform-able. I'm using it here and turned off everything else and for the first time ever - I feel confident that Network Neighbourhood is indeed, "Just Working" (TM).
The second is something called Shell Extensions, which are plugins that can add items to Windows Explorer. I mention these because perhaps, some day, someone will use this to write a shell extension that manages file shares and network neighbourhood even better. Perhaps.
Last I should mention that Win10 is apparently getting Bonjour/mDNS/dns-sd added, so maybe that'll help some people, in some situations. But not all. There will be plenty of Windows users who still just need good old Network neighbourhood to work as it should.
How to do it
Some concrete steps. As always, take a backup first, or at least, be sure you remember what you did so you can undo it if required. I've given both registry entries and CLI commands - use whichever you prefer
- First kill off SMB1 on all your Windows machines (this won't take effect until the next reboot). In a registry editor file or command prompt, run either of these as you prefer:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\mrxsmb10] "Start"=dword:00000004 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters] "SMB1"=dword:00000000
or
sc config mrxsmb10 start=disabled reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters /v SMB1 /t REG_DWORD /d 0 /f
- Next, ensure the services that WSD relly on, are run automatically. Their full names are "Function Discovery Provider Host" and "Function Discovery Resource Publication. The registry edits/commands are either of these, as you prefer:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\fdPHost] "Start"=dword:00000002 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FDResPub] "Start"=dword:00000002
or
sc config fdPHost start=auto sc config FDResPub start=auto
(If either service isn't running, these also won't take effect until either the next reboot, or until the services are manually started) - If you have used WINS, you'll also want to switch that off as well. If it's allocated via your router using DHCP, clear that setting. If it's statically defined in Network Adapter Properties, clear those settings.
- You can quickly check that Windows systems are free of WINS, by forcing them to disconnect/reconnect from the network (disconnect/reconnect to Wifi, or unplug then plug in the network cable, or simply disable/enable the network connection), and then open a CMD window and type
ipconfig /all
. Just check there's no WINS server listed below DNS servers. - Next, reboot Windows. None of these changes will have much effect, until you do. When it restarts, check that Network Neighbourhood is still working nicely even with SMB1, NetBIOS and WINS all killed. It should. For me, it was working *better* than before - more snappily, all PCs showing up for the first time. We haven't got TrueNAS on board yet, so for now, that computer will be missing.
- With the PCs dealt with, we've just got left, discovery of the TrueNAS server (and any other non-Windows devices offering SMB/CIFS shares+printers).
First of all, if you have any WINS related settings in your Samba config (including "local" or "domain" master), disable/remove/comment them out. If SMB1 is enabled, kill that as well. (There's already a control to disable SMB1 added in TrueNAS 11.2-U2 - it's about 5 items down in the "SMB Service" panel. A control to disable Samba's NetBIOS name server ('nmbd', handles NetBIOS discovery) will be added in 11.2-U3) If you have custom Samba config, be sure to also remove or comment out any custom config that might enable SMB1, because it'll override that setting.
It's worth remembering your old settings in case you have to revert, but you shouldn't need to. - Make sure any Windows boxes renew their IP configs - either by having just rebooted, or by simply disabling/re-enabling the adapter in the Network Control Panel, or disconnecting/reconnecting them briefly from the LAN. (You can also restart Samba if you want to be sure the NAS has ditched WINS as well, but just restarting Samba should be enough)
- You can test your work in a couple of other ways - and I'd suggest it's worthwhile to do so. First, in WIndows Explorer, select "Network" or "Network Neighbourhood", on the left, to see your networked devices. Then choose "View" -> "Detail" to see them in tabular (detail) layout. Now right click on the column headings, and you'll see an option to choose columns. You can add columns for "discovery method" and "IP Address". If it's all worked, you'll get "WSD" as discovery method for your file shares. Second, you can use Wireshark or any other packet sniffer. If it's worked, you will see traffic on ports such as 1702 (or is it 3702), rather than NetBIOS ports 137-139,445.
If you want to test beyond doubt, go to your network adapter's properties, go into IPv4 properties, and choose the "ADVANCED" button near the bottom. Go to the "WINS" tab, and disable NetBIOS over TCP. Close properties. Go to command prompt, type "ipconfig/all" and config no WINS server and NetBIOS over TCP disabled. You should find Network Neighbourhood *still* works.
- There's only one last thing to do - on any non-Windows platforms offering SMB/CIFS shares (TrueNAS etc, but also any other *nix systems you operate), you'll want to run wsdd.py (direct download), which allows Windows Explorer Network Neighbourhood to discover your Samba/NAS/SMB/CIFS shares natively in Vista/7/8/8.1/10/11/server versions, without relying on NetBIOS.
wsdd is built into TrueNAS from 12.0 (it's supposed to be in FreeNAS11.3 and earlier TrueNAS but didn't work for me, but works perfectly on 12), and can be enabled from the SMB service config. Controls will also be added to kill off Samba's NetBIOS discovery daemon, as mentioned. But if you want it on any other platform, you'll need to download and run it manually. The next 3 bullets cover "how to do that". - On FreeNAS/TrueNAS 11.3 enable wsdd with this aux parameter to the Samba service:
enable web service discovery=yes
On TrueNAS 12+, no need to do anything apart from disable SMB1/NetBIOS on WIndows. WSD and wsdd should run by default on your NAS box. You can stop reading now..... - Firewalling shouldn't be an issue either on Windows or on your NAS. But if it is, then the rules you need are: Incoming and outgoing multicast traffic on port 3702 allowed. For IPv4, the multicast address is 239.255.255.250, for IPv6 the link local SSDP multicast address (fe02::c) is used. Also, incoming TCP traffic (and related outgoing traffic) on port 5357 must be allowed. But if it's just running on TrueNAS locally in your LAN, you probably don't need to worry.
- I haven't yet figured the neatest way to package and daemonise wsdd if needed, but you can test + run it manually from the command line pretty easily. Suppose you download it to
~/wsdd.py
for testing, then you'd do something like this:
chmod 0555 ~/wsdd.py
(allow read+execute)
~/wsdd.py --help
(to view help)
then one of these two commands:
~/wsdd.py [options]
(to run)
~/wsdd.py [options] &
(to run in background and return to console)
Or see this post for a.workaround script.
To terminate:
If running in the foreground: CTRL-C is easiest. If s running in the background, then you can find and kill it using:
ps -awx | grep wsdd
and look for the line with command "python3 .... wsdd.py"
kill <pid number, shown on left>
On my system, the moment I ran it, my FreeNAS 11.3 server showed up in Network Neighbourhood again.
[/B]
And that's about it! It took me ages to find this out and work out how to make it work, and a lot of that was spent believing that I had to do it somehow via NetBIOS or WINS. Maybe you're get there quicker than I did.
Enjoy your SMB1+NetBIOS + WINS free life!
2019-06-15: Updated to add link to daemonise scripting on forum, as a workaround for now
2019-11-05: Ian Carson has also written a "How to" resource on this, that's worth reading too. ("How to get your FreeNAS Shares to appear in Windows Network Neighborhood")
2020-08-28: Updated to reflect changes in TrueNAS 12+.