Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

SOLVED Painfully slow performance in WebGUI, SSH, terminal, when running Syncthing

Status
Not open for further replies.

Ajedi32

Dabbler
Joined
Oct 30, 2016
Messages
28
I just finished a new home server build with FreeNAS, and I'm getting ridiculously slow performance on pretty much everything. Simple commands (ls, whoami) entered into the terminal through the physical keyboard attached to the server take several seconds to complete, and more complex operations (like loading the WebGUI) can take up to a minute, if they complete at all before timing out.

The problem seems to be related to the Syncthing plugin: whenever I'm running it, everything is extremely slow. When I stop the jail, performance returns to normal. Syncthing doesn't appear to be doing anything out of the ordinary though; the performance metrics under the "Reporting" section of the WebGUI seem normal, and Syncthing does not appear to be using an unusual amount of RAM or CPU. Plus I have a near-identical setup running on an older FreeNAS system and Syncthing does not seem to cause poor performance when installed there.

System info:

NQJTc4l.png


FIXED: I actually _just_ fixed this as I was typing the post, but I'm gonna post it anyway in case someone else has the same problem.

Syncthing has a (somewhat new) feature that allows it to watch for changes to files rather than periodically scan for them. I don't know why, but it seems this feature was what was causing the issue; I turned it off for all folders on my server and performance returned to normal. Then I tried enabling it for a couple folders on my old FreeNAS server and performance there quickly tanked.

If someone wants to investigate this further to find out why Syncthing's watch feature is causing massive performance issues, be my guest. I've spent more than enough time on this already though; so I'm content to leave it as that.
 

mb99

Cadet
Joined
Jun 1, 2018
Messages
1
Thanks, Ajedi32, your post saved me a lot of time!

I'd recently noticed super-slow performance on my Freenas system, too, where loading the web GUI or changing pages was painful (tens of seconds), and accessing files on SMB shares was similarly slow, sometimes timing out.

Going into the Syncthing plugin's web GUI (which performed at its usual slick speed) and disabling the 'watch for changes' flag on each folder has resulted in Freenas returning to its normal speed of operation. I've left the watch flag enabled on my various client PCs - the server is mainly a receiver of content.

At the time I was on 11.1-U4 but have now updated to U5

Code:
Build		 FreeNAS-11.1-U5	
Platform	  Intel(R) Core(TM) i3-4360T CPU @ 3.20GHz	
Memory		16307MB	
System Time   Fri, 1 Jun 2018 21:50:55 +0100
Uptime		9:50PM up 1:41, 1 user
Load Average  0.57, 0.62, 0.63
 

Bearmangler

Cadet
Joined
May 22, 2018
Messages
2
Thank you Ajedi32!

Disabling the Folders setting under Advanced > Scanning > Watch for Changes looks like it's fixed my issue. I've also left this setting on my PCs too as they're the source of content, and seems to be running fine. I was on U4 and had the issue, and upgraded to U5 which had the same problem. At a guess it could be an issue with Syncthing and this feature.

I'm new to FreeNAS and was scratching my head all weekend trying to work out why Syncthing was tanking it to the point where I'd have to restart my unresponsive FreeNAS after a few minutes of Syncthing running just to be able to click on things to troubleshoot.

Previously I was trying to pause all folders except for 1 and it seemed to run relatively okay, was still slow. But I had one folder that would just freeze up every time with a message under Rescans stating that the Watching could not be established. Seems to be scanning by itself with Watch for Changes disabled.

I was getting a lot of "samba4.samba4 alert object at 0x814d0a438 failed timed out", and "handshake failed" errors. I'm not sure if they're related, but I haven't seen this error pop up over the last hour of having the watching feature disabled.
 

LIGISTX

Guru
Joined
Apr 12, 2015
Messages
505
I experienced this same issue. I posted in syncthing forum as well a didn’t really get much info. Not sure if it’s a FreeNAS issue or a syncthing issue. But I can confirm turning off the folder watcher fixes the slowdowns.


Sent from my iPhone using Tapatalk
 

Baenwort

Explorer
Joined
Feb 19, 2015
Messages
77
With the Emby plugin the cause of watching the local file system causing problem is that FreeBSD doesn't have a good method of pushing this info. On Emby they have to modify mono to provide the ability and it takes customization of each release to accomplish this.

I would suspect that Syncthing has a similar problem with FreeBSD and ZFS notification of file system changes but no one has customized anything.

I've performed the same disabling of the function as like everyone else my FreeNAS setup is mostly receiving changes,
 

Stookdog

Cadet
Joined
Oct 19, 2018
Messages
4
Thank you SO MUCH, seriously. This has been giving me grief for a week now and causing me constant freezes. I tried reinstalling the OS, changing a bunch of settings, but didn't see this option until now. I hope this keeps the issue fixed for me too.
 
Status
Not open for further replies.
Top