btsync and immediate tracking of file changes does not work in FreeNAS

[wingstar]

Hi,
I have recently setup a FreeNAS 9.2.0 server with btsync in a jail. btsync have two ways to detecting that a file has changed or has been added.
1. It scans the shared folders at regular intervals (can be changed in config file).
2. Using a mechanism to be notified when any change is made by the OS/file system.

I decided to test this on FreeNAS. I deliberately set the scan interval to 36000s (10 hours). Then I copied a file via cifs/samba to a share used by btsync. The issue is that btsync did not detect the change until 10 hours had passed. I wrote an question in the btsync forum:
http://forum.bittorrent.com/topic/26430-detecting-file-changes-on-freenas-seems-to-have-issues/
I got a response saying:

Unfortunately FreeBSD version of BTSync cannot track immediate changes on file system due to OS peculiarities. So, sync folders on FreeBSD are updated only during every full rescan interval, in your case - every 10 hours.

Does any one know what kind of support FreeBSD/FreeNAS has for file system notification?

 

[dlavigne]

Can you get more technical information from them on what the OS peculiarities are? That way we can see if it is a bug that is fixable.

 

[wingstar]

Hi,
I asked the question in the btsync forum and got the following reply:

FreeBSD does not provide an option to get directory change notifications like other OSes do. Therefore, the only way for BTSync to detect changes is a folder rescan.

Does anyone know if this is the case for FreeBSD/FreeNAS?

 

[fracai]

BSD doesn't support inotify, which is what they're probably referring to with "directory change notifications". BSD does have kqueue, but that would require code changes in the BTSync application.

 

[wingstar]

BSD doesn't support inotify, which is what they're probably referring to with "directory change notifications". BSD does have kqueue, but that would require code changes in the BTSync application.

I basically know nothing of either inotify or kqueue. I did some googling and found this at the following link (https://en.wikipedia.org/wiki/Gamin):

Under Linux, Gamin uses inotify or dnotify to monitor filesystem activity. Under FreeBSD, Gamin uses kqueue/kevent as the native FreeBSD kernel event notification mechanism, but in this case kqueue has one major drawback as compared to inotify: inotify is the filename-based monitoring facility and kqueue uses file descriptors for identification of monitored files, so, under FreeBSD, gam_server has to open each file in the monitored directory. This can be an issue when monitoring directories with many files stored, because system could easily reach its kernel limits on maximum file descriptor count (kern.maxfiles and kern.maxfilesperproc).

If this information is correct, which I don't know, there can be issues with kqueue if you are monitoring folders with alot of files which typically is the case for btsync.

 

[raidflex]

I basically know nothing of either inotify or kqueue. I did some googling and found this at the following link (https://en.wikipedia.org/wiki/Gamin):

If this information is correct, which I don't know, there can be issues with kqueue if you are monitoring folders with alot of files which typically is the case for btsync.

The big question here is then what is this limit? Are we talking about 10k's, 100k's or more?

 

[wingstar]

The big question here is then what is this limit? Are we talking about 10k's, 100k's or more?

If it is has something to do with kern.maxfiles and kern.maxfilesperproc the default values for these on my FreeNAS 9.2.0 x64 are:
# sysctl -a | grep kern.maxfiles kern.maxfiles: 12328 kern.maxfilesperproc: 11095
These can as far as I can see be changed but it seems that the default values are too small. I have over 100k files synced via btsync.

 

[raidflex]

If it is has something to do with kern.maxfiles and kern.maxfilesperproc the default values for these on my FreeNAS 9.2.0 x64 are:
# sysctl -a | grep kern.maxfiles kern.maxfiles: 12328 kern.maxfilesperproc: 11095
These can as far as I can see be changed but it seems that the default values are too small. I have over 100k files synced via btsync.

So I would think these changes would need to be applied to the jail where the BtSync plugin is installed?

 

[Joshua Parker Ruehlig]

So I would think these changes would need to be applied to the jail where the BtSync plugin is installed?

Well, you'd also need to convince the BitTorrent Sync developers to modify their code to use kqueue.

 

[Huug Brinkers]

May I ask how you set the scan interval to 36000s (10 hours) wingstar? It is in the config file i found, but where is that file and how to change it ? With 'ee' or someting ?

 

[Joshua Parker Ruehlig]

don't edit the config file directly. all btsync settings are available in the freenas webui btsync menu or btsync webui

 

[Huug Brinkers]

How can I be so stupid....? Thanks! I was searching for hours....