Spotlight with elasticsearch backend on SMB shares

ak111pro

Dabbler
Joined
Feb 20, 2020
Messages
10
Hey guys, maybe you can help me out.
I'm really not sure if I getting this right. I'm trying to get an SMB share with proper spotlight working accessing form macos clients, according to here you can use elasticserach as backend: https://wiki.samba.org/index.php/Spotlight_with_Elasticsearch_Backend
So I was trying to set this up in a jail, following this guide:
Getting through all of this I'm not sure what to do now?

In theory I should add a mountpoint to the jail, then build an index with elasticserach and access the smb share through a macos client...

How do I access an SMB share through a jail?
How do I connect elasticserach to spotlight?
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
Hey guys, maybe you can help me out.
I'm really not sure if I getting this right. I'm trying to get an SMB share with proper spotlight working accessing form macos clients, according to here you can use elasticserach as backend: https://wiki.samba.org/index.php/Spotlight_with_Elasticsearch_Backend
So I was trying to set this up in a jail, following this guide:
Getting through all of this I'm not sure what to do now?

In theory I should add a mountpoint to the jail, then build an index with elasticserach and access the smb share through a macos client...

How do I access an SMB share through a jail?
How do I connect elasticserach to spotlight?
12.0-BETA2 is compiled with spotlight support / elasticsearch. You just need to configure through auxiliary parameters. Do note that we started compiling with this to give users options in the main install. Spotlight isn't a "supported" feature in the sense that we take steps to QE / fix it.
 

ak111pro

Dabbler
Joined
Feb 20, 2020
Messages
10
12.0-BETA2 is compiled with spotlight support / elasticsearch. You just need to configure through auxiliary parameters. Do note that we started compiling with this to give users options in the main install. Spotlight isn't a "supported" feature in the sense that we take steps to QE / fix it.
cool, I'll try this out.
are those the right aux params to set this up, or where can I look up the right ones?

[global]
spotlight backend = elasticsearch
elasticsearch:address = localhost
elasticsearch:port = 9200

[share]
spotlight = yes
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
cool, I'll try this out.
are those the right aux params to set this up, or where can I look up the right ones?

[global]
spotlight backend = elasticsearch
elasticsearch:address = localhost
elasticsearch:port = 9200

[share]
spotlight = yes
They are documented in the smb.conf manpage. Do note that 12.0-BETA2 (and pre-release versions in general) should not be used in production environments.
 

ak111pro

Dabbler
Joined
Feb 20, 2020
Messages
10
You'll need to look up in the manpage of the samba version installed in TrueNAS. `man smb.conf`
thanks, is the man file only in TrueNAS?

one other thing, when i do the command
Code:
afp -version

I get
Code:
netatalk has been compiled with support for these features:

      Zeroconf support: No
     Spotlight support: No


Do you know how can I enable spotlight?
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
thanks, is the man file only in TrueNAS?

one other thing, when i do the command
Code:
afp -version

I get
Code:
netatalk has been compiled with support for these features:

      Zeroconf support: No
     Spotlight support: No


Do you know how can I enable spotlight?
11.3 It's not possible. 12.0 it's, in principle, possible for TrueNAS for an intrepid BETA tester
 

tmueko

Explorer
Joined
Jun 5, 2012
Messages
82
I did some testing on this: looks promising:

I started a jail with NAT and forwarded 5601 for kibana.
I mountet my dataset to /mnt/my_dataset.
Then I installed elasticsearch and so on:

Code:
pkg install -y elasticsearch7 kibana7 openjdk15 tesseract-data tesseract curl
sysrc elasticsearch_enable="YES"
sysrc kibana_enable="YES"

I edited /usr/local/etc/kibana/kibana.yml to contain:
server.host: "0.0.0.0"

and /usr/local/etc/elasticsearch/elasticsearch.yml for
network.host: 0.0.0.0
discovery.seed_hosts: ["127.0.0.1", "[::1]"]

Code:
cd /usr/local/
fetch  https://oss.sonatype.org/content/repositories/snapshots/fr/pilato/elasticsearch/crawler/fscrawler-es7/2.7-SNAPSHOT/fscrawler-es7-2.7-20210407.064623-168.zip
unzip fscrawler-es7-2.7-20210407.064623-168.zip
ln -s fscrawler-es7-2.7-SNAPSHOT fscrawler
cd fscrawler
setenv JAVA_HOME /usr/local/openjdk15
./bin/fscrawler my_dataset


Then edit ~/.fscrawler/my_dataset/_settings.yaml to your needs (at least the path)

Edit SMB in Services and Shares:
[global]
spotlight backend = elasticsearch
elasticsearch:address = 172.16.0.2
elasticsearch:port = 9200

[my_dataset]
spotlight = yes

Then run fscrawler:
Code:
setenv JAVA_HOME /usr/local/openjdk15
/usr/sbin/daemon -t fscrawler-creativteam -P /var/run/daemon-fscrawler-creativteam.pid -p /var/run/fscrawler-creativteam.pid -S -R 60 /usr/local/fscrawler/bin/fscrawler creativteam


now I'm waiting for the crawler for first run.
 

dak180

Patron
Joined
Nov 22, 2017
Messages
307

kapitainsky

Dabbler
Joined
Sep 30, 2022
Messages
46
Maybe closer than ever to put it to rest and enjoy searching. We think we are on the right path to fix it. Ticket has been raised but to help with its visibility please login to jira and vote for this issue.

 

kapitainsky

Dabbler
Joined
Sep 30, 2022
Messages
46
there is no spotlight support in smbd in truenas

TrueNAS-13.0-U2:

Code:
root@truenas[~]# smbd -b | grep SPOT
   HAVE_SPOTLIGHT_BACKEND_ES
   WITH_SPOTLIGHT


It is also enabled in TrueNAS SCALE 22.02.4 and TrueNAS SCALE 22.12-BETA2
 
Top