Faster Searching from Mac Spotlight or Finder of Freenas Files

TravisT

Patron
Joined
May 29, 2011
Messages
297
So if you can ping 172.16.110.16 from another system in your LAN and netstat -na | grep LISTEN shows port 9200 with tcp4 at *, you really should be able to connect to Elastic on that port.

That's the problem - I can't even get elasticsearch service to start if I change the network.host to anything other than 127.0.0.1. But otherwise, yes, connectivity-wise I agree with you.

netstat output:
Code:
root@crawler:~ # netstat -na | grep LISTEN
tcp4       0      0 127.0.0.1.9200         *.*                    LISTEN
tcp6       0      0 ::1.9200               *.*                    LISTEN
tcp4       0      0 127.0.0.1.9300         *.*                    LISTEN
tcp6       0      0 ::1.9300               *.*                    LISTEN
root@crawler:~ #
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,740
That's the problem - I can't even get elasticsearch service to start if I change the network.host to anything other than 127.0.0.1.
I might have overlooked it, but what happens when you set it to 0.0.0.0 or 0 which should be a valid value for "every address" according to the docs ...
 

TravisT

Patron
Joined
May 29, 2011
Messages
297

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,740
Thanks. This is something I vaguely remember having seen in production on some of our hosts at some time. All pretty vague, sorry.
So Elastic expects to be running in a clustered setup by default if there is connectivity to the outside world. You need to find out how to convince it not to do that. The error message and the Elastic docs are your best source. It's bed time for me around here, good luck :wink:

If you prefer to McGyver it:

1. Create an entry in /etc/services for 9201
2. Create an entry in inetd.conf to forward 9201 (using the service name) to 127.0.0.1 port 9200 using /usr/bin/nc (netcat)
3. Enable and start inetd
4. Use port 9201 from outside

Edit: we have some projects where we run Elastic for cool sh.. in large production environments. I have colleagues who really know this stuff. I'll ask tomorrow.
 

TravisT

Patron
Joined
May 29, 2011
Messages
297
Thanks. This is something I vaguely remember having seen in production on some of our hosts at some time. All pretty vague, sorry.
So Elastic expects to be running in a clustered setup by default if there is connectivity to the outside world. You need to find out how to convince it not to do that. The error message and the Elastic docs are your best source. It's bed time for me around here, good luck :wink:

If you prefer to McGyver it:

1. Create an entry in /etc/services for 9201
2. Create an entry in inetd.conf to forward 9201 (using the service name) to 127.0.0.1 port 9200 using /usr/bin/nc (netcat)
3. Enable and start inetd
4. Use port 9201 from outside

Edit: we have some projects where we run Elastic for cool sh.. in large production environments. I have colleagues who really know this stuff. I'll ask tomorrow.

Thank you! That helped tremendously. All I needed was to configure network.host to 127.0.0.1 and add this parameter to the elasticsearch.yml config:

http.host: 0.0.0.0

I can now access port 9200 remotely and locally.

Not sure that I'm out of the water yet, but this has to be forward progress.
 

Henning Kessler

Contributor
Joined
Feb 10, 2015
Messages
143
Mmhh today I compiled samba by myself and enabled spotlight in the build options. The result of smbd -b | grep -I spot:
Code:
   HAVE_SPOTLIGHT_BACKEND_ES
   HAVE_SPOTLIGHT_BACKEND_TRACKER
   WITH_SPOTLIGHT

no such results on the samba version that is coming with TrueNAS-12.0-U2.1. LOL.
 

volothamp

Explorer
Joined
Jul 28, 2019
Messages
72
I'm almost there, I managed to configure elasticsearch and fscrawler. Thank you for this post it was extremely useful. Especially Patrick.

I can do queries using

Code:
❯ curl -X GET "<ELASTIC_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "query_string": {
      "query": "my query",
      "default_field": "content"
    }
  }
}


From my mac and I see results! That's great.
I also installed Kibana and managed to create some visualisations on my NAS, this is all very nice.

The only thing missing is the actual searching from spotlight :D

So far I've put

Screenshot 2021-10-21 at 22.51.38.png


In the SMB General advanced configuration and in my specific share

Screenshot 2021-10-21 at 22.52.13.png



But it seems like spotlight searching is not working. Any idea? I'm on TrueNAS-12.0-U6
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
I’d consider foxtrot for word files and like documents and and image indexer like acdsee, Lightroom, etc for images.

spotlight is simply inferior to foxtrot for word based searches.
 

Henning Kessler

Contributor
Joined
Feb 10, 2015
Messages
143
Yes, see my post above. Under TrueNAS Core 12U6 smbd -b | grep -I spot still results in "crickets". If they would have compiled samba with spotlight enabled you would see something like this:
Code:
HAVE_SPOTLIGHT_BACKEND_ES
HAVE_SPOTLIGHT_BACKEND_TRACKER
WITH_SPOTLIGHT
 

volothamp

Explorer
Joined
Jul 28, 2019
Messages
72
So
Yes, see my post above. Under TrueNAS Core 12U6 smbd -b | grep -I spot still results in "crickets". If they would have compiled samba with spotlight enabled you would see something like this:
Code:
HAVE_SPOTLIGHT_BACKEND_ES
HAVE_SPOTLIGHT_BACKEND_TRACKER
WITH_SPOTLIGHT

so I guess the only way to proceed is compile SMB and TrueNAS on my own, thank you
 

volothamp

Explorer
Joined
Jul 28, 2019
Messages
72
A quick update on this:

to compile it with the spotlight backend it seems like this option is needed


Which is currently disabled on master

I don't have 80gb to fully recompile TrueNAS on machine, can some of the guys at IXSystem finally clear the situation?

  • #2 says it's compiled in the beta
  • Yesterday at 7:44 PM actually shows that none of the options (defined here) are enabled - it's the same on my machine on TrueNAS-12.0-U6
  • And Kris Moore says it's only enabled only in SCALE.

Can I open a PR to try it for the next U release or am I stuck to compile it on my own?
 

volothamp

Explorer
Joined
Jul 28, 2019
Messages
72
  • #2 says it's compiled in the beta
  • Yesterday at 7:44 PM actually shows that none of the options (defined here) are enabled - it's the same on my machine on TrueNAS-12.0-U6
  • And Kris Moore says it's only enabled only in SCALE.
Another quick update

22 days ago the flag was enabled on the TN-13.0-BETA1 branch, so I'm expecting to have Spotlight support in TrueNAS 13.0


Thank you Andrew @ IXSystems!
 

gegtor

Explorer
Joined
Sep 16, 2017
Messages
99
Could this be at least added to backlog? Other NAS OSes have this for years

Spotlight + built in indexing is super important for macOS users and makes stuff seamless
And in places where TrueNAS is often deployed like in video / graphic editing Macs are the majority
 

cap

Contributor
Joined
Mar 17, 2016
Messages
122
Could this be at least added to backlog? Other NAS OSes have this for years
Which NASOS has spotlight support? I only know that Synlogy has had it for a while.

Anyway, I hope we can get this running relatively easily in the future.

Question: Are the read permissions of the users taken into account in these implementations? Or can you then find things via Spotlight that you really shouldn't.
 

gegtor

Explorer
Joined
Sep 16, 2017
Messages
99
Synology has excellent spotlight / macOS support up to this day with indexing service it even has ocr support
QNAP Also has it
 

kamal juck

Dabbler
Joined
May 13, 2014
Messages
18
Hi All,

Today I was trying to see if there had been any development on Apple Spotlight support, and found this thread that I had started a few years ago LOL.

I'm a bit confused and had trouble following the posts above, my questions:

- Is there Native support for Spotlight now?
- Do I just just use SMB and flip some flags to enable Spotlight?
- Do I have to install elasticsearch?

Sorry for these questions, but I'm an dumb Apple user who likes to be able to find his files quickly and not wait till the next eclipse:wink:

K
 
Top