afp filename search problems

[giox069]

We just migrated a 2,6TB fileserver from MAC OSX to FreeNAS 8.3.0-RELEASE-p1. MACs clients are now accessing the data on FreeNAS via afp.

Unfortunately users are experiencing problems in searching files by name: they used to have a result in few seconds. Now with Freenas the search result comes out after some minutes, or sometimes never appears.

I have read of netatalk's CatalogSearch searchdb optimization here.
Will it help our users ? Is it possible to enable it on freenas ? How can we enable it ? Is it safe ?

Thank you
Giovanni

 

[Stephens]

While you're waiting on an answer, can you tell us how much RAM you have? What CPU? How many/what type HDD's? What layout? (Mirror? RaidZ1?) Virtual machine or raw hardware?

FYI, if you put that information in your forum signature, people will be able to see it whenever you post.

 

[JaimieV]

I believe that when hosted by OSX, the remote OSX machine runs the Finder search queries locally using the local Spotlight database contents and passes the results back to the clients - that's why it's fast, similar to a local Spotlight search.

Now your data is on FreeNAS, so that service isn't available, so slooooow file searches.

I don't know anything about the CatSearch, but the link you give certainly seems to indicate that it's for similar quicksearch functionality as OSX would give. Try it and see - please post back if you get something interesting.

(I just checked to see if the "Database Path" field in the AFP shares setup is connected to this - but no, it's for the CNID database)

 

[giox069]

@Stephens:
Old mac fileserver: X Serve G5, 2GHz PowerPC G5, 1,5GB RAM with external 6 SCSI disk of 700GB each one.
New FreeNAS afp fileserver: it's VM in a VMWare ESXi 5.1 infrastructure. The guest freenas VM configuration is: 4 CPU, 8GB RAM. Disks are on a EqualLogicPS6110XV SAN, connected to the server with dual 10Gbps Dell switches.

I will try to enable searchdb for CatSearch and let you know.

 

[fracai]

I believe that when hosted by OSX, the remote OSX machine runs the Finder search queries locally using the local Spotlight database contents and passes the results back to the clients - that's why it's fast, similar to a local Spotlight search.

I'm fairly certain that isn't correct, or at least is worded poorly. My understanding is that OS X machines are always updating their local Spotlight databases as files are changed. When you perform a search that includes a remote share, if the remote database is available I don't see why Spotlight wouldn't be able to read that remote database and return quick results. If the share is coming from a non-Mac source Spotlight either needs to maintain the database itself (I'm not sure how this works if multiple Macs have the same share mounted) or run a real time search.

If all that is being searched are filenames, it may be a better option to pick a third-party tool that isn't going to also inspect the file contents as Spotlight does.

 

[JaimieV]

The remote OSX Servers' Spotlight index is located in a root folder which is generally outside the share, so the client has no access. OSX Server runs a process called "Spotlight Server", which does exactly what you expect. Your local Spotlight search talks to the Spotliught Server process on the server. Your local Spotlight *will not* use the remote disks Spotlight indexes, even if they're available because the root of the drive is the share point and you're logged into the share as an admin user.

When Spotlight Server is not available from the server, the OSX client automatically runs normal doing-it-the-hard-way directory traversing unix 'find' command style searching.

(There was a bug in OSX 10.7 client where on AFP shares that fallback didn't happen, but that was fixed in 10.7.4 I think)

 

[fracai]

Well, the more you know. Thanks.

Wait, is that only for OS X Server shares? Or is the same process provided with OS X to OS X AFP shares?

 

[JaimieV]

Server only, came in with 10.4 I think. OSX client shares don't help out, so fall back straight to the crap slow version.

Of course, if Glox's Mac was running client, this whole discussion is under a misapprehension!

 

[giox069]

I did some other tests. There is currently no way to speedup searches in netatalk 2.2. Option "searchdb" (CatalogSearch) is useless to search a substring in filename. A new MAC client connecting to my freenas/netatalk server typically takes 40-50 minutes to produce the first results. Some MAC, then, make a cache of the index locally, and are faster.
The only useful way I found to speedup search, is to schedule a script to export the full CNID DB (command dbd -d) to a text file and make a web application to let the users search by filename.
I also came to this page: http://www.helios.de/viewart.html?id=730-en. They clearly say that for netatalk there is "Lack of performance for large volumes (Mac Find File will take hours)".

Anyone knows if upgrading to Netatalk 3.0, unfortunately outside freenas, will bring fast search results ?

 

[slowfranklin]

Anyone knows if upgrading to Netatalk 3.0, unfortunately outside freenas, will bring fast search results ?

It won't.
http://www.netafp.com/spotlight-technology-preview-3-1070/