New in 7.9: Fulltext search

[mherger]

> Is there a preferred way of tidying up a library to avoid scanner /
> server crashes?

Try to understand why it is crashing first. There are a hundred or more
reasons why it might fail, but none should happen. It's important to
understand what's wrong before you try to fix whatever.

So... check your server.log and scanner.log files for hints about
problems. And then post to a relevant thread or start your own with
information about your findings, your OS, your LMS version etc. You
should not be using 7.9 any more - which this thread is about.

[mherger]

New in 7.9: Fulltext search

Something new to play with: full text search (FTS).

Your search no longer is limited to a simple title/name search. With the
FTS you can now easily find track X by artist Y. Just type "hurry love
supremes" if you want to make sure not to get Phil Collins' version.

Other examples:

"mp3" - only find tracks in lossy mp3 (or with mp3 in the name - oh well)

"playlist elvis" - find playlists with tracks by Elvis, or with Elvis in
the name or a track's name

"flac 88.2 mozart" - get that highres classical stuff

The search code tries to weight results: hits in track titles have
higher priority than hits in a track's artist or album name. Eg. "lov"
would rank ABBA's "loving me, loving you" higher than "love me tender"
from the "greatest love songs, ever" album. Playlist names have priority
over track titles included in the playlist etc.

FTS is available from both the web UI's "live search" as well as the
regular search menus. Please give it a try and let me know what you
think about it.


A few caveats:

- FTS most likely will be a bit heavier on your system, with regards to
both the CPU as well as memory. I guess it shouldn't be used without the
highmem preference being set to high or max.

- FTS only works with SQLite. If you decided to use MySQL for whatever
reason, then I'm sorry, this won't work, as it's based around SQLite's
FTS extension

FTS is implemented as a plugin (mostly). If you don't like it, then just
disable the plugin to get the old behaviour back.

--

Michael

[Mnyb]

+ excellent idea

[tufty]

Great new feature, thanks!

[pippin]

Michael, you are awesome!

[atrocity]

Your search no longer is limited to a simple title/name search. With the
FTS you can now easily find track X by artist Y. Just type "hurry love
supremes" if you want to make sure not to get Phil Collins' version.

I've wanted this forever, thank you so much!!!

[mherger]

New in 7.9: Fulltext search

> Michael, you are awesome!

....except that it doesn't work with iPeng... Only tested it after I had
it announced here. Web UI should be fine though. I'll take care of it asap.

--

Michael

[frank1969]

Great idea but...
... after 40 minutes it still says "Volltext Index erstellen (0 von 5) Läuft 00:39:45" - and there is only one "dot" shown - so it should take hours, dates ...
I know, i have a 420k track library, but on a high performance PC.
Any idea how long full text search "usually" should do or should I stop and restart so it maybe "hanged`up"?

last entry in logs is:
[14-10-23 21:12:56.9772] Slim::Music::Import::runImporter (485) Starting Slim::Plugin::FullTextSearch::Plugin scan
some 40 minutes ago ...

[mdconnelly]

Awesome concept! Where is it picking up the hirez "88.2" in a query? Is it actually looking at the files or just if that was stored in a tag? Just wondering if I could do the same for 24bit depth?

[mherger]

New in 7.9: Fulltext search

"usually" is hard to define. My 20k tracks take a second or two to index on a poor Atom based system. Even less on the Macbook Pro with SSD, of course.

Is there any activity going on? Eg. massive memory usage? CPU load? library.db changes?



> Am 23.10.2014 um 21:54 schrieb frank1969 <frank1969.6mggfb (AT) no-mx (DOT) forums.slimdevices.com>:
>
>
> Great idea but...
> ... after 40 minutes it still says "Volltext Index erstellen (0 von 5)
> Läuft 00:39:45" - and there is only one "dot" shown - so it should
> take hours, dates ...
> I know, i have a 420k track library, but on a high performance PC.
> Any idea how long full text search "usually" should do or should I stop
> and restart so it maybe "hanged`up"?
>
>
> ------------------------------------------------------------------------
> frank1969's Profile: http://forums.slimdevices.com/member.php?userid=31156
> View this thread: http://forums.slimdevices.com/showthread.php?t=102347
>
>

[pippin]

> Michael, you are awesome!

....except that it doesn't work with iPeng... Only tested it after I had
it announced here. Web UI should be fine though. I'll take care of it asap.

--

Michael

And it probably won't until you tell me how the CLI command for it works

[frank1969]

Yes, thanks - after 55 minutes it finished full text search (for 420k mp3-files). I will now test the feature and report if it was only the initial scan that took so "long" or if it's the same with every (update-) scan. Great idea, by the way! :-)


[QUOTE=mherger;794047]"usually" is hard to define. My 20k tracks take a second or two to index on a poor Atom based system. Even less on the Macbook Pro with SSD, of course.

Is there any activity going on? Eg. massive memory usage? CPU load? library.db changes?

[mherger]

New in 7.9: Fulltext search

Oh, and you do use the "maximum" memory preference in Settings/Advanced/Performance?

Michael



> Am 23.10.2014 um 21:54 schrieb frank1969 <frank1969.6mggfb (AT) no-mx (DOT) forums.slimdevices.com>:
>
>
> Great idea but...
> ... after 40 minutes it still says "Volltext Index erstellen (0 von 5)
> Läuft 00:39:45" - and there is only one "dot" shown - so it should
> take hours, dates ...
> I know, i have a 420k track library, but on a high performance PC.
> Any idea how long full text search "usually" should do or should I stop
> and restart so it maybe "hanged`up"?
>
>
> ------------------------------------------------------------------------
> frank1969's Profile: http://forums.slimdevices.com/member.php?userid=31156
> View this thread: http://forums.slimdevices.com/showthread.php?t=102347
>
>

[frank1969]

Ok, first thing I recognized:
New artists are not to be found.

Yesterday I added some tracks from (e.g.) Willy Wagner or Tina (02) - I didn't own any tracks of them before (as "co"-artist, separated with ";" as I always do)

If I call the album, I see performer "Willy Wagner", can click on the name and it shows me I own own album he contributes tracks too - fine, works as it should!

If I search for "Willy Wagner", it finds - nothing.

If I browse my artist list, it finds - nothing!


This behaviour seems to affect only new added TRACK ARTIST .

I also added an album by an artist I did own nothing from before (Ric Seroka) so he is mentioned as ("single") track artist and ALBUM artist - and everyhting is finde.
I can search for his name and I find him in the artist's list...

[Eric_2014]

285

Something new to play with: full text search (FTS).

Your search no longer is limited to a simple title/name search. With the
FTS you can now easily find track X by artist Y. Just type "hurry love
supremes" if you want to make sure not to get Phil Collins' version.

Other examples:

"mp3" - only find tracks in lossy mp3 (or with mp3 in the name - oh well)

"playlist elvis" - find playlists with tracks by Elvis, or with Elvis in
the name or a track's name

"flac 88.2 mozart" - get that highres classical stuff

The search code tries to weight results: hits in track titles have
higher priority than hits in a track's artist or album name. Eg. "lov"
would rank ABBA's "loving me, loving you" higher than "love me tender"
from the "greatest love songs, ever" album. Playlist names have priority
over track titles included in the playlist etc.

FTS is available from both the web UI's "live search" as well as the
regular search menus. Please give it a try and let me know what you
think about it.


A few caveats:

- FTS most likely will be a bit heavier on your system, with regards to
both the CPU as well as memory. I guess it shouldn't be used without the
highmem preference being set to high or max.

- FTS only works with SQLite. If you decided to use MySQL for whatever
reason, then I'm sorry, this won't work, as it's based around SQLite's
FTS extension

FTS is implemented as a plugin (mostly). If you don't like it, then just
disable the plugin to get the old behaviour back.

--

Michael

Sweet!