Announcement

Collapse
No announcement yet.

New in 7.9: Fulltext search

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    New in 7.9: Fulltext search

    > Is there a preferred way of tidying up a library to avoid scanner /
    > server crashes?


    Try to understand why it is crashing first. There are a hundred or more
    reasons why it might fail, but none should happen. It's important to
    understand what's wrong before you try to fix whatever.

    So... check your server.log and scanner.log files for hints about
    problems. And then post to a relevant thread or start your own with
    information about your findings, your OS, your LMS version etc. You
    should not be using 7.9 any more - which this thread is about.
    Michael

    "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
    (LMS: Settings/Information)

    #2
    New in 7.9: Fulltext search

    Something new to play with: full text search (FTS).

    Your search no longer is limited to a simple title/name search. With the
    FTS you can now easily find track X by artist Y. Just type "hurry love
    supremes" if you want to make sure not to get Phil Collins' version.

    Other examples:

    "mp3" - only find tracks in lossy mp3 (or with mp3 in the name - oh well)

    "playlist elvis" - find playlists with tracks by Elvis, or with Elvis in
    the name or a track's name

    "flac 88.2 mozart" - get that highres classical stuff

    The search code tries to weight results: hits in track titles have
    higher priority than hits in a track's artist or album name. Eg. "lov"
    would rank ABBA's "loving me, loving you" higher than "love me tender"
    from the "greatest love songs, ever" album. Playlist names have priority
    over track titles included in the playlist etc.

    FTS is available from both the web UI's "live search" as well as the
    regular search menus. Please give it a try and let me know what you
    think about it.


    A few caveats:

    - FTS most likely will be a bit heavier on your system, with regards to
    both the CPU as well as memory. I guess it shouldn't be used without the
    highmem preference being set to high or max.

    - FTS only works with SQLite. If you decided to use MySQL for whatever
    reason, then I'm sorry, this won't work, as it's based around SQLite's
    FTS extension

    FTS is implemented as a plugin (mostly). If you don't like it, then just
    disable the plugin to get the old behaviour back.

    --

    Michael
    Michael

    "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
    (LMS: Settings/Information)

    Comment


      #3
      + excellent idea
      --------------------------------------------------------------------
      Main hifi: Rasbery PI digi+ MeridianG68J MeridianHD621 MeridianG98DH 2 x MeridianDSP5200 MeridianDSP5200HC 2 xMeridianDSP3100 +Rel Stadium 3 sub.
      Bedroom/Office: Boom
      Loggia: Raspi hifiberry dac + Adams
      Bathroom : Radio (with battery)
      iPad with iPengHD & SqueezePad
      (spares Touch, SB3, reciever ,controller )
      server Intel NUC Esxi VM Linux mint 18 LMS 7.9.2

      http://people.xiph.org/~xiphmont/demo/neil-young.html

      Comment


        #4
        Great new feature, thanks!

        Comment


          #5
          Michael, you are awesome!
          ---
          learn more about iPeng, the iPhone and iPad remote for the Squeezebox and
          Logitech UE Smart Radio as well as iPeng Party, the free Party-App,
          at penguinlovesmusic.com
          New: iPeng 9, the Universal App for iPhone, iPad and Apple Watch

          Comment


            #6
            Originally posted by mherger View Post
            Your search no longer is limited to a simple title/name search. With the
            FTS you can now easily find track X by artist Y. Just type "hurry love
            supremes" if you want to make sure not to get Phil Collins' version.
            I've wanted this forever, thank you so much!!!

            Comment


              #7
              New in 7.9: Fulltext search

              > Michael, you are awesome!

              ....except that it doesn't work with iPeng... Only tested it after I had
              it announced here. Web UI should be fine though. I'll take care of it asap.

              --

              Michael
              Michael

              "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
              (LMS: Settings/Information)

              Comment


                #8
                Great idea but...
                ... after 40 minutes it still says "Volltext Index erstellen (0 von 5) Läuft 00:39:45" - and there is only one "dot" shown - so it should take hours, dates ...
                I know, i have a 420k track library, but on a high performance PC.
                Any idea how long full text search "usually" should do or should I stop and restart so it maybe "hanged`up"?

                last entry in logs is:
                [14-10-23 21:12:56.9772] Slim::Music::Import::runImporter (485) Starting Slim::Plugin::FullTextSearch::Plugin scan
                some 40 minutes ago ...
                Last edited by frank1969; 2014-10-23, 21:59.

                Comment


                  #9
                  Awesome concept! Where is it picking up the hirez "88.2" in a query? Is it actually looking at the files or just if that was stored in a tag? Just wondering if I could do the same for 24bit depth?
                  Mike (1 SB3, 1 Duet, 1 Boom, 2 Touch, 1 SB Radio)

                  Comment


                    #10
                    New in 7.9: Fulltext search

                    "usually" is hard to define. My 20k tracks take a second or two to index on a poor Atom based system. Even less on the Macbook Pro with SSD, of course.

                    Is there any activity going on? Eg. massive memory usage? CPU load? library.db changes?



                    > Am 23.10.2014 um 21:54 schrieb frank1969 <frank1969.6mggfb (AT) no-mx (DOT) forums.slimdevices.com>:
                    >
                    >
                    > Great idea but...
                    > ... after 40 minutes it still says "Volltext Index erstellen (0 von 5)
                    > Läuft 00:39:45" - and there is only one "dot" shown - so it should
                    > take hours, dates ...
                    > I know, i have a 420k track library, but on a high performance PC.
                    > Any idea how long full text search "usually" should do or should I stop
                    > and restart so it maybe "hanged`up"?
                    >
                    >
                    > ------------------------------------------------------------------------
                    > frank1969's Profile: http://forums.slimdevices.com/member.php?userid=31156
                    > View this thread: http://forums.slimdevices.com/showthread.php?t=102347
                    >
                    >
                    Michael

                    "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
                    (LMS: Settings/Information)

                    Comment


                      #11
                      Originally posted by mherger View Post
                      > Michael, you are awesome!

                      ....except that it doesn't work with iPeng... Only tested it after I had
                      it announced here. Web UI should be fine though. I'll take care of it asap.

                      --

                      Michael
                      And it probably won't until you tell me how the CLI command for it works
                      ---
                      learn more about iPeng, the iPhone and iPad remote for the Squeezebox and
                      Logitech UE Smart Radio as well as iPeng Party, the free Party-App,
                      at penguinlovesmusic.com
                      New: iPeng 9, the Universal App for iPhone, iPad and Apple Watch

                      Comment


                        #12
                        Yes, thanks - after 55 minutes it finished full text search (for 420k mp3-files). I will now test the feature and report if it was only the initial scan that took so "long" or if it's the same with every (update-) scan. Great idea, by the way! :-)


                        [QUOTE=mherger;794047]"usually" is hard to define. My 20k tracks take a second or two to index on a poor Atom based system. Even less on the Macbook Pro with SSD, of course.

                        Is there any activity going on? Eg. massive memory usage? CPU load? library.db changes?

                        Comment


                          #13
                          New in 7.9: Fulltext search

                          Oh, and you do use the "maximum" memory preference in Settings/Advanced/Performance?

                          Michael



                          > Am 23.10.2014 um 21:54 schrieb frank1969 <frank1969.6mggfb (AT) no-mx (DOT) forums.slimdevices.com>:
                          >
                          >
                          > Great idea but...
                          > ... after 40 minutes it still says "Volltext Index erstellen (0 von 5)
                          > Läuft 00:39:45" - and there is only one "dot" shown - so it should
                          > take hours, dates ...
                          > I know, i have a 420k track library, but on a high performance PC.
                          > Any idea how long full text search "usually" should do or should I stop
                          > and restart so it maybe "hanged`up"?
                          >
                          >
                          > ------------------------------------------------------------------------
                          > frank1969's Profile: http://forums.slimdevices.com/member.php?userid=31156
                          > View this thread: http://forums.slimdevices.com/showthread.php?t=102347
                          >
                          >
                          Michael

                          "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
                          (LMS: Settings/Information)

                          Comment


                            #14
                            Ok, first thing I recognized:
                            New artists are not to be found.

                            Yesterday I added some tracks from (e.g.) Willy Wagner or Tina (02) - I didn't own any tracks of them before (as "co"-artist, separated with ";" as I always do)

                            If I call the album, I see performer "Willy Wagner", can click on the name and it shows me I own own album he contributes tracks too - fine, works as it should!

                            If I search for "Willy Wagner", it finds - nothing.

                            If I browse my artist list, it finds - nothing!


                            This behaviour seems to affect only new added TRACK ARTIST .

                            I also added an album by an artist I did own nothing from before (Ric Seroka) so he is mentioned as ("single") track artist and ALBUM artist - and everyhting is finde.
                            I can search for his name and I find him in the artist's list...

                            Comment


                              #15
                              285

                              Originally posted by mherger View Post
                              Something new to play with: full text search (FTS).

                              Your search no longer is limited to a simple title/name search. With the
                              FTS you can now easily find track X by artist Y. Just type "hurry love
                              supremes" if you want to make sure not to get Phil Collins' version.

                              Other examples:

                              "mp3" - only find tracks in lossy mp3 (or with mp3 in the name - oh well)

                              "playlist elvis" - find playlists with tracks by Elvis, or with Elvis in
                              the name or a track's name

                              "flac 88.2 mozart" - get that highres classical stuff

                              The search code tries to weight results: hits in track titles have
                              higher priority than hits in a track's artist or album name. Eg. "lov"
                              would rank ABBA's "loving me, loving you" higher than "love me tender"
                              from the "greatest love songs, ever" album. Playlist names have priority
                              over track titles included in the playlist etc.

                              FTS is available from both the web UI's "live search" as well as the
                              regular search menus. Please give it a try and let me know what you
                              think about it.


                              A few caveats:

                              - FTS most likely will be a bit heavier on your system, with regards to
                              both the CPU as well as memory. I guess it shouldn't be used without the
                              highmem preference being set to high or max.

                              - FTS only works with SQLite. If you decided to use MySQL for whatever
                              reason, then I'm sorry, this won't work, as it's based around SQLite's
                              FTS extension

                              FTS is implemented as a plugin (mostly). If you don't like it, then just
                              disable the plugin to get the old behaviour back.

                              --

                              Michael
                              Sweet!

                              Comment

                              Working...
                              X