Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 2 12 LastLast
Results 1 to 10 of 13
  1. #1
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821

    Strange behaviour searching with ['songs', ] via jsonrpc

    I notice that if I search my library using ['songs', 0, 500, 'search:' + what, 'tags:Aal'] where what is a single word like Chicane, the search results only match in the title tag. So songs such as 'She's Got a Way (Chicane Mix)' will match because the artist Chicane is mentioned in the title tag.

    However, if what consists of more than one word (e.g. Bob Seger, Armin van Buuren, Everything but the Girl) then the search matches in the artist tag also and I can successfully use it to peruse all tracks by that artist.
    I know I can use artist_id with the API but the problem is that collaborations result in extra artist_ids for an artist in combination with somebody else. The artist_id for the bare artist will miss those tracks.

    The Full Text Search plugin is enabled. This behavious eems to be version independent because I also had this back on 7.9.2 but FWIW I'm on the latest debian nightly.

    Is this a bug or am I somehow misusing the songs query?

  2. #2
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,605

    Strange behaviour searching with ['songs',] via jsonrpc

    > Is this a bug or am I somehow misusing the *songs* query?

    I'm not sure I understand the problem. The FTS would indeed lead to
    finding an artist for tracks. But what is strange in what you're seeing?

  3. #3
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821
    Quote Originally Posted by mherger View Post
    > Is this a bug or am I somehow misusing the *songs* query?

    I'm not sure I understand the problem. The FTS would indeed lead to
    finding an artist for tracks. But what is strange in what you're seeing?
    The results are different when the search item contains a single word vs two or more words. A single word ONLY searches the title tag. Multiple words results in searching ALL tags.

  4. #4
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821
    I also see this behaviour with the ['albums', ] query. With 'search: xxx' as parameter, the results will depend on whether xxx contains whitespace or not. No whitespace in xxx means only the album tag is searched whereas with whitespace the match can occur in any tag.

    Example. I have 20 albums by Chicane in my local LMS library. I perform ['albums', 0, 250, 'search: chicane', 'tags:Aalyj']. There are only 3 results returned and not 20 ("Chicane pres. 'Visions of Ibiza'", The Best of Chicane (1996-2008)", and "Ultimate Dance (Mixed by Chicane)" all of which are matching because the search term is in the ALBUM tag.

    I have 12 albums by Steve Winwood. I perform ['albums', 0, 250, 'search: steve winwood', 'tags:Aalyj']. All 12 are returned. The only one that has Steve Winwood in the ALBUM tag is "Revolutions: The Very Best of Steve Winwood". Yet all the others match in the ARTIST tag and are returned.

    The question is: Why does the search behave differently depending on the number of words in the seach term (i.e. whitespace or not)?

    FYI the same happens with *any* artist with a single word name like Madonna, Sting, U2, Rihanna, Cher, Dido, INXS so it's not just this one isolated case.

  5. #5
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821
    Nobody?

    What it boils down to is that the FTS plugin does not operate unless there is whitespace in the search term.

  6. #6
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,605

    Strange behaviour searching with ['songs',] via jsonrpc

    > What it boils down to is that the FTS plugin does not operate unless
    > there is whitespace in the search term.


    There are some optimizations in place:

    - single character only checks in main columns only (no real FTS)
    - keyword(s) plus single character does respect that character, too
    - single, popular terms (found in more than 500 records) would search in
    main columns only

    "Main columns": FTS can define different weights for different values.
    Eg. for an artist the name would have the heighest weight. For tracks
    where he contributed to he'd be found in a column with lower weight. Now
    if "Chicane" was a contributor on >500 tracks, in comments, album names
    whetever, that would be considered a "popular term". In this case that
    search would only return results where he was in the name or track
    title, but not where he was listed as contributor.

    If you enable debug logging for plugin.fulltext you'll get warnings in
    those cases where the FTS is optimized away.

  7. #7
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821
    Quote Originally Posted by mherger;
    > What it boils down to is that the FTS plugin does not operate unless
    > there is whitespace in the search term.


    There are some optimizations in place:

    - single character only checks in main columns only (no real FTS)
    - keyword(s) plus single character does respect that character, too
    - single, popular terms (found in more than 500 records) would search in
    main columns only

    "Main columns": FTS can define different weights for different values.
    Eg. for an artist the name would have the heighest weight. For tracks
    where he contributed to he'd be found in a column with lower weight. Now
    if "Chicane" was a contributor on >500 tracks, in comments, album names
    whetever, that would be considered a "popular term". In this case that
    search would only return results where he was in the name or track
    title, but not where he was listed as contributor.

    If you enable debug logging for plugin.fulltext you'll get warnings in
    those cases where the FTS is optimized away.
    Thank Michael. Chicane is indeed present in >500 places across the columns so that would explain it being a 'popular term' for optimisation.

    I never realised that FTS was actually a SQLite thing and not LMS_specific. At https://www.sqlite.org/fts5.html#full_text_query_syntax I found that I can force a search in a particular column(s) by including the column name.

    So I can use e.g. "search:{artist,album}:chicane" to specify the columns I want to always include. If, however, FTS is not enabled this fails and returns no matches whatsoever.

    Which leads me to: is there any way from the jsonrpc.js interface I can check if FTS is enabled?
    Last edited by philchillbill; 2021-07-02 at 00:29.

  8. #8
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,605

    Strange behaviour searching with ['songs',] via jsonrpc

    > Which leads me to: is there any way from the jsonrpc.js interface I can
    > check if FTS is enabled?


    Oh... didn't you ask this in another thread already? And I decided to
    silently ignore it?...

    TBH: I don't know of a well defined way to figure this out using any
    existing query. But it would be easy to add one. Would that be of any
    help? Something like 'fulltextsearch ?'?

  9. #9
    Senior Member philchillbill's Avatar
    Join Date
    Jan 2019
    Location
    The Netherlands
    Posts
    821
    Quote Originally Posted by mherger View Post
    > Which leads me to: is there any way from the jsonrpc.js interface I can
    > check if FTS is enabled?


    Oh... didn't you ask this in another thread already? And I decided to
    silently ignore it?...

    TBH: I don't know of a well defined way to figure this out using any
    existing query. But it would be easy to add one. Would that be of any
    help? Something like 'fulltextsearch ?'?
    That would be wonderful 😎

  10. #10
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,605

    Strange behaviour searching with ['songs',] via jsonrpc

    In the upcoming 8.2 build:

    "fulltextsearch ?" => "fulltextsearch 1": FTS available
    "fulltextsearch ?" => "fulltextsearch ?|0": FTS not available

    Always make sure that you check against "1", as if the plugin was
    missing/disabled, LMS would simply echo the request, returning a
    "truthy" value (in some languages like JS) "?" as the result.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •