Home of the Squeezebox™ & Transporter® network music players.
Page 6 of 8 FirstFirst ... 45678 LastLast
Results 51 to 60 of 79
  1. #51
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > Mine's a bit over 700MB so not so sure about that.

    Feel free to upload it to

    https://www.dropbox.com/request/T3RctyzGgNg0oFDubq6a

    > Largest playlist file is 583K -- i'm sure some of the lists are pointing


    I assume that's 583KB, not 583'000 tracks?...

    > to files that may no longer exist. As far as length, some playlists
    > over 8 hours or so if you are talking about length. I have 130 playlsts
    > right now, but nothing changed since maybe 3 months ago as far as new
    > playlists.


    FTS did change. I broke it at some point in the past: it wouldn't index
    tracks in a playlist any more. Now that I've fixed this, users with
    large playlists are suffering the issues you're seeing, as something
    about the indexing is super busy.

    --

    Michael

  2. #52
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > There's nothing unusual in either the scanner or sever logs (attached).

    Ok, figured something out: I was looking up stuff in the FTS index by
    the track ID. But in some library versions this value would NOT be
    indexed. Which means, that in those case the database had to do a slow
    scan instead of a quick lookup.

    I've committed a change to 8.0 which should mitigate this issue. Please
    give it a try and let me know about your experience.

    > One minor problem - practically all of my artists have had face
    > transplants.


    Isn't this a browser side caching issue?

    --

    Michael

  3. #53
    Senior Member kidstypike's Avatar
    Join Date
    Feb 2007
    Location
    Brindle
    Posts
    4,010
    Quote Originally Posted by mherger View Post
    > There's nothing unusual in either the scanner or sever logs (attached).

    Ok, figured something out: I was looking up stuff in the FTS index by
    the track ID. But in some library versions this value would NOT be
    indexed. Which means, that in those case the database had to do a slow
    scan instead of a quick lookup.

    I've committed a change to 8.0 which should mitigate this issue. Please
    give it a try and let me know about your experience.
    Not so good, logs attached

    Name:  Screenshot 2020-09-24 154830.jpg
Views: 104
Size:  160.4 KB
    Attached Files Attached Files
    Server - LMS 8.0.1 Pi4B 4GB/Flirc case/pCP v7.0.0b6 - 74K library, playlists & LMS cache on SSD (ntfs)
    Study - Pi3B/pCP 7.0.0b6/pi screen/Allo Boss DAC Ruark MR1 Mk2
    Lounge - Pi2/pCP 6.0.0 > HiFiBerry DIGI+ > AudioEngine DAC1 > AVI DM5
    Garage - Squeezebox Boom + Fostex sub
    Dining Room - Squeezebox Radio

    Spares - 2xTouch, 1xSB3, 7xRPi

  4. #54
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > Not so good, logs attached

    Thanks. Stupid mistake of mine (hopefully). There should be new builds
    out in a bit. Thanks for testing!

    --

    Michael

  5. #55
    Senior Member kidstypike's Avatar
    Join Date
    Feb 2007
    Location
    Brindle
    Posts
    4,010
    Quote Originally Posted by mherger View Post
    > Not so good, logs attached

    Thanks. Stupid mistake of mine (hopefully). There should be new builds
    out in a bit. Thanks for testing!
    No better I'm afraid.

    Name:  Screenshot 2020-09-24 192308.jpg
Views: 91
Size:  155.2 KB
    Attached Files Attached Files
    Server - LMS 8.0.1 Pi4B 4GB/Flirc case/pCP v7.0.0b6 - 74K library, playlists & LMS cache on SSD (ntfs)
    Study - Pi3B/pCP 7.0.0b6/pi screen/Allo Boss DAC Ruark MR1 Mk2
    Lounge - Pi2/pCP 6.0.0 > HiFiBerry DIGI+ > AudioEngine DAC1 > AVI DM5
    Garage - Squeezebox Boom + Fostex sub
    Dining Room - Squeezebox Radio

    Spares - 2xTouch, 1xSB3, 7xRPi

  6. #56
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > No better I'm afraid.

    This sucks. I'm using the same SQLite version as you, and I can run
    those queries on your library just fine. I'll give it another try in a
    build later today. I'm wondering whether I'm missing something here. The
    log file seems to indicate there's a problem with a string in the query.
    But as I said: I don't see this issue with your library :-(.

    Thanks for testing!


    --

    Michael

  7. #57
    Code:
     Discovering files/directories: /mnt/mother/mp3   (173263 of 173263)   Complete  00:07:59
    
    Scanning new music files: /mnt/mother/mp3   (1 of 1)   Complete  00:00:00
    
    Discovering playlists: /mnt/mother/! Playlists   (130 of 130)   Complete  00:00:01
    
    Building full text index   (3 of 7)   Complete  00:01:42
    
    Media scan terminated unexpectedly (Scanning Progress)
    Total Time: 00:09:42 (Thursday, September 24, 2020 / 7:53 PM)

  8. #58
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > Media scan terminated unexpectedly (Scanning Progress)

    Please share your scanner.log. Thanks!

    --

    Michael

  9. #59
    Junior Member
    Join Date
    Nov 2019
    Posts
    17
    Code:
    [20-09-25 18:29:07.5107] Slim::Plugin::FullTextSearch::Plugin::_rebuildIndex (479) Create fulltext index for playlists
    [20-09-25 18:29:10.5696] Slim::Schema::Storage::throw_exception (121) Error: DBI Exception: DBD::SQLite::db do failed: malformed MATCH expression: [file:///Volumes/Ade_iTunes/iTunes/iTunes Media/Music/Metz/Metz/11 --))--.m4a] [for Statement "
    	INSERT  INTO fulltext (id, type, w10, w5, w3, w1)
    		SELECT playlist_track.playlist, 'playlist', ?, '', ?, GROUP_CONCAT(w10 || ' ' || w5 || ' ' || w3 || ' ' || w1)
    		FROM playlist_track
    			LEFT JOIN tracks ON tracks.url = playlist_track.track
    			LEFT JOIN fulltext ON fulltext MATCH REPLACE(playlist_track.track, '%20', ' ') AND type = 'track'
    		WHERE playlist_track.playlist = ?
    "]
    [20-09-25 18:29:10.5700] Slim::Schema::Storage::throw_exception (121) Backtrace:
    
       frame 0: Slim::Utils::Log::logBacktrace (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Schema/Storage.pm line 121)
       frame 1: Slim::Schema::Storage::throw_exception (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/CPAN/DBIx/Class/Storage/DBI.pm line 1006)
       frame 2: DBIx::Class::Storage::DBI::__ANON__ (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Plugin/FullTextSearch/Plugin.pm line 520)
       frame 3: Slim::Plugin::FullTextSearch::Plugin::_createPlaylistItem (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Plugin/FullTextSearch/Plugin.pm line 488)
       frame 4: Slim::Plugin::FullTextSearch::Plugin::_rebuildIndex (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Plugin/FullTextSearch/Plugin.pm line 157)
       frame 5: Slim::Plugin::FullTextSearch::Plugin::startScan (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Music/Import.pm line 579)
       frame 6: Slim::Music::Import::runImporter (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Music/Import.pm line 455)
       frame 7: Slim::Music::Import::runScanPostProcessing (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/scanner.pl line 344)
       frame 8: (eval) (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/scanner.pl line 344)
       frame 9: main::main (/Library/PreferencePanes/Squeezebox.prefPane/Contents/server/scanner.pl line 499)
    
    [20-09-25 18:29:10.5724] main::main (348) Error: Failed when running scan post-process: [Carp::Clan::__ANON__(): DBI Exception: DBD::SQLite::db do failed: malformed MATCH expression: [file:///Volumes/Ade_iTunes/iTunes/iTunes Media/Music/Metz/Metz/11 --))--.m4a] [for Statement "
    	INSERT  INTO fulltext (id, type, w10, w5, w3, w1)
    		SELECT playlist_track.playlist, 'playlist', ?, '', ?, GROUP_CONCAT(w10 || ' ' || w5 || ' ' || w3 || ' ' || w1)
    		FROM playlist_track
    			LEFT JOIN tracks ON tracks.url = playlist_track.track
    			LEFT JOIN fulltext ON fulltext MATCH REPLACE(playlist_track.track, '%20', ' ') AND type = 'track'
    		WHERE playlist_track.playlist = ?
    "] at /Library/PreferencePanes/Squeezebox.prefPane/Contents/server/Slim/Schema/Storage.pm line 125
    ]
    [20-09-25 18:29:10.5726] main::main (349) Error: Not updating lastRescanTime!

  10. #60
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,472

    Scan taking hours instead ofminutes on recent nightlies

    > [20-09-25 18:29:07.5107] Slim::Plugin::FullTextSearch::Plugin::_rebuildInde x (479) Create fulltext index for playlists
    > [20-09-25 18:29:10.5696] Slim::Schema::Storage::throw_exception (121) Error: DBI Exception: DBD::SQLite::db do failed: malformed MATCH expression: [file:///Volumes/Ade_iTunes/iTunes/iTunes Media/Music/Metz/Metz/11 --))--.m4a] [for Statement "


    What LMS build is this?

    --

    Michael

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •