Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 3 123 LastLast
Results 1 to 10 of 21
  1. #1
    Senior Member jimzak's Avatar
    Join Date
    May 2008
    Location
    Austin TX
    Posts
    656

    Unable to perform full rescan

    I have over 400k files on two 8 TB hard drives.

    Recent complete rescans have stalled at the point you see below:

    Logitech Media Server Version: 7.9.2 - 1565967976 @ Fri Aug 16 17:08:05 WEDT 2019
    Hostname: musicserver2019
    Server IP Address: 192.168.0.xxx
    Server HTTP Port Number: 9000
    Operating system: Windows 10 - EN - cp1252
    Platform Architecture: 8664
    Perl Version: 5.14.1 - MSWin32-x86-multi-thread
    Audio::Scan: 0.95
    IO::Socket::SSL: 2.024
    Database Version: DBD::SQLite 1.58 (sqlite 3.22.0)
    Total Players Recognized: 7

    Media Scan Details
    Discovering files/directories: J: (231441 of 231441) Complete 00:05:00

    Scanning new music files: J: (206239 of 206239) Complete 00:46:43

    Discovering files/directories: K: (272071 of 272071) Complete 00:06:43

    Scanning new music files: K: (244656 of 244656) Complete 01:09:26

    Discovering files/directories: K:\V-Z\Various (1 of 1) Complete 00:00:00

    Discovering playlists: C:\Users\user\Music\Playlists (1 of 1) Complete 00:00:00

    Building full text index (1 of 7) Running 01:37:20

    Scanner log:

    [19-08-29 07:47:11.8381] Slim::Utils::Scanner::Local::__ANON__ (229) Re-build temporary table
    [19-08-29 07:47:13.4713] Slim::Utils::Scanner::Local::__ANON__ (278) Get deleted tracks count
    [19-08-29 07:47:13.4716] Slim::Utils::Scanner::Local::__ANON__ (285) Get new tracks count
    [19-08-29 07:47:13.4719] Slim::Utils::Scanner::Local::__ANON__ (290) Get changed tracks count
    [19-08-29 07:47:13.4720] Slim::Utils::Scanner::Local::__ANON__ (296) Removing deleted audio files (0)
    [19-08-29 07:47:13.4722] Slim::Utils::Scanner::Local::__ANON__ (374) Scanning new audio files (0)
    [19-08-29 07:47:13.4723] Slim::Utils::Scanner::Local::__ANON__ (454) Rescanning changed audio files (0)
    [19-08-29 07:47:13.4725] Slim::Music::Import::endImporter (643) Completed Slim::Music::PlaylistFolderScan Scan in 1 seconds.
    [19-08-29 07:47:13.4739] Slim::Music::Import::runImporter (511) Starting Slim::Plugin::FullTextSearch::Plugin scan
    [19-08-29 07:47:13.4744] Slim::Plugin::FullTextSearch::Plugin::_rebuildInde x (413) Starting fulltext index build
    [19-08-29 07:47:13.4746] Slim::Plugin::FullTextSearch::Plugin::_rebuildInde x (417) Initialize fulltext table
    [19-08-29 07:47:13.4760] Slim::Plugin::FullTextSearch::Plugin::_rebuildInde x (430) Create fulltext index for tracks

    Any diagnostic or therapeutic help would be appreciated.
    http://zzzone.net
    http://have-a-nice-day.org
    http://www.last.fm/user/zzzoneDOTnet
    http://somethingsomethingsomething.net

    SBS 7.9 - i7 nuc - Win 10 64bit
    5 Booms, 2 Radio, 3 Touch, 1 Duet, 5 piCorePlayers including 3 touchscreen, 1 Avy
    2 controllers, various tablets/phones
    Apps including iPeng, Squeeze Ctrl etc.
    Library: 425,000+ FLAC/MP3 files - 18 TB HD

  2. #2
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,315

    Unable to perform full rescan

    > I have over 400k files on two 8 TB hard drives.

    How large are your library.db* files? How much free disk space do you
    have on the drive where your LMS cache is?

    --

    Michael

  3. #3
    Senior Member jimzak's Avatar
    Join Date
    May 2008
    Location
    Austin TX
    Posts
    656
    Quote Originally Posted by mherger View Post
    > I have over 400k files on two 8 TB hard drives.

    How large are your library.db* files? How much free disk space do you
    have on the drive where your LMS cache is?

    --

    Michael
    Thanks for replying.

    The cache is 3.16 GB on an 8 GB RAM drive that I have been using for several years. Full rescans have worked well over the years until now.

    However, I found a workaround.

    After the full scan stalled, I aborted it. I then ran a "new and changed media" scan which completed.

    Total Tracks: 452,925
    Total Albums: 45,690
    Total Artists: 66,298
    Total Genres: 2,236
    Total Playing Time: 49108:12:58

    Media Scan Details
    Discovering files/directories: J: (231441 of 231441) Complete 00:06:05

    Scanning changed files: J: (206192 of 206192) Complete 00:52:33

    Discovering files/directories: K: (272071 of 272071) Complete 00:06:59

    Scanning new music files: K: (1 of 1) Complete 00:00:00

    Scanning changed files: K: (244634 of 244634) Complete 01:20:27

    Discovering files/directories: K:\V-Z\Various (1 of 1) Complete 00:00:00

    Discovering playlists: C:\Users\user\Music\Playlists (1 of 1) Complete 00:00:00

    Building full text index (7 of 7) Complete 00:01:55

    Find updated coverart files (45636 of 45636) Complete 00:05:35

    Pre-caching Artwork (47037 of 47037) Complete 00:34:19

    Database Optimize (2 of 2) Complete 00:00:42

    -----------

    Unclear why I had to do this.
    http://zzzone.net
    http://have-a-nice-day.org
    http://www.last.fm/user/zzzoneDOTnet
    http://somethingsomethingsomething.net

    SBS 7.9 - i7 nuc - Win 10 64bit
    5 Booms, 2 Radio, 3 Touch, 1 Duet, 5 piCorePlayers including 3 touchscreen, 1 Avy
    2 controllers, various tablets/phones
    Apps including iPeng, Squeeze Ctrl etc.
    Library: 425,000+ FLAC/MP3 files - 18 TB HD

  4. #4
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,315

    Unable to perform full rescan

    > The cache is 3.16 GB on an 6 GB RAM drive that I have been using for
    > several years. Full rescans have worked well over the years until now.


    That's a large file, and the RAM drive probably not large enough. At
    some stages the temporary files created by the scanner can become as
    large as the main library. Could you try to increase the RAM disk's
    size, or put the cache back on your main drive? Or at least monitor disk
    usage while in the critical phase of the scan?

    --

    Michael

  5. #5
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,315

    Unable to perform full rescan

    >>> I have over 400k files on two 8 TB hard drives.
    ....
    >
    > The cache is 3.16 GB on an 6 GB RAM drive that I have been using for


    Thinking about this again... 3GB is HUGE, even for 400k tracks. I've got
    20k, library.db is about 40M. Assuming 20x the tracks generate 20x the
    data, this should still be less than 1GB. That's >8k of information per track.
    Even with Lyrics this seems like a lot (artwork is stored separately).

    Next time you plan to do a full wipe & scan I'd suggest you shut down
    LMS and delete library.db*.

    --

    Michael
    Last edited by mherger; 2019-08-30 at 00:42.

  6. #6
    This looks familiar to me: https://forums.slimdevices.com/showt...ems&highlight=

    My .db files were not that big, recently I tried a full rescan after removal all of them, but it still didn't work.
    My workaround is to choose "Looking for new and changed media files" when rescanning.
    Living Room: HifiBerry DAC+ Pro & piCorePlayer
    Attic: HifiBerry DAC+ RCA & piCorePlayer
    Kitchen: SB Radio
    Other rooms: 4x SB Radio

  7. #7
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,315

    Unable to perform full rescan

    > This looks familiar to me:
    > https://forums.slimdevices.com/showt...ems&highlight=


    You're posting too many references to other threads... that thread you
    link here is pointing to yet another one, which in turn would link back
    to yours... Maybe you should be more specific.

    --

    Michael

  8. #8
    Senior Member jimzak's Avatar
    Join Date
    May 2008
    Location
    Austin TX
    Posts
    656
    Quote Originally Posted by mherger View Post
    >>> I have over 400k files on two 8 TB hard drives.
    ....
    >
    > The cache is 3.16 GB on an 6 GB RAM drive that I have been using for


    Thinking about this again... 3GB is HUGE, even for 400k tracks. I've got
    20k, library.db is about 40M. Assuming 20x the tracks generate 20x the
    data, this should still be less than 1GB. That's >8k of information per track.
    Even with Lyrics this seems like a lot (artwork is stored separately).

    Next time you plan to do a full wipe & scan I'd suggest you shut down
    LMS and delete library.db*.

    --

    Michael
    Very interesting.

    I have displayed the contents of my cache below. Let me know if something sticks out as being strange. My RAM drive is 8 GB (6 was a typo). And deleting library.db is exactly how I initiated the failed rescans.

    Name:  cache.jpg
Views: 162
Size:  54.3 KB
    http://zzzone.net
    http://have-a-nice-day.org
    http://www.last.fm/user/zzzoneDOTnet
    http://somethingsomethingsomething.net

    SBS 7.9 - i7 nuc - Win 10 64bit
    5 Booms, 2 Radio, 3 Touch, 1 Duet, 5 piCorePlayers including 3 touchscreen, 1 Avy
    2 controllers, various tablets/phones
    Apps including iPeng, Squeeze Ctrl etc.
    Library: 425,000+ FLAC/MP3 files - 18 TB HD

  9. #9
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,315

    Unable to perform full rescan

    > I have displayed the contents of my cache below. Let me know if
    > something sticks out as being strange. My RAM drive is 8 GB (6 was a
    > typo). And deleting library.db is exactly how I initiated the failed
    > rescans.


    Ah, ok. That 3.16GB file is not library.db, but artwork.db. That makes
    sense (as does the size of library.db).

    But still I'd monitor disk space next time you run a full scan.
    Optimizing those files can double the disk space requirement temporarily.

    --

    Michael

  10. #10
    Senior Member jimzak's Avatar
    Join Date
    May 2008
    Location
    Austin TX
    Posts
    656

    scan does not compete

    I took the RAM disk offline and did a complete resinstall of the server, latest beta.

    The scan still does not complete.

    It seems to be choking on damaged jpeg image files.

    The reason for damaged files relates to an old hard drive enclosure that I was previously using. Apparently the USB controller was damaged or just became ill and started damaging any files that passed through it, likely on the way in from my network.

    I didn't discover the problem for while, because, I think, very few people verify that files are copied correctly. When I finally did, I replaced the hardware, but the damage was done.

    Not only was the damage done, but the backups were damaged too.

    I have the original files, which I am slowly restoring from 25 GB Blu-ray recordables.

    However, in the meantime, it would be nice if I could get the server to ignore the damaged jpegs and complete the scan.

    Any help would be appreciated.

    Addendum: the Squeezesonic plugin at least lets me access my collection while we are sorting this out.

    -------------------------

    Total Tracks: 464,440
    Total Albums: 46,302
    Total Artists: 66,431
    Total Genres: 2,257
    Total Playing Time: 52416:35:56

    Media Scan Details
    Discovering files/directories: J: (235549 of 235549) Complete 00:05:22

    Scanning new music files: J: (210062 of 210062) Complete 00:42:56

    Discovering files/directories: K: (280101 of 280101) Complete 00:07:25

    Scanning new music files: K: (252086 of 252086) Complete 00:53:31

    Discovering files/directories: K:\V-Z\Various (1 of 1) Complete 00:00:00

    Discovering playlists: C:\Users\user\Music\Playlists (1 of 1) Complete 00:00:00

    Building full text index (7 of 7) Complete 00:01:16

    Pre-caching Artwork (16132 of 47561) Complete 00:08:35

    Media scan terminated unexpectedly (Scanning Progress)
    Total Time: 01:59:05 (Sunday, September 15, 2019 / 4:23 PM)

    Excerpt from scanner log:

    [19-09-15 16:16:35.2738] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2739] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2741] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2742] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2744] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2746] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2747] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2749] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2750] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2752] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2753] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2755] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2756] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2758] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2759] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Ignoring bad adaptive filter type (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2764] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Extra compressed data (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:35.2765] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libpng warning: Extra compression data (K:\Various A-L\Hercules and Love Affair - DJ-Kicks\10. In Flagranti - Magojiro (Original Mix).mp3)
    [19-09-15 16:16:40.4238] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libjpeg error: Corrupt JPEG data: premature end of data segment (J:\Misc\Dance\Porterhouse\Deetron - Re-Creation Remixes Compiled-\01 - Deetron, Justin Chapman - Let's Get Over It Feat. Justin Chapman (Deetron Unreleased Remix).flac)
    [19-09-15 16:16:42.4925] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libjpeg error: Corrupt JPEG data: bad Huffman code (K:\T\The Threshold HouseBoys Choir - The Threshold Houseboys Choir Amulet Edition\CD B\02_-_Threshold_HouseBoys_Choir,_The_-_As_X_Is_To_Geff.flac)
    [19-09-15 16:16:43.3792] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libjpeg error: Corrupt JPEG data: 2 extraneous bytes before marker 0xd9 (K:\T\Two Lone Swordsmen - Wrong Meeting (2007) (FLAC)\09 - Two Lone Swordsmen - Get Out Of My Kingdom.flac)
    [19-09-15 16:16:46.6143] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libjpeg error: Corrupt JPEG data: 53 extraneous bytes before marker 0xd9 (K:\T\Tally Hall - Good & Evil\14_-_Tally_Hall_-_Fate_Of_The_Stars.flac)
    [19-09-15 16:16:54.0967] Slim::Utils::GDResizer::resize (205) Warning: Image::Scale libjpeg error: Corrupt JPEG data: 227 extraneous bytes before marker 0xd9 (K:\T\Thanatos - Blisters\11_-_Thanatos_-_Dry_Gray_Whisper.flac)
    Last edited by jimzak; 2019-09-15 at 16:07.
    http://zzzone.net
    http://have-a-nice-day.org
    http://www.last.fm/user/zzzoneDOTnet
    http://somethingsomethingsomething.net

    SBS 7.9 - i7 nuc - Win 10 64bit
    5 Booms, 2 Radio, 3 Touch, 1 Duet, 5 piCorePlayers including 3 touchscreen, 1 Avy
    2 controllers, various tablets/phones
    Apps including iPeng, Squeeze Ctrl etc.
    Library: 425,000+ FLAC/MP3 files - 18 TB HD

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •