Home of the Squeezebox™ & Transporter® network music players.
Page 4 of 5 FirstFirst ... 2345 LastLast
Results 31 to 40 of 47
  1. #31
    Junior Member
    Join Date
    Oct 2021
    Location
    Athens, Greece
    Posts
    16
    Currently I am doing some tests in order to understand the whole scanning process and spot possible problematic files.

    Question 1: Is there a simple way to do a backup (and restore) the scanning results of my music library?

    Question 2: Is there any option to have more detailed scan.log file?
    Watching to scan.log during the scanning process, sometimes I see that stays for a very long time in a specific file and looks like either the file is corrupted or the scanning process has stopped or crashed. I recently noticed that although scanning looks as stopped or crashed (no activity indication or errors, warnings etc. in the scan.log) the process finishes successfully after several hours.
    Last edited by KaPa; 2021-10-16 at 14:08.

  2. #32
    Senior Member
    Join Date
    Apr 2005
    Location
    UK/London
    Posts
    4,904
    1:
    Code:
     ls -lrt Cache/lib*
    -rw-rw-r--    1 tc       staff    155197440 Oct 14 15:05 Cache/library.db
    -rw-rw-r--    1 tc       staff            0 Oct 16 20:28 Cache/library.db-wal
    -rw-rw-r--    1 tc       staff        32768 Oct 16 20:28 Cache/library.db-shm
    Stopping LMS and copying those files should give you a backup. The scanning process can result in other files being created (e.g. downloading artist images) but my guess is that this is not important for your investigation.
    I don't know if it is sufficient to restore old copies of those files.

    2:
    Settings/Advanced/Logging/scan...
    Increase the logging level
    Paul Webster
    Author of "Now Playing" plugins covering Radio France (FIP etc), PlanetRadio (Bauer - Kiss, Absolute, Scala, JazzFM etc), KCRW, ABC Australia and CBC/Radio-Canada
    and, via the extra "Radio Now Playing" plugin, AllZIC Radio, Caroline Flashback, Cesky rozhlas, Dandelion Radio, Europa Radio Jazz, Frequence3, FSK, JukeRadio, KlassikRadio.de, Linn Radio, Mother Earth, Naim Radio, Nova.fr, RadioBlues Flac, Radio Espace, Radio Swiss, RTE Ireland, Somehow Jazz, Supla Finland

  3. #33
    Junior Member
    Join Date
    Oct 2021
    Location
    Athens, Greece
    Posts
    16
    Quote Originally Posted by Paul Webster View Post
    1:
    Code:
     ls -lrt Cache/lib*
    -rw-rw-r--    1 tc       staff    155197440 Oct 14 15:05 Cache/library.db
    -rw-rw-r--    1 tc       staff            0 Oct 16 20:28 Cache/library.db-wal
    -rw-rw-r--    1 tc       staff        32768 Oct 16 20:28 Cache/library.db-shm
    Stopping LMS and copying those files should give you a backup. The scanning process can result in other files being created (e.g. downloading artist images) but my guess is that this is not important for your investigation.
    I don't know if it is sufficient to restore old copies of those files.

    2:
    Settings/Advanced/Logging/scan...
    Increase the logging level
    Thanks a lot Paul.

    1. The full scan of my library took more than 12 hours. My intention is to keep all necessary data to restore it after my tests, if possible. Do you think that a full rescan is better, than restoring my current files when finish my tests?

    2. I suppose that the highest level of logging level is "debug". Isn't it?

  4. #34
    Senior Member
    Join Date
    Apr 2005
    Location
    UK/London
    Posts
    4,904
    Yes
    Paul Webster
    Author of "Now Playing" plugins covering Radio France (FIP etc), PlanetRadio (Bauer - Kiss, Absolute, Scala, JazzFM etc), KCRW, ABC Australia and CBC/Radio-Canada
    and, via the extra "Radio Now Playing" plugin, AllZIC Radio, Caroline Flashback, Cesky rozhlas, Dandelion Radio, Europa Radio Jazz, Frequence3, FSK, JukeRadio, KlassikRadio.de, Linn Radio, Mother Earth, Naim Radio, Nova.fr, RadioBlues Flac, Radio Espace, Radio Swiss, RTE Ireland, Somehow Jazz, Supla Finland

  5. #35
    Senior Member
    Join Date
    May 2010
    Location
    London, UK
    Posts
    923
    Quote Originally Posted by KaPa View Post
    1. The full scan of my library took more than 12 hours.
    Does the "Media Scan Details" section of the Server settings page offer any clue about where it is spending its time ?

    Example, on a test system with a *very* small library:

    Name:  Screenshot 2021-10-17 at 12.34.08.png
Views: 58
Size:  50.3 KB

  6. #36
    Senior Member
    Join Date
    Apr 2005
    Location
    UK/London
    Posts
    4,904
    Yes - they are done in sequence.
    Paul Webster
    Author of "Now Playing" plugins covering Radio France (FIP etc), PlanetRadio (Bauer - Kiss, Absolute, Scala, JazzFM etc), KCRW, ABC Australia and CBC/Radio-Canada
    and, via the extra "Radio Now Playing" plugin, AllZIC Radio, Caroline Flashback, Cesky rozhlas, Dandelion Radio, Europa Radio Jazz, Frequence3, FSK, JukeRadio, KlassikRadio.de, Linn Radio, Mother Earth, Naim Radio, Nova.fr, RadioBlues Flac, Radio Espace, Radio Swiss, RTE Ireland, Somehow Jazz, Supla Finland

  7. #37
    Junior Member
    Join Date
    Oct 2021
    Location
    Athens, Greece
    Posts
    16
    Quote Originally Posted by mrw View Post
    Does the "Media Scan Details" section of the Server settings page offer any clue about where it is spending its time ?

    Example, on a test system with a *very* small library:
    The sequence is the same but I have noticed that the scan sticks for hours (usually in a specific artist and specific song) in the second action line "Scanning new music files". In my previous tests (before summer) I thought that the process will never end and I was trying to abort the scan, taking into account that the scanner.log was referring nothing about these particular files.

    That is the why I want to try different level of details to discover what causes this delay.

    I know that some of my files have problematic tags. But these are recognized in the scanning process and there are warnings in the scanner.log. I am thinking that another reason might be the long time it takes to find pictures for artists (especially Greek). As far as I understand there is an option to disable this information seeking. I will try it. I am very curious to find out the real reason.

    I will also try to eventually improve my (non existent) skills in linux command line.

  8. #38
    Senior Member
    Join Date
    May 2010
    Location
    London, UK
    Posts
    923
    Quote Originally Posted by KaPa View Post
    The sequence is the same but I have noticed that the scan sticks for hours (usually in a specific artist and specific song) in the second action line "Scanning new music files".
    On the log settings web page there is a drop down selector at the top to choose a log set. "Media Scanner" will set all related log levels to "Debug", after you click the "Apply" button. If that is too noisy, you could selectively turn some of them back to "Error", or "Warn". 'formats.audio' and 'formats.playlist' focusses on scanning the actual media file.

    Quote Originally Posted by KaPa View Post
    I am thinking that another reason might be the long time it takes to find pictures for artists (especially Greek). As far as I understand there is an option to disable this information seeking. I will try it. I am very curious to find out the real reason.
    The "Music and Artist Information" can take a relatively long time for me, compared with the rest of the scan. It does have some options that can be set. You could try just disabling the plugin, temporarily, for a test.


    I don't know how many tracks you have in your library. Perhaps someone else with a Raspberry pi4 could comment on expected scan times.

  9. #39
    Junior Member
    Join Date
    Oct 2021
    Location
    Athens, Greece
    Posts
    16
    Quote Originally Posted by mrw View Post
    On the log settings web page there is a drop down selector at the top to choose a log set. "Media Scanner" will set all related log levels to "Debug", after you click the "Apply" button. If that is too noisy, you could selectively turn some of them back to "Error", or "Warn". 'formats.audio' and 'formats.playlist' focusses on scanning the actual media file.

    The "Music and Artist Information" can take a relatively long time for me, compared with the rest of the scan. It does have some options that can be set. You could try just disabling the plugin, temporarily, for a test.
    Thank you very much! I really appreciate your answers! I will try your suggestions and I will come back when I have something new!

  10. #40
    Junior Member
    Join Date
    Oct 2021
    Location
    Athens, Greece
    Posts
    16
    Back again to my problems scanning my music library having file names in Greek. Apologies for the long post.

    Last week I spent a lot of my time doing tests scanning my music library (Greek content) trying to identify the problems I had . My findings so far:

    1) Initially it took 04:51:00 time (4 hours and 51 minutes) to scan one of my directories containing 383 CDs of Greek music. Each album is in a separate sub-directory (all directory names in Greek language) under the directory of the relevant artist. For compilations of more than one CD, each CD is also in a separate sub directory.
    2) Choosing "Media scanner" on the log set web page (as member @mrw suggested), and examining the log file (opened in notepad++) I was able to isolate 37 CDs causing big delays in the scanning process.
    3) When these sub directories deleted from the library, the scanning process took only 00:09:14 to complete (9 minutes and 14 sec versus almost 5 hours of the same library when included the 37 problematic sub directories).

    I tried to understand what was the problem in the problematic sub directories. Unfortunately I do not have the knowledge to technically analyze the scanner.log. So, this was a hit and miss process. Anyway I managed to isolate a (compilation) set of 4 CDs called "Ζεϊμπέκικα που έγραψαν ιστορία" (means "Zeimbekiko dance songs that wrote history"). It takes 1,5 hour to scan just these 4 CDs. The tags of all files looked OK with UTF-8 coding. My attention then turned to the Greek file names of the set. Changing the names of the sub directories (not the file names of tracks) to just CD1,2,3,4 it took only 15 secs to finish scanning the set. Bingo!!!!!

    Main conclusion:
    - The names of the sub directories have big impact in the time it takes to scan my library if with Greek characters. I suspect that the problem is bigger when characters with umlaut (diaeresis) are used. I have to underline that the final results of the scan, although delayed are OK.
    - On the other hand I have many directories with greek names causing no problem to scan very fast.

    So the problem has to be analyzed further by someone who understands the technical details of the log file.

    For those kind fellows who would like to see these details, I uploaded to my Onedrive a copy of the scanner.log and server.log (scanning only Vol.3 and Vol.4 of the above referred compilation of greek music for just having sorter log file). One can see the big time delay between line 1964 and 1976 (line numbers in notepad++).
    https://1drv.ms/u/s!Atz4nm8CRsart0jb...9z4Oy?e=OVVzT6

    I have uploaded also the actual music files, for those wanting to examine them further.
    https://1drv.ms/u/s!Atz4nm8CRsaruAR7...SR7gN?e=a7ONIA

    Other conclusions:
    Greek characters appear with 3 different coding in the log files making difficult to identify problems
    a)ANSI (?) in the "discovering files/directories" scanning process
    b)URL % coding filenames and directories
    c) normal Greek in TAGS
    Last edited by KaPa; 2021-10-21 at 10:46.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •