Home of the Squeezebox™ & Transporter® network music players.
Results 1 to 4 of 4
  1. #1
    Senior Member
    Join Date
    Apr 2008
    Location
    Paris, France
    Posts
    2,205

    Investigating LMS 7.9.1 crash on restart

    Hi all,

    I run a very stable machine (Pi3/raspbian, about 1 year old) with logitechmediaserver 7.9.1~1491339164 on it
    The machine has a cron entry that restarts LMS weekly at 4:00 on Sundays. The gist of it is:
    Code:
    /usr/sbin/service "$SERVICE" stop
    sleep 5
    /usr/sbin/service "$SERVICE" start
    Today I received a mail telling me LMS had failed. I looked in server log and LMS was caught in a restart loop. The problem happened at 4:00.
    Code:
    2018-10-07 04:00:02 squeezeboxserver_safe stopped.
    2018-10-07 04:00:08 squeezeboxserver_safe started.
    2018-10-07 04:00:15 Logitech Media Server died. Restarting.
    2018-10-07 04:00:20 Logitech Media Server died. Restarting.
    ...
    I ran /usr/sbin/squeezeboxserver --help and got this:
    Code:
    Number found where operator expected at /usr/share/perl5/Slim/Schema.pm line 2890, near "} 5"
    	(Missing operator before  5?)
    Bareword found where operator expected at /usr/share/perl5/Slim/Schema.pm line 2892, near "$)sDebug"
    	(Missing operator before sDebug?)
    Unrecognized character \xF4; marked by <-- HERE after 				$at<-- HERE near column 8 at /usr/share/perl5/Slim/Schema.pm line 2893.
    Compilation failed in require at /usr/share/perl5/Slim/Player/Song.pm line 18.
    ...
    Which corresponds to:
    Code:
    tail -2885 /usr/share/perl5/Slim/Schema.pm | more
    
    sub hasLibrary {
    	return $initialized;
    }
    
    sub _connect {
    	my ( $class, $dsn, $sql ) = @_;
    	
    	$sql ||= [];
    	
    	my ($driver, $source, $username, $password) = $class->sourceInformation;
    ...
    I reinstalled logitechmediaserver_7.9.1~1492077569_arm.deb, so not the same version but one I had lying around in the machine.
    This "solved" the problem as the server restarted ok this time.
    The uptime of the machine was 78 days, dmesg or syslog didn't show anything extraordinary. Load was a bit higher than normal due to LMS restarting incessantly, but still very manageable for the system. LMS doesn't work on Sundays on that machine

    I guess something happened on the way to saving the DB when LMS stopped, and then it failed to restart?
    In any case I would like to know how to make my crontab restart script less naive and avoid this problem in the future. Any idea what might have happened and how to make restart more sturdy?

    Thanks in advance.
    3 SB 3 • Libratone Loop, Zipp Mini • iPeng (iPhone + iPad) • LMS 7.9 (linux) with plugins: CD Player, WaveInput, Triode's BBC iPlayer by bpa • IRBlaster by Gwendesign (Felix) • Server Power Control by Gordon Harris • Smart Mix, Music Walk With Me, What Was That Tune? by Michael Herger • PowerSave by Jason Holtzapple • Song Info, Song Lyrics by Erland Isaksson • AirPlay Bridge by philippe_44 • WeatherTime by Martin Rehfeld • Auto Dim Display, SaverSwitcher, ContextMenu by Peter Watkins.

  2. #2
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,081

    Investigating LMS 7.9.1 crash onrestart

    > Number found where operator expected at /usr/share/perl5/Slim/Schema.pm line 2890, near "} 5"
    > (Missing operator before 5?)
    > Bareword found where operator expected at /usr/share/perl5/Slim/Schema.pm line 2892, near "$)sDebug"
    > (Missing operator before sDebug?)
    > Unrecognized character \xF4; marked by <-- HERE after $at<-- HERE near column 8 at /usr/share/perl5/Slim/Schema.pm line 2893.
    > Compilation failed in require at /usr/share/perl5/Slim/Player/Song.pm line 18.


    This looks a lot as if LMS' source code got corrupted somehow? Did you
    fsck your SD card?

    --

    Michael

  3. #3
    Senior Member
    Join Date
    Apr 2008
    Location
    Paris, France
    Posts
    2,205
    Quote Originally Posted by mherger View Post
    This looks a lot as if LMS' source code got corrupted somehow? Did you
    fsck your SD card?
    Hmm. The base is Raspbian, I think it checks rootfs on startup, but how thorough the check is, I wouldn't know. There is a lot of free space on the SD and logging is severely trimmed down.

    Freak accident then; I saved time by not using a more specialized OS, I guess one year on I might have to start paying on that decision...
    I'll prepare a clone SD just in case.

    Thanks, Michael.

    EDIT. Theoretically ok
    Code:
    # systemctl status systemd-fsck*
    ● systemd-fsck-root.service - File System Check on Root Device
       Loaded: loaded (/lib/systemd/system/systemd-fsck-root.service; static)
       Active: active (exited) since Sun 2018-10-07 14:22:29 CEST; 21h ago
         Docs: man:systemd-fsck-root.service(8)
      Process: 112 ExecStart=/lib/systemd/systemd-fsck (code=exited, status=0/SUCCESS)
     Main PID: 112 (code=exited, status=0/SUCCESS)
       CGroup: /system.slice/systemd-fsck-root.service
    
    Oct 07 14:22:29 audiosrv systemd-fsck[112]: e2fsck 1.43.3 (04-Sep-2016)
    Oct 07 14:22:29 audiosrv systemd-fsck[112]: /dev/mmcblk0p2: clean, 71127/937280 files, 518096/3772672 blocks
    Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
    
    ● systemd-fsck@dev-mmcblk0p1.service - File System Check on /dev/mmcblk0p1
       Loaded: loaded (/lib/systemd/system/systemd-fsck@.service; static)
       Active: active (exited) since Sun 2018-10-07 14:22:30 CEST; 21h ago
         Docs: man:systemd-fsck@.service(8)
      Process: 241 ExecStart=/lib/systemd/systemd-fsck %f (code=exited, status=0/SUCCESS)
     Main PID: 241 (code=exited, status=0/SUCCESS)
    Last edited by epoch1970; 2018-10-08 at 02:41.
    3 SB 3 • Libratone Loop, Zipp Mini • iPeng (iPhone + iPad) • LMS 7.9 (linux) with plugins: CD Player, WaveInput, Triode's BBC iPlayer by bpa • IRBlaster by Gwendesign (Felix) • Server Power Control by Gordon Harris • Smart Mix, Music Walk With Me, What Was That Tune? by Michael Herger • PowerSave by Jason Holtzapple • Song Info, Song Lyrics by Erland Isaksson • AirPlay Bridge by philippe_44 • WeatherTime by Martin Rehfeld • Auto Dim Display, SaverSwitcher, ContextMenu by Peter Watkins.

  4. #4
    Senior Member
    Join Date
    Apr 2008
    Location
    Paris, France
    Posts
    2,205
    BTW, I am running a custom watchdog script to test the condition of a few processes. The LMS is very robust and its status is a bit tricky to assess (multiple processes, can take all the CPU during rescan) so my check is lax and infrequent on LMS.

    Now I am thinking I should filter syslog entries "Logitech Media Server died. Restarting." and raise a definitive alarm when that line appears too often.

    I can edit syslog.conf to filter/redirect to a custom logfile, check the freshness that logfile with watchdog and be done with it.
    Is there a more standard (yet lightweight) way of doing this on linux?
    3 SB 3 • Libratone Loop, Zipp Mini • iPeng (iPhone + iPad) • LMS 7.9 (linux) with plugins: CD Player, WaveInput, Triode's BBC iPlayer by bpa • IRBlaster by Gwendesign (Felix) • Server Power Control by Gordon Harris • Smart Mix, Music Walk With Me, What Was That Tune? by Michael Herger • PowerSave by Jason Holtzapple • Song Info, Song Lyrics by Erland Isaksson • AirPlay Bridge by philippe_44 • WeatherTime by Martin Rehfeld • Auto Dim Display, SaverSwitcher, ContextMenu by Peter Watkins.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •