Home of the Squeezebox™ & Transporter® network music players.
Page 16 of 18 FirstFirst ... 61415161718 LastLast
Results 151 to 160 of 172
  1. #151
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,950
    Quote Originally Posted by mvordeme View Post
    Loading this discussion with more stuff: The number of file handles on my Pi has now exceeded 500, memory is at 580 M. I'll keep an eye on it. (For the protocol: Today, I have been listening to Deezer Smart Radio for some time.)
    Here is the code in charge of limits
    Code:
    sub init {
    	$cache = Slim::Utils::Cache->new;
    
    	my $maxPlaylistLengthCB = sub {
    		my ($pref, $max) = @_;
    		
    		if ($prefs->get('dbhighmem')) {
    			$max ||= 2000;
    			$max = 2000 if $max < 2000;
    		}
    		else {
    			$max ||= 500;
    			$max = 500 if $max > 500;
    			$max = 100 if $max < 100;
    		}
    
    		my $cacheObj = tied %Cache;
    		if ($cacheObj->max_size != $max) {
    			$cacheObj->max_size($max);
    		}
    	};
    	
    	$maxPlaylistLengthCB->(undef, $prefs->get('maxPlaylistLength'));
    	
    	$prefs->setChange($maxPlaylistLengthCB, 'maxPlaylistLength');
    }
    So if you have set the "high memory usage" in LMS settings, it's 2000. I did not remember that
    LMS 8.2 on Odroid-C4 - SqueezeAMP!, 5xRadio, 5xBoom, 2xDuet, 1xTouch, 1xSB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, RivaArena 1 & 3

  2. #152
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    Seems that I need to get my head around that funny Perl syntax after all. In the LMS settings, I can choose from normal, high and maximum database size. It is set to "high" for machines with 1+ GB of memory, which should be fine. 2 GB is a little too much.
    Last edited by mvordeme; 2021-02-08 at 14:04.
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  3. #153
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    Meanwhile, the LMS is using almost 800 M of memory. The process is holding a total 696 of open file handles, of which 20 are sockets, 12 are files in /mnt/mmcblk0p2/tce/slimserver/Cache/ and 656 are files in /tmp/. All these files do actually exist, and all existing temp files seem to be associated with open file handles.

    Code:
    Mem: 1820584K used, 133768K free, 102676K shrd, 21016K buff, 867764K cached
    CPU:  0.0% usr  0.0% sys  0.0% nic 99.8% idle  0.0% io  0.0% irq  0.0% sirq
    Load average: 0.00 0.03 0.02 2/137 23884
      PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    32307     1 tc       S     795m 41.6   0  0.1 {slimserver.pl} /usr/bin/perl /usr/local/slimserver/slimserver.pl --daemon --user tc --group staff
    
    $ sudo lsof -p 32307 | grep 32307 | wc -l
    696
    
    $ sudo lsof -p 32307 | grep 32307 | cut -f 3 | grep socket | wc -l
    20
    
    $ sudo lsof -p 32307 | grep 32307 | cut -f 3 | grep /slimserver/Cache/ | wc -l
    12
    
    $ sudo lsof -p 32307 | grep 32307 | cut -f 3 | grep /tmp/ | wc -l
    656
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  4. #154
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    Trying to find a pattern, but no success, so far. I thought that maybe the mechanism of releasing the files is working in general but leaving the odd file behind. In that case, most of those files should be rather new. This doesn't seem to be the case.
    Code:
    Date   | Count
    -------+------
    Jan 18 |  42
    Jan 22 |  29
    Jan 26 |  11
    Jan 27 |  29
    Jan 28 |  64
    Jan 29 |  50
    Jan 30 |  20
    Jan 31 |  17
    Feb 01 |  25
    Feb 02 |  12
    Feb 03 |  40
    Feb 05 |  12
    Feb 06 |  57
    Feb 07 |  13
    Feb 08 |  20
    Feb 09 |  20
    Feb 10 |   8
    Feb 11 |  19
    Feb 12 |   2
    Feb 13 |  37
    Feb 15 |  13
    Feb 16 |  30
    Feb 17 |   4
    Feb 18 |  38
    Feb 19 |  15
    Feb 21 |  13
    Feb 22 |  46
    -------+------
    Total  | 686
    Last edited by mvordeme; 2021-02-23 at 02:10.
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  5. #155
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    Now it is finally dead and does not play anything any more. The file handle count is up to 1024 and memory consumption is above 1 G. The system has almost no space for buffers any more. It already stopped playing hi-res FLAC files weeks ago.
    Code:
    $ sudo lsof -p 32307 | grep 32307 | wc -l
    1024
    $ top
    Mem: 1903428K used, 50924K free, 106604K shrd, 10532K buff, 549776K cached
    CPU:  0.2% usr  0.0% sys  0.0% nic 99.7% idle  0.0% io  0.0% irq  0.0% sirq
    Load average: 0.31 0.19 0.08 3/138 21673
      PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    32307     1 tc       S    1151m 60.2   0  0.2 {slimserver.pl} /usr/bin/perl /usr/local/slimserver/slimserver.pl --daemon
    Going to restart ...

    Edit: Had to restart twice. On the first attempt, the process died again.
    Last edited by mvordeme; 2021-03-17 at 02:19.
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  6. #156
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    20,325
    Quote Originally Posted by mvordeme View Post
    Now it is finally dead and does not play anything any more. The file handle count is up to 1024 and memory consumption is above 1 G.
    If the files are the issue (or possibly a side effect of the issue - objects not being garbage collected), then I think you have to find out what's in the files and so get an indication of where they came from.

  7. #157
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    I had found two different types of files, one of which had been identified somewhere above in the discussion. So far, I have not been able to actually see it happen. Every time I observe the system, all new file handles are closed again after a few minutes. I have been thinking about writing a little watch dog to alert me when it happens.
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  8. #158
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    20,325
    Quote Originally Posted by mvordeme View Post
    I had found two different types of files, one of which had been identified somewhere above in the discussion. So far, I have not been able to actually see it happen. Every time I observe the system, all new file handles are closed again after a few minutes. I have been thinking about writing a little watch dog to alert me when it happens.
    It was a good while back and I think it may be good to look again. I wonder whether the contents of the file somehow results in a scanning error.

    For scanning streamed audio, the audio header (to be scanned) is saved in a temporary file usually created using File::Temp->new() and it is possible to do have better control over the temp file.

    My initial idea would be to use TEMPLATE so that the name of the temp file has additional info such as code module, streamname, format etc. - the "lsof" can then be used to see if any commonality in the files being left open.

    It is also possible override auto deletion when the associated object is destroyed so contents can be examined

    The File::Temp also has its own debugging which can be enabled although not sure what it produces - but ultimately it is the module which is responsible for deleting the file. In this case - it would be used to looking for reasons/times why deletion did NOT take place.

    edit:

    The location of the temp file can be controlled (i.e. use a special dir) and so no need for lsof on a process id. Watchdog need just monitor the special temp dir for growing file count
    Last edited by bpa; 2021-03-17 at 04:20.

  9. #159
    Senior Member mvordeme's Avatar
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    402
    I am not sure about all the debugging stuff, but it appears to me that it is not a matter of configuring things but of changing the code. I have tried to increase the amount of logging for a few items but didn't learn much. Where is all the code? The directory containing the slimserver.pl only contains 9 pl files in total, and only gdresize.pl uses File::Temp.
    Code:
    tc@piCoreServer:/tmp/tcloop/slimserver/usr/local/slimserver$ find . -name "*.pl" -print
    ./CPAN/AnyEvent/Util/idna.pl
    ./CPAN/AnyEvent/Util/uts46data.pl
    ./CPAN/AnyEvent/constants.pl
    ./CPAN/XML/SAX/placeholder.pl
    ./cleanup.pl
    ./gdresize.pl
    ./gdresized.pl
    ./scanner.pl
    ./slimserver.pl
    But I used the opportunity to update the LMS and watch a fresh system playing an album from Tidal. I now understood why just counting the number of open file handles does not help with understanding what is happening. Every track creates and destroys two sockets and two pipes, and there are other activities adding to the noise, like the caching of album art. But when it comes to temporary files, the behaviour is rather straightforward.

    Every single track played leaves a temporary file behind with an open file handle in slimserver.pl. The files are rather uniform. They seem to contain mp4 metadata. I downloaded one of the mp4 tracks, too, and it looks nothing like those temporary files, although with binary data, this is hard to say. All temporary files start with binary data and end like this:
    Code:
    udta�meta-hdlrmdirilst handlerilstr----(meanorg.hydrogenaudio.replaygain&namereplaygain_originator_codedata011011000000i----(meanorg.hydrogenaudio.replaygain!namereplaygain_track_gaindata-7.87 dBi----(meanorg.hydrogenauIuuidDLBY-METADATA-00�meta3hdlrdlbtdolby tags handler�xml<?xml version="1.0" encoding="UTF-8"?>
    <protected xmlns="http://www.dolby.com/ns/mobile/protected-1.0">
      <payload>
        <tags xmlns="http://www.dolby.com/ns/mobile/tags-1.0" id="febbcb871e10fe78fcb42572d432c797" read-only="true">
          <tag name="tool-info" lang="und">
            <data>LS1pbnB1dC1maWxlIDxhcmc+IC0tb3V0cHV0LWZpbGUgPGFyZz4gLS1hdWRpby1lbmNvZGVyIGFhYyAtLWF1ZGlvLWNici1yYXRlIDMyMCAtLWFhYy1tb2RlIGFhYyAtLWlucHV0LXNwZWVjaCBmYWxzZSAtLWF1ZGlvLW9ubHkgLS1yZXBsYXlnYWluIG9uIC0tYXVkaW8tcHJvcC1tZXRhIGxvdWRuZXNzIA==</data>
          </tag>
        </tags>
      </payload>
      <signature algorithm="hmac-md5">a8d5be608e4aca0266c159418d806a4e</signature>
    </protected>
    �meta;hdlrdlbfdolby file metrics handlerqxml <?xml version="1.0" encoding="UTF-8"?>
    <protected xmlns="http://www.dolby.com/ns/mobile/protected-1.0">
      <payload>
        <file-metrics xmlns="http://www.dolby.com/ns/mobile/file-metrics-1.0" id="febbcb871e10fe78fcb42572d432c797">
          <dolby-app-version>Dolby Media Generator 3.8.3</dolby-app-version>
          <creation-date>2016-08-12T17:57:06</creation-date>
          <creation-id>115124</creation-id>
          <stream id="1" type="audio">
            <encoder-description>Dolby Pulse Encoder 1.1.4 sbr: 4.0 / Aug 18 2011</encoder-description>
            <encoder>AAC</encoder>
            <bitrate-mode>VBR</bitrate-mode>
            <bitrate>320000</bitrate>
            <duration unit="frames">9765888</duration>
            <signature algorithm="md5">1c5f05da944468156c140c483176ed9d</signature>
            <audio-info>
              <samplerate>44100</samplerate>
              <channel-count>2</channel-count>
              <encode-mode>None</encode-mode>
              <encode-delay>1600</encode-delay>
            </audio-info>
          </stream>
        </file-metrics>
      </payload>
      <signature algorithm="hmac-md5">0f2340c558a9c8d72d0ce1fadcd2ff28</signature>
    </protected>
    mmeta;hdlrdlbkdolby audio kernel handler&xml <?xml version="1.0" encoding="UTF-8"?>
    <protected xmlns="http://www.dolby.com/ns/mobile/protected-1.0">
      <payload>
        <audio-kernel xmlns="http://www.dolby.com/ns/mobile/audio-kernel-1.0" id="febbcb871e10fe78fcb42572d432c797">
          <configuration>
            <config name="sample-rate" value="44100"/>
          </configuration>
          <parameters>
            <parameter name="tdas.reset" value="1"/>
          </parameters>
        </audio-kernel>
      </payload>
      <signature algorithm="hmac-md5">3c207014a46f955c7c617a330a9d1029</signature>
    </protected>
    3meta7hdlrdlbmdolby metadata handler�xml <?xml version="1.0" encoding="UTF-8"?>
    <protected xmlns="http://www.dolby.com/ns/mobile/protected-1.0">
      <payload>
        <metadata xmlns="http://www.dolby.com/ns/mobile/metadata-1.0" id="febbcb871e10fe78fcb42572d432c797">
          <replaygain stream-id="1">
            <parameter name="track-gain" value="-7.87 dB"/>
            <parameter name="track-peak" value="0.999969"/>
            <parameter name="originator-code" value="011011000000"/>
          </replaygain>
          <parameters stream-id="1">
            <parameter name="dialnorm" value="-10.13"/>
            <parameter name="peak-signal-amplitude" value="1.00"/>
          </parameters>
        </metadata>
      </payload>
      <signature algorithm="hmac-md5">36bbff96c50793db343f56cc653f0adb</signature>
    </protected>
    �)emdat
    The only time I saw one of the files being removed was when my playlist contained only one track and I skipped forward, playing the same song again. Skipping backward leaves the existing file in place, and so does skipping forward to another track or the song ending on its own.
    SCALEO Home Server 2105 & piCorePlayer 6.1.0 | Logitech Media Server 8.2.0 | Server Power Control 20120716.103808 | Transporter & Duet & Touch & Boom & Radio | Rotel RC-995 & RMB-100 | Nubert NuVero 140

  10. #160
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,950

    Memory Leak in Perl Engine on piCorePlayer?

    Quote Originally Posted by mvordeme View Post
    I am not sure about all the debugging stuff, but it appears to me that it is not a matter of configuring things but of changing the code. I have tried to increase the amount of logging for a few items but didn't learn much. Where is all the code? The directory containing the slimserver.pl only contains 9 pl files in total, and only gdresize.pl uses File::Temp.
    Code:
    tc@piCoreServer:/tmp/tcloop/slimserver/usr/local/slimserver$ find . -name "*.pl" -print
    ./CPAN/AnyEvent/Util/idna.pl
    ./CPAN/AnyEvent/Util/uts46data.pl
    ./CPAN/AnyEvent/constants.pl
    ./CPAN/XML/SAX/placeholder.pl
    ./cleanup.pl
    ./gdresize.pl
    ./gdresized.pl
    ./scanner.pl
    ./slimserver.pl
    But I used the opportunity to update the LMS and watch a fresh system playing an album from Tidal. I now understood why just counting the number of open file handles does not help with understanding what is happening. Every track creates and destroys two sockets and two pipes, and there are other activities adding to the noise, like the caching of album art. But when it comes to temporary files, the behaviour is rather straightforward.

    Every single track played leaves a temporary file behind with an open file handle in slimserver.pl. The files are rather uniform. They seem to contain mp4 metadata. I downloaded one of the mp4 tracks, too, and it looks nothing like those temporary files, although with binary data, this is hard to say. All temporary files start with binary data and end like this:
    Code:
    udta�meta-hdlrmdirilst handlerilstr----(meanorg.hydrogenaudio.replaygain&namereplaygain_originator_codedata011011000000i----(meanorg.hydrogenaudio.replaygain!namereplaygain_track_gaindata-7.87 dBi----(meanorg.hydrogenauIuuidDLBY-METADATA-00�meta3hdlrdlbtdolby tags handler�xml
    
      
        
          
            LS1pbnB1dC1maWxlIDxhcmc+IC0tb3V0cHV0LWZpbGUgPGFyZz4gLS1hdWRpby1lbmNvZGVyIGFhYyAtLWF1ZGlvLWNici1yYXRlIDMyMCAtLWFhYy1tb2RlIGFhYyAtLWlucHV0LXNwZWVjaCBmYWxzZSAtLWF1ZGlvLW9ubHkgLS1yZXBsYXlnYWluIG9uIC0tYXVkaW8tcHJvcC1tZXRhIGxvdWRuZXNzIA==
          
        
      
      a8d5be608e4aca0266c159418d806a4e
    
    �meta;hdlrdlbfdolby file metrics handlerqxml 
    
      
        
          Dolby Media Generator 3.8.3
          2016-08-12T17:57:06
          115124
          
            Dolby Pulse Encoder 1.1.4 sbr: 4.0 / Aug 18 2011
            AAC
            VBR
            320000
            9765888
            1c5f05da944468156c140c483176ed9d
            
              44100
              2
              None
              1600
            
          
        
      
      0f2340c558a9c8d72d0ce1fadcd2ff28
    
    mmeta;hdlrdlbkdolby audio kernel handler&xml 
    
      
        
          
            
          
          
            
          
        
      
      3c207014a46f955c7c617a330a9d1029
    
    3meta7hdlrdlbmdolby metadata handler�xml 
    
      
        
          
            
            
            
          
          
            
            
          
        
      
      36bbff96c50793db343f56cc653f0adb
    
    �)emdat
    The only time I saw one of the files being removed was when my playlist contained only one track and I skipped forward, playing the same song again. Skipping backward leaves the existing file in place, and so does skipping forward to another track or the song ending on its own.
    These temp files are the download of the header for remote files. They should be closed/erased after 500 files (tbc) on large memory server. I will revisit the code in the coming days.
    LMS 8.2 on Odroid-C4 - SqueezeAMP!, 5xRadio, 5xBoom, 2xDuet, 1xTouch, 1xSB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, RivaArena 1 & 3

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •