Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 5 123 ... LastLast
Results 1 to 10 of 44
  1. #1
    Junior Member
    Join Date
    Aug 2009
    Posts
    10

    TrackStat and iTunes Statistics Import

    I am wondering, why statistics of some songs are imported and some are not. I havent' found a pattern yet, to which kind of tracks this problem applies. About 70% of my iTunes-rated tracks don't have a TrackStat in Squeezecenter rating after running an import.

    I haven't found any suspicios messages in the server.log, even when I turned debugging mode on for CustomScann and TrackStat logging messages.

    Any ideas?
    Any help is appreciated!

    btw: The problem I have, ist not related to special characters in the file path (like '[', '&', etc.). I fixed it by applying a sed-script to the library-xml.

  2. #2
    Senior Member erland's Avatar
    Join Date
    Dec 2005
    Location
    Sweden
    Posts
    11,038
    Could you find a specific song that doesn't work ?

    In that case, I would like you to enable debug level logging on plugin.trackstat and then post or mail me the section of the log that contains rows related to the server.log file. You can post/send the whole log file if it isn't too large or you can cut out the section related to the file plus 20 extra rows before and after in a text editor.

    If you run TrackStat with debug logging enabled, you should see rows that indicates that ratings or other statistics is imported for each specific track.
    Erland Isaksson (My homepage)
    Developer of many plugins/applets

  3. #3
    Junior Member
    Join Date
    Aug 2009
    Posts
    10
    Hi Erland, let me mention, how much I like your plugin. You are doing a great job!

    Thanks for the fast reply. It lead me to where really the problem was and I have to correct myself: the problem I described *is* related to special characters in the file path:

    My own sed-based workaround for the special-char-problem generated a non-well-formed xml file:

    transformation.sed:
    s/&/&/g
    s/%5B/\[/g
    s/%5D/\]/g

    Log file entry:
    [09-08-08 19:52:45.7814] Plugins::CustomScan::Scanner::exitScanTrack (1707) CustomScan: Failed to call exitScanTrack on module Plugins::TrackStat::Plugin->itunesimport:
    not well-formed (invalid token) at line 18, column 41, byte 817:
    <key>Track ID</key><integer>3040</integer>
    <key>Name</key><string>Dream Machine (Down Tempo Mix)</string>
    <key>Artist</key><string>Mark Farina & Sean Hayes</string>
    ========================================^

    I got rid of all the 'No matching alternative path found' messages, but ignored the error message above. Bugger! The songs with imported rating have been imported *before* I implemented this workaround.

    Deactivating my sed script leads me back to the problem described in Thread "CustomScan - iTunes statistics import notworking". In this thread I haven't found a hint how to excactly deal with those characters like '[]&' in my filenames.
    Nevertheless, focussing (as advised) on the plugin.trackstat-messages, I found out, that for some reason SC does NOT url-Encode the above Characters. Eg. '&' in 'Girls & Boys':

    mysql> select url, rating from tracks_persistent where url like '%Blur%Parklife%Girls%Boys%';
    +------------------------------------------------------------------------+--------+
    | url | rating |
    +------------------------------------------------------------------------+--------+
    | file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20&%20Boys.mp3 | NULL |
    +------------------------------------------------------------------------+--------+
    1 row in set (2.89 sec)

    I would expect this url to be stored as TrackStat does: 'Girls%20%26%20Boys' (thanks again for the hint to the logfile), but SC stores it as 'Girls%20&%20Boys'

    Log Messages:
    [09-08-09 22:12:53.6025] Plugins::TrackStat::Storage::findTrack (495) findTrack(): URL: file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/10_London%20Loves.mp3

    [09-08-09 22:12:53.6195] Plugins::TrackStat::iTunes::Import::normalize_loca tion (689) normalized file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.mp3 to file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.mp3

    [09-08-09 22:12:53.6238] Plugins::TrackStat::iTunes::Import::handleTrack (448) got a track named Girls %26 Boys location: file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.mp3

    [09-08-09 22:12:53.6557] Plugins::TrackStat::Storage::mergeTrack (1065) File file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.mp3 doesn't exist - will try to find on alternative path.
    [09-08-09 22:12:53.6584] Plugins::TrackStat::Storage::findObjectForMovedUrl (899) Find track urls ending with /B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.
    [09-08-09 22:12:59.9177] Plugins::TrackStat::Storage::findObjectForMovedUrl (904) Found 0 matching tracks
    [09-08-09 22:12:59.9196] Plugins::TrackStat::Storage::findObjectForMovedUrl (924) Find track urls ending with /Blur%20-%20Parklife/01_Girls%20%26%20Boys.
    [09-08-09 22:13:06.2219] Plugins::TrackStat::Storage::findObjectForMovedUrl (929) Found 0 matching tracks
    [09-08-09 22:13:06.2237] Plugins::TrackStat::Storage::mergeTrack (1072) No matching alternative path found for file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/01_Girls%20%26%20Boys.mp3.
    [09-08-09 22:13:06.2315] Plugins::TrackStat::iTunes::Import::normalize_loca tion (689) normalized file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/12_Clover%20Over%20Dover.mp3 to file:///mnt/sda1/mp3/A-Z/B/Blur%20-%20Parklife/12_Clover%20Over%20Dover.mp3

    Maybe a bug in SC?

    Cheers
    ElMuto

  4. #4
    Senior Member Chunkywizard's Avatar
    Join Date
    Feb 2009
    Location
    Sindlesham, UK
    Posts
    909
    Hi ElMuto,

    I am the other guy seeing this (I started the other thread). I remember erland saying that this bug isn't present in the Linux build, it only seems to affect Windows builds. I guess that wouldn't point to a bug in SC...

    CW
    Squeezebox: Boom x2 (Kitchen and Office), Radio x4 (Kids, Lounge and Bedroom)
    Server: 7.9.2 for PCP
    Plugins: Dynamic Playlists | SQL Playlist | Music Information Screen | TrackStat
    Remote: iPhone6/iPad Air running iPeng
    Music: Last.fm

  5. #5
    Junior Member
    Join Date
    Aug 2009
    Posts
    10
    Hi ChunkyWizard,

    I was hoping, we could find some synergies :-)

    Think, we're getting closer to the problem: I experience those symptoms on a

    - Linux-Installation: Ubuntu Server 8.04.1 LTS (Hardy)
    - SC 7.3.3 - 27044

    System. It would be interesting to know if on other (than our two) systems, characters like '&' in filepaths are *not* URL-encoded, when stored in SC's tracks.url and tracks_persistent.url columns. If so, I would really like to know the reason for it. Will search other threads about URL encoding in SC tonight.

    ElMuto

  6. #6
    Senior Member Chunkywizard's Avatar
    Join Date
    Feb 2009
    Location
    Sindlesham, UK
    Posts
    909
    Aha, you are not who I thought you were! We are three, see: http://code.google.com/p/erlandplugi...edBy%20Summary

    They are running a WHS system as well. Can you link your findings to the bug report?

    CW
    Squeezebox: Boom x2 (Kitchen and Office), Radio x4 (Kids, Lounge and Bedroom)
    Server: 7.9.2 for PCP
    Plugins: Dynamic Playlists | SQL Playlist | Music Information Screen | TrackStat
    Remote: iPhone6/iPad Air running iPeng
    Music: Last.fm

  7. #7
    Junior Member
    Join Date
    Aug 2009
    Posts
    10
    good idea! done.

    Seems as if I am the missing SC-Linux-User, doesn't it?

  8. #8
    Senior Member erland's Avatar
    Join Date
    Dec 2005
    Location
    Sweden
    Posts
    11,038
    Quote Originally Posted by ElMuto View Post
    It would be interesting to know if on other (than our two) systems, characters like '&' in filepaths are *not* URL-encoded, when stored in SC's tracks.url and tracks_persistent.url columns. If so, I would really like to know the reason for it. Will search other threads about URL encoding in SC tonight.
    Could you check which locale setting SqueezeCenter uses ?
    It's shown in the "Information" tab in the Server Settings as something like:
    "Operating system: Debian - EN - utf8"

    How does the paths look like in the iTunes xml file ? Do they contain & or is it encoded with %26 ?

    With Chunkywizard's test files my development machine seems to only encode # and ' characters, I'm not sure if this was the case earlier or if it behaves differently now.
    Erland Isaksson (My homepage)
    Developer of many plugins/applets

  9. #9
    Junior Member
    Join Date
    Aug 2009
    Posts
    10
    Quote Originally Posted by erland View Post
    Could you check which locale setting SqueezeCenter uses ?
    It's shown in the "Information" tab in the Server Settings as something like:
    "Operating system: Debian - EN - utf8"
    Debian - DE - utf8

    Quote Originally Posted by erland View Post
    How does the paths look like in the iTunes xml file ? Do they contain & or is it encoded with %26 ?
    Interesting point:
    - in the iTunes (WinXP) xml file it's encoded with: & #38; (please ignore the space between the ampersand and the pound symbol)
    - TrackStat (log) reads it as: %26
    - SC stores it as: &

    Quote Originally Posted by erland View Post
    With Chunkywizard's test files my development machine seems to only encode # and ' characters, I'm not sure if this was the case earlier or if it behaves differently now.
    Another interesting observation on my system:
    - iTunes xml: '
    - Trackstat log: '
    - SC: %27

    Let me know, if you need any more infos/things I can do

  10. #10
    Junior Member
    Join Date
    Aug 2009
    Posts
    10
    In order deepen my findings, I modified the method "normalize_location" in /usr/share/perl5/Slim/Plugin/iTunes/Common.pm and wanted to see, what happens. So I inserted following code-snippet at Line 358 (just before returning $url):

    # ElMuto's extension
    # replace %26 by ampersand to get
    # matching file urls in SCs DB
    $log->warn("original url was $url");
    $url =~ s/%26/\&/;
    $log->warn("..changed url is $url");
    # end of ElMuto's extension

    Interestingly enough, even after restart of the squeezecenter-service, this code-modification seems to have no effect in the execution. Not even the messages are written to the log.

    Does anybody know, what I can do to test my code modifications on a standard Ubuntu-SC-installation? I would really like to avoid setting up an SC-dev-environment for testing one line of effective code.
    Last edited by ElMuto; 2009-08-12 at 13:16.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •