TrackStat/Custom Scan/LMS Â/¬/URL problem

[erland]

This is what TrackStat Statistics looks like after I have deleted the data and then restored from my backup file. The number of songs is correct, but I would expect several thousand more songs to have a rating.

Just to be sure, did you went for the restore operation to finish ?
It runs in the background if you trigger it from the TrackStat settings page and you can only see if it's finished by logging in the server log for a message saying: "Restore completed at <date and time>". If you triggered the restore from Custom Scan, you should be able to see if it's finished based on the status in the Custom Scan settings page where all scan/export/import modules are shown.

I'm not exactly sure what you mean by "how the 'Rating' is set." Do you mean is the rating set (that is, does it have a value/1-10/stars/asterisks)? When I browse to a song (or search) and look at its rating, there is no rating (in other words, yes, the rating is empty). Of course, the exception is the mp3s where the rating is correctly being restored.

What I meant was just if the correct ratings were shown in the "Song Info" page or not.
The reason the Song Info page is of interest is because that information is not cached by TrackStat, the TrackStat plugin caches some data for its own browse menus under TrackStat menu but not for the information it provides in the standard "Song Info" page.

Do you see rating stars in the Song Info page (these comes from TrackStat) or do you only see a number (which comes from standard SBS/LMS tables).

If you haven't done so already, look at a section of the log for the TrackStat restore operation and look for a specific track and check if the rating of that tracks is shown with correct number of stars in the "Song Info" page.

The "Song Info" page is reached by browsing to a track using the standard browse menus under "My Music".

It could be that the issue is cased of these duplicated musicbrainz identities, but I want to be sure before we assume that.

[prabbit]

Just to be sure, did you went for the restore operation to finish ?
It runs in the background if you trigger it from the TrackStat settings page and you can only see if it's finished by logging in the server log for a message saying: "Restore completed at <date and time>". If you triggered the restore from Custom Scan, you should be able to see if it's finished based on the status in the Custom Scan settings page where all scan/export/import modules are shown.

I did wait for the restore operation to finish. Just to simplify things I uninstalled Custom Scan and nearly every other plugin for now. Although it was probably unnecessary, I figure that having a configuration that is as simple as possible with help determine where the issue is.

Do you see rating stars in the Song Info page (these comes from TrackStat) or do you only see a number (which comes from standard SBS/LMS tables)?
If you haven't done so already, look at a section of the log for the TrackStat restore operation and look for a specific track and check if the rating of that tracks is shown with correct number of stars in the "Song Info" page.

Here is the section of the TrackStat log for a specific track and what it looks like on the "Song Info" page:

[12-04-25 11:42:20.4311] Plugins::TrackStat::Storage::findTrack (533) findTrack(): URL: file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a
[12-04-25 11:42:20.4314] Plugins::TrackStat::Storage::findTrack (534) findTrack(): mbId: b8bb131f-5428-4067-bb96-31412c9a48d0
[12-04-25 11:42:20.4322] Plugins::TrackStat::Storage::saveTrack (1039) Saving play count, last played and added time in storage: 1, 1325189815, 1323722693
[12-04-25 11:42:20.4333] Plugins::TrackStat::Storage::findTrack (533) findTrack(): URL: file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a
[12-04-25 11:42:20.4336] Plugins::TrackStat::Storage::findTrack (534) findTrack(): mbId: b8bb131f-5428-4067-bb96-31412c9a48d0
[12-04-25 11:42:20.4344] Plugins::TrackStat::Storage::saveTrack (1079) Store rating: 90

It could be that the issue is ca[u]sed of these duplicated musicbrainz identities, but I want to be sure before we assume that.

Although it would be a bit of a hassle, if you need me to delete the musicbrainz IDs from my files, I will. I have already tried restoring from a TrackStat backup file where I removed the musicbrainz IDs from the TrackStat backup file and left the musicbrainz IDs in the m4as/mp3s, but that didn't help either.
To me, my thousands of ratings are more important that the musicbrainz IDs. And if the musicbrainz IDs are indeed the culprit, why do some of my files (some, but not all, of the mp3s) get their ratings restored while others (m4as and some mp3s) don't? <<< Mostly a rhetorical question and one that doesn't require an answer.

I hope this info helps. What next?

[erland]

Although it would be a bit of a hassle, if you need me to delete the musicbrainz IDs from my files, I will.

Do NOT remove musicbrainz ID's from the files, this should not be necessary.

I have already tried restoring from a TrackStat backup file where I removed the musicbrainz IDs from the TrackStat backup file and left the musicbrainz IDs in the m4as/mp3s, but that didn't help either.

Ok, that means it's probably not related to musicbrainz ID's

To me, my thousands of ratings are more important that the musicbrainz IDs. And if the musicbrainz IDs are indeed the culprit, why do some of my files (some, but not all, of the mp3s) get their ratings restored while others (m4as and some mp3s) don't? <<< Mostly a rhetorical question and one that doesn't require an answer.

I hope this info helps. What next?

I tried to rename my file according you the one you posted in the log in the previous message with a ¬ character in the name and this works to backup and restore in my setup, it could still be related to the ¬ character though if your regional/language settings on the computer is different than mine.
1. Have you seen any cases where tracks with ¬ characters works to restore ?
2. Have you seen any cases where tracks without ¬ characters doesn't work to restore ? Do they have some other strange character in the file name ?
3. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from track_statistics where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

4. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from tracks_persistent where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

5. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from tracks_persistent where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

You can click the "CSV" link at the top of the Database Query result page to export the query results to a semi colon separated file.

6. Can you in a text editor search and try to find all occurrences of the above URL in the TrackStat backup file and send them to me. Maybe there are duplicate entries for some reason where one set the rating and one resets it later in the backup file.

[prabbit]

1. Have you seen any cases where tracks with ¬ characters works to restore ?

Yes, some of my mp3s restore correctly. When I restore from the TrackStat backup file I have about 2200 songs get ratings--all of them are mp3 files.

2. Have you seen any cases where tracks without ¬ characters doesn't work to restore ? Do they have some other strange character in the file name ?

I don't have any songs that don't have the ¬ character in the file name. Thinking the ¬ character might be the problem, I had renamed an album's worth of files, updated TrackStat to match my changes, and ran TrackStat restore. Same results...no ratings.

3. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from track_statistics where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

The query returns a blank result.

4. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from tracks_persistent where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

The query returns a blank result.

5. Can you try to install Database Query plugin and create a new "Free form query" and use the following SQL statement in it and post or mail me the result ?

select * from tracks_persistent where url='file:///Volumes/MediaFiles/alac_mp3/alac/lief/The%20Sisters%20Of%20Mercy/Floodland/1987%20%C2%AC%201-1%20%C2%AC%20Dominionmother%20Russia.m4a';

The query returns a blank result.

6. Can you in a text editor search and try to find all occurrences of the above URL in the TrackStat backup file and send them to me. Maybe there are duplicate entries for some reason where one set the rating and one resets it later in the backup file.

The URL appears 44 times. Each time it looks like this (only with no space before the 'y' in Mercy and no line wrapping):

<track>

<url>file%3A%2F%2F%2FVolumes%2FMediaFiles%2Falac_m p3%2Falac%2Flief%2FThe%2520Sisters%2520Of%2520Merc y%2FFloodland%2F1987%2520%25C2%25AC%25201-1%2520%25C2%25AC%2520Dominionmother%2520Russia.m4a </url>

<playCount>1</playCount>

<lastPlayed>1325189815</lastPlayed>

<rating>90</rating>

</track>

I sent you a .zip file to your e-mail address on your website with my original TrackStat file and a couple of modified versions from which I am working.

[erland]

Thanks for the zipped TrackStat files, will take a look and see if I can figure out something based on them.

In the meantime, can you try the following free form queries in Database Query plugin instead ?
1.

select * from track_statistics where url like '%Sisters%Floodland%Dominionmother%'

2.

select * from tracks_persistent where url like '%Sisters%Floodland%Dominionmother%'

3.

select * from tracks where url like '%Sisters%Floodland%Dominionmother%'

[prabbit]

^^^ The above queries also all returned a blank result.

I ran the TrackStat Statistics query again just to double check I still have data in the TrackStat database. I do. It's still the same as in post #9.

[prabbit]

Since I have ~2200 mp3 tracks that did successfully have had their ratings restored, I ran a query for one of them just to confirm that the queries would work with tracks with a rating. The query did in fact work. I'm not sure what that tells us, especially since I have removed all data (which I assume drops the tables) before restoring.

If I run this query:

select * from tracks where url like '%Latifah%'

It returns this result:

xml version="1.0" encoding="utf-8"?
result:
resultitem:
id: 83637
url: file:///Volumes/MediaFiles/alac_mp3/mp3/amazon/Queen%20Latifah/Persona/2009%20%C2%AC%201-3%20%C2%AC%20Cue%20The%20Rain.mp3
title: Cue The Rain
titlesort: CUE THE RAIN
titlesearch: CUE THE RAIN
customsearch:
album: 7495
tracknum: 3
content_type: mp3
timestamp: 1334992406
filesize: 11653496
audio_size: 11586664
audio_offset: 66832
year: 2009
secs: 362.083
cover: 44100
vbr_scale:
bitrate: 256000
samplerate: 44100
samplesize:
channels: 2
block_alignment: 1
endian:
bpm:
tagversion: ID3v2.3.0
drm: 0
disc: 1
audio: 1
remote: 0
lossless: 0
lyrics: Cue The Rain (snipped for posting brevity)
musicbrainz_id: f5ab81c8-3e0d-4312-b494-24d9ed7ace45
musicmagic_mixable:
replay_gain: -8.57
replay_peak: 1.301323
extid:
primary_artist: 5869
urlmd5: bd9ea32f1eb4d1259f6e54f16e2b9037
coverid: cea4ae7b
cover_cached:
virtual:
added_time: 1335369999
updated_time: 1335369999
dlna_profile: MP3
hash:

[erland]

^^^ The above queries also all returned a blank result.

I ran the TrackStat Statistics query again just to double check I still have data in the TrackStat database. I do. It's still the same as in post #9.

This is really strange, especially if the query for the "tracks" table returned an empty result.
Could you lookup this track(1987 ¬ 1-1 ¬ Dominionmother Russia.m4a) in the standard Browse menus under My Music and click on the song title to open the Song Info page and post what the "Location" attribute in the Song Info page says ?

[prabbit]

Location: /Volumes/MediaFiles/alac_mp3/alac/lief/The Sisters Of Mercy/Floodland/1987 ¬ 01-01 ¬ Dominion／mother Russia.m4a



Close inspection of the URL shows that the track number in the URL is 01 (the second 01), but in TrackStat it is just 1, not 01. You'll notice the Queen Latifah track above does not have a leading zero in the track position and the rating worked. I spot checked a few random tracks, and they also have the missing (or extra depending on point of view) leading zero. If the leading zero is a part of the problem, the problem should not manifest itself for tracks 10 and above, but it does. I tried editing a TrackStat backup file to add the leading zero to the URL, restoring, and did not have any success. So, the leading zero could be a problem, but also a red herring.

[erland]

Can you try these queries instead ?

1.

select * from track_statistics where url like '%Sisters%Floodland%Dominion%mother%'

2.

select * from tracks_persistent where url like '%Sisters%Floodland%Dominion%mother%'

3.

select * from tracks where url like '%Sisters%Floodland%Dominion%mother%'

There seems to be some strange / character between "Dominion" and "mother" in the screenshot, I suspect this might be causing the issues as it doesn't exist in the TrackStat backup file. The above queries should ignore the / characters so we can see how the url looks in the database. I think it could be some character encoding issue which causes strange / character to be removed before the TrackStat backup file is written. Do you see this or other strange characters also in the other files which you have problem with ?