Announce: New versions of erlands plugins

[erland]

I found a solution to the trackstat backup. It seems that although a trackstat backup directory is specified, when specifying a file for a manual backup, a full path must be specified for the file as well. It seems strange, and I don't know why, but that seems to work (I found it via the wiki).

I guess it could be considered a bug, at the moment the directory is only used for storing the scheduled TrackStat backups that runs every night. I should probably change it so it uses the backup directory if the file name only contains the file name and not the full path.

[PasTim]

I guess it could be considered a bug, at the moment the directory is only used for storing the scheduled TrackStat backups that runs every night. I should probably change it so it uses the backup directory if the file name only contains the file name and not the full path.

More of a 'feature' really, but it had me fooled for many hours.

If you stated that a full path was needed in the help information, that would help to avoid others being so slow in figuring out what was going on.

[PasTim]

TrackStat don't trust SBS/LMS scanner because it often does the wrong thing regarding ratings, so the refresh operation in TrackStat that runs after a rescan will overwrite the ratings in the scanner. The reason is the following:
1. A user have put RATING tags in his/her music
2. The user changes rating in TrackStat
3. The user performs a rescan either manually or automatically because he/she upgrades SBS/LMS

The result of 3 would be that the ratings would be overwritten if the TrackStat refresh operation wasn't run.

If you really want the ratings from the tags you should use the "Rating Tag" module in Custom Scan, this module will update the TrackStat data properly and won't be overwritten of the TrackStat refresh operation.

I am using the Rating Tag module to take the RATING tag from my flacs. I also have an automatic Rating scan set up (not manual as is the default), along with mixed and custom tags. That's why I am so puzzled as to why a rescan did not take the updated RATING tags (several hundred) but left them as they were.

It got to a point where if I did an enquiry on a track (song) to see the details, the Rating tag half way down was shown as value 60, but the Rating *'s higher up the track info showed it was stubbornly set at 4 * (the previous rating value was 70). Further, SQL code I used to read tags in the range 60 did not see all the tracks with that value (using the track_statistics.rating attribute). I therefore assume that there is some logic which prevents the custom scan rating module from changing ratings to those in the flacs in some instances.

I had to give up on the past trackstat data (much of which I lost anyway due to directory permissions on my VB, so no great loss), clear it all out and start from scratch. All ratings now have their correct value, and my SQL retrieves them all correctly.

[erland]

I am using the Rating Tag module to take the RATING tag from my flacs. I also have an automatic Rating scan set up (not manual as is the default), along with mixed and custom tags. That's why I am so puzzled as to why a rescan did not take the updated RATING tags (several hundred) but left them as they were.

Do you know if you only checked a specific track/album before and after ?
I'm asking because I introduced some caching in a recent TrackStat version so it caches the data for some of its menus in-memory and there might be some bug so this cache is not updated properly after a rescan so it reads the old value from memory instead of getting the new one from the database. If you checked multiple menus and multiple tracks/albums this should not be the issue because it only caches the last viewed TrackStat menu. Could this explain what you saw ?

[PasTim]

Do you know if you only checked a specific track/album before and after ?
I'm asking because I introduced some caching in a recent TrackStat version so it caches the data for some of its menus in-memory and there might be some bug so this cache is not updated properly after a rescan so it reads the old value from memory instead of getting the new one from the database. If you checked multiple menus and multiple tracks/albums this should not be the issue because it only caches the last viewed TrackStat menu. Could this explain what you saw ?

Not being a very methodical person (and not expecting problems) I can't give you a very explicit account. Roughly:

1) I decided to re-rate a whole load of tracks, using mp3tag, setting previous values of 70 to 60
2) Some of the tracks may have been on my 'now playing' list at the time
3) I tried (using the Alpha custom scan) to re-rate them, but selecting scan on the rating line seemed to do nothing, so I did a complete rescan
4) Some tracks had been renumbered (mostly those which had only had any rating tag at all set fairly recently), others had not
5) I had then a period of trying loads of things - somewhat randomly I guess - at one point the Song Info still seemed wrong, but after several attempts at scanning I got to the point where
6) The song info reported the correct numeric value (60) but the Rating line near the top of the song info showed 4*, not 3*
7) I did a rescan overnight, shutdown (automatic) and restart - still wrong
8) I took a copy of the persist.db and had a look - I couldn't figure out what the 3 trackstat tables were meant to have in attribute rating, and gave up, ditching the lot - sorry!
9) A rescan having cleared all trackstat gave the correct ratings.

I do have a copy of 'persist.db' as it was around step (8) and could try to analyse it further if I knew what I was looking for.

[PasTim]

I would really appreciate it if someone could point me to a description of how the content and function of the three "tracks_...." sqlite database tables, and their relationship to custom scan.

The wiki at http://wiki.slimdevices.com/index.php/TrackStat_plugin briefly describes tracks_history and tracks_statistics, but I believe the description may be out of date, and tracks_persistent is not mentioned at all. It is also not clear to me what happens to ratings when custom_scan is run with the ratings module.

If possible I'd also like some clue as to how the ratings in these tables relates to the LMS ratings.

[erland]

I would really appreciate it if someone could point me to a description of how the content and function of the three "tracks_...." sqlite database tables, and their relationship to custom scan.

The wiki at http://wiki.slimdevices.com/index.php/TrackStat_plugin briefly describes tracks_history and tracks_statistics, but I believe the description may be out of date, and tracks_persistent is not mentioned at all. It is also not clear to me what happens to ratings when custom_scan is run with the ratings module.

If possible I'd also like some clue as to how the ratings in these tables relates to the LMS ratings.

The table ownerships:
* tracks_persistent: Owned by SBS/LMS
* track_statistics: Owned by TrackStat
* customscan_track_attributes: Owned by Custom Scan

The relations I try to use are:
* tracks_persistent.url=tracks.url
* track_statistics.url=tracks.url
* customscan_track_attributes.track=tracks.id
* customscan_album_attributes.album=albums.id
* customscan_contributor_attributes.contributor=cont ributors.id

In the Custom Scan tables, there are also an unoptimized relation:
* customscan_track_attributes.url=tracks.url
(But this shouldn't be used unless you have a very specific need where the track/id relation mentioned above doesn't work)

Then there are some shortcuts which are available if you are using SBS 7.6 or later:
* tracks_persistent.urlmd5=tracks.urlmd5
(Reason for this relation is that it uses shorter texts than the *.url relation and due to this can be faster if you have performance issues, but there are bugs related to it, for example for cue sheets it currently doesn't work correctly)

And another shortcut if you are using TrackStat 3.0 or later:
* tracks_statistics.urlmd5=tracks.urlmd5
(Reason for this relation is also performance issues, it's only available in TrackStat 3.0 and later versions, main reason to use it is when you join both track_statistics, tracks and tracks_persistent in a single query)

Regarding ratings they are stored as follows:
* track_statistics.rating: TrackStat rating storage, should newer be overwritten by incorrect logic in SBS/LMS scanner
* tracks_persistent.rating: SBS/LMS scanner rating, this is always overwritten with RATING tags in the music files or iTunes library (if using iTunes scanner). Note that this will ALWAYS be overwritten with information the scanner finds, so if you change the rating through TrackStat and then do a rescan it will be overwritten by the scanner. The track_statistics.rating will survive this and TrackStat provided iTunes importer will have some smart logic that only overwrite if the rating in iTunes is later then the one in SBS/LMS.
* customscan_track_attributes.value: When customscan_track_attributes.attr='RATING' the value colum will contain the value scanned by the Custom Scan "Rating Tag" scanning module. However, note that "Rating Tag" module will also write to the above tables if you have enabled the "Write ratings to Squeezebox Server" option in the Rating Tag settings page.

Most of the menus and playlists I've made and provided templates for uses the track_statistics.rating value because that's the only one I trust.
The tracks_persistent table will survive a full rescan, it's problem is that it will always be overwritten with what the scanner finds in RATING tags, so basically it can be used in two scenarios:
* If your master rating storage is RATING tags and you always change the tag when you change a rating
* If you are sure you don't have any RATING tags in your music file and you aren't using the iTunes integration when scanning.

When changing a rating with TrackStat it will be written to both track_statistics.rating and tracks_persistent.rating.

[PasTim]

Erland - thanks very much indeed. After a quick read I already realise I have misunderstood several things.

[PasTim]

Erland - one question and one comment:

Which attribute is the source of - the *** rating near the top of the song info web page? I am no longer using your CB web interface, but the standard one, albeit accessed via Custom Browse menus.

When using the alpha custom scan, some tracks were not being updated in track_statistics with the RATING tag, even when the RATING module in Custom Scan was being run as part of a full rescan. I have looked in the table and there were still tracks there with the original rating. The Custom Scan Rating module is enabled, used in automatic scans, and set to overwrites LMS/SB ones.

[erland]

Which attribute is the source of - the *** rating near the top of the song info web page? I am no longer using your CB web interface, but the standard one, albeit accessed via Custom Browse menus.

Anything with stars comes from TrackStat (track_statistics.rating)
The standard SBS/LMS Rating item is displayed a bit down the on the Song Info page, probably as "Rating: 60" in this case.

When using the alpha custom scan, some tracks were not being updated in track_statistics with the RATING tag, even when the RATING module in Custom Scan was being run as part of a full rescan. I have looked in the table and there were still tracks there with the original rating. The Custom Scan Rating module is enabled, used in automatic scans, and set to overwrites LMS/SB ones.

Thanks for reporting, it could be something with the alpha version, I haven't tested the Rating Tag module much with the alpha version as I've mainly had the focus to get it to work with CustomTag/MixedTag scanning modules now in the beginning.