Handling of rating in MusicMagic, iTunesUpdate, TrackStat and slimserver

[erland]

I feel we need to start thinking about how ratings should be handled in third party plugins and in slimserver.

The main reason I decided to implement the TrackStat plugin is that:
1. Ratings where not supported in slimserver at the time
2. The slimserver database could not be considered a save storage of ratings since the tracks table will be cleared at rescan and track id's reused and in some situations/crashes the slimserversql.db file needs to be deleted.

Actually I think we need to think about more than ratings, all statistic data is interested such as "last played time", "added time", "play counts", "ratings" and maybe something else also. But lets start the discussion with ratings.

Today the situation as far as I know is:

TrackStat plugin
================
- Makes it possible to set/view ratings from SqueezeBox, web ui and cli
- Ratings can be viewed as titleFormats for tracks in SqueezeBox, web ui
- Ratings in TrackStat survives a rescan since they are stored in a separate table which is not cleared during rescan.
- Ratings can be backup/restored to xml file
- TrackStat sets ratings in the standard slimserver tracks table(in 6.5)
- Implements MusicMagic export/import of ratings

iTunesUpdate plugin
===================
- Makes it possible to set/view ratings from SqueezeBox and web ui
- Ratings in iTunesUpdate is sent and stored in iTunes
- iTunesUpdate sets ratings in the standard slimserver tracks table(in 6.5)
- Ratings in iTunesUpdate survives a rescan since they are stored in iTunes

MusicMagic plugin
=================
- Implements import of ratings from MusicMagic during rescan
- Ratings are stored in the standard slimserver tracks table
- Cannot set ratings (besides setting them in MusicMagic of course)
- Ratings survive a rescan by reimporting the ratings from MusicMagic during rescan.

Slimserver
==========
- Cannot set rating
- Ratings is shown in SongInfo menu on SqueezeBox and web interface

My opinion is that some important points are:
=============================================
1. Ratings must survive a rescan
2. Ratings must be possible to backup
3. Ratings must be possible set in SqueezeBox, web ui and cli
4. Ratings must be possible to view in SqueezeBox, web ui and cli
5. It must be possible to subscribe on rating changes for plugins/cli to be able to implement export/import plugin/application to new third party products

Now, what is your opionion regarding handling of ratings and do you have any suggestion how it should be handled ?

[tgoldstone]

IMHO slimserver db needs to be rock solid.

It should never require a db dump except in the most extreme circumstances.

If this was the case then ratings and other static information would be safe and easy to access.

It is better to have one database with all of the information in it then multiple DBs that somehow have to relate to each other in some way.

my two cents...

[kdf]

On 29-Apr-06, at 10:22 AM, erland wrote:
>
> MusicMagic plugin
> =================
> - Implements import of ratings from MusicMagic during rescan
> - Ratings are stored in the standard slimserver tracks table
> - Cannot set ratings (besides setting them in MusicMagic of course)
>
musicmagic API does allow setting. it just needs some mechanism for
implementing it. do you do it from the plugin, or update as part of
slimserver. I'm not sure yet.

>
> Slimserver
> ==========
> - Cannot set rating

I don't believe anything is preventing this. call $ds->updateOrCreate
on the track with the new rating supplied in the args. Search the code
and you should see lots of examples.

> - Ratings is shown in SongInfo menu on SqueezeBox and web interface
>
wonder if RATING might work in titleformat prefs? in theory any track
field should parse.

> My opinion is that some important points are:
> =============================================
> 1. Ratings must survive a rescan
> 2. Ratings must be possible to backup
exporting back to iTunes/musicmagic, etc ought to accomplish that.
>

>
> Now, what is your opionion regarding handling of ratings and do you
> have any suggestion how it should be handled ?
>
the plugin route would be my preferred way, at least until the above
gets worked out. No reason you couldn't dump a Storable archive of
ratings -> url maps for backups, and import after rescan. Moving to
mysql, as slimserver will be doing for 6.5 might make the schema
flexible enough to have a separate rating table that doesn't get wiped
but some other method to dump out ratings would still be useful.

with current implementation, there isn't an easy way to set rating in
web ui. From a plugin, you could create a "mixer" link such as those
used for MusicMagic and Moodlogic. You could hijack this to go to a
rating page for the given song/artist/etc. This way the hooks are
already there, and you don't have to compaign for each skin to make
changes (at least for those that DO support mixers)

i suppose another option would be to have an insertable block in the
songinfo page where rating is currently displayed. For the default
case, it could stay as text (or maybe simple graphics) but allow a
plugin on alter that with html (such as making it a link to adjust
ratings, etc).

-k

[erland]

On 29-Apr-06, at 10:22 AM, erland wrote:
>
> MusicMagic plugin
> =================
> - Implements import of ratings from MusicMagic during rescan
> - Ratings are stored in the standard slimserver tracks table
> - Cannot set ratings (besides setting them in MusicMagic of course)
>
musicmagic API does allow setting. it just needs some mechanism for
implementing it. do you do it from the plugin, or update as part of
slimserver. I'm not sure yet.

I think the user interaction should be handled by slimserver when the user selects to set a rating, slimserver should then set it in the internal slimserver database and post a command/notification that the plugin can subscribe on. When the plugin receives the notification it can set the rating in MusicMagic, iTunes or some other external application.

>
> Slimserver
> ==========
> - Cannot set rating

I don't believe anything is preventing this. call $ds->updateOrCreate
on the track with the new rating supplied in the args. Search the code
and you should see lots of examples.

Maybe I was unclear, I just tried to say that there is currently no way for the user to set the rating using standard slimserver. The TrackStat and iTunesUpdate plugin already today uses updateOrCreate to set the rating, so the support is there internaly its just not available in the user interface yet.

> My opinion is that some important points are:
> =============================================
> 1. Ratings must survive a rescan
> 2. Ratings must be possible to backup
exporting back to iTunes/musicmagic, etc ought to accomplish that.

The problem is that it would require third party software, I would prefer if we had a solution to support ratings without third party software. I can't use iTunes today since I run slimserver on linux and iTunes on a completely different computer on Windows. Another problem with iTunes is that it does not support FLAC files which also means that it can't store rating for flac files. MusicMagic is also a problem since it doesn't work correctly with unicode characters on linux yet.

the plugin route would be my preferred way, at least until the above
gets worked out. No reason you couldn't dump a Storable archive of
ratings -> url maps for backups, and import after rescan.

I think the url won't be enough because if I change a filename the url will change during rescan and the rating would be lost. But url together with musicbrainzid should be good enough, thats what TrackStat uses today for storing statistics.

i suppose another option would be to have an insertable block in the
songinfo page where rating is currently displayed. For the default
case, it could stay as text (or maybe simple graphics) but allow a
plugin on alter that with html (such as making it a link to adjust
ratings, etc).

I would love to be able to insert info in the songinfo page both in web ui and in SqueezeBox ui. This would not just be useful for rating but also for plugins such as Biography and AlbumReview and other plugins that show info related to a specific song.

[erland]

The problem I am starting to get with TrackStat is that TrackStat stores its ratings in its own table while other plugins like iTunesUpdate/MusicMagic just updates the tracks tables in the standard slimserver.

I have some simple synchronization code in TrackStat that moves ratings from the tracks table to the TrackStat(track_statistics) table. But the problem with this synchronization is that I am afraid of overwriting already existing ratings since I can't be sure which rating "database" that is the master for the user. Some will prefer to have an external source such as iTunes/MusicMagic as master, others will prefer TrackStat or some other plugin. The standard slimserver tracks table can't be the master today since it is cleared during rescans.

One way would be to just add an option in TrackStat where the user can specify if he wants the slimserver tracks table to be the master or if he wants the master to be the TrackStat(track_statistics) table.

I'm starting to feel that it might be better to start adding some rating functionallity to standard slimserver instead of extending TrackStat to handle all the rating stuff. So this is why I wanted to start a discussion related to this, so all of you that have ideas of how you would like the rating handling to work, keep posting.

[Yannzola]

IMHO slimserver db needs to be rock solid.

It should never require a db dump except in the most extreme circumstances.

If this was the case then ratings and other static information would be safe and easy to access.

It is better to have one database with all of the information in it then multiple DBs that somehow have to relate to each other in some way.

my two cents...

That's seriously wishful thinking... slimserver db is always getting rebuilt. What if ratings and playcount info were stored in a backup file from slimserver db directly... or what if the option of writing the ratings data directly to meta tag were provided? Potentially to multiple RATINGS tags related to user profile(s)... when and if Profiles are ever implemented.

[Yannzola]

I think the url won't be enough because if I change a filename the url will change during rescan and the rating would be lost. But url together with musicbrainzid should be good enough, thats what TrackStat uses today for storing statistics.

Erland... This is a bit of a tangent but isn't there some convergence occuring between musicbrainzid and music magic fingerprints occuring? http://blog.musicbrainz.org/archives...ngerprint.html

[erland]

What if ratings and playcount info were stored in a backup file from slimserver db directly... or what if the option of writing the ratings data directly to meta tag were provided?

If you by "metatag" mean that it should write a tag directly in the music file I think that might not be a good idea. The reason is that it is in my opinion to big risk to modify the music files from within slimserver. If a bug occurred in this logic of the code it could potentially destroy someones music.
Storing the info in a backup file at regular intervals might be an option though.

Erland... This is a bit of a tangent but isn't there some convergence occuring between musicbrainzid and music magic fingerprints occuring? http://blog.musicbrainz.org/archives...ngerprint.html

I think you are talking about the PUID which is a new identifier defined that can be calculated by analyzing the contents of the musicfiles. PUID's are as far as I know calculated by latest MusicMagic version and latest musicbrainz picard tagger and the resulting PUID is globaly unique and registered on musicdns.org.
I think it might be worth to consider to add this sort of id also for tracks in slimserver, but thats a completely different discussion.

[Yannzola]

I think you are talking about the PUID which is a new identifier defined that can be calculated by analyzing the contents of the musicfiles. PUID's are as far as I know calculated by latest MusicMagic version and latest musicbrainz picard tagger and the resulting PUID is globaly unique and registered on musicdns.org.
I think it might be worth to consider to add this sort of id also for tracks in slimserver, but thats a completely different discussion.

Yes... it was a tangent. But it seemed a good way to assign a permanent and unique id to tracks. Ratings and playcount data could be associated to these id's... which would survive renaming, relocation, etc. Otherwise, even if properly backed up this data could easily lose it's track association and become useless.

[erland]

Is there no one else that have any ideas regarding the handling of ratings in slimserver and the mentioned plugins ?