PDA

View Full Version : Handling of rating in MusicMagic, iTunesUpdate, TrackStat and slimserver



erland
2006-04-29, 10:22
I feel we need to start thinking about how ratings should be handled in third party plugins and in slimserver.

The main reason I decided to implement the TrackStat plugin is that:
1. Ratings where not supported in slimserver at the time
2. The slimserver database could not be considered a save storage of ratings since the tracks table will be cleared at rescan and track id's reused and in some situations/crashes the slimserversql.db file needs to be deleted.

Actually I think we need to think about more than ratings, all statistic data is interested such as "last played time", "added time", "play counts", "ratings" and maybe something else also. But lets start the discussion with ratings.

Today the situation as far as I know is:

TrackStat plugin
================
- Makes it possible to set/view ratings from SqueezeBox, web ui and cli
- Ratings can be viewed as titleFormats for tracks in SqueezeBox, web ui
- Ratings in TrackStat survives a rescan since they are stored in a separate table which is not cleared during rescan.
- Ratings can be backup/restored to xml file
- TrackStat sets ratings in the standard slimserver tracks table(in 6.5)
- Implements MusicMagic export/import of ratings

iTunesUpdate plugin
===================
- Makes it possible to set/view ratings from SqueezeBox and web ui
- Ratings in iTunesUpdate is sent and stored in iTunes
- iTunesUpdate sets ratings in the standard slimserver tracks table(in 6.5)
- Ratings in iTunesUpdate survives a rescan since they are stored in iTunes

MusicMagic plugin
=================
- Implements import of ratings from MusicMagic during rescan
- Ratings are stored in the standard slimserver tracks table
- Cannot set ratings (besides setting them in MusicMagic of course)
- Ratings survive a rescan by reimporting the ratings from MusicMagic during rescan.

Slimserver
==========
- Cannot set rating
- Ratings is shown in SongInfo menu on SqueezeBox and web interface

My opinion is that some important points are:
=============================================
1. Ratings must survive a rescan
2. Ratings must be possible to backup
3. Ratings must be possible set in SqueezeBox, web ui and cli
4. Ratings must be possible to view in SqueezeBox, web ui and cli
5. It must be possible to subscribe on rating changes for plugins/cli to be able to implement export/import plugin/application to new third party products

Now, what is your opionion regarding handling of ratings and do you have any suggestion how it should be handled ?

tgoldstone
2006-04-29, 10:30
IMHO slimserver db needs to be rock solid.

It should never require a db dump except in the most extreme circumstances.

If this was the case then ratings and other static information would be safe and easy to access.

It is better to have one database with all of the information in it then multiple DBs that somehow have to relate to each other in some way.

my two cents...

kdf
2006-04-29, 11:23
On 29-Apr-06, at 10:22 AM, erland wrote:
>
> MusicMagic plugin
> =================
> - Implements import of ratings from MusicMagic during rescan
> - Ratings are stored in the standard slimserver tracks table
> - Cannot set ratings (besides setting them in MusicMagic of course)
>
musicmagic API does allow setting. it just needs some mechanism for
implementing it. do you do it from the plugin, or update as part of
slimserver. I'm not sure yet.

>
> Slimserver
> ==========
> - Cannot set rating

I don't believe anything is preventing this. call $ds->updateOrCreate
on the track with the new rating supplied in the args. Search the code
and you should see lots of examples.

> - Ratings is shown in SongInfo menu on SqueezeBox and web interface
>
wonder if RATING might work in titleformat prefs? in theory any track
field should parse.

> My opinion is that some important points are:
> =============================================
> 1. Ratings must survive a rescan
> 2. Ratings must be possible to backup
exporting back to iTunes/musicmagic, etc ought to accomplish that.
>

>
> Now, what is your opionion regarding handling of ratings and do you
> have any suggestion how it should be handled ?
>
the plugin route would be my preferred way, at least until the above
gets worked out. No reason you couldn't dump a Storable archive of
ratings -> url maps for backups, and import after rescan. Moving to
mysql, as slimserver will be doing for 6.5 might make the schema
flexible enough to have a separate rating table that doesn't get wiped
but some other method to dump out ratings would still be useful.

with current implementation, there isn't an easy way to set rating in
web ui. From a plugin, you could create a "mixer" link such as those
used for MusicMagic and Moodlogic. You could hijack this to go to a
rating page for the given song/artist/etc. This way the hooks are
already there, and you don't have to compaign for each skin to make
changes (at least for those that DO support mixers)

i suppose another option would be to have an insertable block in the
songinfo page where rating is currently displayed. For the default
case, it could stay as text (or maybe simple graphics) but allow a
plugin on alter that with html (such as making it a link to adjust
ratings, etc).

-k

erland
2006-04-29, 13:35
On 29-Apr-06, at 10:22 AM, erland wrote:
>
> MusicMagic plugin
> =================
> - Implements import of ratings from MusicMagic during rescan
> - Ratings are stored in the standard slimserver tracks table
> - Cannot set ratings (besides setting them in MusicMagic of course)
>
musicmagic API does allow setting. it just needs some mechanism for
implementing it. do you do it from the plugin, or update as part of
slimserver. I'm not sure yet.

I think the user interaction should be handled by slimserver when the user selects to set a rating, slimserver should then set it in the internal slimserver database and post a command/notification that the plugin can subscribe on. When the plugin receives the notification it can set the rating in MusicMagic, iTunes or some other external application.



>
> Slimserver
> ==========
> - Cannot set rating

I don't believe anything is preventing this. call $ds->updateOrCreate
on the track with the new rating supplied in the args. Search the code
and you should see lots of examples.

Maybe I was unclear, I just tried to say that there is currently no way for the user to set the rating using standard slimserver. The TrackStat and iTunesUpdate plugin already today uses updateOrCreate to set the rating, so the support is there internaly its just not available in the user interface yet.



> My opinion is that some important points are:
> =============================================
> 1. Ratings must survive a rescan
> 2. Ratings must be possible to backup
exporting back to iTunes/musicmagic, etc ought to accomplish that.

The problem is that it would require third party software, I would prefer if we had a solution to support ratings without third party software. I can't use iTunes today since I run slimserver on linux and iTunes on a completely different computer on Windows. Another problem with iTunes is that it does not support FLAC files which also means that it can't store rating for flac files. MusicMagic is also a problem since it doesn't work correctly with unicode characters on linux yet.



the plugin route would be my preferred way, at least until the above
gets worked out. No reason you couldn't dump a Storable archive of
ratings -> url maps for backups, and import after rescan.
I think the url won't be enough because if I change a filename the url will change during rescan and the rating would be lost. But url together with musicbrainzid should be good enough, thats what TrackStat uses today for storing statistics.




i suppose another option would be to have an insertable block in the
songinfo page where rating is currently displayed. For the default
case, it could stay as text (or maybe simple graphics) but allow a
plugin on alter that with html (such as making it a link to adjust
ratings, etc).

I would love to be able to insert info in the songinfo page both in web ui and in SqueezeBox ui. This would not just be useful for rating but also for plugins such as Biography and AlbumReview and other plugins that show info related to a specific song.

erland
2006-04-29, 13:38
The problem I am starting to get with TrackStat is that TrackStat stores its ratings in its own table while other plugins like iTunesUpdate/MusicMagic just updates the tracks tables in the standard slimserver.

I have some simple synchronization code in TrackStat that moves ratings from the tracks table to the TrackStat(track_statistics) table. But the problem with this synchronization is that I am afraid of overwriting already existing ratings since I can't be sure which rating "database" that is the master for the user. Some will prefer to have an external source such as iTunes/MusicMagic as master, others will prefer TrackStat or some other plugin. The standard slimserver tracks table can't be the master today since it is cleared during rescans.

One way would be to just add an option in TrackStat where the user can specify if he wants the slimserver tracks table to be the master or if he wants the master to be the TrackStat(track_statistics) table.

I'm starting to feel that it might be better to start adding some rating functionallity to standard slimserver instead of extending TrackStat to handle all the rating stuff. So this is why I wanted to start a discussion related to this, so all of you that have ideas of how you would like the rating handling to work, keep posting.

Yannzola
2006-04-29, 14:31
IMHO slimserver db needs to be rock solid.

It should never require a db dump except in the most extreme circumstances.

If this was the case then ratings and other static information would be safe and easy to access.

It is better to have one database with all of the information in it then multiple DBs that somehow have to relate to each other in some way.

my two cents...

That's seriously wishful thinking... slimserver db is always getting rebuilt. What if ratings and playcount info were stored in a backup file from slimserver db directly... or what if the option of writing the ratings data directly to meta tag were provided? Potentially to multiple RATINGS tags related to user profile(s)... when and if Profiles are ever implemented.

Yannzola
2006-04-29, 14:38
I think the url won't be enough because if I change a filename the url will change during rescan and the rating would be lost. But url together with musicbrainzid should be good enough, thats what TrackStat uses today for storing statistics.

Erland... This is a bit of a tangent but isn't there some convergence occuring between musicbrainzid and music magic fingerprints occuring? http://blog.musicbrainz.org/archives/2006/03/new_fingerprint.html

erland
2006-04-29, 21:03
What if ratings and playcount info were stored in a backup file from slimserver db directly... or what if the option of writing the ratings data directly to meta tag were provided?If you by "metatag" mean that it should write a tag directly in the music file I think that might not be a good idea. The reason is that it is in my opinion to big risk to modify the music files from within slimserver. If a bug occurred in this logic of the code it could potentially destroy someones music.
Storing the info in a backup file at regular intervals might be an option though.


Erland... This is a bit of a tangent but isn't there some convergence occuring between musicbrainzid and music magic fingerprints occuring? http://blog.musicbrainz.org/archives/2006/03/new_fingerprint.html
I think you are talking about the PUID which is a new identifier defined that can be calculated by analyzing the contents of the musicfiles. PUID's are as far as I know calculated by latest MusicMagic version and latest musicbrainz picard tagger and the resulting PUID is globaly unique and registered on musicdns.org.
I think it might be worth to consider to add this sort of id also for tracks in slimserver, but thats a completely different discussion.

Yannzola
2006-04-29, 23:38
I think you are talking about the PUID which is a new identifier defined that can be calculated by analyzing the contents of the musicfiles. PUID's are as far as I know calculated by latest MusicMagic version and latest musicbrainz picard tagger and the resulting PUID is globaly unique and registered on musicdns.org.
I think it might be worth to consider to add this sort of id also for tracks in slimserver, but thats a completely different discussion.
Yes... it was a tangent. But it seemed a good way to assign a permanent and unique id to tracks. Ratings and playcount data could be associated to these id's... which would survive renaming, relocation, etc. Otherwise, even if properly backed up this data could easily lose it's track association and become useless.

erland
2006-05-04, 13:42
Is there no one else that have any ideas regarding the handling of ratings in slimserver and the mentioned plugins ?

Volition
2006-07-12, 21:15
I've posted recently in the 3rd party plugins, however i have recently Submitted a request
http://bugs.slimdevices.com/show_bug.cgi?id=3741

In regards to my needs for ratings.

The id3v2 popm field handles ratings within the metatags. Mediamonkey, WMP & foobar as far as i know handles ratings in this way.

1st part; requires Plugin to select The Slimserver scan to include POPM field and add to slimserver DB.

2nd Part; Browse By ratings. With an option to browse by individuals ratings. As the popm field allows you to define whose rating it is by an email field. And display Ratings on ui, per song a la trackstat.

3rd Part; Add to random mix or sqlplaylist options to have a greater than less than specific rating included.

4th Part; Writing back to metatag. This obviously needs more input. As Erland has stated his fears of writing to metatags.
Ideally the final product though would ideally be something like Hold Rating key down on remote, ui prompts for user. Which can be preset in an options item by user. e.g User 1 is John, User 2 is mary. User 3 Can be Mary & john.

This implementation of ratings obviously does not help Itunes users. Personally i don't like itunes. And use media monkey. And my brother uses wmp, at home.

How can we standardise ratings and then ultimately output ratings to the relevant media player.

Personally i have no experience in perl. Though i am keen to implement a plugin, which fulfills my purpose. What i need is some pointers to websites resources to start the steep learning curve. I know i can pick up the knowledge in a hurry. Please help me get the ball rolling.

in saying this it has just come to me that i need to clarify the modules.

1. Scanning Database for Popm Ratings. Select on or off.
2. Displaying ratings
3. PLaylists with ratings (Sql Playlist)(done)
4. Writing to Db's and/or metatags.
4a. Itunes (itunesupdate)(done)
4b. Metatags - WMP, Media monkey, etc

Suggestions, contributions, ideas, direction?



4th Part. Some way of

Pale Blue Ego
2006-07-13, 09:07
A few thoughts:

I like the idea of a ratings tag in the actual file itself. That way, it moves with the file. But rating files from the slimserver interface becomes a problem, then. You don't ever want slimserver to write tags. But you could get slimserver to write to the ratings field in the database. Then the main problem is how to populate the file tags from the ratings field. I suppose it could be done with a separate script that calls a command-line tagger.

A larger problem with ratings as a whole is that, for me, the 5-star system is too coarse. I've been using TrackStat for months and I find that 90% of my ratings are "3". So the only real benefit is in excluding the 5% of tracks that are 1-2 or including the 5% of tracks that are 4-5. I would much rather have a 0-9 rating system, or even a 00-99 system.

I see that the POPM rating field is a value between 1-255.

If we go to all the trouble of implementing ratings in slimserver, I feel we should design a finer-grained system than the iTunes 5-star system.

Thanks for reading my suggestions.

erland
2006-07-13, 10:33
...the main problem is how to populate the file tags from the ratings field. I suppose it could be done with a separate script that calls a command-line tagger.

This shouldn't be a problem, its easy to access the slimserver database from outside in the 6.5 release since mysql is used. So a separate script could get all tracks from the slimserver db and launch an external tagger to set the tags in the file.


A larger problem with ratings as a whole is that, for me, the 5-star system is too coarse. I've been using TrackStat for months and I find that 90% of my ratings are "3". So the only real benefit is in excluding the 5% of tracks that are 1-2 or including the 5% of tracks that are 4-5. I would much rather have a 0-9 rating system, or even a 00-99 system.

First of all I have seen a similar behaviour at my own music collection, I tend to set the ratings in TrackStat as follows:
4 - A good track
3 - A standard track, most of my tracks end up here
2 - A track I feel I probably don't want to listen to again.

I have so far never used rating 1 and 5. Most of my tracks are rated as 3 and maybe 10-20% are rated as 4. I have been thinking about to start to use rating 5 also but the problem is that it takes a lot of time to listen through all music and change the ratings correctly.

I am not sure our behaviour would be solved by a fine grained rating system, instead I feel that it maybe could be even even more coarse than it is today. Are you really sure you would set the ratings to 52, 51, 50, 49, 48 if we had a fine grained system and that you wouldn't still just set all track ratings to 50 ?

Another question is if its really important to differ ratings in so small steps as 52,51,50,49,48, as an example do you see yourself making a playlist to just include the tracks rated as 51 and feel that its important that the 52,50,49,48 aren't included as well ?

Pale Blue Ego
2006-07-13, 11:27
I seriously doubt I would ever want to rate tracks as finely as 48, 49, 50, but I could see myself using ratings of 50, 55, 60.

At the very least, a 0-9 system would be a lot more useful than 1-5. With 1-5, loading up a playlist of 4+ gives me only a tiny portion of my collection (maybe 5%), while 3+ gives me almost the whole thing (95%).

A 0-9 system would allow me a lot more control. 8+ would truly be the cream, 7+ would be superb, and 6+ would give me all-day listening with no bad songs. Even 5+ would give me most of my library without the 15% - 20% of clinkers that can buzz-kill a listening session or par-tay vibe.

Rating filters combined with genre selection would allow a lot of control without a lot of effort.

erland
2006-07-13, 12:16
Slimserver internally and also TrackStat uses a rating system between 1-100, so the 1-5 limitation today is just in the user interface and CLI interface of TrackStat.
If more people requests it I can quite easy change it so it supports 1-10 ratings, the main problem is probably just that everyone has to learn the remote control commands again so they use 8 instead of 4 for a good track. It might even be possible to make the behaviour configurable if needed.

Personally I still feel that 1-5 is enough in most cases, its just a matter of start using the whole scale. But I can imagine that this can differ depending on your taste and size of library.

As long as you almost always put all tracks at a rating of 3 I can understand that it feels a bit limited. Think about your example, you actually only used 5 different rating values:
8+ : The cream (Could be rated as 5 today)
7+ : Superb (Could be rated as 4 today)
6+ : No bad songs (Could be rated as 3 today)
5+ : Whole library without the "clinkers" (Could be rated as 2 today)
4- : The "clinkers" that kills a listening session (Could be rated as 1 today)

Another problem is that ratings tends to change over time, usally you feel a track is good just after you have purcased it, a bit later its not so good and some months/years later it can be good again because it was a long time since you listend to it. If the rating scale is fine graned it will probably result in that most tracks has the wrong rating because you don't take the time to go through it and change the rating.

jania
2006-07-13, 13:29
If the rating scale was change from 1-5 to 1-10 it would be a lot of work for those of us that have rated most of our music on the 1-5 scale.
I'm not too eager to go through and re-rate all my music.

Personally I find 1-5 to be enough, sometimes I have trouble deciding if a song should be a 3 or a 4, etc. When in doubt I give the song the higher rating so I'm more likely to hear it again and can re-rate it at that time.

just my 2 cents.

-jason

erland
2006-07-13, 13:38
If the rating scale was change from 1-5 to 1-10 it would be a lot of work for those of us that have rated most of our music on the 1-5 scale.
I'm not too eager to go through and re-rate all my music.

If you are talking about TrackStat or slimserver ratings it would not be neccesary to do any re-rating, the values in the database would not change, just the meaning of the buttons on the remote. Your 4 ratings would automatically be shown as 8 ratings after the change.

But as I said previously I don't think I will change anything regarding this in TrackStat unless there are more users who feel that 1-10 is better than 1-5.

Volition
2006-07-13, 18:06
I personally like the 1-10 rating system. It lets me fine tune my Three ratings that's all. when i start getting towards the 8's & above it does the same. Though like stated above my rating of a track depends on what i ate for breakfast, what time of the day & even what underwear i'm wearing. The POPM field is from 1 -255.
Currently, the values are assigned as such. Media monkey uses Half stars as well. Therefore any changes to how fine the rating is. does not really affect your current ratings.

Where 0 stars is 0
Where 1 star is 1
Where 2 Stars is 64
Where 3 stars is 128
Where 4 stars is 196
Where 5 stars is 255

In regards, to the tagging of files, i believe that this is a module that can be added, as a seperate entity. After i have worked on ways of just displaying popm ratings, and managing them.

Pale Blue Ego
2006-07-14, 07:44
I didn't actually use only 5 ratings (in the proposed 10-point system). I merely gave an example of 5 different ratings FILTERS

With a 10-point rating system, I would certainly use all 10. With a 20,000 track library, it would break down something like this:

9 - 1% [200]
8 - 4% [800]
7 - 12% [2400]
6 - 32% [6400]
5 - 27% [5400]
4 - 11% [2200]
3 - 6% [1200]
2 - 4% [800]
1 - 2% [400]
0 - 1% [200]

So, a ratings FILTER of 9+ would give me the best 1% of my tracks
8+ would give the best 5%
7+ would give the best 17%
6+ would give the best 49%
5+ would give the best 76%
4+ would give the best 87%

You can see that this gives a lot more control.

As you have said, going to a 10-point system would not affect any currently rated tracks - their values would just be doubled. And even those who do like the current 5-star system have said they sometimes can't decide how to rate a track.

Well, you guys know where I stand :) Thanks for listening.