[Announce] Music Similarity DSTM mixer

[cpd73]

Sigur Rós present another set of interesting characters that seem to give problems:

I've updated the python code, can you checkout the latest from github and copy the contents of "lib" over your current install? Saves me making another alpha release...

[mruddo]

That's great - thanks... I got an additional 60 or so tracks added after those changes (turns out another issue was double quotes - so having 12" in the name for example was also causing exlcusions prior to this).

Additionally, I see you've also got the check for track length in there - which is fantastic, as I no longer have to wait for it to analyse whole audio-book chapters.

And the extra logging to differentiate any Essentia vs. Musly failures too.

I have just short of 31,000 tracks analysed now, so time to listen to some mixes!

[cpd73]

turns out another issue was double quotes - so having 12" in the name for example was also causing exlcusions prior to this

Odd. I cannot create a filename with a double quote on Windows. How do you create such a file?

[mruddo]

I think like apostropes which have both "straight" and cuved (opening and closing) variations, you can use the curved variation in a filename.

So you have " vs ˮ... and variations on that theme. I can't be too sure which is actually being used in all cases though, as looking at charmap in Windows you can see a few similar looking variations.

I think EAC and/or Foobar may automatically substitute some of these characters from metadata in the file.

[cpd73]

I think like apostropes which have both "straight" and cuved (opening and closing) variations, you can use the curved variation in a filename.

So you have " vs ˮ... and variations on that theme. I can't be too sure which is actually being used in all cases though, as looking at charmap in Windows you can see a few similar looking variations.

Can you detail a filename that failed? As I've just tried "Test (12ˮ Remix).ogg" and that worked.

[mruddo]

Just to reiterate, these failures were prior to your most recent utf-8 change - so they're OK now!

All tracks on this album failed:

Angel Dream (Songs and Music from The Motion Picture “She's the One”)

And there were a few other 12" remix type names that failed too...

They were just some of the 60 or so additional tracks added on the most recent analysis pass.

[cpd73]

Just to reiterate, these failures were prior to your most recent utf-8 change - so they're OK now!

Ah! OK, that explains why things work now!

[mruddo]

So far I'm impressed... It's playing a good mix from most starting points, and no odd behaviour to report so far. What I like about the analysis approach is that it seems to pull in a more broad range of alternative tracks than a simple grouping determined by a website like LastFM - which is my default DSTM mix source (for local files).

I've not set up any genre groups yet, so I'm still ignoring the genre. Will give that a go next, as I think it makes sense to have certain groups to hone in some of those choices.

BTW, I love the idea of excluding Christmas until December!

Out of curiosity I also had a look at the db entries and compared the key/BMP analysis with Mixed In Key, which as you may recall I'd considered as an alternate DSTM source. Suffice to say, this is a far superior in terms of practicality, and the key/BMP analysis is pretty much in line with MiK, so I don't think I'll be going back there.

[cpd73]

So far I'm impressed... It's playing a good mix from most starting points, and no odd behaviour to report so far.

Glad to hear is working for someone else. Perhaps I'll formalize the release...

[prabbit]

I just had AcounticBrainz add mood and BPM metadata to my files in hopes of using that information for mixes. It appears I should have looked at the forums a bit closer before doing that. Apparently, Essentia/Musly is the way to go (Roland0 has a similar project), even for this long-time Spicefly Sugarcube user. My LMS server is pCP, but I'll be doing Essentia analysis and running my API server on Windows. It'll be my weekened/next week project.

But first a couple of questions, asked out of ignorance but will set my expectations.

MUSICSIMILARITY_ATTRIBUTE_HANDLING_NOTE
If you have analysed your tracks with Essentia, then this can be used to filter tracks based upon attributes (e.g. BPM, key, danceability, aggressiveness, etc.) and also for the similarity score.

Filter on atttributes
Only acccept tracks that have similar attributes.

1. Will 'mood' (or similar) be one of the attributes that can be used?

2. Are all attributes (BPM, key, danceability, aggressiveness, etc.) combined by Essentia into a single score and is that what used for matching? If so, is there a way to add weight individual attributes (i.e., I want danceability to get +10 but aggressiveness to not change)?

[cpd73]

Essentia analysis and running my API server on Windows. It'll be my weekened/next week project.

If you use windows, then you can download the alpha3 release from the initial post but then get the latest python code from the github repo - as I made some changes that are not in a release.

1. Will 'mood' (or similar) be one of the attributes that can be used?

2. Are all attributes (BPM, key, danceability, aggressiveness, etc.) combined by Essentia into a single score and is that what used for matching? If so, is there a way to add weight individual attributes (i.e., I want danceability to get +10 but aggressiveness to not change)?

What this mixer does is, foreach seed track:

1. Query Musly for similar tracks
2. Filter these tracks by metadata - title, artist, album
3. Filter by Essentia key
4. Filter by Essentia BPM
5. Filter by Essentia 'loundness'
6. If tracks have been analysed with Essentia highlevel models then; find the 4 Essentia attributes (danceable, aggressive, etc) of the seed track with which Essentia has very high (80%) or very low (20%) confidence, and check the similar tracks against these. I take Essentia very low value to mean high probability that its not - e.g. danceable 0.8 => is danceable, danceable 0.2 => is not danceable. (There is no 'mood' attribute)

The filtering against high-level Essentia attributes seems to filter out way too many tracks for me, so I'm not sure how useful it is. Also, if you have analysed your tracks with Essentia high level (so, on Linux) then you can opt to use these attributes for similarity.

However, please bear in mind that I'm in no way an expert in this - I would never have heard of Essentia or Musly if not for Roland0 - and I'm just experimenting with trying to create a useful, for me, DSTM mixer that is not MusicIP. And, again for me, so far it seems to work well and is what I currently use.

[AF1]

Hey.

I thought I'd catch up with your latest changes and tried the latest GitHub version of server and plugin. Update all python modules and installed 2 new ones.
I don't use Essentia and the only Essentia-related item in the config.json has always been

 "essentia":{
  "enabled":false
 },

Creating a similarity mix gives me

E Exception on /api/similar [POST]
Traceback (most recent call last):
File "/Users/af/lmsmusly-pyenv/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
response = self.full_dispatch_request()
File "/Users/af/lmsmusly-pyenv/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/af/lmsmusly-pyenv/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/af/lmsmusly-pyenv/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/Users/af/Music/MusicSimilaritySERVER/music-similarity-master/lib/app.py", line 444, in similar_api
ess_cfg = get_essentia_cfg(cfg, params)
File "/Users/af/Music/MusicSimilaritySERVER/music-similarity-master/lib/app.py", line 241, in get_essentia_cfg
ess_cfg={'enabled': config['essentia']['enabled'], 'highlevel': config['essentia']['highlevel']}
KeyError: 'highlevel'

Since it mentions essentia (although my config should disable it) I have to ask: is it no longer possible to use MusicSimilarity just with Musly?

[mruddo]

Glad to hear is working for someone else. Perhaps I'll formalize the release...

That'd be great.

Whilst I can't spend all day listening to music, I'm very impressed with the mixes I'm getting...

I did say I would try utilising the Genre groups, but I'm not thinking that's necessary at the moment as it does not steem to stray to far off course, and I'd rather not restrict my options for varying playback at this time. What interesting is that if I go and play what I'd consider a minority genre, like "Classical", which I don't have a gread deal of, it seems to follow with mostly classical tunes, or piano based pop that flows quite well - which demonstrates a remarkably effective matching ability in place with your default settings.

I do have a couple of questions in respect of genre though:

i) Will it handle multiple genres on a track? (e.g. if I have a track with "Rock" and "Soundtrack", would a group with either of these still potentially pick the track?
ii) In the Server config, you have "Restrict genres", Yes/No. Does this override the choice of DSTM - "Similarity" or "Similarity (Ignore Genre)". I'm not to sure if both are required, unless you want different players utilising different options.

And a couple of thoughts on what may be useful additional options in future:

i) Can you add a basic genre exclusion option? (e.g. Exclude "Audio Book", "Language Course" to exclude things you're never realistically going to want in a mix.) This could also be applied when analysing tracks. As it happens, the matching's so good, that maybe it's never going jump from a song to a book, but there's probably not a lot of point analysing such genres anyway.
ii) What about considering playcount in the selection? I'm thinking this could be used to prioritise tracks that have not been listened to before or recently.

Admittedly, "recently" is tricky as there's likely no "last date played" to work with. Something like the traditional radio feature, "haven't heard this in ages" implementation would be nice - although likely that'd require more detail to pick out songs played multiple times, but not in the past x months - or something along those lines, so is likely not so simple.

[cpd73]

Since it mentions essentia (although my config should disable it) I have to ask: is it no longer possible to use MusicSimilarity just with Musly?

Should be fixed now.

[cpd73]

i) Will it handle multiple genres on a track? (e.g. if I have a track with "Rock" and "Soundtrack", would a group with either of these still potentially pick the track?

It should handle multiple genres - at least the code is written to try and handle this, but I have no files tagged this way.

ii) In the Server config, you have "Restrict genres", Yes/No. Does this override the choice of DSTM - "Similarity" or "Similarity (Ignore Genre)". I'm not to sure if both are required, unless you want different players utilising different options.

Yeah, I'm not sure if I'm too keen on the 2 DSTM entries - this was added as a request by "afriend" to enable quick switching of using genres or not.

i) Can you add a basic genre exclusion option? (e.g. Exclude "Audio Book", "Language Course" to exclude things you're never realistically going to want in a mix.) This could also be applied when analysing tracks. As it happens, the matching's so good, that maybe it's never going jump from a song to a book, but there's probably not a lot of point analysing such genres anyway.

I could add this for the analysis stage.

[Edit] I've added code to handle this. See docs/OtherConfig.md

ii) What about considering playcount in the selection?

The python API code treats the DB as read-only, so has no way to store playcounts. What I could do is have the LMS plugin ask for more tracks than are required and then have the plugin choose tracks that have been played the least. I'l have to think about this, as its not something I particularly care about.

Admittedly, "recently" is tricky as there's likely no "last date played" to work with. Something like the traditional radio feature, "haven't heard this in ages" implementation would be nice - although likely that'd require more detail to pick out songs played multiple times, but not in the past x months - or something along those lines, so is likely not so simple.

Again, the API code has no knowledge of when a track was last played, etc.