Genres, moods, themes and styles

[lauret]

>> A lot of files in my collection have mood tags, some files have as much
>> as 10 moods stored, so your solution to create a library per mood seems
>> unfeasible.
>>
>
> I wonder if it would be possible to adjust the browsing logic in the
> core so you for each library optionally can specify a parent menu which
> the library should be placed inside ?
> That way you would be able to specify that your libraries should be
> placed inside a “Moods” menu and that would maybe make the virtual
> libraries approach a bit more feasible since you mood libraries wouldn’t
> be showed together with all other virtual libraries.

As I just mentioned that should already be feasible with today's tools.
Look at the "Additional Browse Modes" menu.

--

Michael

[lauret]

Genres, moods, themes and styles

As metadata providers get better and better I find I have a need to be able to differentiate more into the types of genres I am storing for my music collection. Rateyourmusic has a genre and descriptor (see https://rateyourmusic.com/release/al...back-in-black/), Discogs has a genre and style (see https://www.discogs.com/master/8471), and Allmusic goes over the top with genre, style, mood, and theme (see https://www.allmusic.com/album/mw0000188862). You used to be able to use CustomScan to load these extra tags, but this plugin is not working for me anymore with LMS 8.0.

Is there a broader wish for a functionality where different types of genres can be stored and searched for in the database? If so, could this eventually become a part of main LMS or is this functionality too marginal to be considered? The upside of integrating it into LMS is that for instance mood can also be used in different places, for instance to create a random mix based on a certain mood (or moods).

What do you think?

--
lauret

[bernt]

LastMix

The plugin LastMix maybe?

[lauret]

The plugin LastMix maybe?

My use case lies with metadata stored in my local files. AFAIK LastMix uses Last.fm to create mixes and does not use local tag information, certainly not the tag mood, which is not stored in the LMS db.

[Roland0]

As metadata providers get better and better I find I have a need to be able to differentiate more into the types of genres I am storing for my music collection.
Is there a broader wish for a functionality where different types of genres can be stored and searched for in the database? If so, could this eventually become a part of main LMS or is this functionality too marginal to be considered? The upside of integrating it into LMS is that for instance mood can also be used in different places, for instance to create a random mix based on a certain mood (or moods).

A flexible mechanism to add metadata (basically, an arbitrary number of key-value pairs associated with a track) would be very useful. As I needed this for the LMS Essentia Integration, I wrote a plugin (which includes functionality to upload metadata and to query it). The obvious disadvantage of this is that a client (e.g. a GUI) has to support this specific method to access it (so Erland's plugins' metadata is used only by his plugins, LMS Essentia's only by my apps etc.), therefore having an official standard would be nice to have.
That being said, it may very well be rather marginal and it seems to me the core development focus nowadays is mostly on streaming, which would not benefit from having such a functionality.

btw, this may be of interest.

[lauret]

A flexible mechanism to add metadata (basically, an arbitrary number of key-value pairs associated with a track) would be very useful. As I needed this for the LMS Essentia Integration, I wrote a plugin (which includes functionality to upload metadata and to query it). The obvious disadvantage of this is that a client (e.g. a GUI) has to support this specific method to access it (so Erland's plugins' metadata is used only by his plugins, LMS Essentia's only by my apps etc.), therefore having an official standard would be nice to have.
That being said, it may very well be rather marginal and it seems to me the core development focus nowadays is mostly on streaming, which would not benefit from having such a functionality.

Very interesting project where you are working on. Did you end up storing the extra metadata in some custom tables? I'm wondering how the extra information from Essentia is queried/browsed from the user perspective. Is that also part of your plugin?

btw, this may be of interest.

Looks interesting, thanks!

[Roland0]

Did you end up storing the extra metadata in some custom tables?

Yes (in LMS' existing persistent DB)

I'm wondering how the extra information from Essentia is queried/browsed from the user perspective.

see screenshots

Is that also part of your plugin?

No, the Essentia plugin only provides the API, admin stuff, data visualization and a JSON upload facility (so you can import metadata from any source. I actually planned on modifying whatlastgenre to output JSON files in a compatible format, but I am currently focusing instead on using neural networks to classify music (which is already possible with Essentia+Tensorflow, but the system I'm developing let's you define your own categories and learns from your own music collection))

[prabbit]

I am currently focusing instead on using neural networks to classify music (which is already possible with Essentia+Tensorflow, but the system I'm developing let's you define your own categories and learns from your own music collection))

That sounds cool!

[erland]

Is there a broader wish for a functionality where different types of genres can be stored and searched for in the database? If so, could this eventually become a part of main LMS or is this functionality too marginal to be considered? The upside of integrating it into LMS is that for instance mood can also be used in different places, for instance to create a random mix based on a certain mood (or moods).

What do you think?

I guess I’m a bit biased since I’ve spent a considerable time to work on this in CustomScan/CustomBrowse plugins but to me it would be very useful to some kind of support for it in LMS core (or broadly supported plugin). The issue with having it as a plugin is that it’s hard to get support for searching/browsing in all third party user interfaces/apps. Support for advanced search is especially important since the virtual library functionality is based on advanced search.

For me the main usage scenario would be to be able to set “tags/genres” without having to modify music files and to be able to find and play music with a certain “tag/genre” (or create a smart playlist based on it). Importing genres/tags from third party sources such as musicbrainz, lastfm and similar would also be very useful. With latest support for integrating streaming service favorites in your library I’d like the functionality both for local music and integrated songs from streaming services.

However, there are probably a few obstacles that need to be handled before it can be integrated in the core:
- who will maintain the functionality ? My guess is that it’s less than 20% of the LMS users that is interested in this kind of functionality and this makes it less interesting for core developers to maintain.
- one main principle with LMS is that database can be discarded without loosing any information. If this is true in the future you would be limited to genres/tags that can be integrated from third party sources which would make it a bit less useful IMHO.

I think the core functionality (if implemented) could focus on storage, scanner hooks (mostly there already), search and ability to view genres/tags of a song. Functionality to import genres/tags from third party sources and maintain locally stored genres/tags could be handled by plugins. Functionality for browse/smart playlists in core would be nice but it could also be handled by third party plugins. Browse/smart playlists tends to get complicated so supporting it in core will probably take up too much time.

CustomScan/CustomBrowse was implemented a long time ago, before the current scanner hooks and browsedb support existed in core. They would be possible to simplify a lot and made to work better if newly implemented today. Unfortunately I’m unlikely to get the time to do it myself but I’d love if someone else does something regarding it.

[lauret]

Erland, thanks for your response!

However, there are probably a few obstacles that need to be handled before it can be integrated in the core:
- who will maintain the functionality ? My guess is that it’s less than 20% of the LMS users that is interested in this kind of functionality and this makes it less interesting for core developers to maintain.

If this functionality could be a flexible facility for track based tags I'm guessing this could also be used in a couple of plugins and subsequently increases usage across LMS users. This however does not solve the problem of maintaining it of course, but should increase the incentive to keep it working.

- one main principle with LMS is that database can be discarded without loosing any information. If this is true in the future you would be limited to genres/tags that can be integrated from third party sources which would make it a bit less useful IMHO.

I do not think this would be limited to third party sources. You should also be able to use tags which are stored in your local music, these tags can be discarded without losing information. But the possibility to enrich these tags with third-party information sounds very helpful.

I think the core functionality (if implemented) could focus on storage, scanner hooks (mostly there already), search and ability to view genres/tags of a song. Functionality to import genres/tags from third party sources and maintain locally stored genres/tags could be handled by plugins. Functionality for browse/smart playlists in core would be nice but it could also be handled by third party plugins. Browse/smart playlists tends to get complicated so supporting it in core will probably take up too much time.

I was thinking about duplicating the tag handling and search/browse handling of genres, but making it configurable. This should then be user-configurable, but also configurable from a plugin perspective. For instance, I would want to add the handling of the tag mood, which I have stored into my local collection. Roland0 above would want to do the same, but then enable the handling of mood from inside his plugin so it can be stored using the scanning hooks.

I guess the main question is if this functionality would be considered as part of LMS core.

[erland]

I do not think this would be limited to third party sources. You should also be able to use tags which are stored in your local music, these tags can be discarded without losing information. But the possibility to enrich these tags with third-party information sounds very helpful.

Definitely, what I meant was that they could be imported from anything outside LMS. Could be tags in files, metadata(xml) files in the file system, other software you run on your local network or cloud based services/data sources.

I was thinking about duplicating the tag handling and search/browse handling of genres, but making it configurable. This should then be user-configurable, but also configurable from a plugin perspective. For instance, I would want to add the handling of the tag mood, which I have stored into my local collection. Roland0 above would want to do the same, but then enable the handling of mood from inside his plugin so it can be stored using the scanning hooks.

In Custom Scan I used a database structure based on key/value pairs. This makes the storage very generic, you can basically store attribute in it. However, the issue is that browsing tends to get less optimized from a performance perspective since all genres/tags are stored in a single database table, especially when you want to filter based on multiple type of genres/tags in a single query (browsing moods->themes->albums for example). The alternative is to instead create the database tables based on the user specific configuration, with separate table structures for each type of genre/tag, so moods would be in one table and themes in another. Each user would have a different set of database tables based on their configuration. This would likely improve performance but instead introduce extra complexity if you want to use DBIx to abstract the raw SQL towards the database. It can probably be handled but I never figured it out with my limited Perl knowledge. If you want to go the route with separate tables with might be worth to consider adding support for new type of attributes(genres) through plugins via Perl code rather than configuration, this likely makes it easier to use DBIx.

Another consideration is how to optimize indices. In CustomScan, if I remember correctly, I selected to not have a integer primary key for each attribute(for example mood) value. This is obviously less optimized but I selected it because if you like to use it in random mix, smart playlists and similar you need to store the attribute value (Mood=Happy) anyway since the integer value(Mood=42) would change if you perform a rescan.

CustomScan supports key/value pairs on both artist, album and track level but I think a new attempt probably should limit itself to track level to make it a bit less complex.

I guess the main question is if this functionality would be considered as part of LMS core.

My feeling is that someone need to prove the concept as a plugin first, when it has been proved that it can be done without adding too much complexity it can be considered for pull requests and inclusion in LMS. I believe this is what happened with the virtual library support which is now part of LMS core, first I did a way too complex attempt with MultiLibrary/CustomBrowse plugin then Michael made a more simplified plugin with less flexibility but also significantly less complexity/code and after that something based on Michael’s attempt got included in LMS core.

Michael, if you are reading this, please let us know if you disagree.

The key for it to be included in LMS is to make sure it follows the LMS code style and is based on relatively little code. If it’s too much code or doesn’t follow the LMS code style it won’t be included in LMS code. These are likely the main reasons why my plugins never will be part of LMS core, they are just too flexible/complex and doesn’t use LMS coding styles (for example they don’t use DBIx for database abstraction)

Having said that, almost no plugin functionality made by non core developers have historically been included in LMS core as it is. There are some cases where similar functionality has been included but in those cases the code has often been rewritten more or less totally before included in the core. The exception I can think of is Extension Downloader made by Triode but he was more or less a core developer at that time if I remember correctly. The reason for this is likely that core developer(s) need to understand and be able to maintain all the code in LMS core and this is sometimes hard if written by someone else. However, nothing bad with having functionality as a plugin and Michael has historically been open to adding api/hooks in core to make a plugin possible. A plugin developed as a joint effort by several developers is also more likely to survive over time.

If you want to try to do something and like to discuss design choices feel free to keep posing and I’ll give you answers based on my experience.

[mherger]

Genres, moods, themes and styles

> In Custom Scan I used a database structure based on key/value pairs.
> This makes the storage very generic, you can basically store attribute
> in it. However, the issue is that browsing tends to get less optimized
> from a performance perspective since all genres/tags are stored in a
> single database table, especially when you want to filter based on
> multiple type of genres/tags in a single query (browsing
> moods->themes->albums for example). The alternative is to instead create

If you could live with a single intermediary (mood or theme or whatever)
you could create a library view based on whatever value you want, then
create a custom browse item limited to this library?

> lauret wrote:
>> I guess the main question is if this functionality would be considered
>> as part of LMS core.
>>
> My feeling is that someone need to prove the concept as a plugin first,
> when it has been proved that it can be done without adding too much
> complexity it can be considered for pull requests and inclusion in LMS.

A plugin would be great. Whether it be a POC, or actually an optional
feature can be debated. As you said: most users probably wouldn't want
to add the overhead for this additional complexity.

But then there's definitely a use case for the online library
integration, too, as you don't have control over their metadata. Many
eg. don't offer any genre information at all.

Some kind of post-scan importer manipulating metadata in the tables
might be very welcome. I added some primitive way to set genres based on
album titles or group names to the online library integration. But
that's rather simplistic.

> Having said that, almost no plugin functionality made by non core
> developers have historically been included in LMS core as it is. There

There actually were quite a few. And that taught us a lesson or two,
too. Eg. at some point there were lots of different skins. But their
original authors left, and nobody really cared about those any more.
Which means we were left with semi functional skins nobody wanted to
take care of. MusicIP integration started as a 3rd party contribution,
too. But you're right: more code needs more maintenance. As long as
there aren't too many contributors dedicated to maintaining LMS around,
it'll be hard to convince me to include major changes for a small group
of users...

--

Michael

[lauret]

If you could live with a single intermediary (mood or theme or whatever)
you could create a library view based on whatever value you want, then
create a custom browse item limited to this library?

But this wouldn't work if the mood of the track is stored inside the mood tag, right? These tags are not stored inside the library database. If it would be possible, this idea is by far the most simple.

A plugin would be great. Whether it be a POC, or actually an optional
feature can be debated. As you said: most users probably wouldn't want
to add the overhead for this additional complexity.

I will try to fit my ideas into a plugin and then we'll see from there.

But then there's definitely a use case for the online library
integration, too, as you don't have control over their metadata. Many
eg. don't offer any genre information at all.

Some kind of post-scan importer manipulating metadata in the tables
might be very welcome. I added some primitive way to set genres based on
album titles or group names to the online library integration. But
that's rather simplistic.

I can see the value in this concept. This idea would be complementary to the possibility to add a custom key (such as mood) to store a certain mood for a track, especially as metadata providers nowadays do not only provide a genre, but also styles etc.

[mherger]

Genres, moods, themes and styles

>> If you could live with a single intermediary (mood or theme or whatever)
>> you could create a library view based on whatever value you want, then
>> create a custom browse item limited to this library?
>
> But this wouldn't work if the mood of the track is stored inside the
> mood tag, right? These tags are not stored inside the library database.
> If it would be possible, this idea is by far the most simple.

If you provided code to read that tag and store it, then sure you could.

--

Michael

[erland]

> In Custom Scan I used a database structure based on key/value pairs.
> This makes the storage very generic, you can basically store attribute
> in it. However, the issue is that browsing tends to get less optimized
> from a performance perspective since all genres/tags are stored in a
> single database table, especially when you want to filter based on
> multiple type of genres/tags in a single query (browsing
> moods->themes->albums for example). The alternative is to instead create

If you could live with a single intermediary (mood or theme or whatever)
you could create a library view based on whatever value you want, then
create a custom browse item limited to this library?

Do you mean creating a virtual library for each individual Mood value ?
So you would get a “Happy” library, a “Sad” library, a “Tired” library if you have these three Mood values in you tags ?

That’s probably a good start for a POC plugin since implementing full browse support in a plugin is going to be a bit of work and such implementation would be able to ignore browsing since its already implemented in the core for virtual libraries.

Would the library_track table be filled by the plugin during scanning or is it handled some other way ?