[Announce] Music Similarity DSTM mixer

[cpd73]

When I started supporting squeezelite, I've found the plethora of different libc versions to be a constant issue. I eventually discovered that building on an older OS virtually eliminated the problem.

The intel 32 and 64bit binaries are still built on CentOS 6 VMs.

Thanks. I tried installing a centos6 VM, but then installing packages failed - something todo with repos not found. Was about to try centos7, but installing ffmpeg in an Ubuntu VM seems to have resolved the library loading. So, the library itself was OK, just needed its dependencies to be installed.

[bakker_be]

It's not that the library cannot be found, its that it cannot be opened. The reason for this is that you need to install the ffmpeg libraries. You can do this by just installing ffmpeg iteself (sudo apt install ffmpeg) or install the libavformat, libavcodec, and libavutil libraries.

I've installed 20.04 in a VM, installed ffmpeg, and analysis works.

Indeed, now it seems to work. Thanks!

[mruddo]

If you re-run then newly, or un-analysed, tracks will be analysed and added. Any tracks that are in the DB but not on disk will be removed from the DB. If you want to keep these non-existant tracks in the DB the pass "--keep-old"

Thanks for confirming... sure enough, a re-run was very quick - and it failed to add that file again. This was the one:

musly_track_analyze_audiofile failed for M:\TestLibrary\Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac

I may have been mistaken, and can't be sure it made it in on the previous (Alpha2) build.

It's not so obvious to me what's wrong with this one. The "-" after Interlude is actually a ":" that's been substituted in the filename already, but given there are a number of other similarly named tracks on that album it does not look obviously different in any way.

Anyway, 680 out of 681 trackes analysed this time in 2 hours 5 mins.... so that's not too bad.

I'm tempted to give my whole library a go, but may play around with some more sample mixes first.

I'm not sure where you are in the world, but Happy New Year!

[cpd73]

musly_track_analyze_audiofile failed for M:\TestLibrary\Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac

Odd. Can you run ffprobe against this file? ffbrobe.exe should be in the 'windows' folder - and it should print info such as the track's duration. Also, if you edit musly.py (in the lib folder) and uncomment (remove the '#') lines 71 and 72 (these lines have ....musly_debug....) and re-run, then you should have more debug (comming from libmusly). This might provide more info.

The "-" after Interlude is actually a ":" that's been substituted in the filename already, but given there are a number of other similarly named tracks on that album it does not look obviously different in any way.

Yeah, the contents of the tags shouldn't matter - and I have a few with a colon (mind you I'm on Linux, so my filesystem accepts a colon in a filename).

Anyway, 680 out of 681 trackes analysed this time in 2 hours 5 mins.... so that's not too bad.

Not bad? To me that's slow! On my 6 (or 7?) year old laptop I get 1140 tracks/hour. Perhaps its because my libmusly uses the ffmpeg libraries, so does not need to start the executables for each track. Still, that's quite a difference!

[ralphy]

Thanks. I tried installing a centos6 VM, but then installing packages failed - something todo with repos not found. Was about to try centos7, but installing ffmpeg in an Ubuntu VM seems to have resolved the library loading. So, the library itself was OK, just needed its dependencies to be installed.

Since it's just be a library dependency perhaps linking the static ffmpeg libraries would be an alternative to needing the shared ffmpegs libs installed on target systems.

Sorry, centos 6 has been EOL for several years now. I've no reason to update the packages so hadn't noticed that.

Centos 7 glibc is likely old enough these days, but only if you start having lots of reports of like.

squeezelite: /lib64/libc.so.6: version `GLIBC_x.xx' not found (required by squeezelite)

[mruddo]

Sorry, I got distracted trying to get the high-level essentia analysis working in Windows. (No joy yet with that one.)

Odd. Can you run ffprobe against this file? ffbrobe.exe should be in the 'windows' folder - and it should print info such as the track's duration. Also, if you edit musly.py (in the lib folder) and uncomment (remove the '#') lines 71 and 72 (these lines have ....musly_debug....) and re-run, then you should have more debug (comming from libmusly). This might provide more info.

Here's the output from ffprobe:
----------------------------------------------------------------------------------------------------------------------------------
ffprobe version n4.4.1-2-gcc33e73618-20211215 Copyright (c) 2007-2021 the FFmpeg developers
built with gcc 11.2.0 (crosstool-NG 1.24.0.498_5075e1f)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --disable-avisynth --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --disable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --disable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --disable-libvidstab --disable-vulkan --disable-libx264 --disable-libx265 --disable-libxavs2 --disable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20211215
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
Input #0, flac, from 'M:\TestLibrary\Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac':
Metadata:
ALBUM : Humanz (Deluxe)
album_artist : Gorillaz
ARTIST : Gorillaz
REPLAYGAIN_TRACK_PEAK: 0.943359
DATE : 2017
GENRE : Trip-Hop
TITLE : Interlude: Elevator Going Up
TRACKTOTAL : 26
track : 9
REPLAYGAIN_ALBUM_GAIN: -9.17 dB
REPLAYGAIN_ALBUM_PEAK: 0.998840
REPLAYGAIN_TRACK_GAIN: -1.36 dB
comment : 10A - Energy 4
BPM : 83
Duration: 00:00:04.85, start: 0.000000, bitrate: 531 kb/s
Stream #0:0: Audio: flac, 96000 Hz, stereo, s16
Side data:
replaygain: track gain - -1.360000, track peak - 0.000022, album gain - -9.170000, album peak - 0.000023,
----------------------------------------------------------------------------------------------------------------------------------

And this is what I get when re-running with those lines commented out:
----------------------------------------------------------------------------------------------------------------------------------
2022-01-01 13:30:42 D [1/1 100%] Gorillaz/Humanz (Deluxe)/09-Interlude- Elevator Going Up.flac
DEBUG: Decoding: M:\TestLibrary/Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac started.
DEBUG: Running command:ffprobe -hide_banner "M:\TestLibrary/Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac" 2>&1
DEBUG: File duration:4
musly_track_analyze_audiofile failed for M:\TestLibrary/Gorillaz\Humanz (Deluxe)\09-Interlude- Elevator Going Up.flac
----------------------------------------------------------------------------------------------------------------------------------

Out of curiosity, I've just listened to the track... It's only four seconds long, with the spoken words, "Elevator going up"... so possibly too little for any meaningful analysis - and certainly no great loss in terms of any mix.

It's also an interesting point about the speed of my analysis too - which is indeed relatively slow, but still quite feasible.

I've been testing with LMS on my desktop Intel I7 3770S @ 3.1GHz with 4 cores and 8 logical processors. It's quite an old machine - about 8 yrs+, but I figured it would likely be quicker analysing than my mini-PC that I normally run the server on, which is a Intel Celeron J4115 @ 1.8GHz with 4 cores and 4 logical processors... This one's only about a year old, and although for some stange reason it seemed quicker at first, it is definitely slower.

I'm analysing all 30k tracks in my library for a full scan... I'll copy the db back to the mini-PC when done.

[cpd73]

Sorry, I got distracted trying to get the high-level essentia analysis working in Windows. (No joy yet with that one.)

You will need to compile your own Essentia extractor - the pre-built ones don't support the models. To be honest, I'm not 100% convinced on the need for the filtering with high-level attributes.

Out of curiosity, I've just listened to the track... It's only four seconds long

Ah! Its short length is probably why the analysis fails.

[mruddo]

You will need to compile your own Essentia extractor - the pre-built ones don't support the models. To be honest, I'm not 100% convinced on the need for the filtering with high-level attributes.

I think you may be right...hence I've moved on, to see what sort of mixes I get on the full library. I also see you've enabled genre group configuration, so that's something that should enable pretty good control of how far the mix deviates.

I'm also not so sure about the analysis speed on the smaller box now - it seems to have slowed down (possibly throttling because of the CPU temp?!). I'll leave it running though and see how it goes.

[mruddo]

Ah! Its short length is probably why the analysis fails.

Certainly looks like it. I'm still working through the analysis of my library... several similarly titled "Interlude" tracks have also failed. Generally though, it's progressing well.

[mruddo]

Analysis (almost) complete...

Just an update on full my library analysis...

As mentioned previously, I was running this on a desktop PC. I use this to rip CDs/download music etc. then have a robocopy clone job that sends the files to the mini-PC where I run LMS.

So the plan was to simply analyse on the more powefull PC, then copy the db across when done. And I started what should have been the final run of analysis last night, only to find it seemed to have stalled this morning. i.e. the dubug logging just showed the last track it looked at, but there was no confirmation of the number of records written to the db.

As I need to use my desktop for work today, I decided to run the final bactch of analysis on my mini-PC, and one thing I noticed was that it's re-analysing some files because of case differences in the filename. So in one library I might have "08-Catch the Sun.flac" and in the other "08-Catch The Sun.flac" - and it will then analyse this again on the second PC ...which is fair enough, and likely a limitation of robocopy/Windows not being specific with filenames when cloning - so not really your problem.

Secondly, regardless of which PC I run the analysis on it seems to analyse certain files repeatedly - and not report any errors. Now we've already established that some files are simply too short (< 6 seconds at a guess, maybe a few more), but these other files it will try to analyse again and again regardless. This seemed to be happening on some AudioBook tracks - which are often quite long, so it appears we also have an issue with analysis of exceptionally long songs... but without an error. Again though, this is hardly an issue for any mix.

I did wonder though if you could pull the track length from the file and skip files that were, for example < 10 secs or > 30 minutes?

[cpd73]

I did wonder though if you could pull the track length from the file and skip files that were, for example < 10 secs or > 30 minutes?

Yeah, should be possible. I already read the duraiton from the file's tags. However, this is currently doen after analysis. I'll update the code to read the tags first, and then only analyse if tags are valid and duration is in range.

[mruddo]

Sigur Rós present another set of interesting characters that seem to give problems:

2022-01-06 09:19:59 D [ 114/2495 4%] Sigur Rós/Ágætis Byrjun/01-Intro.flac
2022-01-06 09:20:59 D [ 115/2495 4%] Sigur Rós/Ágætis Byrjun/02-Svefn-g-englar.flac
2022-01-06 09:21:23 D [ 116/2495 4%] Sigur Rós/Ágætis Byrjun/03-Starálfur.flac
2022-01-06 09:24:21 D [ 117/2495 4%] Sigur Rós/Ágætis Byrjun/04-Flugufrelsarinn.flac
2022-01-06 09:25:12 D [ 118/2495 4%] Sigur Rós/Ágætis Byrjun/05-Ný batterí.flac
2022-01-06 09:25:16 D [ 119/2495 4%] Sigur Rós/Ágætis Byrjun/06-Hjartað hamast (bamm bamm bamm).flac
2022-01-06 09:26:41 D [ 120/2495 4%] Sigur Rós/Ágætis Byrjun/07-Viðrar vel til loftárasa.flac
2022-01-06 09:28:40 D [ 121/2495 4%] Sigur Rós/Ágætis Byrjun/08-Olsen Olsen.flac
2022-01-06 09:29:13 D [ 122/2495 4%] Sigur Rós/Ágætis Byrjun/09-Ágætis byrjun.flac
2022-01-06 09:29:41 D [ 123/2495 4%] Sigur Rós/Ágætis Byrjun/10-Avalon.flac

That's the output... i.e. analysis does not report an issue, but perhaps there's an issue writing to the db, so each time you re-run they're picked up again.

It's also doing the same for the few "Now That's What I Call Music!" compilation albums I have - I suspect the exclamation mark may be the issue there. I'm not yet sure though - this could be the scenario you described where not all tracks were written on the first pass. I'll re-analyse one of these more specifically.

re. my previous point about inconsistent cases in filenames. I've just noticed it's re-analysing all files in a "War On Drugs" folder where the other PC had "War on Drugs", so not surprisingly, both the path and the filename are sensitive to this issue. What's interesting about that is that the analysis is deemed necessary, and yet no tracks (with the different case) are removed from the db (i.e. Num old tracks = 0). Which means I'll likely have duplicates in the db now. Perhaps all filnames need to be considered case insensitive for comparisons when identifying old tracks? (Not essential - I appreciate this may be a Windows perculiarity.)

[cpd73]

Sigur Rós present another set of interesting characters that seem to give problems:

2022-01-06 09:19:59 D [ 114/2495 4%] Sigur Rós/Ágætis Byrjun/01-Intro.flac
2022-01-06 09:20:59 D [ 115/2495 4%] Sigur Rós/Ágætis Byrjun/02-Svefn-g-englar.flac
2022-01-06 09:21:23 D [ 116/2495 4%] Sigur Rós/Ágætis Byrjun/03-Starálfur.flac
2022-01-06 09:24:21 D [ 117/2495 4%] Sigur Rós/Ágætis Byrjun/04-Flugufrelsarinn.flac
2022-01-06 09:25:12 D [ 118/2495 4%] Sigur Rós/Ágætis Byrjun/05-Ný batterí.flac
2022-01-06 09:25:16 D [ 119/2495 4%] Sigur Rós/Ágætis Byrjun/06-Hjartað hamast (bamm bamm bamm).flac
2022-01-06 09:26:41 D [ 120/2495 4%] Sigur Rós/Ágætis Byrjun/07-Viðrar vel til loftárasa.flac
2022-01-06 09:28:40 D [ 121/2495 4%] Sigur Rós/Ágætis Byrjun/08-Olsen Olsen.flac
2022-01-06 09:29:13 D [ 122/2495 4%] Sigur Rós/Ágætis Byrjun/09-Ágætis byrjun.flac
2022-01-06 09:29:41 D [ 123/2495 4%] Sigur Rós/Ágætis Byrjun/10-Avalon.flac

Can you manually run essentia against these? e.g.

streaming_extractor_music.exe "Sigur Rós/Ágætis Byrjun/10-Avalon.flac" out.json

Does that produce any errors? If not then its Musly which is failing - but I'm not sure there is much I can do. Perhaps what I need to do is use the windows 'short' filename - e.g. its DOS equivalent.

re. my previous point about inconsistent cases in filenames. I've just noticed it's re-analysing all files in a "War On Drugs" folder where the other PC had "War on Drugs", so not surprisingly, both the path and the filename are sensitive to this issue. What's interesting about that is that the analysis is deemed necessary, and yet no tracks (with the different case) are removed from the db (i.e. Num old tracks = 0). Which means I'll likely have duplicates in the db now. Perhaps all filnames need to be considered case insensitive for comparisons when identifying old tracks? (Not essential - I appreciate this may be a Windows perculiarity.)

Sorry, but no. The problem here is Windows. It shows files/folders using mixed-case but then treats them case insensitively. Linux, however is fully case insensitive - where you can have "Temp" and "temp" in the same folder.

[mruddo]

Can you manually run essentia against these? e.g.
Does that produce any errors? If not then its Musly which is failing - but I'm not sure there is much I can do. Perhaps what I need to do is use the windows 'short' filename - e.g. its DOS equivalent.

No errors:

streaming_extractor_music.exe "m:\my music\FLAC\Sigur Rós\Ágætis Byrjun\10-Avalon.flac" out.json
Process step: Read metadata
Process step: Compute md5 audio hash and codec
Process step: Replay gain
Process step: Compute audio features
Process step: Compute aggregation
All done
Writing results to file out.json

So that looks good. I've attached the JSON FYI. Interestingly, it also looks like much of the low level I did not expect to see in the output... e.g.

"danceability": 0.955245614052

...this may explain in part at least why my analysis appears to be slower.

I can live with the Windows filename case issues. I'm not keen on using the idea of the DOS filename... I'd have to re-analyse all my tracks!

Incidentally, the "Now That's What I Call Music!" tracks also appear to analyse just fine with essentia - so I'm not sure why they're being done again - aside from the possibility they were not written on the first pass. (I've since checked the db, and sure enough most of these had not been written so they're being re-analaysed for that reason. Hopefully they'll be applied correctly this time. It looks as though the output to the db stopped mid-way through this point, and nothing more was written - despite the analysis appearing to continue for a while longer.)

[cpd73]

[B]streaming_extractor_music.exe "m:\my music\FLAC\Sigur Rós\Ágætis Byrjun\10-Avalon.flac" out.json

I think it's an encoding issue. Essentia writes its output to a JSON file. The script then reads this file to extract items to save to the DB. Linux is all (?) utf-8, so has no issues. With Windows it fails to load the JSON if it has utf-8 characters - as utf-8 is not Windows' default encoding. What I need to do is tell python it's utf-8 before opening the file. Will fix later.

Interestingly, it also looks like much of the low level I did not expect to see in the output... e.g.

"danceability": 0.955245614052

No, this danceability is not the same as the "danceable" attribute. I thought about using it, but it appears very inconsistent.