Online music service integration - how to test

[HighPeakRider]

Genres = Tidal

Sorry if this has been covered I've only just started my LMS journey.
Seems when I enable the plugin my Genre's get updated to Tidal.
Any work around for this?

[mherger]

Online music service integration - how to test

> Seems when I enable the plugin my Genre's get updated to Tidal.

Can you elaborate?

[HighPeakRider]

> Seems when I enable the plugin my Genre's get updated to Tidal.

Can you elaborate?

I enabled the Tidal plugin and also 'Online music service integration' and this resulted in only having one genre which was Tidal.

I removed the plugin the deleted library.db and once the database rebuilt I have all the genres back.

I now think it could be corruption as I've just enabled the plugin to test and get screenshots and it all seems OK.

Thanks for checking back with me........

[mherger]

Online music service integration - how to test

> I enabled the Tidal plugin and also 'Online music service integration'
> and this resulted in only having one genre which was Tidal.

And you do have local music by artists which aren't in your TIDAL
collection? Would you mind sharing a scanner.log.zip with us after a
full scan with online music integration enabled?

[HighPeakRider]

> I enabled the Tidal plugin and also 'Online music service integration'
> and this resulted in only having one genre which was Tidal.

And you do have local music by artists which aren't in your TIDAL
collection? Would you mind sharing a scanner.log.zip with us after a
full scan with online music integration enabled?

I re-imported everything and its now OK.
Thanks for coming back to me.......

[thniels]

>That's really odd, as 4GB should be more than enough to handle 100k
tracks. Could you please disable all 3rd party plugins, and the online
scanner, to see whether this would run a full scan? If that's ok, then
re-enable the TIDAL integration and test again. etc.

Okay, I don't get very consequent results. I have tried disabling all but the Tidal plugin and sometimes it scans through, other times it doesn't. I managed to make a few scans where I killed the Bridge plugin processes and appeared to get somewhere, then suddenly that stopped working.

I am on the brink of simply ditching the pCPs and go back to the PC based linuxes. They work splendidly with the same setup and the same amount of RAM. The pCPs have 2GB and 4GB respectively. I have been unable to get one with 8GB to test that. They appear to be still sold out everywhere.

For the record, the pCP setup itself is default out-of-the-box. I don't suppose there is anything there I can tweak to remedy this?

p.s. Running without the Tidal integration it scans through.

[RobbH]

...I am on the brink of simply ditching the pCPs and go back to the PC based linuxes. They work splendidly with the same setup and the same amount of RAM. The pCPs have 2GB and 4GB respectively. I have been unable to get one with 8GB to test that. They appear to be still sold out everywhere.
...

If you are comfortable with Linux, your Pi can run Raspberry Pi OS (formerly called Rasbian), which is based on Debian, instead of pCP. I have LMS running on a 4 GB Pi 4 with RPOS. No problems with online music integration, BUT I am not using Tidal.

[mherger]

Online music service integration - how to test

> Okay, I don't get very consequent results. I have tried disabling all
> but the Tidal plugin and sometimes it scans through, other times it
> doesn't. I managed to make a few scans where I killed the Bridge plugin
> processes and appeared to get somewhere, then suddenly that stopped
> working.

At which stage would the scan fail?

[thniels]

Within seconds...

[22-12-15 12:44:13.7718] main::main (202) Starting Logitech Media Server scanner (v8.3.1, 1670141275, Sun 04 Dec 2022 09:30:06 AM CET) perl 5.032001
[22-12-15 12:44:14.1013] Carp::Clan::__ANON__ (216) Warning: DBIx::Class::ResultSet::update_or_create(): Query returned more than one row.  SQL that returns multiple rows is DEPRECATED for ->find and ->single at /usr/local/slimserver/Slim/Schema.pm line 1927
[22-12-15 12:44:14.3828] Slim::Music::Import::runImporter (579) Starting Plugins::MusicArtistInfo::Importer scan
[22-12-15 12:44:14.7230] Plugins::MusicArtistInfo::Importer::_getAlbumCoverURL (198)   finished in 0.326 seconds
[22-12-15 12:44:14.7523] Plugins::MusicArtistInfo::Importer::_scanAlbumGenre (306)     finished in 0.002 seconds
[22-12-15 12:44:34.5952] Plugins::MusicArtistInfo::Importer2::_getArtistPhotoURL (198) finished in 19.793 seconds
[22-12-15 12:44:34.5958] Slim::Music::Import::endImporter (712) Completed Plugins::MusicArtistInfo::Importer Scan in 20.213 seconds.
[22-12-15 12:44:34.5968] Slim::Music::Import::runImporter (579) Starting Slim::Plugin::ExtendedBrowseModes::Libraries scan
[22-12-15 12:44:34.6223] Slim::Music::Import::endImporter (712) Completed Slim::Plugin::ExtendedBrowseModes::Libraries Scan in 0.025 seconds.
[22-12-15 12:44:34.6231] Slim::Music::Import::runImporter (579) Starting Slim::Music::VirtualLibraries scan
[22-12-15 12:44:37.5197] Slim::Music::Import::endImporter (712) Completed Slim::Music::VirtualLibraries Scan in 2.897 seconds.
[22-12-15 12:44:37.5535] Slim::Music::Artwork::updateStandaloneArtwork (234) Starting updateStandaloneArtwork for 0 albums
[22-12-15 12:44:37.5541] Slim::Music::Import::endImporter (712) Completed updateStandaloneArtwork Scan in 0.031 seconds.
[22-12-15 12:44:37.5922] Slim::Music::Artwork::precacheAllArtwork (657) Starting precacheArtwork for 0 albums
[22-12-15 12:44:37.5928] Slim::Music::Import::endImporter (712) Completed precacheArtwork Scan in 0.038 seconds.
[22-12-15 12:44:37.5937] Slim::Music::Import::runScanPostProcessing (481) Starting Database optimization.
[22-12-15 12:44:48.8133] Slim::Music::Import::endImporter (712) Completed dbOptimize Scan in 11.219 seconds.

With all but the online plugins disabled it mostly finishes. If I disable only the bridge plugins (which appear to be the ones spending most memory) it seems to be the same behaviour - mostly completing. With no stats to back it up, the latter situation slightly less likely to finish than the complete plugin disablement.

I have tried making a fresh pCp with the same results. Still 4GB.

I don't suppose there is a way to disable the plugins from the command line? Using --failsafe disables online integration too.

[thniels]

[I think I accidentally deleted my post, instead of editing it. Anyway... ]

Line 95 in Plugins/WiMP/Importer.pm returns an error 504 (gateway timeout) and this appears to correllate with successfull/failed scan attempts. When mysqueezebox.com returns this error instead of a json string, LMS parses the error message as an empty json string and fails, subsequently removing all Tidal entries from the library ([63] deleteRemovedTracks(), I presume).

my $albumsResponse = $http->get(Slim::Networking::SqueezeNetwork::Sync->url(sprintf(ALBUMS_URL, $account)));

In other words, this error appears to stem from mysqueezebox.com and not the LMS server itself.

[mherger]

thniels - I'm trying to summarize you postings: you often fail to import your TIDAL library because the request for albums returns incorrectly? How often would this happen vs. a valid scan? And how many albums do you have?

[thniels]

thniels - I'm trying to summarize you postings: you often fail to import your TIDAL library because the request for albums returns incorrectly? How often would this happen vs. a valid scan? And how many albums do you have?

Unfortunately I don't have statistics on how often it fails. As I have mentioned earlier, I thought it was a memory problem on my Pcps because it didn't appear to happen when running LMS on other machines. Consequently, I set up a VM with 16GB allocated to only do scans and simply copy over the library.db to the Pcp running the actual LMS. But having that running nightly, it was obvious that this failed just as often. I must have simply been lucky when it worked.

I haven't measured the successes yet but will set up a metric now to do so and log them - I will get back with some numbers for you. FWIW; Yesterday afternoon I had 1 or 2 completed scans out of appr. a dozen attempts. The rest failed. So now I contemplate simply exiting the scan if it fails, to at least keep previous successful scans intact in the database. Otherwise LMS deletes all the tracks.

There are 2450 albums on Tidal that amount to roughly 10MB of json from the mentioned request. When it fails it is within a few minutes, with mysqueezebox.com returning the 504 error.

When it works, the json appears wellformed and not truncated.

[mherger]

There are 2450 albums on Tidal that amount to roughly 10MB of json from the mentioned request. When it fails it is within a few minutes, with mysqueezebox.com returning the 504 error.

Most likely (as the 504 suggests) the backend is timing out paging through your extensive collection: we are allowed a maximum of 50 items per request. Which means we have to do about 50 requests to get all of your collection. That takes some time... Would the 10MB be the raw data size, or the actual downloaded data size (it should be compressed)? I'll see whether I can increase the timeout on the proxy.

[thniels]

The 10MB is the size of the json string itself. It doesn't appear to have any compressed properties and I didn't check the actual raw content from mysqueezebox.com.

Do you want me to check the raw size?

[mherger]

The 10MB is the size of the json string itself. It doesn't appear to have any compressed properties and I didn't check the actual raw content from mysqueezebox.com.

Do you want me to check the raw size?

Not necessary, thanks. Compression should be handled transparently. I only wanted to be sure it's not 10MB compressed (which could result in 100MB to process).