Online music service integration - how to test

[thniels]

I am sorry . This is getting weird. I went through the protocol to have a fresh log to post here and deleted the library, restarted LMS and copy'pasted the logs. But now it turns out it loads the Tidal albums after all. It's been failing miserably for a couple of weeks and now when someone is looking over its shoulders, it appears to run. It hasn't finished indexing yet, so I will wait for that to happen to be certain it doesn't go bonkers on the finishing line (even if that is unlikely). It usually takes a couple of hours to do a full rescan (pcp).

[thniels]

Well, it didn't last long. Around the next scan they all disappeared again. I simply cannot see in the logs anything suspicious, except for this little tidbit (XYZZY is a 10-digit number)...

[22-06-13 16:35:08.8712] Slim::Plugin::WiMP::Importer::scanAlbums (90) Reading albums for XYZZY...
[22-06-13 16:35:34.0420] Slim::Plugin::WiMP::Importer::scanAlbums (102) Importing album tracks for 1 albums...
[22-06-13 16:35:34.0429] Slim::Plugin::WiMP::Importer::scanAlbums (103) ["XYZZY"]
[22-06-13 16:35:34.0434] Slim::Plugin::WiMP::Importer::scanAlbums (106) Invalid data: XYZZY
[22-06-13 16:35:34.0564] Slim::Plugin::WiMP::Importer::scanArtists (144) Reading artists for XYZZY...
[22-06-13 16:35:34.3633] Slim::Plugin::WiMP::Importer::scanArtists (154) Importing artist tracks for 32 artists...
...

For what it's worth, I do see a curious message when watching the scan progress in the Information panel. After roughly an hour, it says something like "removing online tracks" and then counts down from 14,000 or thereabouts. It does look very much like it actually rolls back the additions.

The modules that I log are have checked for debugging are plugin.onlinelibrary, plugin.tidal and scan. Attached is the complete (500 lines) scanner.log when refreshing online libraries. The server.log is empty.

I think what I really would like to know, is how to come about debugging this. Tidal blocked me for several hours over this for acting like a robot (I was pretty darn close to switching to Quboz &#129325.

[mherger]

> Well, it didn't last long. Around the next scan they all disappeared
> again. I simply cannot see in the logs anything suspicious, except for
> this little tidbit (XYZZY is a 10-digit number)...

In all of those messages? And it's not your account ID?

> For what it's worth, I do see a curious message when watching the scan
> progress in the Information panel. After roughly an hour, it says
> something like "removing online tracks" and then counts down from 14,000
> or thereabouts. It does look very much like it actually rolls back the
> additions.

Yes, as it doesn't find any album information it's removing all of those
tracks. That's not great.

Are you saying that you have albums worth 14k tracks in your library on
TIDAL? If that's the case, then we might see some issue due to this
rather large size.

[thniels]

Yes, XYZZY is just a placeholder for my ID which is my phone number. I just didn't want that to appear in the forum. My subscription is tied to my cell phone account. I have seen automatic type conversion run afoul over large numbers when expecting strings (looking at you Tcl). But that was many years ago and a bit of a subliminal reaction on my part. I honestly don't expect that to be an issue in this day and age.

I haven't found a way to reliably count my Tidal favorites. I simply have a habit of favoriting almost every album for a new artist when I spot them. But doing a rough count in the Naim app, there are 3,000 and a handful of albums on Tidal. So I suppose it amounts to a few tracks. The mainstay of our collection is locally ripped CDs, though.

[thniels]

I was looking at Importer.pm...

sub scanAlbums { if (main::SCANNER) {
        my ($class, $accounts) = @_;

        my $progress;

        foreach my $account (@$accounts) {
                if ($progress) {
                        $progress->total($progress->total + 1);
                }
                else {
                        $progress = Slim::Utils::Progress->new({
                                'type'  => 'importer',
                                'name'  => 'plugin_tidal_albums',
                                'total' => 1,
                                'every' => 1,
                        });
                }

                main::INFOLOG && $log->is_info && $log->info("Reading albums for $account...");
                $progress->update(string('PLUGIN_TIDAL_PROGRESS_READ_ALBUMS', $account));

                my $accountName = "tidal:$account" if scalar @$accounts > 1;

                my $albumsResponse = $http->get(Slim::Networking::SqueezeNetwork::Sync->url(sprintf(ALBUMS_URL, $account)));
                my $albums = eval { from_json($albumsResponse->content) } || [];

                $@ && $log->error($@);

                $progress->total($progress->total + scalar @$albums);

                main::INFOLOG && $log->is_info && $log->info(sprintf("Importing album tracks for %s albums...", scalar @$albums));
                main::DEBUGLOG && $log->is_debug && $log->debug(Data::Dump::dump($albums));
                foreach my $album (@$albums) {
                        if (!ref $album) {
                                $log->error("Invalid data: $album");
                                next;
                        }

                        $progress->update($account . string('COLON') . ' ' . $album->{title});
                        Slim::Schema->forceCommit;

                        my $tracks = delete $album->{tracks};

                        $class->storeTracks([
                                map { _prepareTrack($_, $album) } @$tracks
                        ], undef, $accountName);
                }

                Slim::Schema->forceCommit;
        }

        $progress->final() if $progress;
} }

I presume this is the source of the error message from the previous log file. If the array @$albums contains my account number as well as $albums then I find that a bit odd. I think it's time I find a computer I can use as a testing ground.