SBS 7.6: please test automatic artwork download

[mherger]

I've added some (experimental) code to 7.6 which allows you to have the music scanner automatically download missing album artwork for your music. It's working good enough to give it some broader testing. Scanning my test library of about 670 albums it was able to find roughly 90% of all artwork automatically.

The artwork download is an additional step during the scanning process. It's disabled by default, but I'd love to hear some feedback how it's working for you.

In order to enable the automatic artwork download, please follow these instructions:
- install SBS 7.6/r31180 or later
- make sure it's using test.mysqueezebox.com: you should have use_sn_test:1 in your server.prefs file. If you don't, add it, while SBS is shut down, then restart the server
- enable the download in Settings/Advanced/Formatting
- re-run a scan

If you're an iTunes user (using our iTunes integration) I would be interested to know how this compares in performance and results with our iTunes artwork importer (which can be enabled/disabled on the iTunes settings page).

A few caveats:
- automatic artwork download will take some time, as it's communicating with external resources
- mysb.com is using last.fm as the source for the covers. Thus the quality of the artwork can vary a lot (I've seen >1600x1600px PNG as well as 100x100px JPG)
- mysb.com will currently limit the artwork size to max. 500x500px. Larger files are being scaled down. This value is a deliberate decision based on the experience during testing, but it's not set in stone.

Have fun!

--

Michael

[pippin]

Any chance for 768x768? That would be cool for iPad...

[mherger]

> Any chance for 768x768? That would be cool for iPad...

Try the following patch to override the default value:

Index: Artwork.pm
===================================================================
--- Artwork.pm	(revision 31181)
+++ Artwork.pm	(working copy)
@@ -638,6 +638,7 @@
  				my $url = $snURL
  					. '?album=' . URI::Escape::uri_escape_utf8( $albumname )
  					. '&artist=' . URI::Escape::uri_escape_utf8( $contributor )
+					. '&width=768'
  					. '&mbid=' . $album_mbid;

Unfortunately test.mysb.com is encountering some issues right now. The service isn't working :-(. I hope we'll manage to fix it before the weekend...

--

Michael

[erland]

I've added some (experimental) code to 7.6 which allows you to have the music scanner automatically download missing album artwork for your music. It's working good enough to give it some broader testing. Scanning my test library of about 670 albums it was able to find roughly 90% of all artwork automatically.

The artwork download is an additional step during the scanning process. It's disabled by default, but I'd love to hear some feedback how it's working for you.

Where will the artwork be stored ? In cache, music folder or somewhere else ?
Can I somehow run it on my library without removing all cover.jpg file I currently have in the folder of all albums ? I want to make it retrieve artwork for all albums just to see how good it works.

[mherger]

> Where will the artwork be stored ? In cache, music folder or somewhere
> else ?

[sb cachefolder]/DownloadedArtwork

> Can I somehow run it on my library without removing all cover.jpg file
> I currently have in the folder of all albums ? I want to make it
> retrieve artwork for all albums just to see how good it works.

Good plan - what I did too. Simplest solution: disable the filter on the query.

Index: Artwork.pm
===================================================================
--- Artwork.pm	(revision 31181)
+++ Artwork.pm	(working copy)
@@ -558,7 +558,7 @@
  	# Find distinct albums to check for artwork.
  	my $tracks = Slim::Schema->search('Track', {
  		'me.audio'   => 1,
-		'me.coverid' => { '='  => undef },
+#		'me.coverid' => { '='  => undef },
  	}, {
  		'join'     => 'album',
  	});

You'd then be able to inspect the downloaded files in the aforementioned folder. Unfortunately SBS would still show whatever artwork it did find, not only the downloaded. This might require a larger patch (which I haven't tested before posting, but I did something similar earlier today):

Index: Artwork.pm
===================================================================
--- Artwork.pm	(revision 31181)
+++ Artwork.pm	(working copy)
@@ -558,7 +558,7 @@
  	# Find distinct albums to check for artwork.
  	my $tracks = Slim::Schema->search('Track', {
  		'me.audio'   => 1,
-		'me.coverid' => { '='  => undef },
+#		'me.coverid' => { '='  => undef },
  	}, {
  		'join'     => 'album',
  	});
@@ -688,6 +688,15 @@
  			
  			else {
  				main::DEBUGLOG && $importlog->is_debug && $importlog->debug( "Failed to download artwork for $albumname" );
+
+				# remove any artwork if nothing has been found
+				$track->cover( undef );
+				$track->update;
+
+				$track->coverid( undef );
+				$track->update;
+
+				$track->album->artwork( undef );
  				
  				$cache{"artwork_download_failed_$albumid"} = 1;
  			}

This will basically remove existing artwork data from tracks/albums which were not found online.

test.mysb.com seems to be back online...

--

Michael

[copperstate]

Hi Michael,

here are my results. Each test was done using a clean rescan of the whole library:

Version: 7.6.0 - r31181 @ Sun Aug 8 02:00:52 PDT 2010
Operating system: Mac OS X 10.6.4 - EN - utf8
Database Version: MySQL 5.0.22-standard


1. Without any deliberate artwork scan:

Music Scan Details
Discovering files (3912 of 3912) Complete 00:00:06
Scan new files (2756 of 2756) Complete 00:02:35
Discovering playlists (1 of 1) Complete 00:00:00
iTunes Import (2947 of 2947) Complete 00:00:28
iTunes Playlist Import (217 of 217) Complete 00:00:35
Pre-caching Artwork (19 of 19) Complete 00:00:31

The server has finished scanning your music collection.
Total Time: 00:04:15 (Sunday, 8 August 2010 / 13.23)


2. Artwork scan using iTunes (done yesterday thus still 7.6.0 - r31176):

Music Scan Details
Discovering files (3912 of 3912) Complete 00:00:09
Scan new files (2756 of 2756) Complete 00:02:47
Discovering playlists (1 of 1) Complete 00:00:01
iTunes Import (2947 of 2947) Complete 00:00:38
iTunes Playlist Import (217 of 217) Complete 00:00:54

iTunes Downloaded Artwork Import (2955 of 2955) Complete 00:01:51
iTunes User-Added Artwork Import (2674 of 2674) Complete 00:18:38
Pre-caching Artwork (306 of 306) Complete 00:00:34

The server has finished scanning your music collection.
Total Time: 00:25:32 (Saturday, 7 August 2010 / 15.46)


3. Artwork scan using test.mysb.com / last.fm:

Music Scan Details
Discovering files (3912 of 3912) Complete 00:00:07
Scan new files (2756 of 2756) Complete 00:01:52
Discovering playlists (1 of 1) Complete 00:00:00
iTunes Import (2947 of 2947) Complete 00:00:46
iTunes Playlist Import (217 of 217) Complete 00:00:42

Downloading Artwork (2674 of 2674) Complete 00:14:40
Pre-caching Artwork (119 of 119) Complete 00:00:06

The server has finished scanning your music collection.
Total Time: 00:18:13 (Sunday, 8 August 2010 / 13.45)

I mostly listen to classical music so that might an explanation why iTunes seems to find more covers than last.fm.

[mherger]

> 3. ARTWORK SCAN USING TEST.MYSB.COM / LAST.FM:
>
> Music Scan Details
> Discovering files (3912 of 3912) Complete 00:00:07
> Scan new files (2756 of 2756) Complete 00:01:52
> Discovering playlists (1 of 1) Complete 00:00:00
> iTunes Import (2947 of 2947) Complete 00:00:46
> iTunes Playlist Import (217 of 217) Complete 00:00:42
>
> Downloading Artwork (2674 of 2674) Complete 00:14:40
> Pre-caching Artwork (119 of 119) Complete 00:00:06

Thanks. Did you apply any of the above patches before running these tests? Can you give an estimate how well it worked (matches vs. misses or false matches)?

--

Michael

[copperstate]

Thanks. Did you apply any of the above patches before running these tests? Can you give an estimate how well it worked (matches vs. misses or false matches)?

--

Michael

I used the stock 7.6 SBS for MacOS X and did not apply any of the patches. One thing I noticed is that the library scan takes (almost) forever to finish if not internet is present (e.g. the ADSL modem is turned off):

Music Scan Details
Discovering files (3911 of 3911) Complete 00:00:22
Scan new files (39 of 39) Complete 04:58:34

Downloading Artwork (1663 of 1663) Complete 04:40:19
Pre-caching Artwork (4 of 4) Complete 00:00:02

The server has finished scanning your music collection.
Total Time: 09:39:17 (Monday, 9 August 2010 / 09.32)

An other thing that I noticed is that either the computed scanning time or the time it has finished seem to be incorrect, as the daily scan starts at 04:30 a.m. I will look into the artwork later on.

[verypsb]

It's working good enough to give it some broader testing. Scanning my test library of about 670 albums it was able to find roughly 90% of all artwork automatically.

If I could get 7.6 running on my system, I would gladly test it. However, I haven't been able to run 7.6 (or 7.5e) for a very long time. (I never had a working 7.6 (or 7.5e) squeezeboxserver) All builds that I've tried are unusable (scanner crashes, scanner stalls, squeezeboxserver unresponsive, squeezeboxserver crashes, etc)

All I have to do is installing a squeezeboxserver version that doesn't use sqlite, and all is up & running again.

OS: Windows 7 x64

[copperstate]

If I could get 7.6 running on my system, I would gladly test it. However, I haven't been able to run 7.6 (or 7.5e) for a very long time. (I never had a working 7.6 (or 7.5e) squeezeboxserver) All builds that I've tried are unusable (scanner crashes, scanner stalls, squeezeboxserver unresponsive, squeezeboxserver crashes, etc)

All I have to do is installing a squeezeboxserver version that doesn't use sqlite, and all is up & running again.

OS: Windows 7 x64

If sqlite seems to be the problem, that may be easily remedied by selecting mysql in the 7.6 web ui settings (advanced/performance). Never tried it with Windows though.