PDA

View Full Version : adding 2 albums to my library takes 30 minutes



Torgo
2008-12-03, 19:14
surely there is a better way. below is the report generated after doing a "look for new and changed" scan of my library in order to add 2 new CDs.

Seems very silly to me to scan over 100,000 songs just to add something. Could SqueezeCenter not be pointed at exactly what needs to be added in order to sidestep this phenomenon?

--------------------------
Library Statistics

Total Tracks: 103,786
Total Albums: 8,920
Total Artists: 3,133
Total Genres: 217
Total Playing Time: 7354:19:14

Music Scan Details
Directory Scan (24 of 24) Complete 00:09:40
Playlist Scan (1 of 1) Complete 00:00:30
Merge Various Artists (8216 of 8216) Complete 00:04:15
Artwork Scan (2 of 2) Complete 00:00:02
Database Cleanup #1 (103786 of 103786) Complete 00:04:38
Database Cleanup #2 ( of ) Complete 00:03:40
Database Optimize ( of ) Complete 00:07:02

SqueezeCenter has finished scanning your music collection. Total Time:00:29:47 (Wednesday, December 3, 2008 / 5:47 PM)

Mnyb
2008-12-03, 19:24
You can "browse music folder" to the folders containing the new albums they will then be added to the dB.

Beware you still have to do a real scan from time to time.
SC has a tendency to sometimes miss stuff like artwork.

Torgo
2008-12-03, 21:05
oh nice! I'll give that a go. All these years and I never knew that!

JadeMonkee
2008-12-03, 21:13
You can "browse music folder" to the folders containing the new albums they will then be added to the dB.

Beware you still have to do a real scan from time to time.
SC has a tendency to sometimes miss stuff like artwork.

That's a very helpful hint, thank you.
I was going to say that you can start playing the music before the scan has finished (just keep your eye on the "recently added" menu), but that little tip is now moot.

Thanks again, Mnyb!

Torgo
2008-12-03, 21:54
yup, verified. it indeed works. very nice indeed.

after being added into the library it at first misses the album art when viewing the listings for that artist, but if you click into the album to view the tracks then go back and refresh the browser the artwork pops up.

one other hint here, although it takes a while to load in, it is far easier to browse the music folder through the Duet controller than through the web interface since the controller loads in the list for the entire folder. The web interface breaks up the page into many many pages that are labeled with only a number, making finding the artist you are interested in difficult if you have a large library. Alternately you can circumvent this behavior by setting the "items per page" setting to a VERY large number, but that makes the page load in very slowly, again making the duet controller a better choice as strangely it gets the full list far faster. Perhaps changing the web interface skin to "light" just for this procedure might not be a terrible idea.

snarlydwarf
2008-12-03, 22:14
one other hint here, although it takes a while to load in, it far is easier to browse the music folder through the Duet controller than through the web interface since the controller loads in the list for the entire folder. The web interface breaks up the page into many many pages that are labeled with only a number, making finding the artist you are interested in difficult if you have a large library. Alternately you can circumvent this behavior by setting the "items per page" setting to a VERY large number, but that makes the page load in very slowly, again making the duet controller a better choice as strangely it gets the full list far faster. Perhaps changing the web interface skin to "light" just for this procedure might not be a terrible idea.

You may find better performance by nesting your directories.

Large directories are generally slower to process than a series of smaller directories. (Think about it: which would be easier to find a file in: a set of 10 file cabinets, labelled with an iniitial digit, and each holding 1000 files... or one huge cabinet with 10,000 files.)

If Browse Music Folder is showing you 'many many pages' of 50 items, this is part of why your performance is bad.

Torgo
2008-12-03, 22:48
thanks for the tip, but I don't think its gonna happen. With a library that big any step towards automation is a good one. I don't know of any programs that will automagically nest ripped music any deeper than root_directory/artist/album. And if I had to maintain it manually I just might end up chipping my teeth on the barrel of a shotgun. Though I use EAC to rip my music I tend to use iTunes (and The Godfather) to keep it all organized (really the only thing I think iTunes does well)

Though if you had suggestions for better options I'm all ears :)

snarlydwarf
2008-12-03, 23:02
thanks for the tip, but I don't think its gonna happen. With a library that big any step towards automation is a good one. I don't know of any programs that will automagically nest ripped music any deeper than root_directory/artist/album. And if I had to maintain it manually I just might end up chipping my teeth on the barrel of a shotgun. Though I use EAC to rip my music I tend to use iTunes (and The Godfather) to keep it all organized (really the only thing I think iTunes does well)

Though if you had suggestions for better options I'm all ears :)

Surely your ripper can make 36 directories, ie, a hash of the first character of the artist name.

ie /root/a/artist/.. /root/b/bartist etc.

SMTP and NNTP servers have done simple hashed directories for years for this reason. A single level of hashing can, even with English patterns, reduce directory search time by 50% or more.

peter
2008-12-04, 01:53
snarlydwarf wrote:
> Torgo;366179 Wrote:
>
>> thanks for the tip, but I don't think its gonna happen. With a library
>> that big any step towards automation is a good one. I don't know of any
>> programs that will automagically nest ripped music any deeper than
>> root_directory/artist/album. And if I had to maintain it manually I
>> just might end up chipping my teeth on the barrel of a shotgun. Though
>> I use EAC to rip my music I tend to use iTunes (and The Godfather) to
>> keep it all organized (really the only thing I think iTunes does well)
>>
>> Though if you had suggestions for better options I'm all ears :)
>>
>
> Surely your ripper can make 36 directories, ie, a hash of the first
> character of the artist name.
>
> ie /root/a/artist/.. /root/b/bartist etc.
>
> SMTP and NNTP servers have done simple hashed directories for years for
> this reason. A single level of hashing can, even with English patterns,
> reduce directory search time by 50% or more.
>

Undoubtedly, but how much time does a modern OS/FS spend looking through
directories? And how much time (in ms) are we actually talking about?
I'm betting it won't be a lot. Having said that, I actually split my
artists in alphabetical groups, for folder browsing convenience.

Regards,
Peter

Philip Meyer
2008-12-04, 08:32
Vote for enhancement #3673 (http://bugs.slimdevices.com/show_bug.cgi?id=3673), which I raised ages ago.

I suggested that it should be possible to configure multiple SC music folders, instead of/as well as support for shortcut/links, and then the scan type could be set for each defined music folder.

Then the music scan functionality could be extended to only rescan parts of the library.

This could be done during development of version 8.0, because the database schema and scanner are going to be significantly reworked.

MrSinatra
2008-12-04, 08:56
i just voted for it...

but i am the ONLY vote for it. doesn't hurt to vote for your own bug. i hope others here do as well. seems to me it should get the new schema keyword even tho its a scanner issue, since they are so closely related and as u said both are being worked on.

i also voted for 607, which now has 10 votes, and has the keyword.

maggior
2008-12-04, 09:14
You can "browse music folder" to the folders containing the new albums they will then be added to the dB.

Beware you still have to do a real scan from time to time.
SC has a tendency to sometimes miss stuff like artwork.

Yes this is a very handy tip. With a pretty large library myself, it saved me a lot of scanning time.

The only trouble is that this does not work if you are using MusicIP or iTunes integration (i.e. you haven't specified a directory for your music). But the MusicIP scan time is supposed to improve in 7.3.

Torgo
2008-12-04, 11:16
Surely your ripper can make 36 directories, ie, a hash of the first character of the artist name.

ie /root/a/artist/.. /root/b/bartist etc.

Unless I have been missing something all these years I have never seen any ripper capable of doing this. Now I've only used a few rippers but honestly you say this as if its the most common feature in the world, and yet I've never seen it.

EAC, for example has the following options for creating file/directory names:

Track Title
Track NUmber
CD or track artist
Release year
CD title
ID3 music type
freeDB music type
CD artist
freedb ID

Care to explain how I am supposed to get it to create a hash of the first character of the artist name with these as my options?

Philip Meyer
2008-12-04, 11:47
>Care to explain how I am supposed to get it to create a hash of the
>first character of the artist name with these as my options?
Mp3tag could do this, as you can call a function to get the first character of the artist name, and store that as part of the path/filename.

Torgo
2008-12-04, 12:11
>Mp3tag could do this, as you can call a function to get the first character of the artist name, and store that as part of the path/filename.

MP3Tag is a tag editor, not a ripper. As I said before automation is key and retagging/reorganizing everything as an additional step is not on the table. It's gotta do its thing as it rips the file. And the statment in question was "Surely your ripper can..."

MeSue
2008-12-04, 16:23
I'm pretty sure J. River can do it, but I don't know the syntax offhand.