PDA

View Full Version : Poll: Browse Music Folder Speed



Dan Sully
2005-06-15, 18:49
I've been hacking up the internals of SlimServer as relating to playlists and browse music folder. I've gotten the speed for browsing to the music folder with 560 top level directories down to 1 second with a numeric pagebar.

I'm going for a balance between existing features and speed/window explorer type functionality.

I'd like some input from people before I go too much further:

From what I've read previously on the list, the majority of people would like BMF to behave more like Windows explorer. IE: No fancy sorting, or displaying of metadata.

Right now I still have cover art being displayed if it exists in a .jpg form, but that's about it in terms of "extra" features. I've not yet tackled the Buttons BMF, that's next.

This will create a new browsetree() function in Pages.pm - and remove browser() and browser_addtodone(). If the directory that you browse into has changed from the timestamp we have in the db, a non-recursive scan will be done immediately - ie: not in the background via the scanner.

Caleb Epstein
2005-06-16, 06:21
On Wed, Jun 15, 2005 at 06:49:13PM -0700, Dan Sully wrote:

> >From what I've read previously on the list, the majority of people
> would like BMF to behave more like Windows explorer. IE: No fancy
> sorting, or displaying of metadata.

One thing I *would* like to see in BMF would be some more info
about the folder, specifically the last modified time. This
would be awesome. Being able to sort on it would be even
better.


--
Caleb Epstein | bklyn . org |
cae at | Brooklyn Dust | Never give an inch!
bklyn dot org | Bunny Mfg. |

Harald Wilhelm
2005-06-16, 08:19
Hi,

please create a feature request for this (sortable last modified time on
browse music folder). I would vote for it immediately.

Thanks.


Caleb Epstein wrote:
> On Wed, Jun 15, 2005 at 06:49:13PM -0700, Dan Sully wrote:
>
>
>>>From what I've read previously on the list, the majority of people
>>would like BMF to behave more like Windows explorer. IE: No fancy
>>sorting, or displaying of metadata.
>
>
> One thing I *would* like to see in BMF would be some more info
> about the folder, specifically the last modified time. This
> would be awesome. Being able to sort on it would be even
> better.

Caleb Epstein
2005-06-16, 09:18
On Thu, Jun 16, 2005 at 05:19:29PM +0200, Harald Wilhelm wrote:

> Hi,
>
> please create a feature request for this (sortable last modified time on
> browse music folder). I would vote for it immediately.

http://bugs.slimdevices.com/show_bug.cgi?id=1679

--
Caleb Epstein | bklyn . org | Check it out, send me comments, and dance
cae at | Brooklyn Dust | joyously in the streets,
bklyn dot org | Bunny Mfg. | -- Linus Torvalds announcing 2.0.27

MrC
2005-06-16, 09:56
I would suggest two improvements: a) use a threaded approach where you first collect only the necessary information quickly (eg. folder names), and update additional information in the background, and b) use a lazy approach where you only get the information relevant to the current view, continuing with a) above in the background.

Is this possible?

Dan Sully
2005-06-16, 10:14
* MrC shaped the electrons to say...

>I would suggest two improvements: a) use a threaded approach where you
>first collect only the necessary information quickly (eg. folder
>names), and update additional information in the background, and b) use
>a lazy approach where you only get the information relevant to the
>current view, continuing with a) above in the background.

This is more like it is currently - which isn't what we want.

My new way will fire of a immediate scan (otherwise one can't display
results), only if the directory level that you are at has changed.

-D
--
<moof> I always think about the production machines when I think of olivia - because god, she has such a nice rackspace.

Harald Wilhelm
2005-06-16, 10:40
Hi,

hmm, I'm missing something ;-)

- Add metadata to Browse Music Folder
- Add metadata to Browse Music Folder and allow sorting on metadata

Metadata could be the last modification time


MrC wrote:
> ------------------------------------------------------------------------
> A poll associated with this post was created, to vote and see the
> results, please visit http://forums.slimdevices.com/showthread.php?t=14689
> ------------------------------------------------------------------------
> Question: Refactoring Browse Music Folder Features
>
> - More like Windows Explorer/ls - no metadata
> - Keep it like it is.
> - I don't care - just make it go faster!
> - Remove Browse Music Folder completely.
> ------------------------------------------------------------------------
>
> I would suggest two improvements: a) use a threaded approach where you
> first collect only the necessary information quickly (eg. folder
> names), and update additional information in the background, and b) use
> a lazy approach where you only get the information relevant to the
> current view, continuing with a) above in the background.
>
> Is this possible?

holthaus
2005-06-16, 11:30
I want it fast as can be. All my music is in
Genre\Artist\Album\Tracks
so, for me, going through the Browse Music Folder is the
fastest way since the collection is pretty large.
The other thing I think would be nice, is for the music to start playing a bit faster. If I hit play on an artist, selecting all the albums by that artist, it takes a bit to get going.
Is it possible to atleast put one song in the cue and playing, then add the other songs later?

I really don't give a darn about the scanning for metadata since I add music in bulk and then tell the server to rescan at that time. Typically before I go to bed, so it can do whatever it wants at night.

I just want it much faster

Richie
2005-06-16, 11:44
I'd like to be able view any other ARTwork that may also be present in the folder. It doesn't have to be displayed by SlimServer, I'd be happy with a filename and a link that opens in a new window.

Richard

JJZolx
2005-06-16, 23:44
Hmm, you definitely need to think this through...

I've been one asking for the explorer-like functionality, but do you want to replace the current behavior, make the behavior configurable, or add a whole new 'Browse By'?

If browsing through the actual file system, would you get rid of the current "add new tracks and metadata to the database while browsing" behavior of BMF? Or could it be made an option? I'm sure people use this feature and many wouldn't want to lose the functionality. But it has to be expensive - you're still hitting the database pretty hard, especially in large directories.

If browsing the file system without metadata, I assume you display file and directory names. But is this going to be usable from the remote UI? From the web UI, seeing actual file names would be great. But on the little Squeezebox screen, would it always be workable? I know many people use file naming conventions such as "Artist - Album - TrackNo - Title". That wouldn't be very handy if all you could see was part of the Artist and Album names while scrolling through on the remote screen.

In both the remote and web UIs, I think you should still be able to view detailed track and file information, including tag data, once you've drilled down and clicked on a music file name. I would think this should easily be doable (performance-wise) by reading directory info, file headers and tags in real time, without hitting the database at all.

Dan Sully
2005-06-17, 00:07
* JJZolx shaped the electrons to say...

>I've been one asking for the explorer-like functionality, but do you
>want to replace the current behavior, make the behavior configurable,
>or add a whole new 'Browse By'?

I am replacing the current behavior - yet still retaining some of the features.

>If browsing through the actual file system, would you get rid of the
>current "add new tracks and metadata to the database while browsing"
>behavior of BMF? Or could it be made an option? I'm sure people use
>this feature and many wouldn't want to lose the functionality. But it
>has to be expensive - you're still hitting the database pretty hard,
>especially in large directories.
>
>If browsing the file system without metadata, I assume you display file
>and directory names. But is this going to be usable from the remote UI?
>>From the web UI, seeing actual file names would be great. But on the
>little Squeezebox screen, would it always be workable? I know many
>people use file naming conventions such as "Artist - Album - TrackNo -
>Title". That wouldn't be very handy if all you could see was part of
>the Artist and Album names while scrolling through on the remote
>screen.
>
>In both the remote and web UIs, I think you should still be able to
>view detailed track and file information, including tag data, once
>you've drilled down and clicked on a music file name. I would think
>this should easily be doable (performance-wise) by reading directory
>info, file headers and tags in real time, without hitting the database
>at all.

The browsing is actually done almost entirely via the database - which is
what makes it so fast. The previous mechanism was a frankenstein of using the
regular file scanner, coupled with the scheduler *and* the database.

I only kick off a non-scheduler based non-recusive scan if the currently
browsed directory has changed it's mtime - ie: something was added, removed
or altered. So that "side-effect feature" stays, as it's useful.

You bring up a good point regarding the potential length of file-level names
on the player UI. I don't think it's going to make too much of a difference
however, as I'd imagine the usage pattern is actually drilling down to
track-1, and pressing play. Where -1 is the album, or however your structure
is arranged. In any case - the filename would scroll, just as all long-length
strings do in the player UI.

Once you do get down to the track level - yes, the metadata information via
songinfo is available as if you were going through the other "Browse by"
mechanisms.

I'm trying to get this patch ready for a release - unfortunately this and the
playlists-in-the db is a tightly coupled patch - which will actually uncouple
them. I hope to have something to send out either Friday evening (PST) or
over the weekend.

FYI: Fred - if you're reading this thread, the 'playlists' CLI has changed
slightly - and I've updated the documentation as well.

-D
--
<nil> It sucks to discover that you are the foremost authority on some set of things when you've got a problem.

JJZolx
2005-06-17, 08:14
I am replacing the current behavior - yet still retaining some of the features.

Consider adding a new Browse By, perhaps only accessible to the web interface. Also consider some user configuration ability. Certainly not for this iteration, but in the future.


The browsing is actually done almost entirely via the database - which is
what makes it so fast.

It is? I thought many of the problems you were trying to address were performance issues.

One reason to being able to see actual files will be to work around any errors that SlimServer has made in its cataloging of tracks and metadata. Seeing what SlimServer "thinks" is your file structure can be frustrating when there are errors. I realize that in theory they should be the same, but until then...

Thinking a bit to the future, another big reason for being able to browse the actual directories will be to someday use SlimServer as a bona fide music library manager. Being able to manipulate artwork, rename, delete and move files, tag music files, and edit playlists will only be possible once a user is given access to what's actually there.


The previous mechanism was a frankenstein of using the
regular file scanner, coupled with the scheduler *and* the database.

I only kick off a non-scheduler based non-recusive scan if the currently
browsed directory has changed it's mtime - ie: something was added, removed
or altered. So that "side-effect feature" stays, as it's useful.

I'd never suggests getting rid of it. But keep in mind that like many things it's only useful to _some_ people. Ideally, it would be great if it could be disabled by those who want to optimize speed and who don't need to add data by this means.


You bring up a good point regarding the potential length of file-level names
on the player UI. I don't think it's going to make too much of a difference
however, as I'd imagine the usage pattern is actually drilling down to
track-1, and pressing play. Where -1 is the album, or however your structure
is arranged. In any case - the filename would scroll, just as all long-length
strings do in the player UI.

Here I'm a bit confused. If you're actually browsing through the database, why not then display metadata instead of file names in the remote UI? Giving only the appearance of browsing through the file system is of limited use. I think it's fantastic that you're cleaning up this code, but if the end result to the user interface (beside performance improvements) is only to display file and directory names instead of metadata, then I say just keep displaying metadata until a real explorer-like interface is developed.

Dan Sully
2005-06-17, 09:08
* JJZolx shaped the electrons to say...

>> The browsing is actually done almost entirely via the database - which
>> is what makes it so fast.
>
>It is? I thought many of the problems you were trying to address were
>performance issues.

I'm addressing performance issues & code resuability / cleanup issues.

>One reason to being able to see actual files will be to work around any
>errors that SlimServer has made in its cataloging of tracks and
>metadata. Seeing what SlimServer "thinks" is your file structure can
>be frustrating when there are errors. I realize that in theory they
>should be the same, but until then...

After thinking over this - I believe I have a fast, general purpose case
implementation for reading the filesystem, doing the appropriate DB lookups,
and keeping the code clean. I do need to test handling of Windows Shortcuts
though, which might foul up everything. But hopefully not.

>Thinking a bit to the future, another big reason for being able to
>browse the actual directories will be to someday use SlimServer as a
>bona fide music library manager. Being able to manipulate artwork,
>rename, delete and move files, tag music files, and edit playlists will
>only be possible once a user is given access to what's actually there.

This is a non-goal of SlimServer in general.

Playlists however are being moved into the database - retaining the
rudimentary reordering and management that we have now.

>I'd never suggests getting rid of it. But keep in mind that like many
>things it's only useful to _some_ people. Ideally, it would be great
>if it could be disabled by those who want to optimize speed and who
>don't need to add data by this means.

I can add a pref for this.

>Here I'm a bit confused. If you're actually browsing through the
>database, why not then display metadata instead of file names in the
>remote UI? Giving only the appearance of browsing through the file
>system is of limited use. I think it's fantastic that you're cleaning
>up this code, but if the end result to the user interface (beside
>performance improvements) is only to display file and directory names
>instead of metadata, then I say just keep displaying metadata until a
>real explorer-like interface is developed.

This is where I have issue - and we would hit the db a lot more for multiple
queries across multiple tables - and having to deal with the infoFormat()
strings, which I want to avoid. Again - I might consider a pref for this.

-D
--
<Nigel> Please refrain from fearing the reaper.