PDA

View Full Version : Database Plans (was: Removing duplicate tracks)



JJZolx
2005-05-02, 16:51
One thing I think the "Removing duplicate tracks" thread brings up - something I've been wondering about - is the possibility of using outside maintenance programs on the track database.

Say you had a task that was too complex or too low priority to be integrated into SlimServer itself. Removing duplicate tracks might be one such task, compiling album covers might be another. Maybe even picking up multiple artist tags in FLAC files and cataloging them. Ideally, now that the data is in a neat little RDBMS, we might develop scripts or programs to maintian the track catalog independant of SlimServer. The idea of having to muck about with the SlimServer code itself and then having the code integrated and tested within the server isn't very attractive when you could knock out a script in an hour and a few lines of your favorite language.

But I realize that this isn't likely to work very well with the way SlimServer currently operates. Doing a wipe database (which I tend to do several times a week these days) will likely wipe out any changes made to the data.

Is it in the plans that sometime in the future this will be doable? This was one of the reasons I was excited about SlimServer moving to the use of a database backend, but as it currently stands, about the only thing you can do in an outside app is read-only.
________
XT350 (http://www.cyclechaos.com/wiki/Yamaha_XT350)

pfarrell
2005-05-02, 18:28
On Mon, 2005-05-02 at 16:51 -0700, JJZolx wrote:
> One thing I think the "Removing duplicate tracks" thread brings up -
> something I've been wondering about - is the possibility of using
> outside maintenance programs on the track database.

>Is it in the plans that sometime in the future this will be doable?

This is one of the major reasons behind the use of an external
database, so lots of cool utilities could be written without
changing any of the SlimServer code.

Part of the early design work was a long discussion about
what a song (or track) is, so that the database can
be built up with external data sources. For simple pop
songs, using the internal tags is fine, but
for classical works, there is no realistic way
to keep all the important information in the file.

The "duplicate track" thread clearly shows that differing folks
have different definitions of what a duplicate track is.
My personal definition is bit identical music bytes, without
tags or filenames or other stuff. But the whole idea
is to remove those theological arguments from the
code that accesses the music bytes.

> the use of a database backend, but as it currently stands, about the
> only thing you can do in an outside app is read-only.

I've not seen anything about this in the developer lists.
It is a relational database accessable by SQL from nearly
any language or tool (perl, php, java, etc.).

Of course, once you populate the database with lots
of external metadata, you don't want a 'wipe cache' to
zap it all. I'd want that button to be removed :-)



--
Pat
http://www.pfarrell.com/music/slimserver/slimsoftware.html

Jack Coates
2005-05-02, 18:34
Pat Farrell wrote:
> On Mon, 2005-05-02 at 16:51 -0700, JJZolx wrote:
>
>>One thing I think the "Removing duplicate tracks" thread brings up -
>>something I've been wondering about - is the possibility of using
>>outside maintenance programs on the track database.
>
>
>>Is it in the plans that sometime in the future this will be doable?
>
>
> This is one of the major reasons behind the use of an external
> database, so lots of cool utilities could be written without
> changing any of the SlimServer code.
>
> Part of the early design work was a long discussion about
> what a song (or track) is, so that the database can
> be built up with external data sources. For simple pop
> songs, using the internal tags is fine, but
> for classical works, there is no realistic way
> to keep all the important information in the file.
>
> The "duplicate track" thread clearly shows that differing folks
> have different definitions of what a duplicate track is.
> My personal definition is bit identical music bytes, without
> tags or filenames or other stuff. But the whole idea
> is to remove those theological arguments from the
> code that accesses the music bytes.
>
>
>>the use of a database backend, but as it currently stands, about the
>>only thing you can do in an outside app is read-only.
>
>
> I've not seen anything about this in the developer lists.
> It is a relational database accessable by SQL from nearly
> any language or tool (perl, php, java, etc.).
>
> Of course, once you populate the database with lots
> of external metadata, you don't want a 'wipe cache' to
> zap it all. I'd want that button to be removed :-)
>

The wipe cache is a nuclear strike from orbit, which is handy now but
hopefully will become less desirable in future releases. But it's going
to be a while til we get there -- the Browse Music Folder issues are
relevant.

--
Jack at Monkeynoodle dot Org: It's a Scientific Venture...
Riding the Emergency Third Rail Power Trip since 1996!

Dave Strickler
2005-05-03, 03:39
For my 2 cents...

I have done a reasonable amount of programming work in creating
"de-dupe" code for all kinds of databases over the years. The trick is
always to define what criteria counts as a 'for sure' dupe, and which
counts as a 'ask before you delete' dupe, as no single criteria will
ever be sufficient for everyone's needs. Being flexible/configurable is
best.

I have been working on some PHP code to do the de-dupe as my collection
has passed the 10k song mark, and dupes are starting to appear. I do
agree that it would be great to have it integrate into SlimServer
database (which mine does not yet), and a few other features, but I
think we should take this need for a de-dupe slowly before someone
delete half their music collection by accident. ;-)

I'd be happy to share my code once I've shaken out most of the bugs.
Its not in Perl (in PHP - see reference above), so I doubt it can be
integrated with the SlimServer code. I just run it as a nightly batch
file.

Anyway, my 2 cents...

Dave Strickler
MailWise LLC
617 267-0044 x810
www.mailwise.com


>>> pfarrell (AT) pfarrell (DOT) com 5/2/2005 9:28:08 PM >>>
On Mon, 2005-05-02 at 16:51 -0700, JJZolx wrote:
> One thing I think the "Removing duplicate tracks" thread brings up -
> something I've been wondering about - is the possibility of using
> outside maintenance programs on the track database.

>Is it in the plans that sometime in the future this will be doable?

This is one of the major reasons behind the use of an external
database, so lots of cool utilities could be written without
changing any of the SlimServer code.

Part of the early design work was a long discussion about
what a song (or track) is, so that the database can
be built up with external data sources. For simple pop
songs, using the internal tags is fine, but
for classical works, there is no realistic way
to keep all the important information in the file.

The "duplicate track" thread clearly shows that differing folks
have different definitions of what a duplicate track is.
My personal definition is bit identical music bytes, without
tags or filenames or other stuff. But the whole idea
is to remove those theological arguments from the
code that accesses the music bytes.

> the use of a database backend, but as it currently stands, about the
> only thing you can do in an outside app is read-only.

I've not seen anything about this in the developer lists.
It is a relational database accessable by SQL from nearly
any language or tool (perl, php, java, etc.).

Of course, once you populate the database with lots
of external metadata, you don't want a 'wipe cache' to
zap it all. I'd want that button to be removed :-)



--
Pat
http://www.pfarrell.com/music/slimserver/slimsoftware.html