PDA

View Full Version : README - split-scanner changes, new features, etc



Dan Sully
2006-05-23, 23:03
All - I want to give a little overview of what the split-scanner changes/6.5
will mean to you, the end user, and the (plugin) developer.

The backend is now MySQL - SQLite is gone. MySQL operates in one of two modes:

* You have MySQL installed already, and you have a dbi:mysql dbsource, dbuser &
dbpassword config in your slimserver.pref. SlimServer will use these values,
and your existing database.

* You don't have the above - We include the mysqld 5.0.19 binary for Win32,
OSX & x86 Linux. MySQL will be launched in the background on a non-standard
port, as to not conflict with any future MySQL installations.

-----

= What this means for you: If you are on a platform not listed above, you
will need to install your own copy of MySQL and update slimserver.pref

= MySQL is much faster than SQLite, and allows concurrent queries to the
database. This means we have real iterators/cursors - so not all the data
needs to be fetched at once. This saves on CPU & memory consumption.

Additionally, MySQL supports procedures such as SUM(), MAX(), LEFT(), etc.
This allows specialized queries, for generating the alpha page bar without
inflating all the rows in the database as we had to for SQLite.

-----

We are now using DBIx::Class, replacing Class::DBI as our Object-Relational Mapper.

= What this means for you: Much more flexibility. DBIx::Class is faster, and
allows much more specialized queries with a sane API and good code underneath.

See http://search.cpan.org/dist/DBIx-Class/ for all the docs.

As a (Plugin) Developer - lots has changed here:

Slim/DataStores/* is gone. Slim/Schema/* has replaced it.

Some portions of Slim/DataStores/DBI/DBIStore.pm is now in Slim/Schema.pm

The massive (and fragile) $ds->find() method is gone. I've bubbled up
queries as close to the callers as possible. This allows for specific queries
suited to the callers, and changes are localized. There is much less worry
about breaking something far away when changing a DB query now.

For callers of the old code: $ds->updateOrCreate() or $ds->objectForUrl()
now you will call:

my $track = Slim::Schema->resultset('Track')->updateOrCreate()

You can also use 'rs' in place of 'resultset' as a shortcut.

There is no backwards compatability code for this - I wanted to make a
clean break, as this is a large change. You will need to update your code.

DBIx::Class is based on the notion of resultsets, which can be any
combination of queries, joins, etc. You can change resultset queries to
refine them. This is much more flexible than the strict row-based data of CDBI.

Other examples include:

sub totalTime {
my $self = shift;

return $self->search('Track', undef, {

'select' => [ \'SUM(secs)' ],
'as' => [ 'sum' ],

})->single->get_column('sum');
}

Which will return the sum of the secs column for the track table.

Each object (Track, Album, etc) now has a corresponding ResultSet class,
which has methods you can call on it, such as ->title, etc. The equivalent
old world code would be the massive hash that was Slim::DataStores::Base.

Browsing the DB is now implemented in terms of descending resultset methods.

Check out the code in Slim/Schema/ResultSet/* for more details.

-----

* The sum of the two items above allowed me to split the music scanning
functionality into a separate process. This has many advantages, including:

- Scanning won't interrupt audio of the main slimserver process. CPUs are
very good at context switching between two processes.

- Scanning is now a serial process - the scheduler & it's complexity are gone.

- A progress bar when run from the command line with --progress. This
functionality can find it's way into a web progress bar as well.
(patches welcome!)

- Scanning can be run completely separately from the main server - Unix
users can run scanner.pl from cron, etc.

- When the scanning process goes away, that memory is reclaimed by the OS.
The main slimserver process should also be smaller.

As always, this is an evolving code base - I expect there will be many
changes over the next few days to shore up the bugs that still exist, and

Remember - ask questions early, and often! Thanks.

-D
--
( ( ( [ ] ) ) )
In Stereo Where
Available

mherger
2006-05-23, 23:25
> * You don't have the above - We include the mysqld 5.0.19 binary for
> Win32,

When running a local MySQL server, what are the requirements? Does it have
to be >=5.0? InnoDB?

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

mherger
2006-05-23, 23:26
One more thing...

> Remember - ask questions early, and often! Thanks.

Do you prefer filling Bugzilla or reporting here?

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2006-05-24, 07:40
* Michael Herger shaped the electrons to say...

>>* You don't have the above - We include the mysqld 5.0.19 binary for
>>Win32,
>
>When running a local MySQL server, what are the requirements? Does it have
>to be >=5.0? InnoDB?

4.1 should work, but yes InnoDB is what the schema specifies.

-D
--
"You can usually recover from production flaws...but you can never recover from a bad design".

Dan Sully
2006-05-24, 07:41
* Michael Herger shaped the electrons to say...

>>Remember - ask questions early, and often! Thanks.
>
>Do you prefer filling Bugzilla or reporting here?

Here is good, and if/when it turns into a bug, into Bugzilla it goes.

-D
--
"You can usually recover from production flaws...but you can never recover from a bad design".

Robin Bowes
2006-05-24, 07:56
Dan Sully wrote:
> * Michael Herger shaped the electrons to say...
>
>>> * You don't have the above - We include the mysqld 5.0.19 binary for
>>> Win32,
>>
>> When running a local MySQL server, what are the requirements? Does it
>> have to be >=5.0? InnoDB?
>
> 4.1 should work, but yes InnoDB is what the schema specifies.

Small point: the DBix migration table is created as whatever the DB
default is.

In my case, this was/is MyISAM.

A "fix" would be to patch migration.pm to explicitly make the migration
table InnoDB.

R.

Dan Sully
2006-05-24, 07:59
* Robin Bowes shaped the electrons to say...

>Small point: the DBix migration table is created as whatever the DB
>default is.
>
>In my case, this was/is MyISAM.
>
>A "fix" would be to patch migration.pm to explicitly make the migration
>table InnoDB.

There's not much point in doing for that table.

Also, I've attached possible fix for the socket issue to that bug.

-D
--
"You can usually recover from production flaws...but you can never recover from a bad design".

Robin Bowes
2006-05-24, 08:09
Dan Sully wrote:
> * Robin Bowes shaped the electrons to say...
>
>> Small point: the DBix migration table is created as whatever the DB
>> default is.
>>
>> In my case, this was/is MyISAM.
>>
>> A "fix" would be to patch migration.pm to explicitly make the migration
>> table InnoDB.
>
> There's not much point in doing for that table.

I figured as much.

> Also, I've attached possible fix for the socket issue to that bug.

s/\$dsn/\$source/

It doesn't seem to add the socket to the datasource in slimserver.conf,
but it seems to work.

R.