PDA

View Full Version : Rescan performance



Triode
2005-08-23, 17:51
Dan,

Could of rescan issues - current subversion (4035):

1) The first phase of rescanning currently blocks the server. On my slow server this takes over a minute and players loose
connection. [~1200 flacs on P3 500MHz]

The following is an output from --d_sql --d_select:

2005-08-24 01:35:12.9912 writeNoBlock: writing a segment of length: 1290
2005-08-24 01:35:13.0244 Start and End node: [default:default]
2005-08-24 01:35:13.0264 Running SQL query: [SELECT DISTINCT tracks.id AS id,tracks.album AS album,tracks.url AS
url,tracks.multialbumsortkey AS multialbumsortkey,tracks.ct AS ct FROM tracks ORDER BY tracks.titlesort]
2005-08-24 01:36:21.2000 Start and End node: [default:default]
2005-08-24 01:36:21.2025 Running SQL query: [SELECT DISTINCT tracks.id AS id,tracks.album AS album,tracks.url AS
url,tracks.multialbumsortkey AS multialbumsortkey,tracks.ct AS ct FROM tracks WHERE ( tracks.ct = ? ) ORDER BY tracks.titlesort]
2005-08-24 01:36:21.2033 Bind arguments: [ssp]

2005-08-24 01:36:22.8635 writeNoBlock: writing a segment of length: 1290

The select loop is blocked out for 1 minute 10 secs in this case.

Is there anything that can be done about this prior to moving to mysql and a separate process? [other than me buying a faster
server!]

If not should we put some warning messages on the web interface to explain this could occur on slow servers?

2) Second one is easy - each rescan adds ~1200 to the number of songs - i.e. it adds all songs in the music folder to the current
number of songs..

Adrian

Triode
2005-08-24, 10:02
Dan,

Turns out it is not the sql query. The reason my server freezes when rescanning is that clearExternalPlaylists takes 70 seconds on
a rescan.

When it runs it seems to perform a delete on all tracks in the music folder. I think this is because:

$self->getPlaylists('external')

Seems to return all tracks in the database, which it then proceeds to delete.

Should getPlaylist trap the case of a null playlists array and not do the find? [I think the current find sets ct to nothing and
matches everything? This is when it is called with 'external' and no external importers are used.]

Adrian

> The following is an output from --d_sql --d_select:
>
> 2005-08-24 01:35:12.9912 writeNoBlock: writing a segment of length: 1290
> 2005-08-24 01:35:13.0244 Start and End node: [default:default]
> 2005-08-24 01:35:13.0264 Running SQL query: [SELECT DISTINCT tracks.id AS id,tracks.album AS album,tracks.url AS
> url,tracks.multialbumsortkey AS multialbumsortkey,tracks.ct AS ct FROM tracks ORDER BY tracks.titlesort]
> 2005-08-24 01:36:21.2000 Start and End node: [default:default]
> 2005-08-24 01:36:21.2025 Running SQL query: [SELECT DISTINCT tracks.id AS id,tracks.album AS album,tracks.url AS
> url,tracks.multialbumsortkey AS multialbumsortkey,tracks.ct AS ct FROM tracks WHERE ( tracks.ct = ? ) ORDER BY tracks.titlesort]
> 2005-08-24 01:36:21.2033 Bind arguments: [ssp]
>
> 2005-08-24 01:36:22.8635 writeNoBlock: writing a segment of length: 1290
>
> The select loop is blocked out for 1 minute 10 secs in this case.

Dan Sully
2005-08-24, 10:10
* Triode shaped the electrons to say...

>When it runs it seems to perform a delete on all tracks in the music
>folder. I think this is because:
>
>$self->getPlaylists('external')
>
>Seems to return all tracks in the database, which it then proceeds to
>delete.
>
>Should getPlaylist trap the case of a null playlists array and not do the
>find? [I think the current find sets ct to nothing and matches everything?
>This is when it is called with 'external' and no external importers are
>used.]

Yes - it used to do this, but I think Fred's change to combine the functions
broke that functionality. Take a look at the previous revisions of DBIStore.pm

-D
--
<nil> It sucks to discover that you are the foremost authority on some set of things when you've got a problem.

Triode
2005-08-24, 10:37
> Yes - it used to do this, but I think Fred's change to combine the functions
> broke that functionality. Take a look at the previous revisions of DBIStore.pm
>
Thanks - traping this gets back to <1 second of blocking on initiating a rescan.