PDA

View Full Version : CLI: "rescan done" notice not sent



gerry_duprey
2006-03-11, 12:21
Howdy,

After issuing a "listen 1" on a CLI connection, I will get a simple "rescan" notice/report when a rescan has been started (in this case, by using the slimserver web interface).

However, I never receive a "rescan done" notice. Do I need to do something more than "listen 1" to enable receiving "rescan done" messages?

This is with 6.5b1 6554 (3/11 build)

Thanks,

Gerry

Fred
2006-03-11, 17:00
No. Will look into it.

Fred

Fred
2006-03-19, 15:07
Can't reproduce that. What's your config, using iTunes?

Fred

gerry_duprey
2006-03-19, 15:15
No itunes -- just one root directory with music (music is organzied in subfolders by artist, then album) and one directory where my M3U playlists is kept.

My entire session works like this:

1) telnet 10.0.0.1 9090
2) type "listen 1" -- "listen 1" is echoed back
3) type "rescan ?" -- "rescan 0" is echoed back
4) Press the "Rescan" button on the Web interface
5) Receive a "rescan" report/notice on the telnet connection
6) Watch slimserver.pl via "top" until it stops consuming CPU
7) CPU consumption drops to 0, no report or anything on connection
8) Issue a "rescan ?" -- "rescan 0" is echoed back confirming it's no longer rescanning

In terms of the settings, I'm issuing a rescan, but there are no actual changes to be found (just issued it to test what the server would send, not because I needed it). It was set to the "Look for new and changed" setting when I pressed the button.

Reported slimserver version is 6.5b1 6554

Let me know if there is any info I can get you.

Gerry

Fred
2006-03-19, 16:10
Works for me on 6619 using the steps above... Can you try it again next nightly? Thanks.

Fred

gerry_duprey
2006-03-20, 09:48
Howdy,

Just downloaded, installed and started last nights build (3/20 build, Web interface shows 6.5b1 6626. Running perl 5.8.8

Unfortunatly, there doesn't appear to be any change. I checked the log files and there were no messages/errors during the rescan.

A "rescan ?" done before, during and after the scan did report the right thing, but I never received a "rescan done" or any form of report when the rescan completed.

I understand it's pretty tough to diagnose a problem when you can't see it yourself. If it would help, I can provide ssh login to the linux server slimserver.pl is running on and from there you can look at the config/logs and telnet into the CLI to issue a rescan command and watch the results. In understand if you'd rather not, just putting the option out there.

Gerry

Fred
2006-03-20, 13:45
Before we go the ssh route, can you enable d_import and d_command and try it again?

Thanks

Fred

gerry_duprey
2006-03-21, 19:38
Howdy,

Okay -- ran it with both d_import and d_command. I can send you the log if you'd like. What I think are relevant entries are:

Start of scan:


2006-03-21 21:26:39.9424 Import: Wiped all in-memory caches.
2006-03-21 21:26:43.8080 Import: Starting PLAYLIST scan
2006-03-21 21:26:43.8470 Import: Starting FOLDER scan
2006-03-21 21:26:44.7840 Request: Command [rescan] (Done)
2006-03-21 21:26:44.8319 Request: Notifying CODE(0x9cf9470) of rescan =~ (no filter)
2006-03-21 21:26:44.8324 Request: Don't notify CODE(0x8dbdbe0) of rescan !~ [['playlist']]
2006-03-21 21:26:44.8325 Request: Notifying CODE(0x98a1094) of rescan =~ (no filter)
2006-03-21 21:26:44.9633 Import: Scanning with 2 import plugins
2006-03-21 21:26:44.9634 PLAYLIST scan started at: Tue Mar 21 21:26:43 2006
2006-03-21 21:26:44.9635 FOLDER scan started at: Tue Mar 21 21:26:43 2006

One of the warning/errors that occur during the scan (unrelated, I beleive, to the CLI issue, but just in case):

2006-03-21 21:27:17.1100 Slim::Formats::Parse::readM3U:
WARNING:
file:///usr/opt/mp3/music/XTC/Black%20Sea/Track%205.mp3 found in playlist:
file:///usr/opt/mp3/slim/stock/Entertaining.m3u doesn't exist on disk - skipping!


When I sent a "rescan ?" from my CLI client during the scan:


2006-03-21 21:27:34.5948 Request: Query [rescan] from CLI (Done)
2006-03-21 21:27:34.5949 Result: [_rescan] = [1]


End of the rescan/log:


2006-03-21 21:30:36.7207 Import: Completed PLAYLIST Scan in 232 seconds.
2006-03-21 21:30:48.3254 Import: Scanning with 1 import plugins
2006-03-21 21:30:48.3257 FOLDER scan started at: Tue Mar 21 21:26:43 2006
2006-03-21 21:30:48.3258
2006-03-21 21:30:48.3768 Import: Completed FOLDER Scan in 244 seconds.


As before, I did not get any "rescan done" (or "rescan 0") or any other report when the rescan completed.

Gerry

Fred
2006-03-23, 16:56
Just a hunch: is look for artwork off and what happens if you turn it on?

Fred

gerry_duprey
2006-03-23, 17:27
Just a hunch: is look for artwork off and what happens if you turn it on?

Bingo!

No, I don't use the artwork stuff. So as you asked, I put in an empty directory and told slimserver to use artwork, then did a rescan.

With the artwork in place, I now get a "rescan done" upon completion of the rescan! Yeah!

Is artwork (or at least a dummy artwork directory) a slimserver requirement or just something the CLI was expecting?

Also, and this is in the gravy department, is there anyway a "rescan done" report can indicate if the last scan actually found any changes or not? I ask because my client caches a lot of state and right now, whenever I get a rescan done, I plan to dump it all and reload it (a time consuming process). If I knew the rescan detected no changes, I could reasonably assume my local cached data is still valid and ignore this. Of course, going completely nutz, it would be nice to know what changed (i.e. genres, playlists, albums, tracks, artists, etc). But now I'm off the deep end. Just getting "rescan done" is a great step forward!

Thanks!

Gerry

Fred
2006-03-23, 17:53
Is artwork (or at least a dummy artwork directory) a slimserver requirement or just something the CLI was expecting?

No, it's a bug in the scan code. There are very minor things happening when a rescan is complete, among them the notification, but as coded now, this state is never reached if artwork scanning is disabled. Given there are very minor side effects and given most installs have artwork on, this never got noticed until today. Will fix it. Strictly speaking it's not a CLI but at all, but CLI users are good at digging our strange non CLI related server behaviours :-)


Also, and this is in the gravy department, is there anyway a "rescan done" report can indicate if the last scan actually found any changes or not?

I'd need to investigate but I doubt there is an easy way. It's an "add stuff and clean up at the end" kind of algorithm...
And in any case it would not help you much IMHO. As I mentioned, the server checks the files when you use them (f.e. play them and get them with the songs command), and will update the DB if it catches a change at that point. Try changing the genre of a file behind slimserver's back then play it, and slimserver will update the DB with the new genre. So "rescan done" would not really be "complete" indication of change anyway...

The use case for the CLI was not really to cache all of the data, it was to use it from the API directly (i.e. if you had a scroll list with 10 items, you would get only those 10 and get the next 10 if the user goes down in the list).

I am not saying this is the "right way", just that this is the way it was designed, and the reason why f.e. rescan done is not notified in status even though rescan is in the command. And in any case, some serious rework of the DB mechanisms would be needed to enable some SW to keep a synchronized DB.

HTH

Fred

gerry_duprey
2006-03-23, 18:02
The use case for the CLI was not really to cache all of the data, it was to use it from the API directly (i.e. if you had a scroll list with 10 items, you would get only those 10 and get the next 10 if the user goes down in the list).
Understood -- though it does work pretty well for this, with the caveat being that when you get a "rescan done", you need to reload the cached data.

In my case, I'm building both a CLI API (in Java) to wrap the slimserver stuff up, but also a Touch screen based app for kiosk use in a bar. To make it as snappy as possible and provide quick "browsing", I need to cache things locally. The delay to refresh the client side, while annoying, isn't horrible (about 1 minute) and infrequent (on the order of a few times a week). So overall, I expect it to be more than "adequate" for this app.

Thanks again for all your great work on the CLI!!

Gerry

Jeff
2006-03-23, 19:36
To make it as snappy as possible and provide quick "browsing", I need to cache things locally.
You may want to take a quick look at the current performance of the CLI code for browsing.

In the AMX SlimServerMod module I wrote (see http://sourceforge.net/project/screenshots.php?group_id=62827 for a screen shot), I found that performance was quite good using the CLI directly. There was no need to do caching, and performance is excellent (way faster than the WWW pages).

Just a thought. This would allow you to dramatically simplify your code ...

-- Jeff

gerry_duprey
2006-03-24, 06:02
Thanks, but I did the caching as a result of the response speed. My original take used no caching, fetching what was needed on the fly. But the delays and impact on the user interface in some areas eventually led me to caching.

While response of most CLI commands is very good and bordering on instantaneous, some of the commands that dealt with database activity had very perceptable delays. In particular, requesting data via the "songs" command (or tracks or titles) resulted in seeing a 1 to 2 second delay between issuing the command and getting a result. This is with about 3,500 tracks under 6.5. I don't think this is the CLI per-se, but the result of how data is fetched from the database and the delays things like SQLLite impose on resolving a query (and, in particular, resolving the "get X records starting at offset Y into a result set" stuff.

That isn't bad, but not quite good enough for me to abandon caching for a highly interactive app.

Fortunatly, adding caching wasn't that complicated. I already had object to represent playlists, albums, artists, tracks, etc. And the CLI makes fetching the data and parsing it pretty easy. The last steps were just creating a few lookup tables to cache in. Worked really well.

Gerry

Fred
2006-03-25, 06:02
Fixed in 6713. If this works for you, I will apply it to 6.2.2 as well.

Fred

gerry_duprey
2006-03-26, 19:33
Not sure if there is anything related (likely not), but after pulling the nightly (3/26 nightly), the slimserver runs for a while, but eventually dies after an hour or two with the error:

internal inconsistency: event should have been found at /usr/local/slimserver/CPAN/POE/Queue/Array.pm line 221.

This is on a completely fresh slimserver install (I uninstalled the last version, purged the cache and everything in the /usr/local/slimserver and reinstalled).

Gerry

Ben Sandee
2006-03-26, 20:04
On 3/26/06, gerry_duprey <
gerry_duprey.25b48b1143426901 (AT) no-mx (DOT) forums.slimdevices.com> wrote:
>
>
> Not sure if there is anything related (likely not), but after pulling
> the nightly (3/26 nightly), the slimserver runs for a while, but
> eventually dies after an hour or two with the error:
>
> internal inconsistency: event should have been found at
> /usr/local/slimserver/CPAN/POE/Queue/Array.pm line 221.
>
>
I had to disable LastFM/SlimScrobbler plugin to get rid of this problem just
recently. Something changed that triggered the problem but I doubt it's a
CLI issue. It would be nice if that error message, wherever it's generated
from, actually included a full stack trace rather than this particular
message which doesn't give context. It took me a long time to track down
LastFM/SlimScrobbler (on my system).

Ben

Triode
2006-03-27, 10:54
Actually I believe this error message is due to bug 3204 - fixed in svn 6737.

Adrian
internal inconsistency: event should have been found at
/usr/local/slimserver/CPAN/POE/Queue/Array.pm line 221.

I had to disable LastFM/SlimScrobbler plugin to get rid of this problem just recently. Something changed that triggered the problem but I doubt it's a CLI issue. It would be nice if that error message, wherever it's generated from, actually included a full stack trace rather than this particular message which doesn't give context. It took me a long time to track down LastFM/SlimScrobbler (on my system).

Ben

Fred
2006-03-27, 15:23
Can anyone confirm the rescan done occurs even if artwork scan is off?

Fred

gerry_duprey
2006-04-02, 08:36
Howdy Fred,

Could you tell me how to turn artwork off? After I turned it on for my older 3/20 build, I did get "rescan done" reports. Then, when the build you posted about with the fix was ready, I wanted to turn artwork off on my existing 3/20 build to get the config back to the state where it didn't issue "rescan done" so I could prove the update fixed the issue.

I removed the directory I had added for artwork, but didn't see anything else to "turn it off". However, even after I removed it and wiped the cache and did a reload, the 3/20 build was still consistantly sending a "rescan done" (where it hadn't before). In short, it appeared that while the 3/20 build didn't send a "rescan done" at first, once I added artwork, even if I removed it, it did work and thus I can't prove that next update actually fixed it or not.

I really don't do anything with artwork, so it's a part of slimserver I'm entirely unfamiliar with. If there is a way to turn it off (other than blanking the directory), I'll do that and test again with the old version (to demonstrate that rescan done is not working), then upgrade and prove/test that it is working with a newer release.

Hope that all made sense,

Gerry

Fred
2006-04-03, 01:26
Server Settings -> Performance -> Look For Artwork. If this was off, rescan never finished.

HTH

Fred