How does the scanner detect changed files?

[mbg]

Can anyone tell me how the scanner detects changed files? Is it by filesystem modification timestamp or something else?

I am wondering whether I can change tags within files without changing the modification timestamp and still have the scanner detect the changes.

Thanks.

[paulster]

You need to change the timestamps, which has the unfortunate side-effect of bringing the album into the New Music screen.

What I do now is to set MP3tag to leave the timestamps unchanged and then wrote a little piece of VBscript to bump the timestamps forward by 2 seconds (some filesystems have only 2 seconds' granularity) so that the time has changed according to Squeezebox Server, but not so much that it'll be bumped ahead of the next album that I'd ripped.

Doing it this way Squeezebox Server detects the changes but also keeps everything in correct chronological order so your New Music genuinely shows new music.

[mbg]

You need to change the timestamps, which has the unfortunate side-effect of bringing the album into the New Music screen.

What I do now is to set MP3tag to leave the timestamps unchanged and then wrote a little piece of VBscript to bump the timestamps forward by 2 seconds (some filesystems have only 2 seconds' granularity) so that the time has changed according to Squeezebox Server, but not so much that it'll be bumped ahead of the next album that I'd ripped.

Doing it this way Squeezebox Server detects the changes but also keeps everything in correct chronological order so your New Music genuinely shows new music.

Thanks for the info!

I'm using MP3tag, too, so we had the same thing in mind. I was trying to avoid the album being bumped up in "New Music". Specifically, all the Christmas music I just dumped in there from the archives and did some Genre tag fixing on :)

Bumping the original timestamp forward only slightly is a great idea... thanks!

[andyg]

In 7.6, changes are detected by a combination of timestamp and file size, so it's more likely a tagger that doesn't change mtime will change the file size. May not always work due to tag padding schemes but it's better than just using mtime.

[paulster]

Bumping the original timestamp forward only slightly is a great idea... thanks!

' Bumps file modified dates for FLAC files in the directory specified by 2 seconds
' in order to get Squeezebox Server to recognise the changed timestamp but to keep
' the albums in the correct added sequence.
' 2 seconds is chosen because of granularity limits on certain file systems.

Dim strDirectory
Dim strFilename
Dim objShell
Dim dtModifiedDate
Dim fsoFileSystem
Dim objDirectory
Dim objFile

strDirectory = InputBox ("Directory to process", "Choose Directory")

Set fsoFileSystem = CreateObject ("Scripting.FileSystemObject")
Set objDirectory = fsoFileSystem.getFolder (strDirectory)

For Each objFile In objDirectory.Files
strFilename = objFile.Name
If UCase (Right (strFilename, 5)) = ".FLAC" Then
dtModifiedDate = objFile.DateLastModified
Set objShell = CreateObject ("Shell.Application")
' Change the last modified date to previous date plus 2 seconds
objShell.NameSpace (strDirectory).ParseName (strFilename).ModifyDate = DateAdd ("S", 2, dtModifiedDate)
Set objShell = Nothing
End If
Next

Set objDirectory = Nothing
Set fsoFileSystem = Nothing
This is very rough-and-ready as I only wrote it as a quick proof of concept and it did what it needed to. It'll only process .flac files in a single directory as I didn't bother adding recursion or other error checking to it at this stage.

I should probably make it more flexible and add recursion to traverse the directory tree, etc. but for the most part it does what I need. I copy the pathname from MP3tag into the popup window and off it goes.


In 7.6, changes are detected by a combination of timestamp and file size, so it's more likely a tagger that doesn't change mtime will change the file size. May not always work due to tag padding schemes but it's better than just using mtime.
This is definitely an improvement but if you're an MP3tag user then you'll find that the file size only changes when it needs to be increased. Shorten a tag or embed a smaller image and it'll leave the file size the same.

I don't think there's a universal solution you can (realistically) apply in the scanner, unfortunately.

[mbg]

This is very rough-and-ready as I only wrote it as a quick proof of concept and it did what it needed to. It'll only process .flac files in a single directory as I didn't bother adding recursion or other error checking to it at this stage.

I should probably make it more flexible and add recursion to traverse the directory tree, etc. but for the most part it does what I need. I copy the pathname from MP3tag into the popup window and off it goes.

Thanks for the code. I am alright with VBS so can probably tailor it to what I need.

I wonder... whether it would work in conjunction with MP3Tag's "Tools" feature. It has the ability to process all selected files, so I wonder whether, while you already have your files selected, you could pass it to a custom tool (i.e. the VB script). Haven't tried it, so not sure...but looks like it's meant to behave that way.


I don't think there's a universal solution you can (realistically) apply in the scanner, unfortunately.

File system notification is probably the way... but that's a big change.

The other one is something like MD5 digests...but I realize you said "realistic" :).

[mbg]

In 7.6, changes are detected by a combination of timestamp and file size, so it's more likely a tagger that doesn't change mtime will change the file size. May not always work due to tag padding schemes but it's better than just using mtime.

Thanks, this is useful to know!

[JJZolx]

What I do now is to set MP3tag to leave the timestamps unchanged and then wrote a little piece of VBscript to bump the timestamps forward by 2 seconds (some filesystems have only 2 seconds' granularity) so that the time has changed according to Squeezebox Server, but not so much that it'll be bumped ahead of the next album that I'd ripped.

That's exactly what I've done (VBScript and all). I've been lobbying Florian to add a mode to Mp3tag that would bump the timestamps, but have had no luck. I only really use the bump script occasionally, when I need to immediately see a change. With SbS 7.6 and a fast server a full clear & rescan for my 30,000 track Flac library is only about 10 minutes, so I just run a full scan every night to pick up changes.

[paulster]

I wonder... whether it would work in conjunction with MP3Tag's "Tools" feature. It has the ability to process all selected files, so I wonder whether, while you already have your files selected, you could pass it to a custom tool (i.e. the VB script). Haven't tried it, so not sure...but looks like it's meant to behave that way.
I was wondering about that too. I didn't have time to give it a proper look though.

That's exactly what I've done (VBScript and all). I've been lobbying Florian to add a mode to Mp3tag that would bump the timestamps, but have had no luck. I only really use the bump script occasionally, when I need to immediately see a change. With SbS 7.6 and a fast server a full clear & rescan for my 30,000 track Flac library is only about 10 minutes, so I just run a full scan every night to pick up changes.
I hate doing full rescans because I mostly use the web interface on three or four different computers and the cached artwork becomes a real problem every time each album gets assigned a new identifier. That's where bumping the times really comes into its own. That and I'm running my server on a Crusoe-powered HP Thin Client device so full rescans on a 15,000 track library aren't what you'd call speedy.

It would be great if this could be integrated into MP3tag as it's such a useful solution for Squeezebox users.

[erland]

If we are only looking for a way to keep the "New Music" menu ordering, that information is already in the database.

The first time a track is scanned into SBS its timestamp is stored in tracks_persistent.added in database, so if the "New Music" menu would be changed to use "tracks_persistent.added" instead of "tracks.timestamp" we wouldn't have to mess around with separate scripts to modify the timestamps.

Of course, changing New Music logic is probably going to get some users who use the "New Music" menu as a "Last Retagged Music" menu a bit upset, so it might be hard to make everyone happy.

[Mnyb]

If we are only looking for a way to keep the "New Music" menu ordering, that information is already in the database.

The first time a track is scanned into SBS its timestamp is stored in tracks_persistent.added in database, so if the "New Music" menu would be changed to use "tracks_persistent.added" instead of "tracks.timestamp" we wouldn't have to mess around with separate scripts to modify the timestamps.

Of course, changing New Music logic is probably going to get some users who use the "New Music" menu as a "Last Retagged Music" menu a bit upset, so it might be hard to make everyone happy.

This idea is the good imho :) "new" in an sbs context should be "new to sbs" I would love if new music reflected when I added the file to the library, but on the other hand sbs dB is fragile :-/ many people have to resort to cache wipes frequently then this functionality will break.
So while it is a good idea it will not work in practice because I bet that everyone of us have wiped the dB completely the last 2 years or so ?

On the other hand if the dB was a little more sacred and backed-up and possible to migrate between sbs/ss/sc version etc etc.
Than all other good ideas about storing more info than the tags have in the dB could be implemented too :)

[erland]

This idea is the good imho :) "new" in an sbs context should be "new to sbs" I would love if new music reflected when I added the file to the library, but on the other hand sbs dB is fragile :-/ many people have to resort to cache wipes frequently then this functionality will break.
So while it is a good idea it will not work in practice because I bet that everyone of us have wiped the dB completely the last 2 years or so ?

On the other hand if the dB was a little more sacred and backed-up and possible to migrate between sbs/ss/sc version etc etc.

The tracks_persistent table is actually backed up in the "prefs" directory (tracks_persistent.json), so I think the intention is that it should survive a cache wipe. I've never tried if it works though, but hopefully it does.



Than all other good ideas about storing more info than the tags have in the dB could be implemented too :)

I'd love if SBS were able to contain data that survive a rescan, it would open up a lot of possibilities. Unfortunately, I don't think this will ever happen since no one at Slim Devices nor Logitech ever has shown any interest of managing statistics or meta data in SBS.

However, a bit off topic, there might be some people working on a more permanent third party solution...
http://forums.slimdevices.com/showthread.php?t=80910
(Anyone interested in helping with development, let me know)

[Mnyb]

The tracks_persistent table is actually backed up in the "prefs" directory (tracks_persistent.json), so I think the intention is that it should survive a cache wipe. I've never tried if it works though, but hopefully it does.


I'd love if SBS were able to contain data that survive a rescan, it would open up a lot of possibilities. Unfortunately, I don't think this will ever happen since no one at Slim Devices nor Logitech ever has shown any interest of managing statistics or meta data in SBS.

However, a bit off topic, there might be some people working on a more permanent third party solution...
http://forums.slimdevices.com/showthread.php?t=80910
(Anyone interested in helping with development, let me know)

Yeh but actually the prefs get corrupted to sometimes, so you really have to delete everything ?

[erland]

Yeh but actually the prefs get corrupted to sometimes, so you really have to delete everything ?

Fix the bug so it doesn't get corrupted maybe ?

Do you delete everything including database contents and settings in any other application you use ?

[JJZolx]

Yeh but actually the prefs get corrupted to sometimes, so you really have to delete everything

I've been running a lot of versions of Squeezebox Server/Squeeze Center/Slim Server for a long time and have never seen the prefs get corrupted.

However, if you allow mysqueezebox.com to be 'integrated' (synched) with your local server's prefs then just about anything can happen. The syncing with mysb.com is completely f***ed up.

[Mnyb]

I've been running a lot of versions of Squeezebox Server/Squeeze Center/Slim Server for a long time and have never seen the prefs get corrupted.

However, if you allow mysqueezebox.com to be 'integrated' (synched) with your local server's prefs then just about anything can happen. The syncing with mysb.com is completely f***ed up.

Yep however I only noticed player name and alarm setting f***k up and the max volume bug we had for a while, but as usual it's the tip of the iceberg ;)

[paulster]

Of course, changing New Music logic is probably going to get some users who use the "New Music" menu as a "Last Retagged Music" menu a bit upset, so it might be hard to make everyone happy.
You can tell you've been here for some time! ;)

[JJZolx]

Andy expressed some interest in creating an option for which date New Music would be ordered by, but I guess it got lost somewhere along the line.

http://bugs.slimdevices.com/show_bug.cgi?id=1330#c8