PDA

View Full Version : What's the story on accented file names and tag?



agentsmith
2006-04-03, 08:30
I finally found out the root cause of a couple of problems:

1. If I have accented characters in a file name, e.g. Mozart,Le nozze di Figaro-Non so pi cosa son, cosa faccio.FLAC, the song does not show up in Slimserver.

2. If I have accented characters in a tag, the song shows up in Slimserver web interface but the songs wont play. The SB2 says file not found.

BTW I am running 6.5b1 - 6338 - Windows XP - EN - cp1252, and MusicMagic latest.

This is highly annoying, anybody knows the best way to fix it short of retagging and/or renaming close to 10,000 songs in 1,000 albums?

Craig, James (IT)
2006-04-03, 08:40
I would raise a bug or search for an existing one.
I already raised a bug that tracks with accented filenames are not read
from iTunes.
http://bugs.slimdevices.com/show_bug.cgi?id=3073

I never tried playing the ones tagged with an accent but not in the file
name.
Could be the same problem as you see and it's not iTunes specific at
all.

Sounds like this problem is spreading...

James

PS. This bug didn't exist in 6.2 last I checked. A downgrade would be
less painful than renaming all the files I think...
--------------------------------------------------------

NOTICE: If received in error, please destroy and notify sender. Sender does not waive confidentiality or privilege, and use is prohibited.

radish
2006-04-03, 08:47
There have been a number of people complaining about issues with accented characters, but in many (most?) cases it just seems to work. I'm lucky in that regard, I have plenty of accents in both filenames and tags and don't have any issues. In some cases I thnk iTunes integration may be related, so that's worth looking at if you use it. In a similar vein MusicMagic might be involved, but I've no evidence for that.

I noticed the file you mentioned was flac - what did you use to write the tags?

FYI I'm running 6.2.1 on XP, no MM or iTunes.

Siduhe
2006-04-03, 08:54
There have been a number of people complaining about issues with accented characters, but in many (most?) cases it just seems to work. I'm lucky in that regard, I have plenty of accents in both filenames and tags and don't have any issues. In some cases I thnk iTunes integration may be related, so that's worth looking at if you use it. In a similar vein MusicMagic might be involved, but I've no evidence for that.

I noticed the file you mentioned was flac - what did you use to write the tags?

FYI I'm running 6.2.1 on XP, no MM or iTunes.

FYI there is a recent beta of MusicIP which seems to sort out the issues with accented files (at least for me) in that the files show up in Music IP once (unaccented) and once in Slimserver (with full accents in place) - no duplicate copies of or missing files and best of all, I can use Foobar to apply replaygain with annoying little boxes appearing at the start of every tag.

If you go to the MusicIP forums you can subscribe to their beta list. I don't use Itunes.

stevieweevie
2006-04-03, 14:09
Just tested the latest Beta of MusicIP against a small subset of files and this bug seems to be resolved.

tom permutt
2006-04-03, 17:49
This is highly annoying, anybody knows the best way to fix it short of retagging and/or renaming close to 10,000 songs in 1,000 albums?Short of, no; but here is a Perl program that will rename all files with non-ASCII characters. Pretty painless if you have Perl installed. (ActiveState Perl is free if you don't, but installation may be slightly less painless.) You don't need to retag.

use File::Find;
use File::Copy;
find sub {($newname = $_) =~ s/\P{IsASCII}//g &&
move $_, $newname
}, "your_directory_here";

agentsmith
2006-04-03, 17:56
Short of, no; but here is a Perl program that will rename all files with non-ASCII characters. Pretty painless if you have Perl installed. (ActiveState Perl is free if you don't, but installation may be slightly less painless.) You don't need to retag.

use File::Find;
use File::Copy;
find sub {($newname = $_) =~ s/\P{IsASCII}//g &&
move $_, $newname
}, "your_directory_here";

Thanks.

I am a Perl/Unix dummy. Could you help with the following?

1. Do you have a link to download and get Perl installed in WinXP?

2. Do I create the file with a text editor? What extension do I name it with?

3. How do I run it? From command line? Do I specify the directory path as parameter? Will it fix all the subdirectories in that path?

4. From the above program logic, would it move all the accented and renamed file to a single directory? And in which case I move it back to the original directory manually?

Thanks again, this would be really helpful!

John

radish
2006-04-03, 18:22
Did you try the new version of MusicIP yet? Sounds like that may be causing your problems and that's a MUCH better way of fixing it than changing the files (IMHO).

Oh - and I didn't write this script (and I haven't tested it) but it looks to me like it just removes all non-ascii characters from filenames (rather than replacing them). It's up to you whether it's OK to end up with Rykspp instead of Royksopp :)



1. Do you have a link to download and get Perl installed in WinXP?

www.activestate.com



2. Do I create the file with a text editor? What extension do I name it with?

.pl



3. How do I run it? From command line? Do I specify the directory path as parameter? Will it fix all the subdirectories in that path?

Yes, no and yes. You set the directory in the file - change "your_dir_here" to the path.



4. From the above program logic, would it move all the accented and renamed file to a single directory? And in which case I move it back to the original directory manually?

Looks to me like they'll be changed in place.

tom permutt
2006-04-03, 19:51
What radish said. Yes, it traverses subdirectories, and "move" means essentially "rename": you get a new file name in the same directory as the old one. If you prefer, change "move" to "copy" and it will leave the old one there, too. And, yes, it changes accented characters to nothing at all. If you prefer to change them to something, put the something between the consecutive //. Remember, though, it's only the filenames that are changing, which you never look at anyway - not the tags - so you probably don't want to spend a lot of time worrying what they are renamed to, as long as they play.

I respectfully disagree with radish, though, that it's better not to change the filenames. I don't feel any need to have special characters in the _filenames_ just because somebody was kind enough to type them into the tags, and it is not just the one piece of software that may have trouble with them.

radish
2006-04-03, 21:06
I don't feel any need to have special characters in the _filenames_ just because somebody was kind enough to type them into the tags, and it is not just the one piece of software that may have trouble with them.

Sure, and that's fine if you really do only ever look at the tags. Some people (myself included) do use the filenames for some things, and in some cases (e.g. some portable players) you have no choice. I'd always rather fix bugs than work around them.

Oh and with my pedantic code-reviewer hat on there is the (ever so slight) risk of data loss here. Consider two files in the same directories whose filenames differ only in accented characters. I can't imagine it happening, but I've regretted saying that in the past :)

agentsmith
2006-04-04, 04:13
Short of, no; but here is a Perl program that will rename all files with non-ASCII characters. Pretty painless if you have Perl installed. (ActiveState Perl is free if you don't, but installation may be slightly less painless.) You don't need to retag.

use File::Find;
use File::Copy;
find sub {($newname = $_) =~ s/\P{IsASCII}//g &&
move $_, $newname
}, "your_directory_here";

I tried and it worked with certain file names.

But when I tried with file names like:

Monteverdi ~ Le passioni dell'anima (Monteverdi) ~ 08 ~ S dolce 'tormento ~ Classical ~ 1999.flac

It returns with the error message:
"Malformed UTF-8 character (unexpected non-continuation byte 0x20 after start byte 0xec) in substitution (s///) at test.pl line 4."

So effectively the filename crashed the Perl script :)

Any ideas? I really appreciate the help. I am getting there.

tom permutt
2006-04-04, 06:22
That's interesting. I may be over my head here, but I'll try to look into it. It might help if you can find a way to send me exactly what is in the fatal filename: dir >file from the command line might do it.

agentsmith
2006-04-04, 08:12
That's interesting. I may be over my head here, but I'll try to look into it. It might help if you can find a way to send me exactly what is in the fatal filename: dir >file from the command line might do it.

Sure, I tried a file named "'tormento.flac" (no quote)

This is just one file name tried to shorten with the error. The original file name was much longer with spaces and all. But both files crashed with same error.

* Filenames with no accent characters are OK.

The following is a snapshot of the exact command sequence. And a list of the test.pl

*******

C:\TEST>perl test.pl
Malformed UTF-8 character (unexpected non-continuation byte 0x27 after start byt
e 0xe8) in substitution (s///) at test.pl line 4.

C:\TEST>type test.pl
use File::Find;
use File::Copy;
find sub {($newname = $_) =~ s/\P{IsASCII}//g &&
move $_, $newname
}, "C:\\test";

*******

tom permutt
2006-04-04, 11:25
Sure, I tried a file named "'tormento.flac" (no quote)
What I meant was, I couldn't tell exactly what bunch of bits is representing the

""in your filename.

So, I'm flying blind here. You might try this, which will not do anything destructive:

use File::Find;
use Encode;
find sub {($newname = decode("iso-8859-1", $_)) =~
s/\P{IsASCII}//g &&
print "$_ would become $newname\n"
}, "your_directory_here";


This will just print out a list of proposed renamings. Let me know what happens. If it's to your liking, you can make the corresponding change to the other script and actually rename the files.

agentsmith
2006-04-05, 05:29
What I meant was, I couldn't tell exactly what bunch of bits is representing the
in your filename.

So, I'm flying blind here. You might try this, which will not do anything destructive:

use File::Find;
use Encode;
find sub {($newname = decode("iso-8859-1", $_)) =~
s/\P{IsASCII}//g &&
print "$_ would become $newname\n"
}, "your_directory_here";


This will just print out a list of proposed renamings. Let me know what happens. If it's to your liking, you can make the corresponding change to the other script and actually rename the files.

Thanks on the new script I had the error

"Can't locate Encode.pm in @INC (@INC contains: C:/Perl/lib C:/Perl/site/lib .) a
t test22.pl line 2.
BEGIN failed--compilation aborted at test22.pl line 2."

However, I tried using another PC with your original scipt and it worked perfectly, I used it to rename all my music file names and it seemed to have worked flawlessly.

Thanks, you have been more than helpful and saved me a lot of grief.

Not sure why both scripts worked on one computer but not another. But this stuff is way over my head ;)

tom permutt
2006-04-05, 08:20
I wonder if you might have downloaded different versions of Perl to the two computers. I found the ActiveState website a little confusing that way. I'm glad you got what you needed.

Anyway, this has been fun. I've learned some things about how to write this robustly as production code (thanks, radish) if I were ever going to do such a thing. Of course, all I really meant was to share my discovery that renaming a lot of files needn't be as daunting as it had seemed both to you and to me.

In fact, what I originally did was write a program just to find them all and print them out, so I could rename them by hand. In the middle of that I got bored enough to realize I didn't care what they were named and to look up how to script the renaming.

agentsmith
2006-04-05, 23:25
I wonder if you might have downloaded different versions of Perl to the two computers. I found the ActiveState website a little confusing that way. I'm glad you got what you needed.

Anyway, this has been fun. I've learned some things about how to write this robustly as production code (thanks, radish) if I were ever going to do such a thing. Of course, all I really meant was to share my discovery that renaming a lot of files needn't be as daunting as it had seemed both to you and to me.

In fact, what I originally did was write a program just to find them all and print them out, so I could rename them by hand. In the middle of that I got bored enough to realize I didn't care what they were named and to look up how to script the renaming.

That is entirely possible. I was utterly confused by he ActiveState web site.

You script is very cool, I wish I could do something like that. is there a dummies guide online where I can at least attempt to learn how to do simple scripts?

tom permutt
2006-04-06, 06:25
I think you'll find all the documentation you could want at perldoc.perl.org. I also found the book "Programming Perl" both useful and enjoyable, but it is quite dense. "Learning Perl" is also good and more discursive.

I'm a Perl novice myself though an experienced programmer. I took it up just a couple of months ago to solve another Slim-related problem ( http://forums.slimdevices.com/showthread.php?t=22079 ).

agentsmith
2006-04-06, 20:38
I think you'll find all the documentation you could want at perldoc.perl.org. I also found the book "Programming Perl" both useful and enjoyable, but it is quite dense. "Learning Perl" is also good and more discursive.

I'm a Perl novice myself though an experienced programmer. I took it up just a couple of months ago to solve another Slim-related problem ( http://forums.slimdevices.com/showthread.php?t=22079 ).

Saw the other solution. You are a clever dude!

I havent really programmed since my first job after college. So I think there will be a really steep learning curve for me.

axiomatic
2006-04-06, 21:16
The other solution is to drop back to 6.2.2. When I had the OP's problem, Slim staff suggested this, and it worked for me.

agentsmith
2006-04-07, 01:08
The other solution is to drop back to 6.2.2. When I had the OP's problem, Slim staff suggested this, and it worked for me.

I had another problem that was solved by 6.5, but I forget what it was.