PDA

View Full Version : Unicode troubles



aix
2005-12-05, 00:36
I am having issues with Unicode and SlimServer 6.2.1 running on Windows XP. My music library makes extensive use of non-ANSI characters (predominantly Cyrillic). However, almost all Cyrillic text (artists, albums and song titles) is displayed -- and sorted -- incorrectly in the Web GUI.

The library is a mix of WAV and MP3 files. WAVs have no tags; MP3s have a mix of ID3v1 and ID3v2 tags.

One interesting case is an album that is mostly OK, but parts of which are displayed in a wrong character encoding. When examining the files on disk, there is no obvious reasons for why this would happen. All files are named consistently and all MP3 tags are identical in terms of format and encoding.

I would appreciate any ideas for how best to troubleshoot this.

Dan Sully
2005-12-05, 10:52
* aix shaped the electrons to say...

>The library is a mix of WAV and MP3 files. WAVs have no tags; MP3s have
>a mix of IDv1 and IDv2 tags.
>
>One interesting case is an album that is mostly OK, but parts of which
>are displayed in a wrong character encoding. When examining the files
>on disk, there is no obvious reasons for why this would happen. All
>files are named consistently and all MP3 tags are identical in terms of
>format and encoding.

My first step would be to convert all your tags to ID3v2.4 and remove the ID3v1 tags.

-D
--
I'm really looking forward to this hangover.

aix
2005-12-06, 00:00
Thanks for the suggestion.

To make things more manageable, I am focussing in on one album. All files in the album are in the same format, and all tags are ID3v2. There are no ID3v1 tags. Character encoding, both in file names and tags, appears to be consistent. All files contain album, artist and song title tags.

Yet some of the song titles are displayed as gibberish in the Web GUI (most songs look OK). When I do a full rescan of the library, album title alternates between correct Cyrillic text and gibberish.

I would appreciate any suggestions for troubleshooting this particular album as it might shed some light on what's going on in the rest of the library.

Dan Sully
2005-12-06, 00:41
* aix shaped the electrons to say...

>I would appreciate any suggestions for troubleshooting this particular
>album as it might shed some light on what's going on in the rest of the
>library.

Could you open a bug at http://bugs.slimdevices.com/

and attach one of the files in question to it?

Thanks.

-D
--
<iNoah> all your base class are belong to us

aix
2005-12-07, 00:08
I have been able to sort out all my MP3s by removing and re-creating all ID3 tags.

The next step is to figure out why all Cyrillic text is displayed incorrectly for WAVs. Any suggestions?

Thanks

aix
2005-12-07, 13:35
I did a bit more diagnostic. The garbled filenames are basically Russian text encoded in cp1251 that is being displayed as cp1252. Interestingly enough, the bottom line of the "Server Settings" also mentions cp1252:

SlimServer Version: 6.2.1 - 5194 - Windows XP - EN - cp1252

Any suggestions?

P.S. All filenames look fine in every single application I've tried, including Windows Explorer and OS X Finder. This suggests that the problem lies with SlimServer and is not a system-wide issue.

aix
2005-12-09, 00:21
Looks like I am on my own on this one. ;-)

Anyhow, could someone explain the significance of code page 1252 mentioned on the Server Settings page:

SlimServer Version: 6.2.1 - 5194 - Windows XP - EN - cp1252
Given that the HTML page is rendered in UTF-8, the relevance of cp1252 is not immediately clear. For example, is it used in filename processing?

Dan Sully
2005-12-09, 01:04
* aix shaped the electrons to say...

>Looks like I am on my own on this one. ;-)
>
>Anyhow, could someone explain the significance of code page 1252
>mentioned on the Server Settings page:
>> SlimServer Version: 6.2.1 - 5194 - Windows XP - EN - *cp1252*
>Given that the HTML page is rendered in UTF-8, the relevance of cp1252
>is not immediately clear. For example, is it used in filename
>processing?

Yes - that is the local code page of your system.

-D
--
<iNoah> kernel's original recipe: 11 secret args and switches

aix
2005-12-09, 01:16
Yes - that is the local code page of your system.

Does this imply that SlimServer won't be able to handle filenames that have characters not present in cp1252?

Is the code page configurable?

P.S. FWIW, the filenames appear just fine in every other app I've tried.

Dan Sully
2005-12-09, 10:22
* aix shaped the electrons to say...

>> Yes - that is the local code page of your system.
>
>Does this imply that SlimServer won't be able to handle filenames that
>have characters not present in cp1252?

There's currently a bug open:

http://bugs.slimdevices.com/show_bug.cgi?id=2475

And from my investigations, it's a low level Perl issue.

>Is the code page configurable?

Yes - via Control Panel -> Regional Settings.

-D
--
"A good messenger expects to get shot." --Larry Wall

aix
2005-12-09, 11:17
>Is the code page configurable?

Yes - via Control Panel -> Regional Settings.
I spent a couple of hours earlier in the week playing with Regional and Language Options in Windows XP, specificallly "Language for non-UNICODE programs". However, no changes I made there had any effect on the code page displayed under Server Settings in SlimServer.

Could you explain in slightly more detail what I should be changing and what you'd expect to happen?

Not sure if it's relevant, but I run SlimServer as a service using my Windows account instead of LocalSystem so that it can access the NAS share that houses the music library.

aix
2005-12-09, 11:21
>Does this imply that SlimServer won't be able to handle filenames that
>have characters not present in cp1252?

There's currently a bug open:

http://bugs.slimdevices.com/show_bug.cgi?id=2475

And from my investigations, it's a low level Perl issue.

I've had a look, and issue I am experiencing appears to be sufficiently different from the one reported there. I have therefore raised a new bug report, http://bugs.slimdevices.com/show_bug.cgi?id=2702.