PDA

View Full Version : Umlauts & Co. in folder names



mherger
2005-02-14, 08:54
I have some albums with umlauts and other special characters in their
folder names. These make segfault slimserver v6 when accessing the folder
or its content. The scan does finish, though, finding an appropriate
number of artists and albums. The song count is badly wrong, 3376 vs. 4252
(5.4.1) or 4185 using ("find" from the shell).

The last log entries (-d_info):

2005-02-14 16:49:02.4488 Converting
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge to /home/mh/Musig/für
usszegschnigge
2005-02-14 16:49:02.4525 unk file type for
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge
2005-02-14 16:49:02.4585 Converting
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge to /home/mh/Musig/für
usszegschnigge
2005-02-14 16:49:02.4590 unk file type for
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge
2005-02-14 16:49:02.4628 Converting
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge to /home/mh/Musig/für
usszegschnigge
2005-02-14 16:49:02.4633 unk file type for
file:///home/mh/Musig/f%C3%BCr%20usszegschnigge
2005-02-14 16:49:02.5270 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/01%20-%20Verdamp%20lang%20her.mp3
(TTL: 0).
2005-02-14 16:49:02.6466 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/02%20-%20S%C3%BCdstadt%20verz%C3%A4ll%20nix.mp3
(TTL: 0).
2005-02-14 16:49:02.6550 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/03%20-%20Jraaduss.mp3
(TTL: 0).
2005-02-14 16:49:02.6630 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/04%20-%20Waschsalon.mp3
(TTL: 0).
2005-02-14 16:49:02.6712 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/05%20-%20Jupp.mp3
(TTL: 0).
2005-02-14 16:49:02.6800 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/06%20-%20Frau,%20ich%20freu%20mich.mp3
(TTL: 0).
2005-02-14 16:49:02.6881 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/07%20-%20M%C3%BCsli%20M%C3%A4n.mp3
(TTL: 0).
2005-02-14 16:49:02.6969 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/08%20-%20Fuhl%20am%20Strand.mp3
(TTL: 0).
2005-02-14 16:49:02.7054 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/09%20-%20Ens%20em%20Vertraue.mp3
(TTL: 0).
2005-02-14 16:49:02.7135 CacheItem: Checking status of
file:///home/mh/Musig/Bap/f%C3%BCr%20usszegschnigge/10%20-%20Wo%20mer%20endlich%20SOMMER%20hann.mp3
(TTL: 0).
2005-02-14 16:49:02.8572 isFile(/home/mh/Musig/Bap/für usszegschnigge/01 -
Verdamp lang her.mp3) == 1
2005-02-14 16:49:02.8614 found JPEG image
2005-02-14 16:49:02.8707 found JPEG image
2005-02-14 16:49:02.9271 isFile(/home/mh/Musig/Bap/für usszegschnigge/02 -
Südstadt verzäll nix.mp3) == 1
2005-02-14 16:49:02.9813 isFile(/home/mh/Musig/Bap/für usszegschnigge/03 -
Jraaduss.mp3) == 1
2005-02-14 16:49:03.0299 isFile(/home/mh/Musig/Bap/für usszegschnigge/04 -
Waschsalon.mp3) == 1
2005-02-14 16:49:03.0875 isFile(/home/mh/Musig/Bap/für usszegschnigge/05 -
Jupp.mp3) == 1
2005-02-14 16:49:03.1398 isFile(/home/mh/Musig/Bap/für usszegschnigge/06 -
Frau, ich freu mich.mp3) == 1
2005-02-14 16:49:03.1924 isFile(/home/mh/Musig/Bap/für usszegschnigge/07 -
Müsli Män.mp3) == 1

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2005-02-14, 08:57
* Michael Herger shaped the electrons to say...

>I have some albums with umlauts and other special characters in their
>folder names. These make segfault slimserver v6 when accessing the folder
>or its content. The scan does finish, though, finding an appropriate
>number of artists and albums. The song count is badly wrong, 3376 vs. 4252
>(5.4.1) or 4185 using ("find" from the shell).

What is your LC_CTYPE set to?

-D
--
"Hey, careful, man, there's a beverage here!"

mherger
2005-02-14, 09:02
> * Michael Herger shaped the electrons to say...
>
>> I have some albums with umlauts and other special characters in their
>> folder names. These make segfault slimserver v6 when accessing the
>> folder or its content. The scan does finish, though, finding an
>> appropriate number of artists and albums. The song count is badly
>> wrong, 3376 vs. 4252 (5.4.1) or 4185 using ("find" from the shell).
>
> What is your LC_CTYPE set to?

de_CH

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2005-02-14, 09:05
* Michael Herger shaped the electrons to say...

>>>I have some albums with umlauts and other special characters in their
>>>folder names. These make segfault slimserver v6 when accessing the
>>>folder or its content. The scan does finish, though, finding an
>>>appropriate number of artists and albums. The song count is badly
>>>wrong, 3376 vs. 4252 (5.4.1) or 4185 using ("find" from the shell).
>>
>>What is your LC_CTYPE set to?
>
>de_CH

Try making that de_CH.UTF-8 ?

-D
--
( ( ( [ ] ) ) )
In Stereo Where
Available

mherger
2005-02-14, 09:37
> * Michael Herger shaped the electrons to say...
>
>>>> I have some albums with umlauts and other special characters in
>>>> their folder names. These make segfault slimserver v6 when
>>>> accessing the folder or its content. The scan does finish, though,
>>>> finding an appropriate number of artists and albums. The song count
>>>> is badly wrong, 3376 vs. 4252 (5.4.1) or 4185 using ("find" from
>>>> the shell).
>>>
>>> What is your LC_CTYPE set to?
>>
>> de_CH
>
> Try making that de_CH.UTF-8 ?

This improves the song count considerably: 4162. But browsing one of those
folders still crashes SlimServer.

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2005-02-14, 09:39
* Michael Herger shaped the electrons to say...

>>>>What is your LC_CTYPE set to?
>>>
>>>de_CH
>>
>>Try making that de_CH.UTF-8 ?
>
>This improves the song count considerably: 4162. But browsing one of those
>folders still crashes SlimServer.

Did it spew a backtrace of some type?

If you could send the output running with --d_scan --d_files --dpaths, that'd be great.

-D
--
It's the wrong trousers Gromit, and they've gone wrong!

mherger
2005-02-14, 09:57
On Mon, 14 Feb 2005 08:39:25 -0800, Dan Sully <dan (AT) slimdevices (DOT) com> wrote:

> * Michael Herger shaped the electrons to say...
>
>>>>> What is your LC_CTYPE set to?
>>>>
>>>> de_CH
>>>
>>> Try making that de_CH.UTF-8 ?
>>
>> This improves the song count considerably: 4162. But browsing one of
>> those folders still crashes SlimServer.
>
> Did it spew a backtrace of some type?
>
> If you could send the output running with --d_scan --d_files --dpaths,
> that'd be great.
>
> -D

Sent you the file offline. BTW: since I change LC_CTYPE to de_CH.UTF-8 my
filenames aren't what they were before...

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2005-02-14, 09:59
* Michael Herger shaped the electrons to say...

>Sent you the file offline. BTW: since I change LC_CTYPE to de_CH.UTF-8 my
>filenames aren't what they were before...

How do you mean, exactly? When you do a 'ls', or something else?

What type of system is it? Do you have a default locale set?

-D
--
Minds are like parachutes... they work best when open.

mherger
2005-02-14, 10:28
On Mon, 14 Feb 2005 08:59:29 -0800, Dan Sully <dan (AT) slimdevices (DOT) com> wrote:

> * Michael Herger shaped the electrons to say...
>
>> Sent you the file offline. BTW: since I change LC_CTYPE to de_CH.UTF-8
>> my filenames aren't what they were before...
>
> How do you mean, exactly? When you do a 'ls', or something else?

ls, konqueror, file selector...

> What type of system is it? Do you have a default locale set?

Mandrake 10.1. How to know about the default locale?

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

Dan Sully
2005-02-14, 10:29
* Michael Herger shaped the electrons to say...

>ls, konqueror, file selector...
>
>>What type of system is it? Do you have a default locale set?
>
>Mandrake 10.1. How to know about the default locale?

I'd imagine something under the control panel type app.

It's odd that SS would be able to access the files when you are in UTF-8
mode, but other apps don't. Which indicates to me that you may not really be
in UTF-8 or ISO-8859-1

-D
--
<iNoah> I think someone should create a magazine for computer peripherals, called Card & Driver

mherger
2005-02-14, 11:56
> * Michael Herger shaped the electrons to say...
>
>> ls, konqueror, file selector...
>>
>>> What type of system is it? Do you have a default locale set?
>>
>> Mandrake 10.1. How to know about the default locale?
>
> I'd imagine something under the control panel type app.

Then I'd say it's de_CH. Or has been until I changed it. I tried to
activate utf8 using that tool but it set de_CH.

> It's odd that SS would be able to access the files when you are in UTF-8
> mode, but other apps don't. Which indicates to me that you may not
> really be in UTF-8 or ISO-8859-1

I think it's rather the contrary: in the default configuration (de_CH)
apps were able to access the file, but SS was not. After the change I can
still access them, but the names with special characters in them are
unreadable (question marks everywhere). And imho SS did not always have
that problem.

Maybe I should say that they are accessed from a share (SMB or NFS - tried
both).

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

K Suominen
2005-02-14, 14:17
The LC_CTYPE settings only tells the programs how to interpret
strings. It doesn't change the strings themselves, e.g. filenames.

If you have entered filenames with ISO-8859-1 encoding, setting
LC_CTYPE will result in those question marks when displaying them.
This is because the 8-bit characters result in invalid byte sequences
for UTF-8. (If you set LC_CTYPE to another ISO 8859 standard, e.g.
ISO-8859-15, the results are less drastic: you just see "wrong"
letters).

In summary, you should set LC_CTYPE to match the encoding you used
when creating the filenames.

I'm using ISO-8859-1 both in filenames and in the tags, and it has
worked ok. I haven't tested a lot of folder browsing, though. (Using
trunk revision 1916.)

Regards,
+ Kim


On Mon, 14 Feb 2005 19:56:34 +0100, Michael Herger <slim (AT) herger (DOT) net> wrote:
> > * Michael Herger shaped the electrons to say...
> >
> >> ls, konqueror, file selector...
> >>
> >>> What type of system is it? Do you have a default locale set?
> >>
> >> Mandrake 10.1. How to know about the default locale?
> >
> > I'd imagine something under the control panel type app.
>
> Then I'd say it's de_CH. Or has been until I changed it. I tried to
> activate utf8 using that tool but it set de_CH.
>
> > It's odd that SS would be able to access the files when you are in UTF-8
> > mode, but other apps don't. Which indicates to me that you may not
> > really be in UTF-8 or ISO-8859-1
>
> I think it's rather the contrary: in the default configuration (de_CH)
> apps were able to access the file, but SS was not. After the change I can
> still access them, but the names with special characters in them are
> unreadable (question marks everywhere). And imho SS did not always have
> that problem.
>
> Maybe I should say that they are accessed from a share (SMB or NFS - tried
> both).
>
> --
>
> Michael
>
> -----------------------------------------------------------
> Help translate SlimServer by using the
> SlimString Translation Helper (http://www.herger.net/slim/)
>
>

mherger
2005-02-14, 14:40
[..]
> In summary, you should set LC_CTYPE to match the encoding you used
> when creating the filenames.

Thanks, I'd really like to leave it as it was, because I'm not the only
one accessing those shares. I also have a laptop running Windows.

> I'm using ISO-8859-1 both in filenames and in the tags, and it has
> worked ok. I haven't tested a lot of folder browsing, though. (Using
> trunk revision 1916.)

That's interesting as I thought I did not encounter that problem with
earlier revisions. Would you mind browsing some folders with accented
characters, than updating to the latest revision and do the same thest
again? I would be interested if it's only me.

Regards,

Michael

>
> Regards,
> + Kim
>
>
> On Mon, 14 Feb 2005 19:56:34 +0100, Michael Herger <slim (AT) herger (DOT) net>
> wrote:
>> > * Michael Herger shaped the electrons to say...
>> >
>> >> ls, konqueror, file selector...
>> >>
>> >>> What type of system is it? Do you have a default locale set?
>> >>
>> >> Mandrake 10.1. How to know about the default locale?
>> >
>> > I'd imagine something under the control panel type app.
>>
>> Then I'd say it's de_CH. Or has been until I changed it. I tried to
>> activate utf8 using that tool but it set de_CH.
>>
>> > It's odd that SS would be able to access the files when you are in
>> UTF-8
>> > mode, but other apps don't. Which indicates to me that you may not
>> > really be in UTF-8 or ISO-8859-1
>>
>> I think it's rather the contrary: in the default configuration (de_CH)
>> apps were able to access the file, but SS was not. After the change I
>> can
>> still access them, but the names with special characters in them are
>> unreadable (question marks everywhere). And imho SS did not always have
>> that problem.
>>
>> Maybe I should say that they are accessed from a share (SMB or NFS -
>> tried
>> both).
>>
>> --
>>
>> Michael
>>
>> -----------------------------------------------------------
>> Help translate SlimServer by using the
>> SlimString Translation Helper (http://www.herger.net/slim/)
>>
>>

K Suominen
2005-02-14, 20:16
Directory browsing showed all the ISO-8859-1 entries just fine, e.g.

Aku ja köyhät pojat
För väntan

Interestingly I noticed that some entries are broken in the "Browse
Artists" and "Browse Albums" trees, while others are ok. Maybe it is
time to recreate the db...

I upgraded to revision 2069, and got errors about the db:

DBD::SQLite::db do failed: table tracks has 42 columns but 39 values
were supplied at
/net/grendel/u-9/kim/squeezebox/current/trunk/server/Slim/DataStores/DBI/DataModel.pm
line 67, <$fh> line 120.

I guess this db thing doesn't know how to upgrade itself...

I tried to browse the music folder. I went to a second level
directory, and it showed me the first entry. I pressed 222, and
instead of displaying Cajsa Stina Åkerström, the server died.

The log had this to say:

Can't call method "tracknum" on an undefined value at /net/grendel/u-9/kim/squee
zebox/current/trunk/server/Slim/Buttons/Browse.pm line 143.

Regards,
+ Kim


On Mon, 14 Feb 2005 22:40:21 +0100, Michael Herger <slim (AT) herger (DOT) net> wrote:
> [..]
> > In summary, you should set LC_CTYPE to match the encoding you used
> > when creating the filenames.
>
> Thanks, I'd really like to leave it as it was, because I'm not the only
> one accessing those shares. I also have a laptop running Windows.
>
> > I'm using ISO-8859-1 both in filenames and in the tags, and it has
> > worked ok. I haven't tested a lot of folder browsing, though. (Using
> > trunk revision 1916.)
>
> That's interesting as I thought I did not encounter that problem with
> earlier revisions. Would you mind browsing some folders with accented
> characters, than updating to the latest revision and do the same thest
> again? I would be interested if it's only me.