PDA

View Full Version : Special characters (accents, etc)



jhurley
2005-12-10, 11:38
I have two problems through the Squeezebox UI (not with Slimserver as such) with special characters appearing in artist and album names as well as track titles.

1. When browsing artists and scrolling through the list, when I reach the first artist whose names contains a special character (the beloved Bjrk) my Slim.EXE (running as a hidden application not a Windows Service) just stops.

2. I discovered today that it does the same thing when coming to a track name in a playlist containing accented characters. I was happily playing a double various artists album through 2 SB1's sync'd and each time the track "La poupe qui fait non" got cued up the slim application on my PC just stopped again.

I posted seperately once before on the beta forum about this (as I'm running 6.5b1) and saw the log entries generated on the failure which were only produced by my iTunesUpdate plugin - a quick dialog with James Craig identifying that he could fix those messages but not the main failure.

Do I need to do something special to get my setup to manage special/accented characters?

ajmitchell
2006-01-01, 04:33
I agree completely with this post. I was running slimerver 6.1 on SB1 and SB2. i was having a crash everytime I tried anything from my favourite album (Le Fabuleux Destin d'Amlie Poulin by Yann Tiersen) (Amelie Soundtrack). I tried upgrading to 6.5b also but no luck. I even deleted the album and re-ripped it in better quality 320 - no luck again.

Finally I realised that slimserver (I have it running as a service slim.exe) will crash with any accented character. Slim.exe simply disappears from the screen and the browser has nothing to find. I tried logging the error with -d but the crash happens so fast I couldn't catch the report.

Is this problem know? Is it registered as a bug? Is there a work around or fix?

thanks

alex
example of a song causing a crash:
"J'y suis jamais all from Le Fabuleux Destin d'Amlie Poulin by Yann Tiersen "

Michaelwagner
2006-01-01, 20:44
Please clarify the situation for me.

Are the special characters in the file name or in the tagging for the file? Or both?

You gave this example:
"J'y suis jamais all from Le Fabuleux Destin d'Amlie Poulin by Yann Tiersen "

What is the character after the "n" in Tiersen? Is it a space?

If these are file names, can you create such a file name in windows explorer?

ezkcdude
2006-01-01, 22:09
Mine works fine with Sigur Ros, and if that works, I would have to imagine you must have the tags wrong.

ajmitchell
2006-01-02, 03:04
OK, after some experimentation it seems the liability to crash is when the special character is in the song title. For example I have quite a few artists such as Prozzk or Ryksopp or indeed Bjrk which appear to play fine (see above post). I use itunes, and itunes uses the artists as the filing folder but not in the file name which is based on the song.

However, I am having crashes with special characters in title. These are in the id3 tag and I believe in the file as well. For example Shakira [artist] Que Me Quedes T [title] laudrey service [album] appears to cause a crash, I presume because of the "" which can be corrected by renaming the => u. Similarly the above Amelie album causes a crash when the character is is in the song title (Amlie).

Thats about as far as I can get right now. I know there appears to be a work around by removing all special characters - but this isn't very user friendly!

Michaelwagner
2006-01-02, 07:13
Do I need to do something special to get my setup to manage special/accented characters?
No, that shouldn't be necessary.

I was going to search for accents in my collection, but the search box doesn't like accents, for some reason, so I couldn't find any to try out.

But I'm sure I have songs with accents, and they play.

Can both of you describe your setup?
OS
Slim version, including subversion (i.e. 6.2.2. 5448)
are you both using iTunes?
is this the first version of slim you have run or is this an upgrade from a previous version?

Michaelwagner
2006-01-02, 07:32
I just played Was Wri Wnn on my SB3 and it worked.

So let's figure out what's different between our systems.

martinfowler
2006-01-02, 07:59
I had this kind of behavior happen to me, see http://bugs.slimdevices.com/show_bug.cgi?id=2404, which is now fixed, although you may need to pick up a nightly or something to get a version with the fix.

I have a lot of music with odd accents (since I listen to a lot of world music). I don't have problems with non-ascii characters in the id3 tags. I have had problems with non-ascii characters in file names, I've worked around those by removing them.

Michaelwagner
2006-01-02, 08:14
I am having crashes with special characters in title. These are in the id3 tag and I believe in the file as well. For example Shakira [artist] Que Me Quedes T [title] laudrey service [album] appears to cause a crash, I presume because of the "" which can be corrected by renaming the => u. Similarly the above Amelie album causes a crash when the character is is in the song title (Amlie).

So just to be sure I've got this straight.

If there is an accent in the file name, for instance,
Que Me Quedes T.mp3
that's when it crashes?

If so, can each of you describe your system in more detail:
what operating system
where are the files stored (i.e. in a local disk in the same computer, or off on a network device somewhere)

This is starting to sound like a problem with file systems not liking odd characters.

(although, my Was Wri Wnn is called Was Wri Wnn.mp3 on the disk, accents and all, and that worked ....
but if it's a file system problem, there may be individual allowed and disallowed characters in file system names)

radish
2006-01-02, 09:10
I have the exact same Shakira album, with the same accents (in both the tags and the filenames) and it works just fine here (Win XP, 6.2.1). One difference is that my file is OGG not MP3, so the tags aren't ID3.

Michaelwagner
2006-01-02, 10:14
I have the exact same Shakira album, with the same accents (in both the tags and the filenames) and it works just fine here (Win XP, 6.2.1). One difference is that my file is OGG not MP3, so the tags aren't ID3.
Another difference is 6.2.1 vs 6.2.2

mherger
2006-01-02, 12:12
> (although, my Was Wri Wnn is called Was Wri Wnn.mp3 on the disk,

Wow! Who's that song's artist? This must be swiss german or something :-)

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

mherger
2006-01-02, 12:24
>> (although, my Was Wri Wnn is called Was Wri Wnn.mp3 on the disk,
>
> Wow! Who's that song's artist? This must be swiss german or something :-)

Do you really have Subzonic in your collection?!? Amazing...

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

Michaelwagner
2006-01-02, 12:46
Yup!

It's from a Sony collection called Sounds of the Century or something like that. I'm not at home now, so I can't check.

I listened to it this morning and it's certainly no language that I speak. Michael, now that you say that it's Swiss German, that clarifies everything.

(to paraphrase Pygmalion http://www.imdb.com/title/tt0030637/)

"Why can't the Swiss teach their children how to speak?"

:-) :-)

mherger
2006-01-02, 13:18
> It's from a Sony collection called Sounds of the Century or something
> like that. I'm not at home now, so I can't check.

"Soundtrack for a century"

Must be quite satisfying for a swiss band singing in swiss german to be
included in a sampler of that title :-)

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

ajmitchell
2006-01-03, 03:10
Michael,

I cannot get much further with the experimentation myself as I have been systematically renaming all special characters I come across. As a result I have none left!

Yes I am running itunes, which incidentally has no problem searching for special characters providing you can remember which ones to put in the search box. After renaming, I have to ask slimserver to re-scan.

You asked about my system:
Software. Slimserver 6.51b running for a couple of months (cannot give sub-version until I return home). Ran at least 2 previous versions which seemed to have the same issue. Itunes is v6.
WindowsXP Sp2 + updates
Files all in local drive c:\my music (typical itunes place)
Files. All Mp3s various bit rates from 128 to 320.

-----suggestion-------

A quick experiement would be to load a couple of mp3s with different types of special characters via itunes and see if slimserver has a problem. Then do the same but allow slimserver to find the file directly on the hard disk. Then do the same re-naming the file but keeping the ID3 the same (itunes will not allow me to do this because I allow it to keep control over re-naming).

R

alex

Michaelwagner
2006-01-03, 06:55
A good suggestion, but I can't run the experiment because I don't know anything about iTunes and don't use it. Anyone else want to try this?

Can I ask why you're running 6.5b1? At the moment, 6.5 is experimental, mostly for the developers themselves to try out new things, and as such has a higher statistical likelyhood of bugs, issues that aren't worked out yet, etc.

Sometimes developers will suggest that someone run a pre-release version to fix a specific problem.

The text beside the link says:


This is the latest pre-release development version of SlimServer. Download this version if you are interested in new features, the latest bug fixes, and don't mind reporting new bugs.

Does that description fit you?

When you upgraded to 6.5b1, what did you come from?

Did you do a "clean" install (i.e. erase all slim directories or install in a new directory) or install in place?

ajmitchell
2006-01-03, 15:35
Michael,

You seem to be focussing on 6.51b (my build is 5064 incidentally) as the potential problem, whereas I said it had occured previously. I had run 6.0 and 6.1 (and yes they were fresh installs).

You want to know why I upgraded to 6.51? I'll tell you. It was in the optimistic hope that someone had fixed what I consider to be a much more serious bug in all versions I have tried to date. That is the total inability to crossfade whilst more than one player is synced. Yes I run 4 SBs in different rooms. -and this considerably reduces my listening pleasure. I reported this as a possible fault thorugh tech suport email 6 months ago within 1 week of buying the boxes (yes I am also fairly new to this). Kindly someone entered it as a bug (see http://bugs.slimdevices.com/show_bug.cgi?id=1845). I hope this shows that I am willing to enter comments and bugs when I spot a problem. Hey thats what I am trying to do right now!

Now perhaps you have evidence that 6.51 is the problem but from Martin's comment above its seems I am being advised to 1. get the latest nightly and yet 2. Stay away from those nightlys. Tricky.

This should be a simple problem to test, I would offer myself but I am having to re-build my PC at the moment after a serious crash and I have also renamed most of those pesky special characters.

Michaelwagner
2006-01-03, 17:20
You seem to be focussing on 6.51b (my build is 5064 incidentally) as the potential problem

I'm sorry if somehow I gave that impression. That wasn't my intention.

I expressed no opinion about whether 6.5b1 caused the problem. In fact, I think it's unlikely. But I said nothing in that direction.

I was, however, asking why you went there.

And now you've answered my question.


You want to know why I upgraded to 6.51? I'll tell you. It was in the optimistic hope that someone had fixed what I consider to be a much more serious bug in all versions I have tried to date. That is the total inability to crossfade whilst more than one player is synced. http://bugs.slimdevices.com/show_bug.cgi?id=1845.

I've looked over that bug report. It says it is slated to be fixed in 6.5 but I see nothing to indicate a fix has been put in place yet.


Now perhaps you have evidence that 6.51 is the problem
Nope.


from Martin's comment above its seems I am being advised to 1. get the latest nightly and yet 2. Stay away from those nightlys. Tricky.
Let me try to unconfuse the situation.

There are 2 nightlies generated every night. At the moment, they are 6.2.2, which is meant to include small fixes to the "main" released version (6.2.1) and 6.5b1, which is primarily for developers (and people who have been directed by developers to try something special).


This should be a simple problem to test
Except I can't reproduce it. Maybe someone else can.

ajmitchell
2006-01-04, 02:20
Thanks for that additional clarification and your input on the crossfade when synched bug. Let me be clear. I really enjoy the slimserver/squeezebox product. I run 4 boxes and am on the edge of adding a 5th (SB3). The community here generally seems willing to help - so lets see if anyone else can provide some input here. If not, I am rebuilding my main PC as we speak and can probably try some experimentation next week.

R

alex

mikerob
2006-01-04, 06:03
I've had an accented character in a track title crash Slimserver a while back when using 6.1 nightlies (http://bugs.slimdevices.com/show_bug.cgi?id=1709) and the points I noticed were:

- the crash happened when playing the track on a SB1 but not a SB2
- I had ripped the track using iTunes. The offending character was an e acute however if I manually replaced this with an e, or manually replaced it with an e acute using the Mac keyboard, then the crash didn't happen.
- I had other tracks with an e acute and other accented characters that didn't crash

I know very little about this subject, but can only surmise that maybe iTunes downloaded tag data where an e acute was encoded in a particular form that the Mac could display OK, but a SB1 couldn't handle. That would imply that the encoding of an e acute for that particular track in CDDB or where ever iTunes gets its info from was different from the encoding if you enter that character directly using the Mac keyboard.

Rather than replace the accented character with an unaccented one, you could try to replace accented character with the same character using whatever the keyboard command is for the accented character.

Michaelwagner
2006-01-04, 07:08
Mikerob:

If I remember your problem correctly (and I'm thinking of the right one), it was a crash during playing (actually probably displaying) the song with the odd character in it.

And it was some problem with looking up the right character to output to the screen.

It might be related ... I guess I should try again with an SB1 ... I was playing on the SB3.

ajmitchell, jhurley:

There are several SB1 models - with graphic displays or with the old displays. Which do you have?

Michaelwagner
2006-01-04, 07:27
I have 3 SB1s, but not all are connected at the moment. The one that is connected played Was Wari Wann OK (with accents, but I don't have time to put them into this email).

I need to go to work now but in the evening I will retry this test with all 3.

And I will try to sync them.

That may not be enough of a test, because it may be only certain accents, etc. But I will try to reproduce it if I can.

Otherwise, perhaps one of you can send me one of the files that won't play.

michael at thewagners dot ca

thanks.

mikerob
2006-01-04, 08:54
Mikerob:

If I remember your problem correctly (and I'm thinking of the right one), it was a crash during playing (actually probably displaying) the song with the odd character in it.

And it was some problem with looking up the right character to output to the screen.



That's correct - the song started to play. The SB display size and track title and artist length were such that the accented character wasn't displayed on the SB initially. Then when the SB display scrolled to show the complete track title, when it reached the accented character, it repeated itself so the display was filled with the same character. The track stopped playing after a time - I assume when the SB buffer emptied after Slimserver stopped.

Michaelwagner
2006-01-04, 09:33
Hi Mike:

I just checked out your bug report. It was reported fixed quite some time ago. In theory, both of the other guys reporting problems now should have this fix.

More sleuthing tonight when I'm at home. ...

Michaelwagner
2006-01-04, 17:51
Well, I spent an hour today hacking away at this problem. I can't reproduce the failure.

I played Michael's favorite swiss guys on an SB3, on 2 different SB1s, on several SB1s synched together, on an SB3 synched with an SB1.

I made up a song with a title that was some 30 unicode characters, letters with tildes, accents egu, grave, umlauts, a circonflex or 2 for good measure. A C cedile.

The song plays fine on an SB1, an SB3, the two synched together.

I can't get it to fail.

You're going to have to send me one of these files if I have any chance of reproducing it.

jhurley
2006-01-05, 04:29
Mikerob:

ajmitchell, jhurley:

There are several SB1 models - with graphic displays or with the old displays. Which do you have?

I have 2 original unmodified SB1's.

ajmitchell
2006-01-05, 04:54
I have a SB1 wired, SB1 wireless and 2x SB2s. The Sbs 1 have a standard graphical display approx half the resolution of the 2's.

From your preliminary tests above Michael, it may be a squeezebox/itunes integration problem

R

alex

Michaelwagner
2006-01-05, 06:30
From your preliminary tests above Michael, it may be a squeezebox/itunes integration problem
Anyone with iTunes care to attempt to reproduce this problem?

Triode
2006-01-05, 07:35
Coming to this thread late. Re crashing on SB1 with character displays [original SB1]. There was something added to svn r3620/r3628 to avoid non latin1 characters crashing the server. However this only does it for the bottom line ["line1" of the display] and not for the overlay [bit on the right of the line].

Are you still seeing this - if so is the special character somewhere other than on the bottom left of the display?

Can't comment on the general itunes integration issue.

Michaelwagner
2006-01-05, 08:43
Hi Triode:

I don't know where the bad character is/was, because I couldn't reproduce. However ...
Can you explain what you mean by "the overlay" and maybe it will become clear if my test was incomplete.

Triode
2006-01-05, 10:31
The display is built up from 6 components:

Line1 - left justified text on top line
Line2 - same for bottom line [this bit can scroll]

overlay1 - right justified text on top line [normall used for progress bar/count]
overlay2 - right justified text on bottom line [normally used for arrow/notes symbols]

center1/center2 - centered text on line 1 or 2

Now only line1 has the code to protect against utf8 strings being sent to the display code. Are you problem strings appearing elsewhere on the display? [what screen does the problem occur in?]

Michaelwagner
2006-01-06, 13:38
I found this bug report
http://bugs.slimdevices.com/show_bug.cgi?id=2677
It's both similar and different.

ajmitchell
2006-01-07, 14:29
Got my PC up again today, and loaded some more albums - decided to test the special characters problems more systematically.

Heres the results.

[a] Slimserver Synch with itunes 6

Andril [song title] by Howard Shore [artist] from The Lord of the Rings - The Return of the King [album] => crashes slim.exe after 1-5seconds.

[b] Slimserver Synch with itunes 6

Anduril [RENAMED song title] by Howard Shore [artist] from The Lord of the Rings - The Return of the King [album] => PLAYS PERFECTLY

[c] Slimserver direct read off hard disk

Andril [renamed song title] by Howard Shore [artist] from The Lord of the Rings - The Return of the King [album] => crashes slim.exe almost immediately


[b] Slimserver Synch direct read off hard disk

Anduril [renamed song title] by Howard Shore [artist] from The Lord of the Rings - The Return of the King [album] => PLAYS PERFECTLY

I then repeated the experiment with the most complicated song name I could find: "Vrar Vel Til Loftrsa [title] Sigur Rs [artist] gtis Byrjun[album].

Results were the same.

I conclude....this is not an itunes integration issue but a more serious problem in slimserver itself. Clearly not everyone has this problem so there are probably some other variables.

Next step?

I am happy to forward the files.

Alex

Michaelwagner
2006-01-07, 15:41
michael at thewagners dot ca
oh, and what version of slimserver are you running now on this rebuilt machine?

Michaelwagner
2006-01-08, 12:20
Alex: I got your file and it plays on my machine. Doesn't crash the server.

Now it plays a little oddly, because the title shows up as Andril by Howard Shore.

I'm not sure I got the right number of trailing ''s.

That needs to be fixed. Whether fixing this will fix the crash on your machine ... well, it's hard to predict.

I can tell you, just from the left hand pane in the slim ui that you have both id3v1 and id3v2 tags in this file. That may be a good thing or it may not, depending on whether you're happy the erroneous padding stopped at 30 characters or not.

There are some other odd things about these tags. This claims to be track 255, which seems unlikely but I don't think anything in slim cares, except for sorting purposes.

According to Tag & Rename, the title really is stored as Andril and also Howard Shore's name was similarly butchered. This is in both the id3v1 and id3v2 tags. Also the comment field is a string of what look like hex digits.

Tag & Rename, for the v2 tags, shows a longer string of hex digits in the comment field. Slim doesn't show the comment field, although I'm not sure why. And in the V2 tags, it puts 255/-1 for the track #. I've never seen that before. I'll have to look at the actual tag encodings to see what's going on there.

Someone has put a BPM tag into this file. 135.18. Did you do that?

Does this file you sent me still crash your system? If so, I'll need more details of when it crashed, what player(s) you have tried to play it on, when the server crashes, how you have your display set up, what operating system, etc.

Triode: you wrote:

The display is built up from 6 components:

Line1 - left justified text on top line
Line2 - same for bottom line [this bit can scroll]

overlay1 - right justified text on top line [normall used for progress bar/count]
overlay2 - right justified text on bottom line [normally used for arrow/notes symbols]

center1/center2 - centered text on line 1 or 2

Now only line1 has the code to protect against utf8 strings being sent to the display code.

Did you really mean only line1 has code to protect it? Or did you mean line 2 (according to your naming system). AFAIK, nothing from the song goes into line one. It only ever says Playing or Stopped and (1 of 1) and those sorts of things.

I was about to go off and do something else now, but over the next few days I'll examine the tags more and see how slim handles them and see what I can figure out.

Triode
2006-01-08, 15:26
Did you really mean only line1 has code to protect it? Or did you mean line 2 (according to your naming system). AFAIK, nothing from the song goes into line one. It only ever says Playing or Stopped and (1 of 1) and those sorts of things.

Yes sorry its line2 which has the protection - which is the one that normally gets the song title.

Note I've not been following the rest of the discussion, but if you find a crash in any of the display code I can look at it..

Michaelwagner
2006-01-08, 17:30
if you find a crash in any of the display code I can look at it..
Thanks for the offer ... If I nail it down to that area, I'll call.

ajmitchell
2006-01-09, 02:34
Michael,

Thanks for testing that file.

The Id3 tag is odd because I edited it in order to test the liability to crash. However, I am not sure why track no 255 is there because I wanted no track information at all in that field? As to the BPM I have entered this last week according to Mixmeister a DJ programme. You can rule this out of the investigation because I only got this programme since the new PC settings and therefore it did not take part in this issue.

As this file does not crash on your system but does on mine, I think the next step is a more general poll to see who is having problems with special characters. For now, I am re-naming most files, and making some offline to come back to later.

Alex

Michaelwagner
2006-01-09, 07:49
Alex: What version of slimserver are you running when it crashes?

Michael

P.S. I use MixMeister's BPM function too. Didn't know too many others who do it. So I know that's not the problem, because 99% of my files are tagged that way. :-)