PDA

View Full Version : Problem with international characters with filenames



Yiannis
2007-08-04, 06:59
Hi all. I purchased an SB3 last week and am very happy with the way it was setup and performed until now. Various small problems were rather easily resolved by searching the Wiki or the Forum.

Today I realized there is an issue with international ASCII characters contained in pathnames (with accents, umlauts etc.). The SlimServer seems to just ignore those files. I found several threads to this but none seems to resolve the problem, so I thought of addressing the forum.

FYI, my collection (or most of it, at least) is organized as "\Artist Name - Album Name\TrackNum - TrackTitle" and mostly contains FLAC, some MP3 files and some OGG files. I notice the following:

- If the accented character is anywhere in the folder pathname, the folder is ignored completely.

For example, the folder named

Egberto Gismonti - Dança dos Escravos (ECM) (1989)[FLAC]

is completely ignored during a music library scanning probably because of the "ç" character in the pathname

- If the folder name doesn't have any 'international' characters but a track filename has so, then the specific track does not appear. The rest of the tracks in the folder without special characters are scanned normally.

All these folders and files show regularly in Windows explorer, btw.

Interestingly, from what I see this problem does not pertain to special characters that are included in tag info. This is displayed ok, and files are scanned and play. For instance I have the folder: "Luis Bonfa - Jacaranda". All the filenames don't have special characters, but the tag info (for Titles, Artists etc) contains portuguese accented letters. All these files are scanned & play correctly, and both on SS and on SB screen I see the special characters in tags: for instance, the album title is tagged as "Jacarandá" and I see it.

I am using Windows XP SP2 (english version), my primary regional settings are Greek. The version info for SS reads

SlimServer Version: 6.5.3 - 12376 - Windows XP - EN - cp1253

I have disabled the iTunes support and all my files are stored in multiple external USB drives with links in a music library folder.

I'd appreciate some insight on this issue. I have quite a few files with French, Spanish, Portuguese, Swedish etc. characters in their filenames and it would not be an easy task to go through each of 25,000 files to rename. Plus, I suspect this will be an issue with Greek filenames and I have a lot of Greek-named files that I'd like to import at some stage (I dunno yet whether this is possible, 'cause I haven't tried).

Thanks in advance for some tips on the issue,

Yiannis
Athens, Greece

andyg
2007-08-04, 07:10
Please try the latest 6.5.4 nightly build, your issue sounds like bug 2475 (http://bugs.slimdevices.com/show_bug.cgi?id=2475) which should be fixed.

http://www.slimdevices.com/downloads/nightly/SlimServer_6_5_x_v2007-08-04/

auronthas
2007-08-04, 15:49
Yup, bug 2475 is fixed.

I have mine SS upgrade to 6.5.4 nightly build from 6.5.3, all my folders and filenames in chinese character able to be scanned and recognized.

Well done!

Yiannis
2007-08-05, 04:40
Hi again. AndyG, much thanks for the swift action & response.

I dl'ed & installed the proposed 6.5.4 SS build. I am afraid that this resolved the problem ONLY PARTLY, at least in my case. In particular:

a. As far as international characters included in FILENAMES (and NOT in Folder names), the problem seems to have been resolved, at least from a practical POV. I noticed that files with international characters that were previously omitted from scanning, are now included and play normally:

Example: This is a listing in SlimServer for the album folder Baden Powell - A Vontade (note that this is not tagged, therefore SS shows only the file names for a listing of this folder):

All Songs
01- Garota de Ipanema (Jobim-Vinicius).mp3
02- Berimbau (Baden-Vinicius).mp3
03- O astronauta (Baden-Vinicius).mp3
04-CON~1.MP3
05- Sorongaio (P Santos).mp3
06-SAM~1.MP3
07- Saudade da Bahia (Caymmi).mp3
08-CAN~1.MP3
09- Conversa de poeta (Queiroz-Santos-Vinicius).mp3
10- Samba triste (Baden-Blanco).mp3

All songs, including those with tildes are read and played. The three tracks 04, 06, 08 with special characters, which were omitted from the SS database with my previous version, now appear and play. Note that the actual filename in Windows for
04-CON~1.MP3
is
04- Consolação (Baden-Vinicius)

I went through the Command prompt and ran a dir/p command in this folder and saw that the 'DOS name' in for the same file is

04- Consola??o (Baden-Vinicius) [The 'special characters' appear as question marks].

So, probably SS does some sort of 'internal file renaming' or similar; but, from an end-user POV this doesn't seem to matter as long as the songs play correctly.

B. The problem seems to persist (but somehow differently than in 6.5.3) in the case that the special character is included in the folder name. Taking the very same example that I used in my previous post, the folder

Egberto Gismonti - Dança dos Escravos (ECM) (1989)[FLAC]

As said, this folder was completely omitted from the music folder list with 6.5.3. Now in 6.5.4. it did appear in a Music Folder listing as

Egberto Gismonti - Danca dos Escravos (ECM) (1989)[FLAC].

(the "ç" automatically converted to a "c").

Clicking on this folder, doesn't show any files in it. What I get is

Title: Egberto Gismonti - Danca dos Escravos (ECM) (1989) [FLAC]
File Format: Unknown
Location: H:\Music\Egberto Gismonti - Danca dos Escravos (ECM) (1989) [FLAC] (Download)

Obviously, I understand that the SS doesn't see the right path. This folder in MS-DOS is named

H:\Music\Egberto Gismonti - Dan?a dos Escravos (ECM) (1989) [FLAC]

(again, the special character is replaced with "?").

I checked that the same was the case with any folders including a special character in their name. When renaming the folder to remove special characters and rescanned, it worked.

I don't know if it has to do with settings on my computer, but from what I mentioned above it seems that the bug is only partially fixed. If anyone has any suggestion, pls let me know.

I can now deal with the issue, as I could rather easily rename 'problematic' folders; however, from a programmatical point of view, I guess you will consider too that the bug isn't fully fixed.

Best,

Yiannis

giulianoz
2007-08-06, 00:39
I still have problems while reading uft-8 filenames.
the
structure is like
c:\music\folder\folder\folder\utf-8-filename.mp3

ie:
file 05.美しい朝の輝き.mp3 is not found by the scanner, file 07.Sail Over.mp3 is found and works.

I selected to clean the database and do a full scan on a slimserver 6.5.4 (updated from 6.5.2) on win xp sp2

giuliano

auronthas
2007-08-06, 07:25
I still have problems while reading uft-8 filenames.
the
structure is like
c:\music\folder\folder\folder\utf-8-filename.mp3

ie:
file 05.美しい朝の輝き.mp3 is not found by the scanner, file 07.Sail Over.mp3 is found and works.

I selected to clean the database and do a full scan on a slimserver 6.5.4 (updated from 6.5.2) on win xp sp2

giuliano

I renamed my file name 01 Track 1.wma instead of file in Japanese character with maintaining the music title and properties in Japanese character such as 輝いた季節へ旅立とう, did a clear and full rescan, SS managed to recognise and detect the files and gave me the song title, artists, album in Japanese character.

My SS and OS version as follow: -

SlimServer Version: 6.5.4 - 12485 - Windows Vista - EN - cp1252
Server IP address: 192.168.1.3
Perl Version: 5.8.8 MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt

Give it a try.

Yiannis
2007-08-06, 08:01
To Auronthas: the purpose of my post (and I guess the relevant bug report) is to find a solution WITHOUT HAVING TO RENAME files or folders. Depending on the size & nature of one's collection, this can be a tedious, or even impossible, task. Furthermore, I for one do not want by any means to modify filenames to "01 Track01.mp3" or similar (I guess a lot of other users do not fancy this idea, too).

As I mentioned in my previous post, at least in my case, version 6.5.4 resolved the problem partly - in those cases where the folders did not include any 'foreign characters'. I see that the bug is still investigated by the team.

Giuliano:

a. when you do a dir through MS-DOS prompt at the folder of your example, what's the name that you get for the problematic file 05.美しい朝の輝き.mp3?

b. I have the, maybe silly, suggestion that your problem may lie in the "." separator that you use. Maybe this is treated as an unrecognized file type when a file name conversion takes place. Have you tried to replace the "." in the file name with e.g. an underscore (such as 05_美しい朝の輝き.mp3)? Could you do a sample test to verify?

Although I am partly content with the solution found, I do hope that a more solid solution covering also folder names will be found.

Yiannis

Spies
2007-08-06, 10:05
I have disabled the iTunes support and all my files are stored in multiple external USB drives with links in a music library folder.

Yiannis, can you describe how you did this? It may help in diagnosing the behavior you describe.

Yiannis
2007-08-06, 14:10
Spies:

Thanks for the question. It actually made me do a test which may be of help in further diagnosing the situation. Here are the details:

My collection is organized in various folders which reside in various external (USB) HDD. The folders, in general, follow the structure:
\Artist - Album\Tracknum - Tracktitle or

\Artist - Album\Disknum\Tracknum - Tracktitle for multiple diek albums

The SS Music Folder is set to a folder on my SS running PC (Music Library). This Music Library folder contains a shortcut for each actual album folder,

thus the Music Folder is structured in as a list of .lnk folders

\Music Library\Artist - Album.lnk

This works fine, with the exception I mentioned in the preceding.

Now, I did a sample test and copied an album folder with a problematic character DIRECTLY into the Music Folder (no shortcut). In particular, I copied the very same Egberto Gismonti folder of my previous examplewhich contains a foreign character.

Even without a rescan, the SS database was updated with the new folder, which was renamed to EGBERT~1 and was scanned and read correctly.

Therefore, the question now is why doesn't the same thing occur with the shortcut folder. Any insights anyone?

Yiannis

Spies
2007-08-06, 14:25
Therefore, the question now is why doesn't the same thing occur with the shortcut folder. Any insights anyone?

Yiannis, I wonder if it has something to do with the format of the USB drives and how they are mounted. Do you know what format the USB drives are in and what was used to do the formating? I just tried using shortcuts from USB drives formated as NTFS and FAT32 without any problems here.

Yiannis
2007-08-06, 16:29
Spies, it shouldn't be the USB HDDs; all of them are NTFS formatted with Windows QuickFormat; my local HDD is also NTFS, I doubt if anyone is using FAT32, at least for data storage. To eliminate this possibility, I copied the specific folder on my PC HDD and created a new shortcut in my Music Folder. This time, it did not work; the behaviour was exactly the same as in the case of storing it on the USB drive - that is in SS the international accented characters in the folder name were replaced with the english non-accented characters (see my second post here for exact descriptions).

I guess it's somehow connected to the way shortcuts are read/interpreted. I saw in the bug report that it worked with a folder with Cyrillic characters but does it also work if you create a shortcut in Music Folder pointing to that folder?

Thanks in any case for the valuable assistance and feedback
Yiannis

Spies
2007-08-14, 15:46
Yiannis, you may want to add your name to the bug list if you have not already done so at http://bugs.slimdevices.com/show_bug.cgi?id=2475

jim_merry
2009-11-03, 07:28
SS ignores filenames during scanning if they have international characters. Like the following song:

"C:\Documents and Settings\jmerry\My Documents\My Music\iTunes\iTunes Music\Jordi Savall\Tous Les Matins Du Monde\01 Marche Pour La Cérémonie Des Turc.m4a"

It looks like the same issue that was discusses and resoloved in this post thread of 2 years back. Am I missing a setting somewhere in SS to enable scanning of foreign characters?

-Jim