[GHC] #3307: System.IO and System.Directory functions not Unicode-aware under Unix

[GHC]

#3307: System.IO and System.Directory functions not Unicode-aware under Unix
-------------------------------+--------------------------------------------
 Reporter:  YitzGale           |          Owner:                  
     Type:  bug                |         Status:  new             
 Priority:  normal             |      Milestone:                  
Component:  libraries/base     |        Version:  6.11            
 Severity:  normal             |     Resolution:                  
 Keywords:  directory unicode  |       Testcase:                  
       Os:  Unknown/Multiple   |   Architecture:  Unknown/Multiple
-------------------------------+--------------------------------------------
Comment (by duncan):

 Replying to [comment:3 YitzGale]:

 A good reference on what glib does and recommends is here:
 http://library.gnome.org/devel/glib/stable/glib-Character-Set-
 Conversion.html
 See the description section, after the synopsis.

 > > The problem with making `getArgs` and `openFile` return Unicode is it
 > > may be impossible to open certain files passed on the command line
 > > (those for which the decoding is lossy).
 >
 > On the other hand, they are decoded on other platforms.

 They use the native Unicode representation on other platforms. I don't see
 that that is an argument to use a non-native representation on Unix
 platforms.

 > We don't want to make it impossible to write platform-independent code
 for any
 > program that reads its args.

 Unfortunately as it stands it is impossible for platform-independent code
 to have both of these properties simultaneously:

  * Read all files passed on the command line

  * Display file names to humans accurately in a user interface.

 Currently we get the first property and you're proposing to drop that and
 switch to the second.

 It's pretty well ingrained that `FilePath` is the type for specifying
 files eg to open them (it's specified by H98). It's
 a much more recent problem that we want to display Unicode file names in
 user interfaces. For portable code, how about we add a function:
 {{{
 filePathToString :: FilePath -> String
 }}}
 On Unix this would decode. On Windows and OSX it'd be the identity since
 on those platforms the string would already have been decoded.

 It means we treat `FilePath` as if it were an ADT (with differing
 representation on different platforms) but without actually switching to
 an opaque type.

 > Would that actually happen for users using any normal UI and any normal
 > input method?

 Generating new names is not a huge problem. The user selects a name in
 Unicode and if the conversion to a FilePath is impossible or lossy then
 the user can be prompted to select a different name. Note that does need
 another function:

 {{{
 filePathFromString :: String -> Maybe FilePath
 }}}

 > It has always been possible in Unix to create weird file
 > names that are very difficult to deal with, but it won't happen in
 normal
 > usage. We can provide a Unix-specific hack for the odd case.

 The most frustrating thing for a user would be selecting a file, having
 the app read it, but be unable to save back to the exact same file because
 of lossy decoding. That's why such apps are supposed to save the real file
 name, and translate that into a string, but they must keep the original
 name because the decoding can be lossy.

 Unfortunately that's not a case we can just provide Unix-specific hacks
 for, it can happen for almost any portable app. Eg consider apps that
 translate `.foo` file into `.bar` files (like, say a compiler,
 preprocessor). If we decode `filename.foo` into Unicode but it's a lossy
 conversion then saving `filename.bar` may work, but the file names will no
 longer correspond which could break things (think chars replaced by '?').

 So my suggestion basically is, keep `FilePath` as a file path, and convert
 to/from String for human consumption.

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/3307#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler