[Haskell-cafe] Ready for testing: Unicode support for Handle I/O

[Simon Marlow]

Ross Paterson wrote:
> On Tue, Feb 03, 2009 at 04:42:44PM +0000, Simon Marlow wrote:
>> Unicode-aware Handles
>> ~~~~~~~~~~~~~~~~~~~~~
>>
>> This is a significant restructuring of the Handle implementation with
>> the primary goal of supporting Unicode character encodings.
>>
>> The only change to the existing behaviour is that by default, text IO
>> is done in the prevailing encoding of the system.  Handles created by
>> openBinaryFile use the Latin-1 encoding, as do Handles placed in
>> binary mode using hSetBinaryMode.
> 
> I would like to see the API make a type distinction between text
> handles with Char-based operations, and binary handles with Word8-based
> operations.  (And phase out openBinaryFile, hSetBinaryMode etc.)  It's a
> vital distinction that often trips people up.  Treating binary data as
> Chars with a Latin-1 encoding just feels like a kludge.

Yes I agree, but I think we should tackle that separately.  The important 
thing right now is that we can do I/O in the local encoding, and that 
existing programs that depend on reading/writing binary data using 
openBinaryFile will continue to work.

Let's get this in now, and then talk about a binary I/O layer later.  The 
restructuring I've done in the IO library should make it easier to separate 
the binary and text layers now too.

Cheers,
	Simon