Unicode windows console output.

[Simon Marlow]

On 04/11/2010 02:35, David Sankel wrote:
> On Wed, Nov 3, 2010 at 9:00 AM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
>     On 03/11/2010 10:36, Bulat Ziganshin wrote:
>
>         Hello Max,
>
>         Wednesday, November 3, 2010, 1:26:50 PM, you wrote:
>
>             1. You need to use "chcp 65001" to set the console code page
>             to UTF8
>             2. It is very likely that your Windows console won't have
>             the fonts
>             required to actually make sense of the output. Pipe the
>             output to
>             foo.txt. If you open this file in notepad you will see the
>             correct
>             characters show up.
>
>
>         it will work even without chcp. afaik nor ghc nor windows
>         adjusts text
>         being output to current console codepage
>
>
>     GHC certainly does.  We use GetConsoleCP() when deciding what code
>     page to use by default - see libraries/base/GHC/IO/Encoding/CodePage.hs.
>
>
>
> This can actually be quite helpful. I've discovered that if you have a
> console set to code page 65001 (UTF-8) and use WriteConsoleA (the
> non-wide version) with UTF-8 encoded strings, the console displays the
> text properly!
>
> So the solution seems to be, when outputting to a utf8 console use
> WriteConsoleA.

We need someone to rewrite the IO library backend for Win32.  Currently 
it is going via the msvcrt POSIX emulation layer, i.e. using write() and 
pseudo-file-descriptors.  More than a few problems have been caused by 
this, and it's totally unnecessary except that we get to share some code 
between the POSIX and Windows backends.  We ought to be using the native 
Win32 APIs and HANDLE directly, then we could use WriteConsoleA.

This is a prerequisite for having a decent Win32 implementation of the 
IO manager too, and we could get proper support for hGetNonBlocking.

We're not talking about a lot of code here - basically a replacement for 
the modules GHC.IO.FD and GHC.IO.Handle.FD (about 1000 lines in total). 
  Some of the low-level Win32 support might have to be imported from the 
Win32 package though.  Any volunteers?

Cheers,
	Simon