Text I/O library proposal, first draft

Ben Rudiak-Gould benrg@dark.darkweb.com
Tue, 5 Aug 2003 20:24:51 -0700 (PDT)


Some general points about recoders:

1. We don't really need to work out a single recoder interface that's best
for every purpose. All the types people have proposed can happily coexist,
and in many cases it's easy to write functions which convert from one to
another.

2. There are regional encodings in use which are not stateless (JIS is the
only one I know of personally), so I think that a currentLocaleEncoding
function will have to return a stateful interface.

3. I expect that most conversion functions will be implemented in C code,
for three reasons: (a) performance; (b) the C code already exists, is
well-tested, knows about correct locale handling on different systems,
etc., and (c) every Haskell executable will become hundreds of kilobytes
larger if it has to include translation tables for every locale in which
it might be run. We have little choice but to use the C library's tables,
and the only interface to them is the C conversion functions.

4. Existing C conversion libraries (whose interfaces we can't change)
store their state in opaque data structures and provide no way to copy
that state. So all use has to be single-threaded, and if we want to
enforce that statically we can't use an explicitly threaded interface. ST
and IO are the only options.


-- Ben