[Haskell-cafe] Re: OCaml list sees abysmal Language Shootout results

[Peter Simons]

Ketil Malde writes:

 > Couldn't readFile et al. provide the standard interface,
 > but use hGetBuf tricks (e.g. from your 'wc' entry) behind
 > the curtains?

No amount of hGetBuf'ing will speed the program up if the
problem is the algorithm. I/O comes _sequentially_, and
every program that doesn't process the input sequentially
will require some form of buffering. Lazy I/O provides a
great general-purpose buffering mechanism, but it simply
can't be as fast of as efficient as hand-written code.

IMHO, a good I/O API forces you to write a stateful callback
function (that's how I implemented it in BlockIO), so _if_
you want to buffer something, you can use your state for
that. But the I/O driver doesn't buffer, what has been read
and processed is gone. Then the I/O part will _always_ be as
fast as possible, and the design encourages you to process
the data sequentially without looking forward/backward too
much.

It simply means writing a

      Char -> StateT st IO ()   -- stream processor
or  String -> StateT st IO ()   -- line oriented

function to handle a _part_ of the input every time, rather
than a

    String -> IO ()

function that handles the entire input at once.

Peter