[Haskell-cafe] Re: binary IO

[Peter Simons]

Bulat Ziganshin writes:

 > your BlockIO library is great, but it's usage is limited
 > to very specific sutuations - when we can save pass state
 > between processing of individual bytes

In my experience, any other approach to I/O is slow. If you
don't have an explicit state between processing of
individual bytes, you have an implicit state. Making that
state (the I/O buffer) explicit gives you control over how
it is used and how it is evaluated. With an implicit state
(lazy evaluation), you have no control.

Fast I/O is a matter of application design. BlockIO is fast
because its API forces you to design your applications as
stateful, interruptible computations -- a finite state
machine. If you don't want to design your I/O application as
a finite state machine, then it will be slow regardless of
the I/O library you use. It sucks, but that is my
experience.

This phenomenon isn't specific to Haskell, by the way. C++'s
std::iostream is another fine example for an implicit state
API that is beautiful, elegant, and quite useless for
high-performance I/O.


 > what for (de)serialization tasks? my best attempts is
 > still 10x slower than C version. can you roll up little
 > prototype for such library or just sketch an ideas so i
 > can try to implement it?

The "Fast I/O" article I posted a few days ago is my
unfinished attempt at writing an efficient, general-purpose
binary I/O library for Haskell. I don't know how soon I'll
be able to complete that, nor do I know whether it would be
useful to many Haskell programmers if I do complete it. The
original BlockIO code has been stable (and quite fast) for
over a year or so, but I wouldn't know of anyone actually
using it. Apparently, designing explicit state data types is
nothing the Haskell community is fond of. :-)

Peter