[Haskell-cafe] Re: Left fold enumerator - a real pearl overlooked?

[Duncan Coutts]

On Mon, 2009-03-02 at 11:50 +0000, Bayley, Alistair wrote:
> > From: haskell-cafe-bounces at haskell.org 
> > [mailto:haskell-cafe-bounces at haskell.org] On Behalf Of Manlio Perillo
> > Sent: 02 March 2009 11:01
> > 
> > Eugene Kirpichov ha scritto:
> > > I'm not considering the lazy IO approach, as it doesn't involve any
> > > form of control over resources.
> > 
> > This is not always true.
> > I'm using lazy IO, still having full control over the resources.
> > 
> > parse path = withFile path ReadMode parse'
> >      where
> >        parse' :: Handle -> IO (UArr Xxx)
> >        parse' handle = do
> >          contents <- L.hGetContents handle
> >          let v = toU $ xxx $ L.lines contents
> >          rnf v `seq` return v
> > 
> > All the file is consumed, before the result is returned.

> This only works if the entire file can reasonably fit into memory. If
> you want to process something really big, then you need some sort of
> streaming approach, where you only look at a small part of the file (a
> line, or a block) at a time. And this is where the enumerator-iteratee
> approach looks better, because the IO is strict, but you still take a
> stream-like approach to processing the contents.

This can still be done using withFile and hGetContents. You just have to
put the consumer inside the scope of withFile. The consumer can work in
a streaming fashion. With lazy bytestrings this can be both efficient,
work in constant memory and guarantee the file is closed.

We guarantee the file is closed by using withFile. The only thing to
watch out for is a consumer that doesn't consume as much as you were
expecting before the file does get closed. You should notice that pretty
quickly though since it should happen every time (whereas resource leaks
are not so immediately visible).

Duncan