[Haskell-cafe] Need some advice around lazy IO

[Petr Pudlák]

Hi Kashyap,

you could also use iteratees or conduits for a task like that. The beauty
of such libraries is that they can ensure that a resource is always
properly disposed of. See this simple example:
https://gist.github.com/anonymous/5183107
It prints the first line of each file given as an argument. After each line
is printed, the `fileConduit` pipe ensures that the handle is closed. It
also makes the program nicely composable.

Best regards,
Petr


import Control.Monad
import Control.Monad.Trans.Class
import Control.Monad.IO.Class
import Data.Conduit
import Data.Conduit.List
import System.Environment
import System.IO

{- | Accept file paths on input, output opened file handle, and ensure that the
 - handle is always closed after its downstream pipe finishes whatever
work on it. -}
fileConduit :: MonadResource m => IOMode -> Conduit FilePath m Handle
fileConduit mode = awaitForever process
  where
    process file = bracketP (openFile file mode) closeWithMsg yield
    closeWithMsg h = do
        putStrLn "Closing file"

        hClose h

{- | Print the first line from each handle on input. Don't care about
the handle. -}
firstLine :: MonadIO m => Sink Handle m ()
firstLine = awaitForever (liftIO . (hGetLine >=> putStrLn))

main = do
    args <- getArgs

    runResourceT $ sourceList args =$= fileConduit ReadMode $$ firstLine




2013/3/17 C K Kashyap <ckkashyap at gmail.com>

> Hi,
>
> I am working on an automation that periodically fetches bug data from our
> bug tracking system and creates static HTML reports. Things worked fine
> when the bugs were in the order of 200 or so. Now I am trying to run it
> against 3000 bugs and suddenly I see things like - too  many open handles,
> out of memory etc ...
>
> Here's the code snippet - http://hpaste.org/84197
>
> It's a small snippet and I've put in the comments stating how I run into
> "out of file handles" or simply file not getting read due to lazy IO.
>
> I realize that putting ($!) using a trial/error approach is going to be
> futile. I'd appreciate some pointers into the tools I could use to get some
> idea of which expressions are building up huge thunks.
>
>
> Regards,
> Kashyap
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20130317/10dcfbbd/attachment.htm>