[Haskell-cafe] How to deal with huge text file?

[Magicloud Magiclouds]

This is the function. The problem sure seems like something was
preserved unexpected. But I cannot find out where is the problem.

seperateOutput file =
  let content = lines file
      indexOfEachOutput_ = fst $ unzip $ filter (\(i, l) ->
                                                 " Log for " `isPrefixOf` l
                                               ) $ zip [0..] content
      indexOfEachOutput = indexOfEachOutput_ ++ [length content] in
  map (\(a, b) ->
         drop a $ take b content
      ) $ zip indexOfEachOutput $ tail indexOfEachOutput

On Tue, May 25, 2010 at 10:12 AM, Ivan Miljenovic
<ivan.miljenovic at gmail.com> wrote:
> On 25 May 2010 12:02, Magicloud Magiclouds
> <magicloud.magiclouds at gmail.com> wrote:
>> U is for UTF8 module. And I will try the modules you mentioned.
>> Although I thought Haskell IO is lazy enough....
>
> If you're only streaming data, it probably would be.  However, you
> seem to keep some of it in memory, which is what the problem is.  You
> might be able to fix this by doing "main = readFile filename >>= liftM
> separateOutput >>= mapM_ foo".  However, it depends on what
> separateOutput does.
>
> Also, consider using when (from Control.Monad) instead of your if statement.
>
> --
> Ivan Lazar Miljenovic
> Ivan.Miljenovic at gmail.com
> IvanMiljenovic.wordpress.com
>



-- 
竹密岂妨流水过
山高哪阻野云飞