[Haskell-beginners] getting memory usage to be constant when mixing wav files

[Felipe Lessa]

On Sun, Feb 14, 2010 at 07:22:52PM +0900, Renick Bell wrote:
> addInBase :: IOCArray Int Double -> IOCArray Int Double -> (Int, Int) -> IO ()
> addInBase longer shorter (!li,!si) = do
>     x <- readArray longer li
>     y <- readArray shorter si
>     let result =  x + y
>     writeArray longer li result
>
> addIn :: IOCArray Int Double -> IOCArray Int Double -> Int -> Int ->
> Int -> IO ()
> addIn longer shorter shorterBounds !li !si
>     | si >  shorterBounds = return ()
>     | si <= shorterBounds = addInBase longer shorter (li,si)
>             >> addIn longer shorter shorterBounds (li+1) (si+1)

These functions don't have any lazyness problem.  They run in the
IO monad which enforces execution order.

> Basically, the addIn function is called by mapM_ across a list of all
> of the occurrences of the input wav files.

I'd guess that the mapM_ is where the problem lies in.  If I
understood correctly, you're doing something like (in
pseudo-Haskell):

  do files <- mapM readFile filenames    -- read files lazily
     mapM_ addIn files                   -- do something
     mapM_ writeFile files               -- write files (strictly)

In other words, I guess you're reading the files lazily, right?
If that's the case, then mapM_ is your culprit.  Each 'addIn'
will force your inputs into memory.  You hold a reference to them
so you can save the results later, so after mapM_ has completed
everything is in memory.

The solution is simple for that pseudo-Haskell above:

  do mapM process filenames
  where process filename = readFile filename >>= addIn >>= writeFile

In other words, read, process and write the files in one IO
action.  After the first 'process' finishes, every data is ready
to be garbage collected.  Of course you will still have those
'IOCArray's in memory during the 'process' call.

--
Felipe.