[Haskell-beginners] Memory usage problem

[Sami Liedes]

On Mon, Mar 22, 2010 at 03:06:08AM +0200, Sami Liedes wrote:
> On Sun, Mar 21, 2010 at 08:28:23PM -0400, Patrick LeBoutillier wrote:
> > - Also, does memory usage improve if you remove the "sort"?
> 
> Yes. Then it only takes a few megabytes, regardless of how large the
> file is.

This led me to investigate further. The following hack (and nothing
less crude I could invent) seems to significantly lessen the amount of
memory needed by the program. Now it takes maybe 180 megabytes instead
of 1.5 gigabytes. I guess the 180 megabytes consists of the output
stream (string), which is now explicitly computed entirely before
sorting and outputting it. It's still far from perfect.

I think there definitely _should_ be a better way of doing this
(forcing some computation to be strict so it takes less memory). Any
ideas? And I especially despise the (sum (map (const 0) b)) thing, but
it seems nothing less ugly does the trick.

It seems seq doesn't force the entire computation to finish, only the
next level. Is there some way to force the entire computation subtree
to finish? That way I could say just seq' b (if seq' was the function
that does that).

(see my original posting for the entire code)

------------------------------------------------------------
-- Return the unsorted package list and the sorted package list
processFile :: String -> (String,String)
processFile str =
  let l = lines str
      pkgs = readRecords l
  in (unlines (map sprintPackage pkgs),
      unlines (map sprintPackage (sort pkgs)))

-- Force the unsorted pkg list to be computed first (outputting it also
-- does the trick, but mere (map id b) doesn't). This is really ugly.
-- A mere "do putStrLn c" however eats 1.5 GB of memory instead of 180 MB.
main = do a <- getContents
          let (b,c) = processFile a in
            seq (sum (map (const 0) b)) (
              do putStrLn c)
------------------------------------------------------------

	Sami
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
Url : http://www.haskell.org/pipermail/beginners/attachments/20100322/9db48cc7/attachment.bin