[Haskell-beginners] Changing evalList to parList creates a space
traviserdman at yahoo.com
Sun Aug 29 01:19:49 EDT 2010
Ok, since I last visited, as a result of the help from contributors on this list, I've reached a nice spot. The serial implementation of my processing algorithm works quite quickly (including the tree pruning that I mentioned last time) and the space leaks have been eliminated without resort to the dreadful foldl'rnf.
Here are simplified versions of key components of my program:
data Tree a = Node a [Tree a]
calcTree :: Tree MyData -> InputData -> Tree MyData
calcTree (Node cargo subtrees) input = Node (process cargo input) recurseChildren
process a b = ....
recurseChildren = map (\t -> calcTree t input) subtrees `using` evalList rseq
newTree = foldl' calcTree starttree (take n inputdatalist)
The <`using` evalList rseq> term at the end of recurseChildren was key to solving a space leak without resorting to foldl'rnf (which is a performance killer). This is also the place in my code where it makes the most sense to introduce some parallelism (ie to traverse the subtrees in parallel).
However, when I simply change the evalList to a parList, a space leak is created anew, which I didn't anticipate at all. (of course, I also made the change of compiling with -threaded and -feager-blackholing flags ... but if I compile with those flags and leave it as evalList, there is no space leak).
I'm using parallel-18.104.22.168, so it's not the space leak from parallel-1.1.x.
Any ideas why this simple change should introduce a space leak? How might I get to where I want to go?
More information about the Beginners