[Haskell-cafe] Trying to fix space leak

[Paul Johnson]

Ben Challenor wrote:
> Hi
>
> I'm learning Haskell through writing a compiler. I'm seeing huge 
> memory use in a function which converts the dataflow graph to the form 
> required by Data.Graph.  [...]
> I assume the allocation is being garbage-collected pretty quickly, 
> because a) 6,616,297,296 bytes is stupid (!) and b) Process Explorer 
> informs me that the peak private bytes of the program is not more than 
> a couple of MB.
Thats not a space leak.  A space leak is when you find that all 6 GB of 
that allocation is being retained.  You are correct that the GC is 
recovering this space as fast as it gets allocated.

The profiler records how much memory is allocated from within various 
bits of your program to help you understand its performance and where 
optimisable hotspots are.  I haven't looked at your code, but I imagine 
that you are traversing lazy data structures, which means that bits of 
the structure are consumed and deallocated as fast as they are created.

The profiler also has options to help you understand what is being 
retained where.

I'd advise against trying to make your program stricter because you 
might suddenly find yourself building an entire 6GB structure in memory 
before traversing it, which would not be a Good Thing.

Paul.