[Haskell-cafe] Space not reclaimed by GC

[Neil Mitchell]

Hi,

I have a program which runs, then calls performGC multiple times. I'd
expect all (or nearly all) of the memory to be freed by that point.
But it isn't.

Given this program:

import Language.Haskell.Exts.Annotated -- from haskell-src-exts
import System.Mem

main :: IO ()
main = do
    let src = fromParseResult $ parseFileContents $ "data C = C {a ::
F {- " ++ replicate 400000 'd' ++ " -}     }"
    putStrLn $ "Total length is: " ++ show (length $ show $ src)
    performGC
    performGC
    performGC

I run with: ghc --make Temp.hs -rtsopts && Temp.exe +RTS -G1 -S (the
use of 1 generation is to make sure all GC's clear up everything, the
leak still happens without it). I am using GHC 7.0.1

I get the output:

    Alloc    Copied     Live    GC    GC     TOT     TOT  Page Flts
    bytes     bytes     bytes  user  elap    user    elap
... lots of values ...
 16187336   9598808   9600428  0.05  0.06    0.47    0.66    0    0  (Gen:  0)
 19513288  11571104  11572724  0.06  0.07    0.55    0.77    0    0  (Gen:  0)
 23519176  13948292  13949912  0.09  0.08    0.67    0.89    0    0  (Gen:  0)
Total length is: 4226
 24645752       548   2091128  0.00  0.00    0.75    0.98    0    0  (Gen:  0)
       20       540   2091120  0.00  0.00    0.75    0.99    0    0  (Gen:  0)
        0       540   2091120  0.00  0.00    0.75    1.00    0    0  (Gen:  0)
     1288       844      3028  0.00  0.00    0.75    1.01    0    0  (Gen:  0)
        0                      0.00  0.00

     162,592,312 bytes allocated in the heap
      81,004,756 bytes copied during GC
      13,949,912 bytes maximum residency (30 sample(s))
         226,344 bytes maximum slop
              63 MB total memory in use (6 MB lost due to fragmentation)

  Generation 0:    30 collections,     0 parallel,  0.48s,  0.51s elapsed

  INIT  time    0.02s  (  0.02s elapsed)
  MUT   time    0.25s  (  0.47s elapsed)
  GC    time    0.48s  (  0.51s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time    0.75s  (  1.01s elapsed)

  %GC time      64.6%  (50.6% elapsed)

  Alloc rate    613,093,182 bytes per MUT second

  Productivity  33.3% of total user, 24.7% of total elapsed

QUESTION 1: My reading of this report is that performGC leaves 2091120
of live data. However, the program seems to run a GC on termination,
when only 3028 bytes of data are left. Why is all the data not
released on performGC?

QUESTION 2: It seems that the haskell-src-exts parser has some kind of
space leak in "data C = C {a :: F {- ddddd -} }", but not in "data C =
C {a :: F} {- ddddd -}". That's fair enough, but is it because it's a
space leak that performGC doesn't get the data? If I change to the
second form then the in use memory after performGC is about 3Kb.

I found that if I did the parseFileContents/print bit once I get 2Mb
of leak, if I do it twice I get 2Mb, and if I do it three times I get
1Mb of leak, which really confused me.

Thanks, Neil