[GHC] #13110: GHC API allocates memory which is never GC'd

[GHC]

#13110: GHC API allocates memory which is never GC'd
-------------------------------------+-------------------------------------
        Reporter:  DanielG           |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  GHC API           |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Description changed by DanielG:

@@ -43,3 +43,3 @@
- process with `pmap $(pgrep Leaky)` that the majority of the memory usage
- I'm seeing comes from address `0x0000000200000000` which is where the RTS
- allocates memory AFAIK.
+ process with `pmap $(pgrep Leaky)` I can see that the majority of the
+ memory usage I'm seeing comes from address `0x0000000200000000` which is
+ where the RTS allocates memory AFAIK.

New description:

 I've run into a very strange sort of memory leak it seems. GHC seems to
 manage to allocate memory which I cannot get rid of no matter what I try
 even after all references have disappeared. The allocated memory doesn't
 show up in heap profiles but is still visibly allocated (as seen in ps or
 pmap).

 I discovered this while investigating reports of large memory usage in
 ghc-mod where this issue is forcing us to dump completion information in a
 separate process to avoid the allocated memory sticking around for the
 rest of the user's session.

 The attached test case demonstrates this problem by first calling
 `getModuleInfo` for all modules in all visible packages and then just
 looping forever in `main`. I would expect all memory GHC allocated to be
 GC'd at this point but this does not happen.

 When run as

 {{{
 $ ./Leaky -hide-all-packages
 }}}

 the test case consumes around 30M of memory on my system, if we however
 load some large package, say GHC itself

 {{{
 $ ./Leaky -hide-all-packages -package ghc
 }}}

 the program will consume around 2-300M of memory which are never
 deallocated. The behaviour is not related to the loaded package though I
 tried it with `Cabal` too with the same result though smaller memory
 usage.


 At first I thought this might be related to large CAFs with IORefs inside
 not being GCd for some reason so I tried calling `resetCAFs` from the RTS
 before entering the loop. This however makes no difference whatsoever. I
 also tried forcing a major GC just to be save to no avail.

 Next I speculated that maybe the allocated memory is beyond the GC's
 control (malloc()ed or something) but looking at the memory map of the
 process with `pmap $(pgrep Leaky)` I can see that the majority of the
 memory usage I'm seeing comes from address `0x0000000200000000` which is
 where the RTS allocates memory AFAIK.

--

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13110#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler