[Haskell-cafe] ghc leaking memory?

[Simon Marlow]

On 08/03/2010 13:30, Joachim Breitner wrote:
> Hi Simon,
>
> Am Montag, den 08.03.2010, 11:07 +0000 schrieb Simon Marlow:
>>> On Sat, Mar 6, 2010 at 11:57 AM, Joachim Breitner<nomeata at debian.org>   wrote:
>>>> Hi,
>>>>
>>>> in Debian, we are having some problems building large libraries (such as
>>>> agda, highlighting-kate, xmonad-contrib) on weaker architectures (sparc,
>>>> armel, s390) which compile with -fvia-C, especially when building the
>>>> profiling libraries.
>>>>
>>>> In the case of highlighting-kate, there were really issues with the
>>>> code: A huge list of words which ghc seemed to try to inline caused it
>>>> consuming a lot of memory.
>>>>
>>>> But in the case of agda, it even fails with relatively small modules:
>>>> https://buildd.debian.org/fetch.cgi?pkg=agda&arch=sparc&ver=2.2.6-3&stamp=1267334936&file=log&as=raw
>>>>
>>>> Now, looking at the memory gauge on my computer while compiling such a
>>>> program with Cabal (which compiles all modules in one ghc call), it
>>>> looks like it is steadily increasing. I’d expect ghc to free a lot of
>>>> memory when it is done with a module, but this does not seem to be the
>>>> case.
>>>>
>>>> Is this a bug (i.e. memory leak) in ghc or expected behavior?
>>
>> Can you verify that it is GHC leaking memory, and not gcc?  I recommend
>> turning off -fvia-C in particular unless you're sure that you need it.
>
> Sorry if I was not clear: The issues arise on “strange” architectures
> (armel, s390, sparc, mips) where, as far as I know, -fvia-C is the only
> option and thus the default. I was not setting this flag myself.

Ah, I see.

> Isn’t GHC calling gcc once for each module? In that case, that can not
> leak across modules, can it?

Right, gcc is only being invoked once per module, but the files being 
sent to gcc can get quite large, particularly when profiling is enabled, 
and even a single run of gcc can grow very large.  This has been more of 
a problem with the newer gcc versions since they changed their memory 
manager and started doing whole-module optimisations (though we do try 
to disable this with -fno-toplevel-reorder, I'm not sure if that 
disbales module-at-a-time compilation completely).

Cheers,
	Simon