GHC memory usage when typechecking from source vs. loading ModIfaces

[Matthew Pickering]

Perhaps I am confused as well. I would have to investigate to see what the
precise situation was.

On Thu, Jan 23, 2025 at 12:40 PM ÉRDI Gergő <gergo at erdi.hu> wrote:

> I'll get back to you with details as I can.
>
> But one thing I noticed is that contrary to my expectations, when
> replacing the ModDetails with fresh ones, the real cost comes not from the
> fact that we have to re-evaluate the various parts of the ModDetails, but
> rather the call to `fillModDetails` itself!
>
> In other words, I can replicate most of the slowdown by calling
> `fillModDetails` on the interfaces in the HPT but throwing its result away
> and keeping the HPT as is. This sounds like bad news to me.
>
> Your remark that there are HomeModInfos referenced outside the HPT
> confuses me. I thought the whole reason this could work, and the whole
> reason I can see memory usage improvement, is because the HomeModInfos are
> not stored outside the HPT? If they are, home come my replacing them in
> the HPT with fresh ModDetails helps with memory usage instead of making it
> worse?
>
> On Thu, 23 Jan 2025, Matthew Pickering wrote:
>
> > That's good news.
> >
> > I don't think the first idea will do very much as there are other
> references to the
> > final "HomeModInfo" not stored in the HPT.
> >
> > Have you constructed a time profile to determine why the runtime is
> higher? With the
> > second approach you are certainly trading space usage for repeating work.
> >
> > If you actually do have a forest, then ideally you would replace the
> ModDetails after
> > it will never be used again.
> >
> > You are likely also missing other patches important for memory usage.
> >
> > * https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12582
> > * https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12347
> >
> > I can't comment about the 17 HPT, what do the retainer stacks look like
> in ghc-debug?
> >
> > PS.  Please use eventlog2html so the profiles are readable! You can use
> it on .hp
> > profiles.
> >
> > Cheers,
> >
> > Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250123/cd04a495/attachment.html>