GHC memory usage when typechecking from source vs. loading ModIfaces

[Matthew Pickering]

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive
closure of each module when you compile it, which obviously results in a
large amount of repeated work. This is why people are investigating ideas
of a persistent worker to at least avoid rehydrating all external
dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <
matthewtpickering at gmail.com> wrote:

> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <gergo at erdi.hu> wrote:
>
>> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>>
>> > I wonder if you have got your condition the wrong way around.
>> >
>> > The only "safe" time to perform rehydration is AFTER the point it can
>> never be used
>> > again.
>> >
>> > If you rehydrate it just before it is used then you will repeat work
>> which has already
>> > been done. If you do this, you will always have a trade-off between
>> space used and
>> > runtime.
>>
>> Oops. Yes, I have misunderstood the idea. I thought the idea was that
>> after loading a given module into the HPT, its ModDetails would start
>> out small (because of laziness) and then keep
>> growing in size as more and more of it are traversed, and thus
>> forced, during the typechecking of its dependees, so at some point we
>> would want to reset that into the small initial representation as created
>> by initModDetails.
>>
>> But if the idea is that I should rehydrate modules when they can't be
>> used
>> anymore, then that brings up the question why even do that, instead of
>> straight removing the HomeModInfos from the HPT?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250212/0b378c8b/attachment.html>