From Gergo.Erdi at sc.com  Tue Apr  1 04:51:17 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Tue, 1 Apr 2025 04:51:17 +0000
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
Message-ID: <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

This sounds extremely interesting, but I don’t understand where you are getting this number from! How do you see in the eventlog HTMLs that I’ve included that there are ~2000 ModuleGraphs? I’ve now tried using ghc-debug to find all ModuleGraph constructors at two points in the run: just before typechecking the first module (after all the extendMG calls) and just after typechecking the last module, and even in the cold case I only see 1 ModuleGraph before and 13 ModuleGraphs after.

Also, what do you mean by “precisely one loaded per interface loaded into the EPS”? Since my repro has 2294 modules, wouldn’t that mean 2294 ModuleGraphs by that metric?

From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Saturday, March 29, 2025 1:53 AM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces


Hi Gergo,

I quickly tried building `Cabal` with the master branch. There is precisely 1 ModuleGraph allocated for the home session, and precisely one loaded per interface loaded into the EPS. No leaky behaviour like you can see in your eventlogs.

It seems there are about 2000 live module graphs in your program, are you doing something with the API to create this many?

Cheers,

Matt

On Fri, Mar 28, 2025 at 12:40 PM Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------


----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/96d935c7/attachment.html>

From Gergo.Erdi at sc.com  Tue Apr  1 06:04:46 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Tue, 1 Apr 2025 06:04:46 +0000
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
 <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <SEYPR03MB7463B4D13EF9C71F4A36DDD4E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

OK scratch that, I was looking at wrong ghc-debug output. Indeed there are 2301 ModuleGraphs in the heap at the end of typechecking :O

From: Erdi, Gergo
Sent: Tuesday, April 1, 2025 12:51 PM
To: Matthew Pickering <matthewtpickering at gmail.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: Re: GHC memory usage when typechecking from source vs. loading ModIfaces

This sounds extremely interesting, but I don’t understand where you are getting this number from! How do you see in the eventlog HTMLs that I’ve included that there are ~2000 ModuleGraphs? I’ve now tried using ghc-debug to find all ModuleGraph constructors at two points in the run: just before typechecking the first module (after all the extendMG calls) and just after typechecking the last module, and even in the cold case I only see 1 ModuleGraph before and 13 ModuleGraphs after.

Also, what do you mean by “precisely one loaded per interface loaded into the EPS”? Since my repro has 2294 modules, wouldn’t that mean 2294 ModuleGraphs by that metric?

From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Saturday, March 29, 2025 1:53 AM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces


Hi Gergo,

I quickly tried building `Cabal` with the master branch. There is precisely 1 ModuleGraph allocated for the home session, and precisely one loaded per interface loaded into the EPS. No leaky behaviour like you can see in your eventlogs.

It seems there are about 2000 live module graphs in your program, are you doing something with the API to create this many?

Cheers,

Matt

On Fri, Mar 28, 2025 at 12:40 PM Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/be3a1ef4/attachment-0001.html>

From matthewtpickering at gmail.com  Tue Apr  1 09:44:16 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Tue, 1 Apr 2025 10:44:16 +0100
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <SEYPR03MB7463B4D13EF9C71F4A36DDD4E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
 <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463B4D13EF9C71F4A36DDD4E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <CALuQ0m9ZJ8z5T2PoANvw2xcy2V1X2e-OO_LRoB8kjYFSmSWA3g@mail.gmail.com>

Hi Gergo,

I looked in the detailed pane, searched for ModuleGraph, hovered my mouse
over the "ModuleGraph` constructor, recorded the number of live bytes,
divided that by 32.

Matt

On Tue, Apr 1, 2025 at 7:04 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:

> PUBLIC
>
> OK scratch that, I was looking at wrong ghc-debug output. Indeed there are
> 2301 ModuleGraphs in the heap at the end of typechecking :O
>
>
>
> *From:* Erdi, Gergo
> *Sent:* Tuesday, April 1, 2025 12:51 PM
> *To:* Matthew Pickering <matthewtpickering at gmail.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* Re: GHC memory usage when typechecking from source vs. loading
> ModIfaces
>
>
>
> This sounds extremely interesting, but I don’t understand where you are
> getting this number from! How do you see in the eventlog HTMLs that I’ve
> included that there are ~2000 ModuleGraphs? I’ve now tried using ghc-debug
> to find all ModuleGraph constructors at two points in the run: just before
> typechecking the first module (after all the extendMG calls) and just after
> typechecking the last module, and even in the cold case I only see 1
> ModuleGraph before and 13 ModuleGraphs after.
>
>
>
> Also, what do you mean by “precisely one loaded per interface loaded into
> the EPS”? Since my repro has 2294 modules, wouldn’t that mean 2294
> ModuleGraphs by that metric?
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Saturday, March 29, 2025 1:53 AM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
>
>
> Hi Gergo,
>
>
>
> I quickly tried building `Cabal` with the master branch. There is
> precisely 1 ModuleGraph allocated for the home session, and precisely one
> loaded per interface loaded into the EPS. No leaky behaviour like you can
> see in your eventlogs.
>
>
>
> It seems there are about 2000 live module graphs in your program, are you
> doing something with the API to create this many?
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 12:40 PM Matthew Pickering <
> matthewtpickering at gmail.com> wrote:
>
> HI Gergo,
>
>
>
> Do you have a (synthetic?) reproducer? You have probably identified some
> memory leak. However, without any means to reproduce it becomes very
> difficult to investigate. I feel like we are getting into very precise
> details now, where speculating is not going to be so useful.
>
>
>
> It seems like this is an important thing for you and your company. Is
> there any budget to pay for some investigation? If that was the case then
> some effort could be made to create a synthetic producer and make the
> situation more robust going into the future if your requirements were
> precisely understood.
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
> Just to add that I get the same "equalizing" behaviour (but in a more
> "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just
> call `hugInstancesBelow` before processing each module. So that's
> definitely one source of extra memory usage. I wonder if it would be
> possible to rebuild the ModuleGraph periodically (similar to the ModDetails
> dehydration), or if there are references to it stored all over the place
> from `HscEnv`s scattered around in closures etc. (basically the same
> problem the HPT had before it was made into a mutable reference).
>
> -----Original Message-----
> From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Erdi, Gergo
> via ghc-devs
> Sent: Friday, March 28, 2025 4:49 PM
> To: Matthew Pickering <matthewtpickering at gmail.com>; GHC Devs <
> ghc-devs at haskell.org>
> Cc: ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <
> Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> Hi,
>
> Unfortunately, I am forced to return to this problem. Everything below is
> now in the context of GHC 9.12 plus the mutable HPT patch backported.
>
> My test case is typechecking a tree of 2294 modules that form the
> transitive closure of a single module's dependencies, all in a single
> process. I have done this typechecking three times, here's what `+RTS -s
> -RTS` reports for max residency:
>
> * "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB
>
> * "cold-top": With all `ModIface`s already on disk, except for the
>   single top-level module: 302 MB
>
> * "warm": With all `ModIface`s already on disk: 211 MB
>
> So my stupidly naive question is, why is the "cold" case also not 302 MB?
>
> In earlier discussion, `ModDetails` unfolding has come up. Dehydrating
> `ModDetails` in the HPT all the time is disastrous for runtime, but based
> on this model I would expect to see improvements from dehydrating "every
> now and then". So I tried a stupid simple example where after every 100th
> typechecked module, I run this function on the topologically sorted list of
> modules processed so far:
>
>
> ```
> dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods
> = do
>     let HPT{ table = hptr } = hsc_HPT hsc_env
>     hpt <- readIORef hptr
>     for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface
> _details _linkable) -> do
>         !details <- initModDetails hsc_env iface
>         pure ()
> ```
>
> Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact,
> the profile looks exactly the same.
>
> Speaking of the profile, in the "cold" case I see a lot of steadily
> increasing heap usage from the `ModuleGraph`. I could see this happening if
> typechecking from scratch involves more `modulesUnder` calls which in turn
> force more and more of the `ModuleGraph`. If so, then maybe this could be
> worked around by repeatedly remaking the `ModuleGraph` just like I remake
> the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing
> the `ModuleGraph` at the start, with the idea being that this should
> "equalize" the three scenarios if this really is a substantial source of
> extra memory usage. This pushes up the warm case's memory usage to 381 MB,
> which is promising, but I still see a `Word64Map` that is steadily
> increasing in the "cold-force-modulegraph" case and contributes a lot to
> the memory usage. Unfortunately, I don't know where that `Word64Map` is (it
> could be any `Unique`-keyed environment...).
>
> So I am now stuck at this point. To spell out my goal explicitly, I would
> like to typecheck one module after another and not keep anything more in
> memory around than if I loaded them from `ModIface` files.
>
> Thanks,
>         Gergo
>
> p.s.: I couldn't find a way in the EventLog output HTML to turn event
> markers on/off or filter them, so to avoid covering the whole graph with
> gray lines, I mark only every 100th module.
>
>
>
>
> From: Matthew Pickering <matthewtpickering at gmail.com>
> Sent: Wednesday, February 12, 2025 7:08 PM
> To: ÉRDI Gergő <gergo at erdi.hu>
> Cc: Erdi, Gergo <Gergo.Erdi at sc.com>; Zubin Duggal <zubin at well-typed.com>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; GHC Devs <
> ghc-devs at haskell.org>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> You do also raise a good point about rehydration costs.
>
> In oneshot mode, you are basically rehydrating the entire transitive
> closure of each module when you compile it, which obviously results in a
> large amount of repeated work. This is why people are investigating ideas
> of a persistent worker to at least avoid rehydrating all external
> dependencies as well.
>
> On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:
> matthewtpickering at gmail.com> wrote:
> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu> wrote:
> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>
> > I wonder if you have got your condition the wrong way around.
> >
> > The only "safe" time to perform rehydration is AFTER the point it can
> > never be used again.
> >
> > If you rehydrate it just before it is used then you will repeat work
> > which has already been done. If you do this, you will always have a
> > trade-off between space used and runtime.
>
> Oops. Yes, I have misunderstood the idea. I thought the idea was that
> after loading a given module into the HPT, its ModDetails would start out
> small (because of laziness) and then keep growing in size as more and more
> of it are traversed, and thus forced, during the typechecking of its
> dependees, so at some point we would want to reset that into the small
> initial representation as created by initModDetails.
>
> But if the idea is that I should rehydrate modules when they can't be used
> anymore, then that brings up the question why even do that, instead of
> straight removing the HomeModInfos from the HPT?
>
> ----------------------------------------------------------------------
>
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our public website.
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/f6a1c04f/attachment.html>

From Gergo.Erdi at sc.com  Tue Apr  1 12:24:14 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Tue, 1 Apr 2025 12:24:14 +0000
Subject: [External] Re: GHC memory usage when typechecking from source vs.
 loading ModIfaces
In-Reply-To: <CALuQ0m9ZJ8z5T2PoANvw2xcy2V1X2e-OO_LRoB8kjYFSmSWA3g@mail.gmail.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
 <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463B4D13EF9C71F4A36DDD4E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9ZJ8z5T2PoANvw2xcy2V1X2e-OO_LRoB8kjYFSmSWA3g@mail.gmail.com>
Message-ID: <SEYPR03MB7463CDCAD948DE28ED6B9C51E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

OK so this `ModuleGraph` leak has the potential to be a huge explainer because of the large number of ModuleNodes and NodeKey_Modules in the heap profile. It turns out the reason we had one ModuleGraph per module was because I was populating the ModuleGraph by repeated calls to `extendMG`, where the new edges were created via `mgModSummaries` on the old `ModuleGraph`, and so the new edge list was a thunk pointing to the old `ModuleGraph`.

I’ve now changed it to a single `mkModuleGraph` call, which gets rid of the extra ModuleGraphs (see attached profile), but unfortunately, the ModuleNodes/ edge NodeKey_Modules are still not shared. Here’s how I build the graph, trying hard with a Data.Map.Map to ensure only use one NodeKey_Module per each target module:

```
registerModulesToGraph :: GhcMonad m => [(UnitId, ModuleName, [ModuleName])] -> m ()
registerModulesToGraph entries = modifySessionM \hsc_env -> do
    mod_deps <- for entries \(unit, mod_name, deps) -> do
        mod <- liftIO $ registerToModuleFinder hsc_env unit mod_name
        let ms = dummyModSummary (hsc_dflags hsc_env) mod deps fingerprint0
        pure (ms, deps)
    let nodes = [ModuleNode (edges deps) ms | (ms, deps) <- mod_deps]
    pure hsc_env{ hsc_mod_graph = mkModuleGraph nodes }
  where
    edge_map :: Map.Map ModuleName NodeKey
    edge_map = Map.fromList [(mod_name, NodeKey_Module $ ModNodeKeyWithUid (notBoot mod_name) uid) | (uid, mod_name, _) <- entries]

    edges deps = map (edge_map Map.!) deps

registerToModuleFinder :: HscEnv -> UnitId -> ModuleName -> IO Module
registerToModuleFinder hsc_env unitId modName = do
    addHomeModuleToFinder (hsc_FC hsc_env) homeUnit (notBoot modName) location
  where
    homeUnit = DefiniteHomeUnit unitId Nothing
    location = dummyModLocation modName
```

But, as you can see in the attached profile, I still get way more than 2k ModuleNodes and NodeKey_Modules….

From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Tuesday, April 1, 2025 5:44 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi Gergo,

I looked in the detailed pane, searched for ModuleGraph, hovered my mouse over the "ModuleGraph` constructor, recorded the number of live bytes, divided that by 32.

Matt

On Tue, Apr 1, 2025 at 7:04 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:

PUBLIC

OK scratch that, I was looking at wrong ghc-debug output. Indeed there are 2301 ModuleGraphs in the heap at the end of typechecking :O

From: Erdi, Gergo
Sent: Tuesday, April 1, 2025 12:51 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: Re: GHC memory usage when typechecking from source vs. loading ModIfaces

This sounds extremely interesting, but I don’t understand where you are getting this number from! How do you see in the eventlog HTMLs that I’ve included that there are ~2000 ModuleGraphs? I’ve now tried using ghc-debug to find all ModuleGraph constructors at two points in the run: just before typechecking the first module (after all the extendMG calls) and just after typechecking the last module, and even in the cold case I only see 1 ModuleGraph before and 13 ModuleGraphs after.

Also, what do you mean by “precisely one loaded per interface loaded into the EPS”? Since my repro has 2294 modules, wouldn’t that mean 2294 ModuleGraphs by that metric?

From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Saturday, March 29, 2025 1:53 AM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces


Hi Gergo,

I quickly tried building `Cabal` with the master branch. There is precisely 1 ModuleGraph allocated for the home session, and precisely one loaded per interface loaded into the EPS. No leaky behaviour like you can see in your eventlogs.

It seems there are about 2000 live module graphs in your program, are you doing something with the API to create this many?

Cheers,

Matt

On Fri, Mar 28, 2025 at 12:40 PM Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/c3387938/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cold-no-modulegraph-leak.zip
Type: application/x-zip-compressed
Size: 1337463 bytes
Desc: cold-no-modulegraph-leak.zip
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/c3387938/attachment-0001.bin>

From george.colpitts at gmail.com  Tue Apr  1 14:48:34 2025
From: george.colpitts at gmail.com (George Colpitts)
Date: Tue, 1 Apr 2025 11:48:34 -0300
Subject: can't move to latest version of llvm on ghc 9.12.2
Message-ID: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>

llvm 20 is out but unlike in earlier versions of ghc moving to it means you
can no longer use llvm:

compiling:

ghc -fllvm hello.hs
Loaded package environment from
/Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
[1 of 2] Compiling Main             ( hello.hs, hello.o )
<no location info>: error: [GHC-66599]
    GHC was not configured with a supported LLVM toolchain
    Make sure you have installed LLVM between [13 and 20) and reinstall GHC
to make -fllvm work


from configure:

configure: We only support llvm 13 upto 20 (non-inclusive) (found 20.1.1).


Can we move to llvm 20 on HEAD and can we revert to the old behavior on ghc
9.12.3?

Should I file an ER?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/53092005/attachment.html>

From allbery.b at gmail.com  Tue Apr  1 16:27:41 2025
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 1 Apr 2025 12:27:41 -0400
Subject: can't move to latest version of llvm on ghc 9.12.2
In-Reply-To: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
References: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
Message-ID: <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>

Have you demonstrated that it works? IIRC the current behavior is because
some LLVM version (16, IIRC) didn't work with GHC (threw errors from opt, I
think).

On Tue, Apr 1, 2025 at 10:49 AM George Colpitts <george.colpitts at gmail.com>
wrote:

> llvm 20 is out but unlike in earlier versions of ghc moving to it means
> you can no longer use llvm:
>
> compiling:
>
> ghc -fllvm hello.hs
> Loaded package environment from
> /Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
> [1 of 2] Compiling Main             ( hello.hs, hello.o )
> <no location info>: error: [GHC-66599]
>     GHC was not configured with a supported LLVM toolchain
>     Make sure you have installed LLVM between [13 and 20) and reinstall
> GHC to make -fllvm work
>
>
> from configure:
>
> configure: We only support llvm 13 upto 20 (non-inclusive) (found 20.1.1).
>
>
> Can we move to llvm 20 on HEAD and can we revert to the old behavior on
> ghc 9.12.3?
>
> Should I file an ER?
>
> Thanks
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/8d947be9/attachment.html>

From george.colpitts at gmail.com  Tue Apr  1 17:05:59 2025
From: george.colpitts at gmail.com (George Colpitts)
Date: Tue, 1 Apr 2025 14:05:59 -0300
Subject: can't move to latest version of llvm on ghc 9.12.2
In-Reply-To: <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>
References: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
 <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>
Message-ID: <CAB-d4A4w-ghx4hCmizaw0Ebdye2i0ohe+KbP06XuQJUx0vpn3Q@mail.gmail.com>

I was trying to find out if it works but I ran into the problem I
described. Yes, there were a few years where the latest version didn't work
but in many years it did. ghc used to give a warning, this is an
unsupported version but we'll try it anyways. I would like to return to
that.

In any case I believe ghc dev should test to see if it works and if so use
it in HEAD. If it doesn't work than we should file a bug and fix it. In the
past we got a few versions behind llvm. I think we want to be on the latest
version available for each new release if possible. The earlier we look
into this the more chance we have to succeed at that.

On Tue, Apr 1, 2025 at 1:27 PM Brandon Allbery <allbery.b at gmail.com> wrote:

> Have you demonstrated that it works? IIRC the current behavior is because
> some LLVM version (16, IIRC) didn't work with GHC (threw errors from opt, I
> think).
>
> On Tue, Apr 1, 2025 at 10:49 AM George Colpitts <george.colpitts at gmail.com>
> wrote:
>
>> llvm 20 is out but unlike in earlier versions of ghc moving to it means
>> you can no longer use llvm:
>>
>> compiling:
>>
>> ghc -fllvm hello.hs
>> Loaded package environment from
>> /Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
>> [1 of 2] Compiling Main             ( hello.hs, hello.o )
>> <no location info>: error: [GHC-66599]
>>     GHC was not configured with a supported LLVM toolchain
>>     Make sure you have installed LLVM between [13 and 20) and reinstall
>> GHC to make -fllvm work
>>
>>
>> from configure:
>>
>> configure: We only support llvm 13 upto 20 (non-inclusive) (found 20.1.1).
>>
>>
>> Can we move to llvm 20 on HEAD and can we revert to the old behavior on
>> ghc 9.12.3?
>>
>> Should I file an ER?
>>
>> Thanks
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> brandon s allbery kf8nh
> allbery.b at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/125e4d5a/attachment.html>

From klebinger.andreas at gmx.at  Tue Apr  1 17:11:24 2025
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Tue, 1 Apr 2025 19:11:24 +0200
Subject: can't move to latest version of llvm on ghc 9.12.2
In-Reply-To: <CAB-d4A4w-ghx4hCmizaw0Ebdye2i0ohe+KbP06XuQJUx0vpn3Q@mail.gmail.com>
References: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
 <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>
 <CAB-d4A4w-ghx4hCmizaw0Ebdye2i0ohe+KbP06XuQJUx0vpn3Q@mail.gmail.com>
Message-ID: <4db7eafd-69b0-476c-bffd-e2a8827d7346@gmx.at>

Looking at https://gitlab.haskell.org/ghc/ghc/-/issues/25011 it seems
the desire was to fail with a proper error when LLVM was not found at all.

To me failing when the llvm version is too new seems like a unintended
side effect of fixing the former.

I agree that simply warning and trying anway is the better default. A
ticket would be appreciated!

Am 01/04/2025 um 19:05 schrieb George Colpitts:
> I was trying to find out if it works but I ran into the problem I
> described. Yes, there were a few years where the latest version didn't
> work but in many years it did. ghc used to give a warning, this is an
> unsupported version but we'll try it anyways. I would like to
> return to that.
>
> In any case I believe ghc dev should test to see if it works and if so
> use it in HEAD. If it doesn't work than we should file a bug and fix
> it. In the past we got a few versions behind llvm. I think we want to
> be on the latest version available for each new release if possible.
> The earlier we look into this the more chance we have to succeed at that.
>
> On Tue, Apr 1, 2025 at 1:27 PM Brandon Allbery <allbery.b at gmail.com>
> wrote:
>
>     Have you demonstrated that it works? IIRC the current behavior is
>     because some LLVM version (16, IIRC) didn't work with GHC (threw
>     errors from opt, I think).
>
>     On Tue, Apr 1, 2025 at 10:49 AM George Colpitts
>     <george.colpitts at gmail.com> wrote:
>
>         llvm 20 is out but unlike in earlier versions of ghc moving to
>         it means you can no longer use llvm:
>
>         compiling:
>
>             ghc -fllvm hello.hs
>             Loaded package environment from
>             /Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
>             [1 of 2] Compiling Main             ( hello.hs, hello.o )
>             <no location info>: error: [GHC-66599]
>               GHC was not configured with a supported LLVM toolchain
>               Make sure you have installed LLVM between [13 and 20)
>             and reinstall GHC to make -fllvm work
>
>
>         from configure:
>
>             configure: We only support llvm 13 upto 20 (non-inclusive)
>             (found 20.1.1).
>
>
>         Can we move to llvm 20 on HEAD and can we revert to the old
>         behavior on ghc 9.12.3?
>
>         Should I file an ER?
>
>         Thanks
>
>         _______________________________________________
>         ghc-devs mailing list
>         ghc-devs at haskell.org
>         http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
>     --
>     brandon s allbery kf8nh
>     allbery.b at gmail.com
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/e8b4524b/attachment.html>

From george.colpitts at gmail.com  Tue Apr  1 20:15:46 2025
From: george.colpitts at gmail.com (George Colpitts)
Date: Tue, 1 Apr 2025 17:15:46 -0300
Subject: can't move to latest version of llvm on ghc 9.12.2
In-Reply-To: <4db7eafd-69b0-476c-bffd-e2a8827d7346@gmx.at>
References: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
 <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>
 <CAB-d4A4w-ghx4hCmizaw0Ebdye2i0ohe+KbP06XuQJUx0vpn3Q@mail.gmail.com>
 <4db7eafd-69b0-476c-bffd-e2a8827d7346@gmx.at>
Message-ID: <CAB-d4A4dLMUocv2ZWp_7yPns0WbRZpHicmfkcn+za9ADC4YK4g@mail.gmail.com>

Thanks, I'll file a ticket today or tomorrow.

On Tue, Apr 1, 2025 at 2:11 PM Andreas Klebinger via ghc-devs <
ghc-devs at haskell.org> wrote:

> Looking at https://gitlab.haskell.org/ghc/ghc/-/issues/25011 it seems the
> desire was to fail with a proper error when LLVM was not found at all.
>
> To me failing when the llvm version is too new seems like a unintended
> side effect of fixing the former.
>
> I agree that simply warning and trying anway is the better default. A
> ticket would be appreciated!
> Am 01/04/2025 um 19:05 schrieb George Colpitts:
>
> I was trying to find out if it works but I ran into the problem I
> described. Yes, there were a few years where the latest version didn't work
> but in many years it did. ghc used to give a warning, this is an
> unsupported version but we'll try it anyways. I would like to return to
> that.
>
> In any case I believe ghc dev should test to see if it works and if so use
> it in HEAD. If it doesn't work than we should file a bug and fix it. In the
> past we got a few versions behind llvm. I think we want to be on the latest
> version available for each new release if possible. The earlier we look
> into this the more chance we have to succeed at that.
>
> On Tue, Apr 1, 2025 at 1:27 PM Brandon Allbery <allbery.b at gmail.com>
> wrote:
>
>> Have you demonstrated that it works? IIRC the current behavior is because
>> some LLVM version (16, IIRC) didn't work with GHC (threw errors from opt, I
>> think).
>>
>> On Tue, Apr 1, 2025 at 10:49 AM George Colpitts <
>> george.colpitts at gmail.com> wrote:
>>
>>> llvm 20 is out but unlike in earlier versions of ghc moving to it means
>>> you can no longer use llvm:
>>>
>>> compiling:
>>>
>>> ghc -fllvm hello.hs
>>> Loaded package environment from
>>> /Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
>>> [1 of 2] Compiling Main             ( hello.hs, hello.o )
>>> <no location info>: error: [GHC-66599]
>>>     GHC was not configured with a supported LLVM toolchain
>>>     Make sure you have installed LLVM between [13 and 20) and reinstall
>>> GHC to make -fllvm work
>>>
>>>
>>> from configure:
>>>
>>> configure: We only support llvm 13 upto 20 (non-inclusive) (found
>>> 20.1.1).
>>>
>>>
>>> Can we move to llvm 20 on HEAD and can we revert to the old behavior on
>>> ghc 9.12.3?
>>>
>>> Should I file an ER?
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> brandon s allbery kf8nh
>> allbery.b at gmail.com
>>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250401/5753cb1a/attachment-0001.html>

From matthewtpickering at gmail.com  Wed Apr  2 09:14:44 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 2 Apr 2025 10:14:44 +0100
Subject: [External] Re: GHC memory usage when typechecking from source vs.
 loading ModIfaces
In-Reply-To: <SEYPR03MB7463CDCAD948DE28ED6B9C51E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <CALuQ0m9Cj3dqaOdxUxn_0Vz3rOChzBGu5xpZMVV=N8gd2cZMsQ@mail.gmail.com>
 <SEYPR03MB74632ECD48E921CA918584E1E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463B4D13EF9C71F4A36DDD4E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9ZJ8z5T2PoANvw2xcy2V1X2e-OO_LRoB8kjYFSmSWA3g@mail.gmail.com>
 <SEYPR03MB7463CDCAD948DE28ED6B9C51E9AC2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <CALuQ0m9xD=rNk6v7ZpmkmQ+c3Qh+dpazHq6p3Q-WDKS9tfYJkQ@mail.gmail.com>

Hi Gergo,

Since this is a non-reproducible issue with your custom code, I think that
we've reached the limit of free debugging via email. There are several
consultancies which can be employed to investigate complicated issues like
this and perform the necessary changes to GHC to make your use-case easier.

If you have a reproducer on a distributed GHC release then it could be
investigated.

Cheers,

Matt

On Tue, Apr 1, 2025 at 1:24 PM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:

> PUBLIC
>
> OK so this `ModuleGraph` leak has the potential to be a huge explainer
> because of the large number of ModuleNodes and NodeKey_Modules in the heap
> profile. It turns out the reason we had one ModuleGraph per module was
> because I was populating the ModuleGraph by repeated calls to `extendMG`,
> where the new edges were created via `mgModSummaries` on the old
> `ModuleGraph`, and so the new edge list was a thunk pointing to the old
> `ModuleGraph`.
>
>
>
> I’ve now changed it to a single `mkModuleGraph` call, which gets rid of
> the extra ModuleGraphs (see attached profile), but unfortunately, the
> ModuleNodes/ edge NodeKey_Modules are still not shared. Here’s how I build
> the graph, trying hard with a Data.Map.Map to ensure only use one
> NodeKey_Module per each target module:
>
>
>
> ```
>
> registerModulesToGraph :: GhcMonad m => [(UnitId, ModuleName,
> [ModuleName])] -> m ()
>
> registerModulesToGraph entries = modifySessionM \hsc_env -> do
>
>     mod_deps <- for entries \(unit, mod_name, deps) -> do
>
>         mod <- liftIO $ registerToModuleFinder hsc_env unit mod_name
>
>         let ms = dummyModSummary (hsc_dflags hsc_env) mod deps fingerprint0
>
>         pure (ms, deps)
>
>     let nodes = [ModuleNode (edges deps) ms | (ms, deps) <- mod_deps]
>
>     pure hsc_env{ hsc_mod_graph = mkModuleGraph nodes }
>
>   where
>
>     edge_map :: Map.Map ModuleName NodeKey
>
>     edge_map = Map.fromList [(mod_name, NodeKey_Module $ ModNodeKeyWithUid
> (notBoot mod_name) uid) | (uid, mod_name, _) <- entries]
>
>
>
>     edges deps = map (edge_map Map.!) deps
>
>
>
> registerToModuleFinder :: HscEnv -> UnitId -> ModuleName -> IO Module
>
> registerToModuleFinder hsc_env unitId modName = do
>
>     addHomeModuleToFinder (hsc_FC hsc_env) homeUnit (notBoot modName)
> location
>
>   where
>
>     homeUnit = DefiniteHomeUnit unitId Nothing
>
>     location = dummyModLocation modName
>
> ```
>
>
>
> But, as you can see in the attached profile, I still get way more than 2k
> ModuleNodes and NodeKey_Modules….
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Tuesday, April 1, 2025 5:44 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
> Hi Gergo,
>
>
>
> I looked in the detailed pane, searched for ModuleGraph, hovered my mouse
> over the "ModuleGraph` constructor, recorded the number of live bytes,
> divided that by 32.
>
>
>
> Matt
>
>
>
> On Tue, Apr 1, 2025 at 7:04 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
>
>
> OK scratch that, I was looking at wrong ghc-debug output. Indeed there are
> 2301 ModuleGraphs in the heap at the end of typechecking :O
>
>
>
> *From:* Erdi, Gergo
> *Sent:* Tuesday, April 1, 2025 12:51 PM
> *To:* Matthew Pickering <matthewtpickering at gmail.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* Re: GHC memory usage when typechecking from source vs. loading
> ModIfaces
>
>
>
> This sounds extremely interesting, but I don’t understand where you are
> getting this number from! How do you see in the eventlog HTMLs that I’ve
> included that there are ~2000 ModuleGraphs? I’ve now tried using ghc-debug
> to find all ModuleGraph constructors at two points in the run: just before
> typechecking the first module (after all the extendMG calls) and just after
> typechecking the last module, and even in the cold case I only see 1
> ModuleGraph before and 13 ModuleGraphs after.
>
>
>
> Also, what do you mean by “precisely one loaded per interface loaded into
> the EPS”? Since my repro has 2294 modules, wouldn’t that mean 2294
> ModuleGraphs by that metric?
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Saturday, March 29, 2025 1:53 AM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
>
>
> Hi Gergo,
>
>
>
> I quickly tried building `Cabal` with the master branch. There is
> precisely 1 ModuleGraph allocated for the home session, and precisely one
> loaded per interface loaded into the EPS. No leaky behaviour like you can
> see in your eventlogs.
>
>
>
> It seems there are about 2000 live module graphs in your program, are you
> doing something with the API to create this many?
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 12:40 PM Matthew Pickering <
> matthewtpickering at gmail.com> wrote:
>
> HI Gergo,
>
>
>
> Do you have a (synthetic?) reproducer? You have probably identified some
> memory leak. However, without any means to reproduce it becomes very
> difficult to investigate. I feel like we are getting into very precise
> details now, where speculating is not going to be so useful.
>
>
>
> It seems like this is an important thing for you and your company. Is
> there any budget to pay for some investigation? If that was the case then
> some effort could be made to create a synthetic producer and make the
> situation more robust going into the future if your requirements were
> precisely understood.
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
> Just to add that I get the same "equalizing" behaviour (but in a more
> "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just
> call `hugInstancesBelow` before processing each module. So that's
> definitely one source of extra memory usage. I wonder if it would be
> possible to rebuild the ModuleGraph periodically (similar to the ModDetails
> dehydration), or if there are references to it stored all over the place
> from `HscEnv`s scattered around in closures etc. (basically the same
> problem the HPT had before it was made into a mutable reference).
>
> -----Original Message-----
> From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Erdi, Gergo
> via ghc-devs
> Sent: Friday, March 28, 2025 4:49 PM
> To: Matthew Pickering <matthewtpickering at gmail.com>; GHC Devs <
> ghc-devs at haskell.org>
> Cc: ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <
> Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> Hi,
>
> Unfortunately, I am forced to return to this problem. Everything below is
> now in the context of GHC 9.12 plus the mutable HPT patch backported.
>
> My test case is typechecking a tree of 2294 modules that form the
> transitive closure of a single module's dependencies, all in a single
> process. I have done this typechecking three times, here's what `+RTS -s
> -RTS` reports for max residency:
>
> * "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB
>
> * "cold-top": With all `ModIface`s already on disk, except for the
>   single top-level module: 302 MB
>
> * "warm": With all `ModIface`s already on disk: 211 MB
>
> So my stupidly naive question is, why is the "cold" case also not 302 MB?
>
> In earlier discussion, `ModDetails` unfolding has come up. Dehydrating
> `ModDetails` in the HPT all the time is disastrous for runtime, but based
> on this model I would expect to see improvements from dehydrating "every
> now and then". So I tried a stupid simple example where after every 100th
> typechecked module, I run this function on the topologically sorted list of
> modules processed so far:
>
>
> ```
> dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods
> = do
>     let HPT{ table = hptr } = hsc_HPT hsc_env
>     hpt <- readIORef hptr
>     for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface
> _details _linkable) -> do
>         !details <- initModDetails hsc_env iface
>         pure ()
> ```
>
> Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact,
> the profile looks exactly the same.
>
> Speaking of the profile, in the "cold" case I see a lot of steadily
> increasing heap usage from the `ModuleGraph`. I could see this happening if
> typechecking from scratch involves more `modulesUnder` calls which in turn
> force more and more of the `ModuleGraph`. If so, then maybe this could be
> worked around by repeatedly remaking the `ModuleGraph` just like I remake
> the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing
> the `ModuleGraph` at the start, with the idea being that this should
> "equalize" the three scenarios if this really is a substantial source of
> extra memory usage. This pushes up the warm case's memory usage to 381 MB,
> which is promising, but I still see a `Word64Map` that is steadily
> increasing in the "cold-force-modulegraph" case and contributes a lot to
> the memory usage. Unfortunately, I don't know where that `Word64Map` is (it
> could be any `Unique`-keyed environment...).
>
> So I am now stuck at this point. To spell out my goal explicitly, I would
> like to typecheck one module after another and not keep anything more in
> memory around than if I loaded them from `ModIface` files.
>
> Thanks,
>         Gergo
>
> p.s.: I couldn't find a way in the EventLog output HTML to turn event
> markers on/off or filter them, so to avoid covering the whole graph with
> gray lines, I mark only every 100th module.
>
>
>
>
> From: Matthew Pickering <matthewtpickering at gmail.com>
> Sent: Wednesday, February 12, 2025 7:08 PM
> To: ÉRDI Gergő <gergo at erdi.hu>
> Cc: Erdi, Gergo <Gergo.Erdi at sc.com>; Zubin Duggal <zubin at well-typed.com>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; GHC Devs <
> ghc-devs at haskell.org>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> You do also raise a good point about rehydration costs.
>
> In oneshot mode, you are basically rehydrating the entire transitive
> closure of each module when you compile it, which obviously results in a
> large amount of repeated work. This is why people are investigating ideas
> of a persistent worker to at least avoid rehydrating all external
> dependencies as well.
>
> On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:
> matthewtpickering at gmail.com> wrote:
> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu> wrote:
> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>
> > I wonder if you have got your condition the wrong way around.
> >
> > The only "safe" time to perform rehydration is AFTER the point it can
> > never be used again.
> >
> > If you rehydrate it just before it is used then you will repeat work
> > which has already been done. If you do this, you will always have a
> > trade-off between space used and runtime.
>
> Oops. Yes, I have misunderstood the idea. I thought the idea was that
> after loading a given module into the HPT, its ModDetails would start out
> small (because of laziness) and then keep growing in size as more and more
> of it are traversed, and thus forced, during the typechecking of its
> dependees, so at some point we would want to reset that into the small
> initial representation as created by initModDetails.
>
> But if the idea is that I should rehydrate modules when they can't be used
> anymore, then that brings up the question why even do that, instead of
> straight removing the HomeModInfos from the HPT?
>
> ----------------------------------------------------------------------
>
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our public website.
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/1268c59a/attachment.html>

From Gergo.Erdi at sc.com  Wed Apr  2 09:25:56 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Wed, 2 Apr 2025 09:25:56 +0000
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
Message-ID: <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

Hi Matt,

I think I have something that might demonstrate that GHC (at least GHC 9.12.1) might have a similar problem!

With the attached vacuous module hierarchy, I tried compiling M2294 from scratch, and then with `.hi` files for everything except the toplevel module. I did the same with our GHC-API-using compiler as well. As you can see from the attached event logs, while the details differ, the overall shape of the memory used by ModuleGraph edges (750k of GWIB and NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct dependency edges) is pretty much the same between our compiler and GHC 9.12, suggesting to me that GHC is duplicating ModuleGraph node information in the dependency edges when building the transitive closure.

Based on these measurements, do you agree that this is a GHC-side problem of memory usage scaling quadratically with the number of dependency edges?

Thanks,
            Gergo

p.s.: Sorry for including the reproducer module tree in this weird format as a patch file, but I am behind a mail server that won’t let me send mails with too many individual files in attached archives…

From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Friday, March 28, 2025 8:40 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/2b24b7ce/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ghc-prof.zip
Type: application/x-zip-compressed
Size: 1722494 bytes
Desc: ghc-prof.zip
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/2b24b7ce/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repro-hs.patch.gz
Type: application/x-gzip
Size: 80437 bytes
Desc: repro-hs.patch.gz
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/2b24b7ce/attachment-0003.bin>

From matthewtpickering at gmail.com  Wed Apr  2 09:38:34 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 2 Apr 2025 10:38:34 +0100
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>

What command do I run to generate the files from this patch file? Perhaps a
link to a git repo would be a suitable way to share the reproducer?

On Wed, Apr 2, 2025 at 10:26 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:

> PUBLIC
>
> Hi Matt,
>
>
>
> I think I have something that might demonstrate that GHC (at least GHC
> 9.12.1) might have a similar problem!
>
>
>
> With the attached vacuous module hierarchy, I tried compiling M2294 from
> scratch, and then with `.hi` files for everything except the toplevel
> module. I did the same with our GHC-API-using compiler as well. As you can
> see from the attached event logs, while the details differ, the overall
> shape of the memory used by ModuleGraph edges (750k of GWIB and
> NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct
> dependency edges) is pretty much the same between our compiler and GHC
> 9.12, suggesting to me that GHC is duplicating ModuleGraph node information
> in the dependency edges when building the transitive closure.
>
>
>
> Based on these measurements, do you agree that this is a GHC-side problem
> of memory usage scaling quadratically with the number of dependency edges?
>
>
>
> Thanks,
>
>             Gergo
>
>
>
> p.s.: Sorry for including the reproducer module tree in this weird format
> as a patch file, but I am behind a mail server that won’t let me send mails
> with too many individual files in attached archives…
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Friday, March 28, 2025 8:40 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
> HI Gergo,
>
>
>
> Do you have a (synthetic?) reproducer? You have probably identified some
> memory leak. However, without any means to reproduce it becomes very
> difficult to investigate. I feel like we are getting into very precise
> details now, where speculating is not going to be so useful.
>
>
>
> It seems like this is an important thing for you and your company. Is
> there any budget to pay for some investigation? If that was the case then
> some effort could be made to create a synthetic producer and make the
> situation more robust going into the future if your requirements were
> precisely understood.
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
> Just to add that I get the same "equalizing" behaviour (but in a more
> "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just
> call `hugInstancesBelow` before processing each module. So that's
> definitely one source of extra memory usage. I wonder if it would be
> possible to rebuild the ModuleGraph periodically (similar to the ModDetails
> dehydration), or if there are references to it stored all over the place
> from `HscEnv`s scattered around in closures etc. (basically the same
> problem the HPT had before it was made into a mutable reference).
>
> -----Original Message-----
> From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Erdi, Gergo
> via ghc-devs
> Sent: Friday, March 28, 2025 4:49 PM
> To: Matthew Pickering <matthewtpickering at gmail.com>; GHC Devs <
> ghc-devs at haskell.org>
> Cc: ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <
> Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> Hi,
>
> Unfortunately, I am forced to return to this problem. Everything below is
> now in the context of GHC 9.12 plus the mutable HPT patch backported.
>
> My test case is typechecking a tree of 2294 modules that form the
> transitive closure of a single module's dependencies, all in a single
> process. I have done this typechecking three times, here's what `+RTS -s
> -RTS` reports for max residency:
>
> * "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB
>
> * "cold-top": With all `ModIface`s already on disk, except for the
>   single top-level module: 302 MB
>
> * "warm": With all `ModIface`s already on disk: 211 MB
>
> So my stupidly naive question is, why is the "cold" case also not 302 MB?
>
> In earlier discussion, `ModDetails` unfolding has come up. Dehydrating
> `ModDetails` in the HPT all the time is disastrous for runtime, but based
> on this model I would expect to see improvements from dehydrating "every
> now and then". So I tried a stupid simple example where after every 100th
> typechecked module, I run this function on the topologically sorted list of
> modules processed so far:
>
>
> ```
> dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods
> = do
>     let HPT{ table = hptr } = hsc_HPT hsc_env
>     hpt <- readIORef hptr
>     for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface
> _details _linkable) -> do
>         !details <- initModDetails hsc_env iface
>         pure ()
> ```
>
> Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact,
> the profile looks exactly the same.
>
> Speaking of the profile, in the "cold" case I see a lot of steadily
> increasing heap usage from the `ModuleGraph`. I could see this happening if
> typechecking from scratch involves more `modulesUnder` calls which in turn
> force more and more of the `ModuleGraph`. If so, then maybe this could be
> worked around by repeatedly remaking the `ModuleGraph` just like I remake
> the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing
> the `ModuleGraph` at the start, with the idea being that this should
> "equalize" the three scenarios if this really is a substantial source of
> extra memory usage. This pushes up the warm case's memory usage to 381 MB,
> which is promising, but I still see a `Word64Map` that is steadily
> increasing in the "cold-force-modulegraph" case and contributes a lot to
> the memory usage. Unfortunately, I don't know where that `Word64Map` is (it
> could be any `Unique`-keyed environment...).
>
> So I am now stuck at this point. To spell out my goal explicitly, I would
> like to typecheck one module after another and not keep anything more in
> memory around than if I loaded them from `ModIface` files.
>
> Thanks,
>         Gergo
>
> p.s.: I couldn't find a way in the EventLog output HTML to turn event
> markers on/off or filter them, so to avoid covering the whole graph with
> gray lines, I mark only every 100th module.
>
>
>
>
> From: Matthew Pickering <matthewtpickering at gmail.com>
> Sent: Wednesday, February 12, 2025 7:08 PM
> To: ÉRDI Gergő <gergo at erdi.hu>
> Cc: Erdi, Gergo <Gergo.Erdi at sc.com>; Zubin Duggal <zubin at well-typed.com>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; GHC Devs <
> ghc-devs at haskell.org>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> You do also raise a good point about rehydration costs.
>
> In oneshot mode, you are basically rehydrating the entire transitive
> closure of each module when you compile it, which obviously results in a
> large amount of repeated work. This is why people are investigating ideas
> of a persistent worker to at least avoid rehydrating all external
> dependencies as well.
>
> On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:
> matthewtpickering at gmail.com> wrote:
> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu> wrote:
> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>
> > I wonder if you have got your condition the wrong way around.
> >
> > The only "safe" time to perform rehydration is AFTER the point it can
> > never be used again.
> >
> > If you rehydrate it just before it is used then you will repeat work
> > which has already been done. If you do this, you will always have a
> > trade-off between space used and runtime.
>
> Oops. Yes, I have misunderstood the idea. I thought the idea was that
> after loading a given module into the HPT, its ModDetails would start out
> small (because of laziness) and then keep growing in size as more and more
> of it are traversed, and thus forced, during the typechecking of its
> dependees, so at some point we would want to reset that into the small
> initial representation as created by initModDetails.
>
> But if the idea is that I should rehydrate modules when they can't be used
> anymore, then that brings up the question why even do that, instead of
> straight removing the HomeModInfos from the HPT?
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our public website.
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/1ed2a0c0/attachment.html>

From Gergo.Erdi at sc.com  Wed Apr  2 09:47:30 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Wed, 2 Apr 2025 09:47:30 +0000
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>
Message-ID: <SEYPR03MB7463AA22D073E313B431B865E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

zcat ../repro-hs.patch.gz |patch -p0

From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Wednesday, April 2, 2025 5:39 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces


What command do I run to generate the files from this patch file? Perhaps a link to a git repo would be a suitable way to share the reproducer?

On Wed, Apr 2, 2025 at 10:26 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:

PUBLIC

Hi Matt,

I think I have something that might demonstrate that GHC (at least GHC 9.12.1) might have a similar problem!

With the attached vacuous module hierarchy, I tried compiling M2294 from scratch, and then with `.hi` files for everything except the toplevel module. I did the same with our GHC-API-using compiler as well. As you can see from the attached event logs, while the details differ, the overall shape of the memory used by ModuleGraph edges (750k of GWIB and NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct dependency edges) is pretty much the same between our compiler and GHC 9.12, suggesting to me that GHC is duplicating ModuleGraph node information in the dependency edges when building the transitive closure.

Based on these measurements, do you agree that this is a GHC-side problem of memory usage scaling quadratically with the number of dependency edges?

Thanks,
            Gergo

p.s.: Sorry for including the reproducer module tree in this weird format as a patch file, but I am behind a mail server that won’t let me send mails with too many individual files in attached archives…

From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Friday, March 28, 2025 8:40 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
________________________________
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/cebeb4f6/attachment.html>

From matthewtpickering at gmail.com  Wed Apr  2 10:02:42 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 2 Apr 2025 11:02:42 +0100
Subject: GHC memory usage when typechecking from source vs. loading
 ModIfaces
In-Reply-To: <SEYPR03MB7463AA22D073E313B431B865E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>
 <SEYPR03MB7463AA22D073E313B431B865E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <CALuQ0m88xhEMWG=OSeKSOo4f2Dm7nEgGkFo-7cKX9txAK=cNKA@mail.gmail.com>

I think you are missing
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13593

On HEAD I get maximum residency of about 200M, NodeKey usage is constant.

On 9.10.1, I get maximum residency of 400M, NodeKey usage looks quadratic.



On Wed, Apr 2, 2025 at 10:47 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:

> PUBLIC
>
> zcat ../repro-hs.patch.gz |patch -p0
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Wednesday, April 2, 2025 5:39 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
>
>
> What command do I run to generate the files from this patch file? Perhaps
> a link to a git repo would be a suitable way to share the reproducer?
>
>
>
> On Wed, Apr 2, 2025 at 10:26 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
>
>
> Hi Matt,
>
>
>
> I think I have something that might demonstrate that GHC (at least GHC
> 9.12.1) might have a similar problem!
>
>
>
> With the attached vacuous module hierarchy, I tried compiling M2294 from
> scratch, and then with `.hi` files for everything except the toplevel
> module. I did the same with our GHC-API-using compiler as well. As you can
> see from the attached event logs, while the details differ, the overall
> shape of the memory used by ModuleGraph edges (750k of GWIB and
> NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct
> dependency edges) is pretty much the same between our compiler and GHC
> 9.12, suggesting to me that GHC is duplicating ModuleGraph node information
> in the dependency edges when building the transitive closure.
>
>
>
> Based on these measurements, do you agree that this is a GHC-side problem
> of memory usage scaling quadratically with the number of dependency edges?
>
>
>
> Thanks,
>
>             Gergo
>
>
>
> p.s.: Sorry for including the reproducer module tree in this weird format
> as a patch file, but I am behind a mail server that won’t let me send mails
> with too many individual files in attached archives…
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Friday, March 28, 2025 8:40 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
> HI Gergo,
>
>
>
> Do you have a (synthetic?) reproducer? You have probably identified some
> memory leak. However, without any means to reproduce it becomes very
> difficult to investigate. I feel like we are getting into very precise
> details now, where speculating is not going to be so useful.
>
>
>
> It seems like this is an important thing for you and your company. Is
> there any budget to pay for some investigation? If that was the case then
> some effort could be made to create a synthetic producer and make the
> situation more robust going into the future if your requirements were
> precisely understood.
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
> Just to add that I get the same "equalizing" behaviour (but in a more
> "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just
> call `hugInstancesBelow` before processing each module. So that's
> definitely one source of extra memory usage. I wonder if it would be
> possible to rebuild the ModuleGraph periodically (similar to the ModDetails
> dehydration), or if there are references to it stored all over the place
> from `HscEnv`s scattered around in closures etc. (basically the same
> problem the HPT had before it was made into a mutable reference).
>
> -----Original Message-----
> From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Erdi, Gergo
> via ghc-devs
> Sent: Friday, March 28, 2025 4:49 PM
> To: Matthew Pickering <matthewtpickering at gmail.com>; GHC Devs <
> ghc-devs at haskell.org>
> Cc: ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <
> Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> Hi,
>
> Unfortunately, I am forced to return to this problem. Everything below is
> now in the context of GHC 9.12 plus the mutable HPT patch backported.
>
> My test case is typechecking a tree of 2294 modules that form the
> transitive closure of a single module's dependencies, all in a single
> process. I have done this typechecking three times, here's what `+RTS -s
> -RTS` reports for max residency:
>
> * "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB
>
> * "cold-top": With all `ModIface`s already on disk, except for the
>   single top-level module: 302 MB
>
> * "warm": With all `ModIface`s already on disk: 211 MB
>
> So my stupidly naive question is, why is the "cold" case also not 302 MB?
>
> In earlier discussion, `ModDetails` unfolding has come up. Dehydrating
> `ModDetails` in the HPT all the time is disastrous for runtime, but based
> on this model I would expect to see improvements from dehydrating "every
> now and then". So I tried a stupid simple example where after every 100th
> typechecked module, I run this function on the topologically sorted list of
> modules processed so far:
>
>
> ```
> dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods
> = do
>     let HPT{ table = hptr } = hsc_HPT hsc_env
>     hpt <- readIORef hptr
>     for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface
> _details _linkable) -> do
>         !details <- initModDetails hsc_env iface
>         pure ()
> ```
>
> Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact,
> the profile looks exactly the same.
>
> Speaking of the profile, in the "cold" case I see a lot of steadily
> increasing heap usage from the `ModuleGraph`. I could see this happening if
> typechecking from scratch involves more `modulesUnder` calls which in turn
> force more and more of the `ModuleGraph`. If so, then maybe this could be
> worked around by repeatedly remaking the `ModuleGraph` just like I remake
> the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing
> the `ModuleGraph` at the start, with the idea being that this should
> "equalize" the three scenarios if this really is a substantial source of
> extra memory usage. This pushes up the warm case's memory usage to 381 MB,
> which is promising, but I still see a `Word64Map` that is steadily
> increasing in the "cold-force-modulegraph" case and contributes a lot to
> the memory usage. Unfortunately, I don't know where that `Word64Map` is (it
> could be any `Unique`-keyed environment...).
>
> So I am now stuck at this point. To spell out my goal explicitly, I would
> like to typecheck one module after another and not keep anything more in
> memory around than if I loaded them from `ModIface` files.
>
> Thanks,
>         Gergo
>
> p.s.: I couldn't find a way in the EventLog output HTML to turn event
> markers on/off or filter them, so to avoid covering the whole graph with
> gray lines, I mark only every 100th module.
>
>
>
>
> From: Matthew Pickering <matthewtpickering at gmail.com>
> Sent: Wednesday, February 12, 2025 7:08 PM
> To: ÉRDI Gergő <gergo at erdi.hu>
> Cc: Erdi, Gergo <Gergo.Erdi at sc.com>; Zubin Duggal <zubin at well-typed.com>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; GHC Devs <
> ghc-devs at haskell.org>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> You do also raise a good point about rehydration costs.
>
> In oneshot mode, you are basically rehydrating the entire transitive
> closure of each module when you compile it, which obviously results in a
> large amount of repeated work. This is why people are investigating ideas
> of a persistent worker to at least avoid rehydrating all external
> dependencies as well.
>
> On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:
> matthewtpickering at gmail.com> wrote:
> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu> wrote:
> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>
> > I wonder if you have got your condition the wrong way around.
> >
> > The only "safe" time to perform rehydration is AFTER the point it can
> > never be used again.
> >
> > If you rehydrate it just before it is used then you will repeat work
> > which has already been done. If you do this, you will always have a
> > trade-off between space used and runtime.
>
> Oops. Yes, I have misunderstood the idea. I thought the idea was that
> after loading a given module into the HPT, its ModDetails would start out
> small (because of laziness) and then keep growing in size as more and more
> of it are traversed, and thus forced, during the typechecking of its
> dependees, so at some point we would want to reset that into the small
> initial representation as created by initModDetails.
>
> But if the idea is that I should rehydrate modules when they can't be used
> anymore, then that brings up the question why even do that, instead of
> straight removing the HomeModInfos from the HPT?
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our public website.
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
>
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/e55205f6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2025-04-02-110039_1254x783.png
Type: image/png
Size: 138132 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/e55205f6/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2025-04-02-110049_1676x558.png
Type: image/png
Size: 95386 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/e55205f6/attachment-0003.png>

From george.colpitts at gmail.com  Wed Apr  2 12:06:05 2025
From: george.colpitts at gmail.com (George Colpitts)
Date: Wed, 2 Apr 2025 09:06:05 -0300
Subject: can't move to latest version of llvm on ghc 9.12.2
In-Reply-To: <4db7eafd-69b0-476c-bffd-e2a8827d7346@gmx.at>
References: <CAB-d4A5WCbBX-G+3RvV8AQ_+5_-fR2P8ED7qO-14PhwMVdKY-g@mail.gmail.com>
 <CAKFCL4WDj31se4=nEo4ts8-OwQRUwaXioncPgB1PUgvA2q1HKQ@mail.gmail.com>
 <CAB-d4A4w-ghx4hCmizaw0Ebdye2i0ohe+KbP06XuQJUx0vpn3Q@mail.gmail.com>
 <4db7eafd-69b0-476c-bffd-e2a8827d7346@gmx.at>
Message-ID: <CAB-d4A4teaamw8dOkR1wLjYhPW0uknorvDJyYVnp0gag+No3QQ@mail.gmail.com>

Filed https://gitlab.haskell.org/ghc/ghc/-/issues/25915

On Tue, Apr 1, 2025 at 2:11 PM Andreas Klebinger via ghc-devs <
ghc-devs at haskell.org> wrote:

> Looking at https://gitlab.haskell.org/ghc/ghc/-/issues/25011 it seems the
> desire was to fail with a proper error when LLVM was not found at all.
>
> To me failing when the llvm version is too new seems like a unintended
> side effect of fixing the former.
>
> I agree that simply warning and trying anway is the better default. A
> ticket would be appreciated!
> Am 01/04/2025 um 19:05 schrieb George Colpitts:
>
> I was trying to find out if it works but I ran into the problem I
> described. Yes, there were a few years where the latest version didn't work
> but in many years it did. ghc used to give a warning, this is an
> unsupported version but we'll try it anyways. I would like to return to
> that.
>
> In any case I believe ghc dev should test to see if it works and if so use
> it in HEAD. If it doesn't work than we should file a bug and fix it. In the
> past we got a few versions behind llvm. I think we want to be on the latest
> version available for each new release if possible. The earlier we look
> into this the more chance we have to succeed at that.
>
> On Tue, Apr 1, 2025 at 1:27 PM Brandon Allbery <allbery.b at gmail.com>
> wrote:
>
>> Have you demonstrated that it works? IIRC the current behavior is because
>> some LLVM version (16, IIRC) didn't work with GHC (threw errors from opt, I
>> think).
>>
>> On Tue, Apr 1, 2025 at 10:49 AM George Colpitts <
>> george.colpitts at gmail.com> wrote:
>>
>>> llvm 20 is out but unlike in earlier versions of ghc moving to it means
>>> you can no longer use llvm:
>>>
>>> compiling:
>>>
>>> ghc -fllvm hello.hs
>>> Loaded package environment from
>>> /Users/avie/.ghc/aarch64-darwin-9.12.2/environments/default
>>> [1 of 2] Compiling Main             ( hello.hs, hello.o )
>>> <no location info>: error: [GHC-66599]
>>>     GHC was not configured with a supported LLVM toolchain
>>>     Make sure you have installed LLVM between [13 and 20) and reinstall
>>> GHC to make -fllvm work
>>>
>>>
>>> from configure:
>>>
>>> configure: We only support llvm 13 upto 20 (non-inclusive) (found
>>> 20.1.1).
>>>
>>>
>>> Can we move to llvm 20 on HEAD and can we revert to the old behavior on
>>> ghc 9.12.3?
>>>
>>> Should I file an ER?
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> brandon s allbery kf8nh
>> allbery.b at gmail.com
>>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/925cd23b/attachment.html>

From teofilcamarasu at gmail.com  Wed Apr  2 17:26:02 2025
From: teofilcamarasu at gmail.com (Teo Camarasu)
Date: Wed, 2 Apr 2025 18:26:02 +0100
Subject: Next steps forward for a stable Template Haskell
Message-ID: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>

Hi folks,

At Tuesday's GHC devs meeting, we talked about having a discussion about
template-haskell stability. I've jotted down some notes to help focus our
future discussion. It's mostly just a list of what I take to be the next
most important issues to tackle, and some details about each.

https://docs.google.com/document/d/1gui1YXOvrNihR3NpXuMVCKV3lTkKPL8y_HDRjbZyRxk/edit?usp=sharing

Cheers,
Teo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250402/07f0d46c/attachment.html>

From Gergo.Erdi at sc.com  Thu Apr  3 10:54:11 2025
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Thu, 3 Apr 2025 10:54:11 +0000
Subject: [External] Re: GHC memory usage when typechecking from source vs.
 loading ModIfaces
In-Reply-To: <CALuQ0m88xhEMWG=OSeKSOo4f2Dm7nEgGkFo-7cKX9txAK=cNKA@mail.gmail.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>
 <SEYPR03MB7463AA22D073E313B431B865E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m88xhEMWG=OSeKSOo4f2Dm7nEgGkFo-7cKX9txAK=cNKA@mail.gmail.com>
Message-ID: <SEYPR03MB74634F6071582CC556AF980AE9AE2@SEYPR03MB7463.apcprd03.prod.outlook.com>

PUBLIC

Yes I was, since I am using 9.12.1.

This is great stuff -- with !13593 backported, I don’t see any ModuleGraph-related stuff showing up in top spots of the heap profile! Thank you for pointing me towards this!

I’ll have to do some more digging to see if there is anything else concretely blameable now. But in theory, morally, abstractly, from the 10,000 feet view, would it be correct to say that there should be no memory usage difference between typechecking a large number of modules from scratch vs. loading the `.hi` files for those same modules? (Modulo, of course, the memory usage during any single module’s typechecking).

From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Wednesday, April 2, 2025 6:03 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

I think you are missing https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13593<https://urldefense.com/v3/__https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13593__;!!ASp95G87aa5DoyK5mB3l!736R3M2loLHwk60BLtBarIhVc0mTMNZ41vyzyjTmqQJ81DibLFqtrJNUeZLxM8YOIGmg8SbzoAYXhGK97EyBWDm9$>

On HEAD I get maximum residency of about 200M, NodeKey usage is constant.

On 9.10.1, I get maximum residency of 400M, NodeKey usage looks quadratic.



On Wed, Apr 2, 2025 at 10:47 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:

PUBLIC

zcat ../repro-hs.patch.gz |patch -p0

From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, April 2, 2025 5:39 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces


What command do I run to generate the files from this patch file? Perhaps a link to a git repo would be a suitable way to share the reproducer?

On Wed, Apr 2, 2025 at 10:26 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:

PUBLIC

Hi Matt,

I think I have something that might demonstrate that GHC (at least GHC 9.12.1) might have a similar problem!

With the attached vacuous module hierarchy, I tried compiling M2294 from scratch, and then with `.hi` files for everything except the toplevel module. I did the same with our GHC-API-using compiler as well. As you can see from the attached event logs, while the details differ, the overall shape of the memory used by ModuleGraph edges (750k of GWIB and NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct dependency edges) is pretty much the same between our compiler and GHC 9.12, suggesting to me that GHC is duplicating ModuleGraph node information in the dependency edges when building the transitive closure.

Based on these measurements, do you agree that this is a GHC-side problem of memory usage scaling quadratically with the number of dependency edges?

Thanks,
            Gergo

p.s.: Sorry for including the reproducer module tree in this weird format as a patch file, but I am behind a mail server that won’t let me send mails with too many individual files in attached archives…

From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Friday, March 28, 2025 8:40 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>; ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

HI Gergo,

Do you have a (synthetic?) reproducer? You have probably identified some memory leak. However, without any means to reproduce it becomes very difficult to investigate. I feel like we are getting into very precise details now, where speculating is not going to be so useful.

It seems like this is an important thing for you and your company. Is there any budget to pay for some investigation? If that was the case then some effort could be made to create a synthetic producer and make the situation more robust going into the future if your requirements were precisely understood.

Cheers,

Matt

On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>> wrote:
PUBLIC

Just to add that I get the same "equalizing" behaviour (but in a more "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just call `hugInstancesBelow` before processing each module. So that's definitely one source of extra memory usage. I wonder if it would be possible to rebuild the ModuleGraph periodically (similar to the ModDetails dehydration), or if there are references to it stored all over the place from `HscEnv`s scattered around in closures etc. (basically the same problem the HPT had before it was made into a mutable reference).

-----Original Message-----
From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Erdi, Gergo via ghc-devs
Sent: Friday, March 28, 2025 4:49 PM
To: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Cc: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; Dijkstra, Atze <Atze.Dijkstra at sc.com<mailto:Atze.Dijkstra at sc.com>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

Hi,

Unfortunately, I am forced to return to this problem. Everything below is now in the context of GHC 9.12 plus the mutable HPT patch backported.

My test case is typechecking a tree of 2294 modules that form the transitive closure of a single module's dependencies, all in a single process. I have done this typechecking three times, here's what `+RTS -s -RTS` reports for max residency:

* "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB

* "cold-top": With all `ModIface`s already on disk, except for the
  single top-level module: 302 MB

* "warm": With all `ModIface`s already on disk: 211 MB

So my stupidly naive question is, why is the "cold" case also not 302 MB?

In earlier discussion, `ModDetails` unfolding has come up. Dehydrating `ModDetails` in the HPT all the time is disastrous for runtime, but based on this model I would expect to see improvements from dehydrating "every now and then". So I tried a stupid simple example where after every 100th typechecked module, I run this function on the topologically sorted list of modules processed so far:


```
dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods = do
    let HPT{ table = hptr } = hsc_HPT hsc_env
    hpt <- readIORef hptr
    for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface _details _linkable) -> do
        !details <- initModDetails hsc_env iface
        pure ()
```

Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact, the profile looks exactly the same.

Speaking of the profile, in the "cold" case I see a lot of steadily increasing heap usage from the `ModuleGraph`. I could see this happening if typechecking from scratch involves more `modulesUnder` calls which in turn force more and more of the `ModuleGraph`. If so, then maybe this could be worked around by repeatedly remaking the `ModuleGraph` just like I remake the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing the `ModuleGraph` at the start, with the idea being that this should "equalize" the three scenarios if this really is a substantial source of extra memory usage. This pushes up the warm case's memory usage to 381 MB, which is promising, but I still see a `Word64Map` that is steadily increasing in the "cold-force-modulegraph" case and contributes a lot to the memory usage. Unfortunately, I don't know where that `Word64Map` is (it could be any `Unique`-keyed environment...).

So I am now stuck at this point. To spell out my goal explicitly, I would like to typecheck one module after another and not keep anything more in memory around than if I loaded them from `ModIface` files.

Thanks,
        Gergo

p.s.: I couldn't find a way in the EventLog output HTML to turn event markers on/off or filter them, so to avoid covering the whole graph with gray lines, I mark only every 100th module.




From: Matthew Pickering <matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>>
Sent: Wednesday, February 12, 2025 7:08 PM
To: ÉRDI Gergő <gergo at erdi.hu<mailto:gergo at erdi.hu>>
Cc: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>; Zubin Duggal <zubin at well-typed.com<mailto:zubin at well-typed.com>>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC Devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] Re: GHC memory usage when typechecking from source vs. loading ModIfaces

You do also raise a good point about rehydration costs.

In oneshot mode, you are basically rehydrating the entire transitive closure of each module when you compile it, which obviously results in a large amount of repeated work. This is why people are investigating ideas of a persistent worker to at least avoid rehydrating all external dependencies as well.

On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:matthewtpickering at gmail.com<mailto:matthewtpickering at gmail.com>> wrote:
Sure, you can remove them once you are sure they are not used anymore.

For clients like `GHCi` that doesn't work obviously as they can be used at any point in the future but for a batch compiler it would be fine.

On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu<mailto:gergo at erdi.hu>> wrote:
On Mon, 10 Feb 2025, Matthew Pickering wrote:

> I wonder if you have got your condition the wrong way around.
>
> The only "safe" time to perform rehydration is AFTER the point it can
> never be used again.
>
> If you rehydrate it just before it is used then you will repeat work
> which has already been done. If you do this, you will always have a
> trade-off between space used and runtime.

Oops. Yes, I have misunderstood the idea. I thought the idea was that after loading a given module into the HPT, its ModDetails would start out small (because of laziness) and then keep growing in size as more and more of it are traversed, and thus forced, during the typechecking of its dependees, so at some point we would want to reset that into the small initial representation as created by initModDetails.

But if the idea is that I should rehydrate modules when they can't be used anymore, then that brings up the question why even do that, instead of straight removing the HomeModInfos from the HPT?

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our public website.

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
________________________________
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
________________________________
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com

----------------------------------------------------------------------
This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries together with Standard Chartered Bank’s Privacy Policy via our main Standard Chartered PLC (UK) website at sc. com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250403/7c5f56ad/attachment-0001.html>

From matthewtpickering at gmail.com  Thu Apr  3 11:01:33 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Thu, 3 Apr 2025 12:01:33 +0100
Subject: [External] Re: GHC memory usage when typechecking from source vs.
 loading ModIfaces
In-Reply-To: <SEYPR03MB74634F6071582CC556AF980AE9AE2@SEYPR03MB7463.apcprd03.prod.outlook.com>
References: <CAO7EwMoZPheo=ubP6S5VkTNEuWxSP-BiG0NkfJnMPwA_sg9AQQ@mail.gmail.com>
 <SEYPR03MB74634DE2A8BDE8A481B919AAE91B2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <pom2vzn7ff2ntiwn7gub7tj2d7dghdfrcqqwjqylrstsv4kvuh@fnspm4m4ji3e>
 <CALuQ0m-fn3rTRs1PgK1WdoV3F8FJnF28HST5aOeEdyrK1L8JyQ@mail.gmail.com>
 <4f980efd-58bc-2814-b4cf-b5742c040b97@erdi.hu>
 <CALuQ0m-6RYuJAMT0kw272E2-R_9LsbJ4ZJPpr7m64ymYnaREVg@mail.gmail.com>
 <SEYPR03MB7463A4ED0988986744263839E9E02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m9FXh4fzhmVRBiFW82rS3W4dPuDnH+7u6yqs3Gwff=B_Q@mail.gmail.com>
 <SEYPR03MB74638EDF52238E57708B6C0EE9F72@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m8VJKXz3y3y3Mv+dK1pFdY2DDVgdWM_oTGPCC51JL5jTg@mail.gmail.com>
 <3a5dd981-7fdc-a814-88c9-6768395cae39@erdi.hu>
 <CALuQ0m__FtS-cfz4TcWXmv2JV1r4-sz1e73RK+JPridQTB0EVQ@mail.gmail.com>
 <CALuQ0m-3iB+2oY1x45FhT+G_OypYo2T0gyzjXUoo3yvm8H6JmA@mail.gmail.com>
 <SEYPR03MB7463AABC8385A5FCCCAC330CE9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <SEYPR03MB7463EC99669F8723A3156872E9A02@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_FHoDPF8ZV+-441AS22bQbLDrS9GsX-+vaUYD9okrKXA@mail.gmail.com>
 <SEYPR03MB7463223D4085BA61D2B64FE7E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m_R_oerbKZ43t+1-RA_9R7ec9D3=O7ZhUECohsC3EAXmQ@mail.gmail.com>
 <SEYPR03MB7463AA22D073E313B431B865E9AF2@SEYPR03MB7463.apcprd03.prod.outlook.com>
 <CALuQ0m88xhEMWG=OSeKSOo4f2Dm7nEgGkFo-7cKX9txAK=cNKA@mail.gmail.com>
 <SEYPR03MB74634F6071582CC556AF980AE9AE2@SEYPR03MB7463.apcprd03.prod.outlook.com>
Message-ID: <CALuQ0m_LFQdNPMWv3SWAqLHx6MDcZS3b9NokUq1CmVuF7oHbPQ@mail.gmail.com>

Yes, but

* With all the caveats around forcing ModDetails and using extra memory.
* When you typecheck from scratch you may load additional interfaces for
external dependencies into the EPS.

After a module is typechecked, everything apart from the interface is
discarded and everything is regenerated from that interface.

It's possible there are some leaks in forcing the interface, since the deep
forcing was incomplete (see
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/14078)

Anyway, best thing is to profile with these invariants in mind, and fix
situations where the invariants are violated.

Cheers,

Matt

On Thu, Apr 3, 2025 at 11:54 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:

> PUBLIC
>
> Yes I was, since I am using 9.12.1.
>
>
>
> This is great stuff -- with !13593 backported, I don’t see any
> ModuleGraph-related stuff showing up in top spots of the heap profile!
> Thank you for pointing me towards this!
>
> I’ll have to do some more digging to see if there is anything else
> concretely blameable now. But in theory, morally, abstractly, from the
> 10,000 feet view, would it be correct to say that there should be no memory
> usage difference between typechecking a large number of modules from
> scratch vs. loading the `.hi` files for those same modules? (Modulo, of
> course, the memory usage during any single module’s typechecking).
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Wednesday, April 2, 2025 6:03 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
> I think you are missing
> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13593
> <https://urldefense.com/v3/__https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13593__;!!ASp95G87aa5DoyK5mB3l!736R3M2loLHwk60BLtBarIhVc0mTMNZ41vyzyjTmqQJ81DibLFqtrJNUeZLxM8YOIGmg8SbzoAYXhGK97EyBWDm9$>
>
>
>
> On HEAD I get maximum residency of about 200M, NodeKey usage is constant.
>
>
>
> On 9.10.1, I get maximum residency of 400M, NodeKey usage looks quadratic.
>
>
>
>
>
>
>
> On Wed, Apr 2, 2025 at 10:47 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
>
>
> zcat ../repro-hs.patch.gz |patch -p0
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Wednesday, April 2, 2025 5:39 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
>
>
> What command do I run to generate the files from this patch file? Perhaps
> a link to a git repo would be a suitable way to share the reproducer?
>
>
>
> On Wed, Apr 2, 2025 at 10:26 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
>
>
> Hi Matt,
>
>
>
> I think I have something that might demonstrate that GHC (at least GHC
> 9.12.1) might have a similar problem!
>
>
>
> With the attached vacuous module hierarchy, I tried compiling M2294 from
> scratch, and then with `.hi` files for everything except the toplevel
> module. I did the same with our GHC-API-using compiler as well. As you can
> see from the attached event logs, while the details differ, the overall
> shape of the memory used by ModuleGraph edges (750k of GWIB and
> NodeKey_Module constructors for the 2321 ModuleNodes and ~60k direct
> dependency edges) is pretty much the same between our compiler and GHC
> 9.12, suggesting to me that GHC is duplicating ModuleGraph node information
> in the dependency edges when building the transitive closure.
>
>
>
> Based on these measurements, do you agree that this is a GHC-side problem
> of memory usage scaling quadratically with the number of dependency edges?
>
>
>
> Thanks,
>
>             Gergo
>
>
>
> p.s.: Sorry for including the reproducer module tree in this weird format
> as a patch file, but I am behind a mail server that won’t let me send mails
> with too many individual files in attached archives…
>
>
>
> *From:* Matthew Pickering <matthewtpickering at gmail.com>
> *Sent:* Friday, March 28, 2025 8:40 PM
> *To:* Erdi, Gergo <Gergo.Erdi at sc.com>
> *Cc:* GHC Devs <ghc-devs at haskell.org>; ÉRDI Gergő <gergo at erdi.hu>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; Dijkstra, Atze
> <Atze.Dijkstra at sc.com>
> *Subject:* [External] Re: GHC memory usage when typechecking from source
> vs. loading ModIfaces
>
>
>
> HI Gergo,
>
>
>
> Do you have a (synthetic?) reproducer? You have probably identified some
> memory leak. However, without any means to reproduce it becomes very
> difficult to investigate. I feel like we are getting into very precise
> details now, where speculating is not going to be so useful.
>
>
>
> It seems like this is an important thing for you and your company. Is
> there any budget to pay for some investigation? If that was the case then
> some effort could be made to create a synthetic producer and make the
> situation more robust going into the future if your requirements were
> precisely understood.
>
>
>
> Cheers,
>
>
>
> Matt
>
>
>
> On Fri, Mar 28, 2025 at 10:12 AM Erdi, Gergo <Gergo.Erdi at sc.com> wrote:
>
> PUBLIC
>
> Just to add that I get the same "equalizing" behaviour (but in a more
> "natural" way) if instead of deepseq-ing the ModuleGraph upfront, I just
> call `hugInstancesBelow` before processing each module. So that's
> definitely one source of extra memory usage. I wonder if it would be
> possible to rebuild the ModuleGraph periodically (similar to the ModDetails
> dehydration), or if there are references to it stored all over the place
> from `HscEnv`s scattered around in closures etc. (basically the same
> problem the HPT had before it was made into a mutable reference).
>
> -----Original Message-----
> From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Erdi, Gergo
> via ghc-devs
> Sent: Friday, March 28, 2025 4:49 PM
> To: Matthew Pickering <matthewtpickering at gmail.com>; GHC Devs <
> ghc-devs at haskell.org>
> Cc: ÉRDI Gergő <gergo at erdi.hu>; Montelatici, Raphael Laurent <
> Raphael.Montelatici at sc.com>; Dijkstra, Atze <Atze.Dijkstra at sc.com>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> Hi,
>
> Unfortunately, I am forced to return to this problem. Everything below is
> now in the context of GHC 9.12 plus the mutable HPT patch backported.
>
> My test case is typechecking a tree of 2294 modules that form the
> transitive closure of a single module's dependencies, all in a single
> process. I have done this typechecking three times, here's what `+RTS -s
> -RTS` reports for max residency:
>
> * "cold": With no on-disk `ModIface` files, i.e. from scratch: 537 MB
>
> * "cold-top": With all `ModIface`s already on disk, except for the
>   single top-level module: 302 MB
>
> * "warm": With all `ModIface`s already on disk: 211 MB
>
> So my stupidly naive question is, why is the "cold" case also not 302 MB?
>
> In earlier discussion, `ModDetails` unfolding has come up. Dehydrating
> `ModDetails` in the HPT all the time is disastrous for runtime, but based
> on this model I would expect to see improvements from dehydrating "every
> now and then". So I tried a stupid simple example where after every 100th
> typechecked module, I run this function on the topologically sorted list of
> modules processed so far:
>
>
> ```
> dehydrateHpt :: HscEnv -> [ModuleName] -> IO () dehydrateHpt hsc_env mods
> = do
>     let HPT{ table = hptr } = hsc_HPT hsc_env
>     hpt <- readIORef hptr
>     for_ mods \mod -> for_ (lookupUDFM hpt mod) \(HomeModInfo iface
> _details _linkable) -> do
>         !details <- initModDetails hsc_env iface
>         pure ()
> ```
>
> Buuut the max residency is still 534 MB (see "cold-dehydrate"); in fact,
> the profile looks exactly the same.
>
> Speaking of the profile, in the "cold" case I see a lot of steadily
> increasing heap usage from the `ModuleGraph`. I could see this happening if
> typechecking from scratch involves more `modulesUnder` calls which in turn
> force more and more of the `ModuleGraph`. If so, then maybe this could be
> worked around by repeatedly remaking the `ModuleGraph` just like I remake
> the `ModDetails` above. I tried getting rid of this effect by `deepseq`'ing
> the `ModuleGraph` at the start, with the idea being that this should
> "equalize" the three scenarios if this really is a substantial source of
> extra memory usage. This pushes up the warm case's memory usage to 381 MB,
> which is promising, but I still see a `Word64Map` that is steadily
> increasing in the "cold-force-modulegraph" case and contributes a lot to
> the memory usage. Unfortunately, I don't know where that `Word64Map` is (it
> could be any `Unique`-keyed environment...).
>
> So I am now stuck at this point. To spell out my goal explicitly, I would
> like to typecheck one module after another and not keep anything more in
> memory around than if I loaded them from `ModIface` files.
>
> Thanks,
>         Gergo
>
> p.s.: I couldn't find a way in the EventLog output HTML to turn event
> markers on/off or filter them, so to avoid covering the whole graph with
> gray lines, I mark only every 100th module.
>
>
>
>
> From: Matthew Pickering <matthewtpickering at gmail.com>
> Sent: Wednesday, February 12, 2025 7:08 PM
> To: ÉRDI Gergő <gergo at erdi.hu>
> Cc: Erdi, Gergo <Gergo.Erdi at sc.com>; Zubin Duggal <zubin at well-typed.com>;
> Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; GHC Devs <
> ghc-devs at haskell.org>
> Subject: [External] Re: GHC memory usage when typechecking from source vs.
> loading ModIfaces
>
> You do also raise a good point about rehydration costs.
>
> In oneshot mode, you are basically rehydrating the entire transitive
> closure of each module when you compile it, which obviously results in a
> large amount of repeated work. This is why people are investigating ideas
> of a persistent worker to at least avoid rehydrating all external
> dependencies as well.
>
> On Mon, Feb 10, 2025 at 12:13 PM Matthew Pickering <mailto:
> matthewtpickering at gmail.com> wrote:
> Sure, you can remove them once you are sure they are not used anymore.
>
> For clients like `GHCi` that doesn't work obviously as they can be used at
> any point in the future but for a batch compiler it would be fine.
>
> On Mon, Feb 10, 2025 at 11:56 AM ÉRDI Gergő <mailto:gergo at erdi.hu> wrote:
> On Mon, 10 Feb 2025, Matthew Pickering wrote:
>
> > I wonder if you have got your condition the wrong way around.
> >
> > The only "safe" time to perform rehydration is AFTER the point it can
> > never be used again.
> >
> > If you rehydrate it just before it is used then you will repeat work
> > which has already been done. If you do this, you will always have a
> > trade-off between space used and runtime.
>
> Oops. Yes, I have misunderstood the idea. I thought the idea was that
> after loading a given module into the HPT, its ModDetails would start out
> small (because of laziness) and then keep growing in size as more and more
> of it are traversed, and thus forced, during the typechecking of its
> dependees, so at some point we would want to reset that into the small
> initial representation as created by initModDetails.
>
> But if the idea is that I should rehydrate modules when they can't be used
> anymore, then that brings up the question why even do that, instead of
> straight removing the HomeModInfos from the HPT?
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our public website.
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ----------------------------------------------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
>
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
>
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
> ------------------------------
> This email and any attachments are confidential and may also be
> privileged. If you are not the intended recipient, please delete all copies
> and notify the sender immediately. You may wish to refer to the
> incorporation details of Standard Chartered PLC, Standard Chartered Bank
> and their subsidiaries together with Standard Chartered Bank’s Privacy
> Policy via our main Standard Chartered PLC (UK) website at sc. com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250403/31fa6fe4/attachment.html>

From simon.peytonjones at gmail.com  Fri Apr  4 21:29:33 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 4 Apr 2025 22:29:33 +0100
Subject: Testsuite tests
Message-ID: <CAJKmMz-xohGkVvaHS3-rPCR95ZS7uo9C+DACnt3=s2CLWR4QDA@mail.gmail.com>

Friends

In CI for !10479 I'm getting test failures for some call-stack tests e.g.
`callstack001`.

But if I try to run this in my build tree

hadrian/build -j8 test --skip=_build/stage1/rts/** --skip=_build/stage1/**
--skip=_build/stage0/** --docs=none --only=callstack001

I get

SUMMARY for test run started at Fri Apr  4 22:22:47 2025
0:00:02.413942 spent to go through
       1 total tests, which gave rise to
       8 test cases, of which
       8 were skipped
       0 had missing libraries

       0 expected passes
       0 expected failures

       0 caused framework failures
       0 caused framework warnings
       0 unexpected passes
       0 unexpected failures
       0 unexpected stat failures
       0 fragile tests

So I can't reproduce the failure.  What am i doing wrong?  Can anyone help?

Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250404/b09eeea8/attachment.html>

From simon.peytonjones at gmail.com  Fri Apr  4 21:43:52 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 4 Apr 2025 22:43:52 +0100
Subject: Late CC profiling
Message-ID: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>

Hi Matthew, Andreas, Ben

In GHC today there is a strange inconsistency:

   - Bindings for data constructor wrappers and class method selectors are
   added at the end of CoreTidy.
   - Bindings for data constructor workers are added at the start of
   CorePrep

This is deeply strange.  In !10479 (unary class patch)
<https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10479> I had to make
some changes in this area and after being baffled I moved all these
implicit bindings to one place at the start of CorePrep.  That is simpler
and neater.  There is no reason for them to be separate.

BUT I find that one thing happens between Tidy and Prep, namely
late-cost-centre injection.  That means we will no longer get
late-cost-centres for data constructor wrappers and class method
selectors.  I think that this is what is causing diffs in the output for
callstack001 and friends in !10479.

I think that this is perfectly fine, and I propose to accept the changes.

Another alternative would be to inject all of these bindings at the end of
Tidy (instead of the start of Prep),  But it's much nicer at the start of
Prep -- we don't want any of these bindings to appear  in interface files.

Can you confirm that the change is ok?  Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250404/54575125/attachment.html>

From simon.peytonjones at gmail.com  Fri Apr  4 22:00:36 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 4 Apr 2025 23:00:36 +0100
Subject: Next steps forward for a stable Template Haskell
In-Reply-To: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>
References: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>
Message-ID: <CAJKmMz-BdXBPMyFdHoTbP_PuNc7_CKPexKGL-1oEn1REGkfSZQ@mail.gmail.com>

Good work!  I have added some comments.

Before long it would be good to update the Wiki roadmap doc, so there is
one source of truth.

Simon

On Wed, 2 Apr 2025 at 18:26, Teo Camarasu <teofilcamarasu at gmail.com> wrote:

> Hi folks,
>
> At Tuesday's GHC devs meeting, we talked about having a discussion about
> template-haskell stability. I've jotted down some notes to help focus our
> future discussion. It's mostly just a list of what I take to be the next
> most important issues to tackle, and some details about each.
>
>
> https://docs.google.com/document/d/1gui1YXOvrNihR3NpXuMVCKV3lTkKPL8y_HDRjbZyRxk/edit?usp=sharing
>
> Cheers,
> Teo
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250404/bd7e3459/attachment.html>

From matthewtpickering at gmail.com  Mon Apr  7 08:56:49 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Mon, 7 Apr 2025 09:56:49 +0100
Subject: Testsuite tests
In-Reply-To: <CAJKmMz-xohGkVvaHS3-rPCR95ZS7uo9C+DACnt3=s2CLWR4QDA@mail.gmail.com>
References: <CAJKmMz-xohGkVvaHS3-rPCR95ZS7uo9C+DACnt3=s2CLWR4QDA@mail.gmail.com>
Message-ID: <CALuQ0m_L9cdQ8CGa5gMc0XsbKEdZdcM6oM5srquQR-SRCV+CgA@mail.gmail.com>

It seems that you have not built profiled libraries? It's impossible to say
without knowing your custom local configuration. Could you share that?

On Fri, Apr 4, 2025 at 10:30 PM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> Friends
>
> In CI for !10479 I'm getting test failures for some call-stack tests e.g.
> `callstack001`.
>
> But if I try to run this in my build tree
>
> hadrian/build -j8 test --skip=_build/stage1/rts/** --skip=_build/stage1/**
> --skip=_build/stage0/** --docs=none --only=callstack001
>
> I get
>
> SUMMARY for test run started at Fri Apr  4 22:22:47 2025
> 0:00:02.413942 spent to go through
>        1 total tests, which gave rise to
>        8 test cases, of which
>        8 were skipped
>        0 had missing libraries
>
>        0 expected passes
>        0 expected failures
>
>        0 caused framework failures
>        0 caused framework warnings
>        0 unexpected passes
>        0 unexpected failures
>        0 unexpected stat failures
>        0 fragile tests
>
> So I can't reproduce the failure.  What am i doing wrong?  Can anyone help?
>
> Thanks
>
> Simon
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/db168c15/attachment.html>

From matthewtpickering at gmail.com  Mon Apr  7 09:08:08 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Mon, 7 Apr 2025 10:08:08 +0100
Subject: Late CC profiling
In-Reply-To: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>
References: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>
Message-ID: <CALuQ0m_8Qu4HYRYNuhe_gb9GgAk79JALM5H1sj3Y9bG4n4Lfpg@mail.gmail.com>

Hi Simon,

It does seem strange (at first glance) that these implicit things happen in
two different places.

However, I think you should do this refactoring in a separate MR to !10479,
it seems unrelated to the purpose of that patch and affects other things
like you have mentioned.

Why do you think the change in callstack is fine? It seems the difference
is `callstack001` is whether there an entry for `>>=` or not, which seems
like quite an important thing to place a cost centre on? Andreas has a
better intuition for this than me. I would quite like to know if there is
an unoptimised call to >>= in my program. You have probably thought about
this a little, so interested to hear your opinion.

Cheers,

Matt


On Fri, Apr 4, 2025 at 10:44 PM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> Hi Matthew, Andreas, Ben
>
> In GHC today there is a strange inconsistency:
>
>    - Bindings for data constructor wrappers and class method selectors
>    are added at the end of CoreTidy.
>    - Bindings for data constructor workers are added at the start of
>    CorePrep
>
> This is deeply strange.  In !10479 (unary class patch)
> <https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10479> I had to make
> some changes in this area and after being baffled I moved all these
> implicit bindings to one place at the start of CorePrep.  That is simpler
> and neater.  There is no reason for them to be separate.
>
> BUT I find that one thing happens between Tidy and Prep, namely
> late-cost-centre injection.  That means we will no longer get
> late-cost-centres for data constructor wrappers and class method
> selectors.  I think that this is what is causing diffs in the output for
> callstack001 and friends in !10479.
>
> I think that this is perfectly fine, and I propose to accept the changes.
>
> Another alternative would be to inject all of these bindings at the end of
> Tidy (instead of the start of Prep),  But it's much nicer at the start of
> Prep -- we don't want any of these bindings to appear  in interface files.
>
> Can you confirm that the change is ok?  Thanks
>
> Simon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/e482c6a5/attachment.html>

From klebinger.andreas at gmx.at  Mon Apr  7 09:36:55 2025
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Mon, 7 Apr 2025 11:36:55 +0200
Subject: Late CC profiling
In-Reply-To: <CALuQ0m_8Qu4HYRYNuhe_gb9GgAk79JALM5H1sj3Y9bG4n4Lfpg@mail.gmail.com>
References: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>
 <CALuQ0m_8Qu4HYRYNuhe_gb9GgAk79JALM5H1sj3Y9bG4n4Lfpg@mail.gmail.com>
Message-ID: <5552bd3f-427d-490c-b024-9e12cc3dce90@gmx.at>

A interesting wrinkle!

I agree with matt that having class methods show up in the profile can
be very helpful at times.

Constructors and their wrappers on the other hand should be of no concern.
We explicitly try to avoid wrapping them in cost centres as they
generally add no useful information to profiles.

I've briefly looked over the code and adding class methods seems to be
the *first* thing tidy does, not at it's end. (in tidyProgram)
That would suggest to me that there is a need to tidy these implicit
bindings so moving them might have further unintended consequences.

But then Note [Injecting implicit bindings] claims we do it at the end
of tidy, but seems outdated.This seems like a proper mess.

If you are sure there is no need to tidy the class method selectors then
it would be clearest to have a explicit "materializeImplicitBinds" step
in the pipeline after tidy and before lateCC runs.
That seems like it would be clearer and more coherent overall. And it
would sidestep this issue completely.

Cheers
Andreas


Am 07/04/2025 um 11:08 schrieb Matthew Pickering:
> Hi Simon,
>
> It does seem strange (at first glance) that these implicit things
> happen in two different places.
>
> However, I think you should do this refactoring in a separate MR to
> !10479, it seems unrelated to the purpose of that patch and affects
> other things like you have mentioned.
>
> Why do you think the change in callstack is fine? It seems the
> difference is `callstack001` is whether there an entry for `>>=` or
> not, which seems like quite an important thing to place a cost centre
> on? Andreas has a
> better intuition for this than me. I would quite like to know if there
> is an unoptimised call to >>= in my program. You have probably thought
> about this a little, so interested to hear your opinion.
>
> Cheers,
>
> Matt
>
>
> On Fri, Apr 4, 2025 at 10:44 PM Simon Peyton Jones
> <simon.peytonjones at gmail.com> wrote:
>
>     Hi Matthew, Andreas, Ben
>
>     In GHC today there is a strange inconsistency:
>
>       * Bindings for data constructor wrappers and class method
>         selectors are added at the end of CoreTidy.
>       * Bindings for data constructor workers are added at the start
>         of CorePrep
>
>     This is deeply strange.  In !10479 (unary class patch)
>     <https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10479> I had
>     to make some changes in this area and after being baffled I moved
>     all these implicit bindings to one place at the start of
>     CorePrep.  That is simpler and neater.  There is no reason for
>     them to be separate.
>
>     BUT I find that one thing happens between Tidy and Prep, namely
>     late-cost-centre injection.  That means we will no longer get
>     late-cost-centres for data constructor wrappers and class method
>     selectors.  I think that this is what is causing diffs in the
>     output for callstack001 and friends in !10479.
>
>     I think that this is perfectly fine, and I propose to accept the
>     changes.
>
>     Another alternative would be to inject all of these bindings at
>     the end of Tidy (instead of the start of Prep),  But it's much
>     nicer at the start of Prep -- we don't want any of these bindings
>     to appear  in interface files.
>
>     Can you confirm that the change is ok?  Thanks
>
>     Simon
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/e02b0e33/attachment.html>

From simon.peytonjones at gmail.com  Mon Apr  7 11:37:42 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 7 Apr 2025 12:37:42 +0100
Subject: Late CC profiling
In-Reply-To: <5552bd3f-427d-490c-b024-9e12cc3dce90@gmx.at>
References: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>
 <CALuQ0m_8Qu4HYRYNuhe_gb9GgAk79JALM5H1sj3Y9bG4n4Lfpg@mail.gmail.com>
 <5552bd3f-427d-490c-b024-9e12cc3dce90@gmx.at>
Message-ID: <CAJKmMz-tcNcL+n9Pu=BTJ-n-q3NY5fEibuaOOZpVr3q3LxE0KQ@mail.gmail.com>

If you are sure there is no need to tidy the class method selectors then it
would be clearest to have a explicit "materializeImplicitBinds" step in the
pipeline after tidy and before lateCC runs.


Yes, I want to put all the materialisation in one place.  I think you are
suggesting Plan A

   - Tidy
   - Materialise implicit bindings (class op selectors, data constructor
   workers, data constructor wrappers)
   - Add late cost centres
   - Run late plugins
   - Prep
   - CoreToStg

But then we'll add cost centres to data constructor workers and wrappers.

Or Plan B

   - Tidy
   - Add late cost centres
   - Materialise implicit bindings (class op selectors, data constructor
   workers, data constructor wrappers
   - Run late plugins
   - Prep
   - CoreToStg

Now we won't get cost centres on class-op selectors, which Matthew says he
wants.

I suppose we could have do Plan A, but have the "add cost centre" pass skip
over data constructor workers and wrappers.

Do you have any views?

I'm a bit inclined to combine the late-cc/materialise/prep combo into the
main function of GHC.CoreToStg.Prep.   But I wonder if the "late plugins"
need to see (a) the materialised implicit bindings and (b) late cost
centres.    I have no idea how late plugins are used.

Matthew asks

I would quite like to know if there is an unoptimised call to >>= in my
program. You have probably thought about this a little, so interested to
hear your opinion.


I was thinking that the class-op selector is a small, cheap function, so I
don't mind not profiling it.  But perhaps your point is that calling it
symptomatic of running overloaded code, and we might want to know that.



Simon


On Mon, 7 Apr 2025 at 10:37, Andreas Klebinger <klebinger.andreas at gmx.at>
wrote:

> A interesting wrinkle!
>
> I agree with matt that having class methods show up in the profile can be
> very helpful at times.
>
> Constructors and their wrappers on the other hand should be of no concern.
> We explicitly try to avoid wrapping them in cost centres as they generally
> add no useful information to profiles.
>
> I've briefly looked over the code and adding class methods seems to be the
> *first* thing tidy does, not at it's end. (in tidyProgram)
> That would suggest to me that there is a need to tidy these implicit
> bindings so moving them might have further unintended consequences.
>
> But then Note [Injecting implicit bindings] claims we do it at the end of
> tidy, but seems outdated. This seems like a proper mess.
>
> If you are sure there is no need to tidy the class method selectors then
> it would be clearest to have a explicit "materializeImplicitBinds" step in
> the pipeline after tidy and before lateCC runs.
> That seems like it would be clearer and more coherent overall. And it
> would sidestep this issue completely.
>
> Cheers
> Andreas
>
>
> Am 07/04/2025 um 11:08 schrieb Matthew Pickering:
>
> Hi Simon,
>
> It does seem strange (at first glance) that these implicit things happen
> in two different places.
>
> However, I think you should do this refactoring in a separate MR to
> !10479, it seems unrelated to the purpose of that patch and affects other
> things like you have mentioned.
>
> Why do you think the change in callstack is fine? It seems the difference
> is `callstack001` is whether there an entry for `>>=` or not, which seems
> like quite an important thing to place a cost centre on? Andreas has a
> better intuition for this than me. I would quite like to know if there is
> an unoptimised call to >>= in my program. You have probably thought about
> this a little, so interested to hear your opinion.
>
> Cheers,
>
> Matt
>
>
> On Fri, Apr 4, 2025 at 10:44 PM Simon Peyton Jones <
> simon.peytonjones at gmail.com> wrote:
>
>> Hi Matthew, Andreas, Ben
>>
>> In GHC today there is a strange inconsistency:
>>
>>    - Bindings for data constructor wrappers and class method selectors
>>    are added at the end of CoreTidy.
>>    - Bindings for data constructor workers are added at the start of
>>    CorePrep
>>
>> This is deeply strange.  In !10479 (unary class patch)
>> <https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10479> I had to
>> make some changes in this area and after being baffled I moved all these
>> implicit bindings to one place at the start of CorePrep.  That is simpler
>> and neater.  There is no reason for them to be separate.
>>
>> BUT I find that one thing happens between Tidy and Prep, namely
>> late-cost-centre injection.  That means we will no longer get
>> late-cost-centres for data constructor wrappers and class method
>> selectors.  I think that this is what is causing diffs in the output for
>> callstack001 and friends in !10479.
>>
>> I think that this is perfectly fine, and I propose to accept the changes.
>>
>> Another alternative would be to inject all of these bindings at the end
>> of Tidy (instead of the start of Prep),  But it's much nicer at the start
>> of Prep -- we don't want any of these bindings to appear  in interface
>> files.
>>
>> Can you confirm that the change is ok?  Thanks
>>
>> Simon
>>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/3dc8723d/attachment.html>

From klebinger.andreas at gmx.at  Mon Apr  7 12:01:12 2025
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Mon, 7 Apr 2025 14:01:12 +0200
Subject: Late CC profiling
In-Reply-To: <CAJKmMz-tcNcL+n9Pu=BTJ-n-q3NY5fEibuaOOZpVr3q3LxE0KQ@mail.gmail.com>
References: <CAJKmMz9bNDS=ZAhM15R4y12SnhhPYpzU-PfWXy6ntBzKzkdp0Q@mail.gmail.com>
 <CALuQ0m_8Qu4HYRYNuhe_gb9GgAk79JALM5H1sj3Y9bG4n4Lfpg@mail.gmail.com>
 <5552bd3f-427d-490c-b024-9e12cc3dce90@gmx.at>
 <CAJKmMz-tcNcL+n9Pu=BTJ-n-q3NY5fEibuaOOZpVr3q3LxE0KQ@mail.gmail.com>
Message-ID: <a5bee2eb-001d-4e4e-a7fd-64fb6e7871e6@gmx.at>

 > But then we'll add cost centres to data constructor workers and
wrappers. ... but have the "add cost centre" pass skip over data
constructor workers and wrappers.

We already do this. See compiler/GHC/Core/LateCC/TopLevelBinds.hs:112
where we already check for constructor bindings.
I've added this to avoid adding ccs to workers but it should work for
wrappers just the same.

That means I think your plan A should work fine, assuming there is no
need to tidy those binds.

 > I'm a bit inclined to combine the late-cc/materialise/prep combo into
the main function of GHC.CoreToStg.Prep.

In general I always found it easier to understand passes when they are
narrow in scope.
Sometimes combining them is worth it for performance reasons but I
suspect this is not so here.

And their concerns are clearly separate so combining them seems odd to
me from code structure point of view as well.

For example LateCC must return a list of inserted cost centres, while
CorePrep doesn't need to care about such a thing at all currently!

So I would rather keep those clearly separated.

 > I was thinking that the class-op selector is a small, cheap function,
so I don't mind not profiling it. But perhaps your point is that calling
it symptomatic of running overloaded code, and we might want to know that.

It has definitely been a helpful indicator that optimizations went wrong
for me in the past, as one wouldn't expect (>>=) or similar to be called
5 million times in a optimized program.

Beyond that one of the ideas behind the current late cost centre
profiling is that it can tell you with fairly high precision which (top
level) functions are actually executed at runtime.
Removing method selectors muddies the water there a bit.

Am 07/04/2025 um 13:37 schrieb Simon Peyton Jones:
>
>     If you are sure there is no need to tidy the class method
>     selectors then it would be clearest to have a explicit
>     "materializeImplicitBinds" step in the pipeline after tidy and
>     before lateCC runs.
>
>
> Yes, I want to put all the materialisation in one place.  I think you
> are suggesting Plan A
>
>   * Tidy
>   * Materialise implicit bindings (class op selectors, data
>     constructor workers, data constructor wrappers)
>   * Add late cost centres
>   * Run late plugins
>   * Prep
>   * CoreToStg
>
> But then we'll add cost centres to data constructor workers and wrappers.
>
> Or Plan B
>
>   * Tidy
>   * Add late cost centres
>   * Materialise implicit bindings (class op selectors, data
>     constructor workers, data constructor wrappers
>   * Run late plugins
>   * Prep
>   * CoreToStg
>
> Now we won't get cost centres on class-op selectors, which Matthew
> says he wants.
>
> I suppose we could have do Plan A, but have the "add cost centre" pass
> skip over data constructor workers and wrappers.
>
> Do you have any views?
>
> I'm a bit inclined to combine the late-cc/materialise/prep combo into
> the main function of GHC.CoreToStg.Prep.   But I wonder if the "late
> plugins" need to see (a) the materialised implicit bindings and (b)
> late cost centres.    I have no idea how late plugins are used.
>
> Matthew asks
>
>     I would quite like to know if there is an unoptimised call to >>=
>     in my program. You have probably thought about this a little, so
>     interested to hear your opinion.
>
>
> I was thinking that the class-op selector is a small, cheap function,
> so I don't mind not profiling it.  But perhaps your point is that
> calling it symptomatic of running overloaded code, and we might want
> to know that.
>
>
>
> Simon
>
>
> On Mon, 7 Apr 2025 at 10:37, Andreas Klebinger
> <klebinger.andreas at gmx.at> wrote:
>
>     A interesting wrinkle!
>
>     I agree with matt that having class methods show up in the profile
>     can be very helpful at times.
>
>     Constructors and their wrappers on the other hand should be of no
>     concern.
>     We explicitly try to avoid wrapping them in cost centres as they
>     generally add no useful information to profiles.
>
>     I've briefly looked over the code and adding class methods seems
>     to be the *first* thing tidy does, not at it's end. (in tidyProgram)
>     That would suggest to me that there is a need to tidy these
>     implicit bindings so moving them might have further unintended
>     consequences.
>
>     But then Note [Injecting implicit bindings] claims we do it at the
>     end of tidy, but seems outdated.This seems like a proper mess.
>
>     If you are sure there is no need to tidy the class method
>     selectors then it would be clearest to have a explicit
>     "materializeImplicitBinds" step in the pipeline after tidy and
>     before lateCC runs.
>     That seems like it would be clearer and more coherent overall. And
>     it would sidestep this issue completely.
>
>     Cheers
>     Andreas
>
>
>     Am 07/04/2025 um 11:08 schrieb Matthew Pickering:
>>     Hi Simon,
>>
>>     It does seem strange (at first glance) that these implicit things
>>     happen in two different places.
>>
>>     However, I think you should do this refactoring in a separate MR
>>     to !10479, it seems unrelated to the purpose of that patch and
>>     affects other things like you have mentioned.
>>
>>     Why do you think the change in callstack is fine? It seems the
>>     difference is `callstack001` is whether there an entry for `>>=`
>>     or not, which seems like quite an important thing to place a cost
>>     centre on? Andreas has a
>>     better intuition for this than me. I would quite like to know if
>>     there is an unoptimised call to >>= in my program. You have
>>     probably thought about this a little, so interested to hear your
>>     opinion.
>>
>>     Cheers,
>>
>>     Matt
>>
>>
>>     On Fri, Apr 4, 2025 at 10:44 PM Simon Peyton Jones
>>     <simon.peytonjones at gmail.com> wrote:
>>
>>         Hi Matthew, Andreas, Ben
>>
>>         In GHC today there is a strange inconsistency:
>>
>>           * Bindings for data constructor wrappers and class method
>>             selectors are added at the end of CoreTidy.
>>           * Bindings for data constructor workers are added at the
>>             start of CorePrep
>>
>>         This is deeply strange.  In !10479 (unary class patch)
>>         <https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10479> I
>>         had to make some changes in this area and after being baffled
>>         I moved all these implicit bindings to one place at the start
>>         of CorePrep.  That is simpler and neater.  There is no reason
>>         for them to be separate.
>>
>>         BUT I find that one thing happens between Tidy and Prep,
>>         namely late-cost-centre injection.  That means we will no
>>         longer get late-cost-centres for data constructor wrappers
>>         and class method selectors.  I think that this is what is
>>         causing diffs in the output for callstack001 and friends in
>>         !10479.
>>
>>         I think that this is perfectly fine, and I propose to accept
>>         the changes.
>>
>>         Another alternative would be to inject all of these bindings
>>         at the end of Tidy (instead of the start of Prep),  But it's
>>         much nicer at the start of Prep -- we don't want any of these
>>         bindings to appear  in interface files.
>>
>>         Can you confirm that the change is ok? Thanks
>>
>>         Simon
>>
>>
>>     _______________________________________________
>>     ghc-devs mailing list
>>     ghc-devs at haskell.org
>>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/713cb74c/attachment.html>

From andreash87 at gmx.ch  Mon Apr  7 12:28:16 2025
From: andreash87 at gmx.ch (Andreas Herrmann)
Date: Mon, 7 Apr 2025 14:28:16 +0200
Subject: Extended deadline - CFP - Haskell Implementors' Workshop 2025
Message-ID: <CA+ouDUh+htuG=_F28V7ETQF5Epc4RSogH+B3vR8jkMsDdTkH3A@mail.gmail.com>

Extended deadline to April 18, 2025

Call for proposals for the
Haskell Implementors' Workshop
https://haskell.foundation/events/2025-haskell-implementors-workshop.html
June 6, 2025

Organized by the Haskell Community
Co-located with ZuriHac 2025 and Haskell Ecosystem Workshop 2025
Hosted by the Haskell Foundation
at Eastern Switzerland University of Applied Sciences (OST)
https://www.ost.ch/en/university-of-applied-sciences/campus/rapperswil-jona-campus

## Overview

* Extended Deadline: April 18, 2025
* Notification: May 5, 2025
* Workshop: June 6, 2025

The 17th Haskell Implementors' Workshop is to be held alongside ZuriHac
2025 this year near Zurich. It is a forum for people involved in the design
and development of Haskell implementations, tools, libraries, and
supporting infrastructure to share their work and to discuss future
directions and collaborations with others.

Talks and/or demos are proposed by submitting an abstract, and selected by
a small program committee. There will be no published proceedings. The
workshop will be informal and interactive, with open spaces in the
timetable and room for ad-hoc discussion, demos, and lightning talks.

In the past the Haskell Implementors’ Workshop was co-located with ICFP
(International Conference on Functional Programming). However, in recent
years it has become more and more challenging to attract a large enough
audience and sufficiently many speakers for an appealing program. ZuriHac
and the Haskell Ecosystem Workshop have become an important annual
gathering of a large part of the Haskell community. This year the Haskell
Implementors’ Workshop will be co-located with these events to be
accessible to a broader audience.

## Scope and Target Audience

The Haskell Implementors' Workshop is an ideal place to describe a Haskell
extension, describe works-in-progress, demo a new Haskell-related tool, or
even propose future lines of Haskell development. Members of the wider
Haskell community are encouraged to attend the workshop - we need your
feedback to keep the Haskell ecosystem thriving. Students working with
Haskell are especially encouraged to share their work.

The scope covers any of the following topics. There may be some topics that
people feel we've missed, so by all means submit a proposal even if it
doesn't fit exactly into one of these buckets:

* Compilation techniques
* Language features and extensions
* Type system implementation
* Concurrency and parallelism: language design and implementation
* Performance, optimization and benchmarking
* Virtual machines and run-time systems
* Libraries and tools for development or deployment

## Talks

We invite proposals from potential speakers for talks and demonstrations.
We are aiming for 20-minute talks with 5 minutes for questions and
changeovers. We want to hear from people writing compilers, tools, or
libraries, people with cool ideas for directions in which we should take
the platform, proposals for new features to be implemented, and half-baked
crazy ideas.

Submissions can be made via the form linked below until April 18, 2025
(anywhere on earth).
https://docs.google.com/forms/d/e/1FAIpQLSdczGbxJYGc4eusvPrxwBbZl561PnKeYnoZ2hYsdw_ZpSfupQ/viewform?usp=header

We will also have a lightning talks session. Lightning talks should be
~7mins and are scheduled on the day of the workshop. Suggested topics for
lightning talks are to present a single idea, a work-in-progress project, a
problem to intrigue and perplex Haskell implementors, or simply to ask for
feedback and collaborators.

## Program Committee

* Luite Stegeman
* Jaro Reinders
* Emily Pillmore
* Rodrigo Mesquita
* Ian-Woo Kim
* Andreas Herrmann (chair)

## Contact

* Andreas Herrmann
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250407/93a75c41/attachment.html>

From simon.peytonjones at gmail.com  Tue Apr  8 09:09:48 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Tue, 8 Apr 2025 10:09:48 +0100
Subject: Testsuite tests
In-Reply-To: <CALuQ0m_L9cdQ8CGa5gMc0XsbKEdZdcM6oM5srquQR-SRCV+CgA@mail.gmail.com>
References: <CAJKmMz-xohGkVvaHS3-rPCR95ZS7uo9C+DACnt3=s2CLWR4QDA@mail.gmail.com>
 <CALuQ0m_L9cdQ8CGa5gMc0XsbKEdZdcM6oM5srquQR-SRCV+CgA@mail.gmail.com>
Message-ID: <CAJKmMz9OJn2Zk5oHXNVgxR3oizz61WvxwK_7FjjiTpjL2T4ykg@mail.gmail.com>

>
> It seems that you have not built profiled libraries? It's impossible to
> say without knowing your custom local configuration. Could you share that?


Sure!  UserSettings.hs is attached.

Funnily enough the build system takes ages (many minutes) building profied
version of the runtime system, despite my attempting to switch that off.
Any advice there?

Separately, it would be great if the testsuite driver explained why it
discards tests.  Eg. "Skipping X, Y, Z because they require profiled
libraries".  Otherwise it's hard to know why something is skipped.

Thanks for helping!

Simon



On Mon, 7 Apr 2025 at 09:57, Matthew Pickering <matthewtpickering at gmail.com>
wrote:

> It seems that you have not built profiled libraries? It's impossible to
> say without knowing your custom local configuration. Could you share that?
>
> On Fri, Apr 4, 2025 at 10:30 PM Simon Peyton Jones <
> simon.peytonjones at gmail.com> wrote:
>
>> Friends
>>
>> In CI for !10479 I'm getting test failures for some call-stack tests e.g.
>> `callstack001`.
>>
>> But if I try to run this in my build tree
>>
>> hadrian/build -j8 test --skip=_build/stage1/rts/**
>> --skip=_build/stage1/** --skip=_build/stage0/** --docs=none
>> --only=callstack001
>>
>> I get
>>
>> SUMMARY for test run started at Fri Apr  4 22:22:47 2025
>> 0:00:02.413942 spent to go through
>>        1 total tests, which gave rise to
>>        8 test cases, of which
>>        8 were skipped
>>        0 had missing libraries
>>
>>        0 expected passes
>>        0 expected failures
>>
>>        0 caused framework failures
>>        0 caused framework warnings
>>        0 unexpected passes
>>        0 unexpected failures
>>        0 unexpected stat failures
>>        0 fragile tests
>>
>> So I can't reproduce the failure.  What am i doing wrong?  Can anyone
>> help?
>>
>> Thanks
>>
>> Simon
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250408/b7f17d16/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: UserSettings-spj.hs
Type: application/octet-stream
Size: 3746 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250408/b7f17d16/attachment.obj>

From matthewtpickering at gmail.com  Tue Apr  8 09:23:46 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Tue, 8 Apr 2025 10:23:46 +0100
Subject: Next steps forward for a stable Template Haskell
In-Reply-To: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>
References: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>
Message-ID: <CALuQ0m8SsitL062tkh64Wk8NDSJwf5R8PXoqiG8aPXmu3sH2QQ@mail.gmail.com>

Hi Teo, I went back and read
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

I think my comment is a very good summary there. Which I will paste into
this thread.

```

Why are these packages not re-installable?

   1.

   If you are not using the internal interpreter (or dynamic loading at
   all), then you are free to reinstall any package as long as the resulting
   package database contains a unit for each of the wired in unit ids which
   exports the relevant wired-in definitions.
   2.

   If you are using template-haskell and the internal interpreter, then
   your program must link against exactly the same version of template-haskell
   (and dependencies) that the
   3.

   If you are using plugins, then your program must link against exactly
   the same version of ghc library (and dependencies)

** Cabal simplifies all of these situations by refusing to reinstall
certain packages and always using the global package database **

   1. Cabal refuses to reinstall wired in units and always uses the global
   package database, therefore the relevant unit is always available with the
   correct ABI.
   2. Cabal refuses to reinstall template-haskell, which forces it to be
   taken from the global package database, which ensures it is the same
   version as the compiler is linked against.
   3. Cabal refuses to reinstall ghc library, which forces it to be taken
   from the global package database, which ensures the same version of the
   library is linked against.

Therefore I don't think it's accurate for the compiler to advertise in
particular which packages are "not reinstallable", as that is not a
property of the compiler itself.

There is no conceptual barrier to the situation where you want to use
template-haskell or ghc library in a situation where you don't want or need
to dynamically load code into the running process. In this situation, it
would be perfectly permissible to reinstall it and use it in your program.
The issue comes that Cabal doesn't allow you to express the constraint that
the version of template-haskell you depend on needs to be fixed to the one
the compiler is built against. As such, it has to be pessimistic and never
reinstall template-haskell or ghc, but against, not an issue in particular
with GHC.

** Cabal can not express the constraint that a library must be the same
version as the one the build compiler is built against **

```

Your point is that

* GHC should advertise to Cabal which (template-haskell, ghc) unit it links
against. Then cabal should ensure that if TemplateHaskell/plugins are used
then it should enforce in the build plan that this version of
`template-haskell`/`ghc` is picked.

My point was

* GHC can only advertise a unit id, it wasn't clear to me about how to map
this information into the cabal solver.

I suppose we will have the assumption that if GHC advertises it is linked
against a specific unit id, then that unit will be in a global package
database. Cabal already reads the global package database.

Perhaps the only thing we need to add to the cabal solver is the ability to
express a constraint that you depend on a precise unit-id. Then when
cabal-install starts the solver it can query GHC to find the unit-ids which
need to be fixed, and feed those constraints into the solver. Then if users
are feeling very brave, they can turn off this behaviour..

Thinking aloud here..

Matt






On Wed, Apr 2, 2025 at 6:26 PM Teo Camarasu <teofilcamarasu at gmail.com>
wrote:

> Hi folks,
>
> At Tuesday's GHC devs meeting, we talked about having a discussion about
> template-haskell stability. I've jotted down some notes to help focus our
> future discussion. It's mostly just a list of what I take to be the next
> most important issues to tackle, and some details about each.
>
>
> https://docs.google.com/document/d/1gui1YXOvrNihR3NpXuMVCKV3lTkKPL8y_HDRjbZyRxk/edit?usp=sharing
>
> Cheers,
> Teo
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250408/5ad6980b/attachment.html>

From teofilcamarasu at gmail.com  Tue Apr  8 10:11:11 2025
From: teofilcamarasu at gmail.com (Teo Camarasu)
Date: Tue, 8 Apr 2025 11:11:11 +0100
Subject: Next steps forward for a stable Template Haskell
In-Reply-To: <CALuQ0m8SsitL062tkh64Wk8NDSJwf5R8PXoqiG8aPXmu3sH2QQ@mail.gmail.com>
References: <CAPBO51EE6JB-tFsuKEKQZwBmnqruq6s=cdTREJBCcpNigRiiZw@mail.gmail.com>
 <CALuQ0m8SsitL062tkh64Wk8NDSJwf5R8PXoqiG8aPXmu3sH2QQ@mail.gmail.com>
Message-ID: <CAPBO51Fp_qBS71WE-e7=x16hU43Hf6qu0o6XbKQzK915u+tS0g@mail.gmail.com>

Thanks both!

I've updated the doc and replied to your comments Simon. I'll also update
the wiki page soon.

Thanks for summarising Matt! I've copied over your email into the doc.
I'll also post your suggestion in the cabal ticket.

Cheers,
Teo



On Tue, Apr 8, 2025 at 10:24 AM Matthew Pickering <
matthewtpickering at gmail.com> wrote:

> Hi Teo, I went back and read
> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297
>
> I think my comment is a very good summary there. Which I will paste into
> this thread.
>
> ```
>
> Why are these packages not re-installable?
>
>    1.
>
>    If you are not using the internal interpreter (or dynamic loading at
>    all), then you are free to reinstall any package as long as the resulting
>    package database contains a unit for each of the wired in unit ids which
>    exports the relevant wired-in definitions.
>    2.
>
>    If you are using template-haskell and the internal interpreter, then
>    your program must link against exactly the same version of template-haskell
>    (and dependencies) that the
>    3.
>
>    If you are using plugins, then your program must link against exactly
>    the same version of ghc library (and dependencies)
>
> ** Cabal simplifies all of these situations by refusing to reinstall
> certain packages and always using the global package database **
>
>    1. Cabal refuses to reinstall wired in units and always uses the
>    global package database, therefore the relevant unit is always available
>    with the correct ABI.
>    2. Cabal refuses to reinstall template-haskell, which forces it to be
>    taken from the global package database, which ensures it is the same
>    version as the compiler is linked against.
>    3. Cabal refuses to reinstall ghc library, which forces it to be taken
>    from the global package database, which ensures the same version of the
>    library is linked against.
>
> Therefore I don't think it's accurate for the compiler to advertise in
> particular which packages are "not reinstallable", as that is not a
> property of the compiler itself.
>
> There is no conceptual barrier to the situation where you want to use
> template-haskell or ghc library in a situation where you don't want or
> need to dynamically load code into the running process. In this situation,
> it would be perfectly permissible to reinstall it and use it in your
> program. The issue comes that Cabal doesn't allow you to express the
> constraint that the version of template-haskell you depend on needs to be
> fixed to the one the compiler is built against. As such, it has to be
> pessimistic and never reinstall template-haskell or ghc, but against, not
> an issue in particular with GHC.
>
> ** Cabal can not express the constraint that a library must be the same
> version as the one the build compiler is built against **
>
> ```
>
> Your point is that
>
> * GHC should advertise to Cabal which (template-haskell, ghc) unit it
> links against. Then cabal should ensure that if TemplateHaskell/plugins are
> used then it should enforce in the build plan that this version of
> `template-haskell`/`ghc` is picked.
>
> My point was
>
> * GHC can only advertise a unit id, it wasn't clear to me about how to map
> this information into the cabal solver.
>
> I suppose we will have the assumption that if GHC advertises it is linked
> against a specific unit id, then that unit will be in a global package
> database. Cabal already reads the global package database.
>
> Perhaps the only thing we need to add to the cabal solver is the ability
> to express a constraint that you depend on a precise unit-id. Then when
> cabal-install starts the solver it can query GHC to find the unit-ids which
> need to be fixed, and feed those constraints into the solver. Then if users
> are feeling very brave, they can turn off this behaviour..
>
> Thinking aloud here..
>
> Matt
>
>
>
>
>
>
> On Wed, Apr 2, 2025 at 6:26 PM Teo Camarasu <teofilcamarasu at gmail.com>
> wrote:
>
>> Hi folks,
>>
>> At Tuesday's GHC devs meeting, we talked about having a discussion about
>> template-haskell stability. I've jotted down some notes to help focus our
>> future discussion. It's mostly just a list of what I take to be the next
>> most important issues to tackle, and some details about each.
>>
>>
>> https://docs.google.com/document/d/1gui1YXOvrNihR3NpXuMVCKV3lTkKPL8y_HDRjbZyRxk/edit?usp=sharing
>>
>> Cheers,
>> Teo
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250408/88159506/attachment.html>

From matthewtpickering at gmail.com  Tue Apr  8 12:51:24 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Tue, 8 Apr 2025 13:51:24 +0100
Subject: Testsuite tests
In-Reply-To: <CAJKmMz9OJn2Zk5oHXNVgxR3oizz61WvxwK_7FjjiTpjL2T4ykg@mail.gmail.com>
References: <CAJKmMz-xohGkVvaHS3-rPCR95ZS7uo9C+DACnt3=s2CLWR4QDA@mail.gmail.com>
 <CALuQ0m_L9cdQ8CGa5gMc0XsbKEdZdcM6oM5srquQR-SRCV+CgA@mail.gmail.com>
 <CAJKmMz9OJn2Zk5oHXNVgxR3oizz61WvxwK_7FjjiTpjL2T4ykg@mail.gmail.com>
Message-ID: <CALuQ0m9V6JxSa1i5K5+2Gs=1J_XfCLdT4HXCJWdJrYg8cvVQMA@mail.gmail.com>

Hi Simon,

You have disabled the build of profiled libraries in your user settings, so
the test doesn't run, because it requires profiled libraries.

If you want to disabled profiled things (including the rts) from your
build, then import `Flavour` and use the `disableProfiledLibs :: Flavour ->
Flavour` function.

```
userFlavour = disableProfiledLibs $ defaultFlavour { ... }
```

If you want to run this test then you need profiled libraries.

```
./hadrian/build --flavour=perf -o_perf_build test --only=callstack001
```

Hope that helps.

Matt

On Tue, Apr 8, 2025 at 10:10 AM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> It seems that you have not built profiled libraries? It's impossible to
>> say without knowing your custom local configuration. Could you share that?
>
>
> Sure!  UserSettings.hs is attached.
>
> Funnily enough the build system takes ages (many minutes) building profied
> version of the runtime system, despite my attempting to switch that off.
> Any advice there?
>
> Separately, it would be great if the testsuite driver explained why it
> discards tests.  Eg. "Skipping X, Y, Z because they require profiled
> libraries".  Otherwise it's hard to know why something is skipped.
>
> Thanks for helping!
>
> Simon
>
>
>
> On Mon, 7 Apr 2025 at 09:57, Matthew Pickering <
> matthewtpickering at gmail.com> wrote:
>
>> It seems that you have not built profiled libraries? It's impossible to
>> say without knowing your custom local configuration. Could you share that?
>>
>> On Fri, Apr 4, 2025 at 10:30 PM Simon Peyton Jones <
>> simon.peytonjones at gmail.com> wrote:
>>
>>> Friends
>>>
>>> In CI for !10479 I'm getting test failures for some call-stack tests
>>> e.g. `callstack001`.
>>>
>>> But if I try to run this in my build tree
>>>
>>> hadrian/build -j8 test --skip=_build/stage1/rts/**
>>> --skip=_build/stage1/** --skip=_build/stage0/** --docs=none
>>> --only=callstack001
>>>
>>> I get
>>>
>>> SUMMARY for test run started at Fri Apr  4 22:22:47 2025
>>> 0:00:02.413942 spent to go through
>>>        1 total tests, which gave rise to
>>>        8 test cases, of which
>>>        8 were skipped
>>>        0 had missing libraries
>>>
>>>        0 expected passes
>>>        0 expected failures
>>>
>>>        0 caused framework failures
>>>        0 caused framework warnings
>>>        0 unexpected passes
>>>        0 unexpected failures
>>>        0 unexpected stat failures
>>>        0 fragile tests
>>>
>>> So I can't reproduce the failure.  What am i doing wrong?  Can anyone
>>> help?
>>>
>>> Thanks
>>>
>>> Simon
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250408/cd10c6ef/attachment.html>

From ben at well-typed.com  Mon Apr 14 19:01:33 2025
From: ben at well-typed.com (Ben Gamari)
Date: Mon, 14 Apr 2025 15:01:33 -0400
Subject: GitLab maintenance this evening
Message-ID: <87msci7ek2.fsf@smart-cactus.org>


Hello all,

Tonight GitLab will be down for maintenance between 17:00 EDT and 18:00
EDT as part of the on-going infrastructure migration.

Cheers,

- Ben

From danihacker.viedma at gmail.com  Tue Apr 15 16:40:03 2025
From: danihacker.viedma at gmail.com (Daniel Trujillo Viedma)
Date: Tue, 15 Apr 2025 18:40:03 +0200
Subject: Questions regarding the RISCV native codegen & performance
Message-ID: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>

Hello, ghc-devs! My name is Daniel Trujillo, I'm a Haskell enthusiast 
from Spain and I'm trying to make my Master's thesis about accelerating 
Haskell programs with a custom ISA extension.


Right now, my focus is in executing software written in Haskell within 
QEMU in order to get traces that tells me, basically, how many times 
each block (not exactly basic blocks, but sort of) of assembly code has 
been executed, with the hope of finding some patterns of RISCV 
instructions that I could implement together into 1 instruction.


As you can see, my method is a bit crude, and I was wondering if the 
people involved with any of the different internal representations (STG, 
Cmm...) and/or native code generators (particularly RISCV) could provide 
me hints about assembly instructions that would have made the work 
easier, by removing the need of "massaging" the Cmm code to make CodeGen 
easier, or the need of particular optimizations, or in general, dirty 
tricks because of lacking of proper support of the standard RISCV ISA.


And of course, I would also appreciate very much other hints from people 
involved in general performance (as oppossed to, for example, libraries 
for SIMD and parallel execution, or Haskell wrappers to lower-level code 
for performance reasons).


P.D. I'm sorry if I broke any netiquette rule, but I'm very new to the 
email list, and haven't received yet any email from it.


Looking forward to hear from you!

Cheers,

Dani.


From matthewtpickering at gmail.com  Wed Apr 16 08:33:49 2025
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 16 Apr 2025 09:33:49 +0100
Subject: Questions regarding the RISCV native codegen & performance
In-Reply-To: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
References: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
Message-ID: <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>

Hi Daniel. I think Sven Tennie and the LoongArch contributors are the
experts in NCG for these kinds of instruction sets. I have cced them.

Cheers,

Matt

On Tue, Apr 15, 2025 at 5:40 PM Daniel Trujillo Viedma <
danihacker.viedma at gmail.com> wrote:

> Hello, ghc-devs! My name is Daniel Trujillo, I'm a Haskell enthusiast
> from Spain and I'm trying to make my Master's thesis about accelerating
> Haskell programs with a custom ISA extension.
>
>
> Right now, my focus is in executing software written in Haskell within
> QEMU in order to get traces that tells me, basically, how many times
> each block (not exactly basic blocks, but sort of) of assembly code has
> been executed, with the hope of finding some patterns of RISCV
> instructions that I could implement together into 1 instruction.
>
>
> As you can see, my method is a bit crude, and I was wondering if the
> people involved with any of the different internal representations (STG,
> Cmm...) and/or native code generators (particularly RISCV) could provide
> me hints about assembly instructions that would have made the work
> easier, by removing the need of "massaging" the Cmm code to make CodeGen
> easier, or the need of particular optimizations, or in general, dirty
> tricks because of lacking of proper support of the standard RISCV ISA.
>
>
> And of course, I would also appreciate very much other hints from people
> involved in general performance (as oppossed to, for example, libraries
> for SIMD and parallel execution, or Haskell wrappers to lower-level code
> for performance reasons).
>
>
> P.D. I'm sorry if I broke any netiquette rule, but I'm very new to the
> email list, and haven't received yet any email from it.
>
>
> Looking forward to hear from you!
>
> Cheers,
>
> Dani.
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250416/0b79de74/attachment.html>

From diegorosario2013 at gmail.com  Wed Apr 16 20:40:58 2025
From: diegorosario2013 at gmail.com (Diego Antonio Rosario Palomino)
Date: Wed, 16 Apr 2025 15:40:58 -0500
Subject: GSoC 2025 proposal : Documenting and Improving Cmm
Message-ID: <CAONcbW+74x_MYrHgdWMMwHM+uqpJR-73Muan1S4moZmrOXS05w@mail.gmail.com>

Hi all,

I've posted a thread on Discourse outlining my GSoC 2025 proposal, which
focuses on documenting and improving GHC's Cmm backend. It covers two
possible directions (LLVM and SSA), along with some refinements I've been
exploring.

You can find the full details here:
👉
https://discourse.haskell.org/t/gsoc-2025-documenting-and-improving-cmm/11870

Feedback is very welcome!

Best regards,
Diego Antonio Rosario Palomino
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250416/75a7ed78/attachment.html>

From sven.tennie at gmail.com  Thu Apr 17 07:19:44 2025
From: sven.tennie at gmail.com (Sven Tennie)
Date: Thu, 17 Apr 2025 09:19:44 +0200
Subject: Questions regarding the RISCV native codegen & performance
In-Reply-To: <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>
References: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
 <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>
Message-ID: <CALgrwrYHdOkQ_5+7D-UqLHrkaR_G3-iC-RxKu4DTmaurYME3Xw@mail.gmail.com>

Hey Daniel 👋

That's really an interesting topic, because we never analyzed the emitted
RISC-V assembly with statistical measures.

So, if I may ask for a favour: If you spot anything that could be better
expressed with the current ISA, please open a ticket and label it as
RISC-V: https://gitlab.haskell.org/ghc/ghc/-/issues
(We haven't decided which RISC-V profile to require. I.e. requiring the
very latest extensions would frustrate people with older hardware...
However, it's in anycase good to have possible improvements documented in
tickets.)

I'm wondering if you really have to go through QEMU. Or, if feeding
assembly code to a parser and then doing the math on that wouldn't be
sufficient? (Of course, tracing the execution is more accurate. However,
it's much more complicated as well.)

To account Assembly instructions to Cmm statements you may use the GHC
parameters -ddump-cmm and -dppr-debug (and to stream this into files
instead of stdout -ddump-to-file.) This will add comments for most Cmm
statements into the dumped assembly code.

At first, I thought that sign-extension / truncation might be a good
candidate. However, it turned out that this is already covered by the
RISC-V B-extension. Which led to this new ticket:
https://gitlab.haskell.org/ghc/ghc/-/issues/25966

Skimming over the NCG code and watching out for longer or repeating
instruction lists might be a good strategy to make educated guesses.

>From a developer's perspective, I found the immediate sizes (usually 12bit)
rather limiting. E.g. the Note [RISCV64 far jumps] (
https://gitlab.haskell.org/ghc/ghc/-/blob/395e0ad17c0d309637f079a05dbdc23e0d4188f6/compiler/GHC/CmmToAsm/RV64/CodeGen.hs?page=2#L1996)
tells a story how we had to work around this limit for addresses in
conditional jumps.

So, you could raise the question if - analog to compressed expressions - it
wouldn't make sense to have extended expressions that cover two words. Such
that the first word is the instruction and the second it's immediate(s).
(Hardware designers would probably hate that, because it means a bigger
change to the instruction decoding unit. However, I got asked as a software
developer ;) )

Other than that, I've unfortunately got no great ideas.

Please feel free to keep us in the loop (especially regarding the results
of your analyses.) And, if you've got any questions regarding the RISC-V
NCG, please feel free to reach out either here or directly to me. There's
also a #GHC "room" on Matrix where you can quickly drop smaller scoped
questions.

I hope that was of any help. Best regards,

Sven

Am Mi., 16. Apr. 2025 um 10:34 Uhr schrieb Matthew Pickering <
matthewtpickering at gmail.com>:

> Hi Daniel. I think Sven Tennie and the LoongArch contributors are the
> experts in NCG for these kinds of instruction sets. I have cced them.
>
> Cheers,
>
> Matt
>
> On Tue, Apr 15, 2025 at 5:40 PM Daniel Trujillo Viedma <
> danihacker.viedma at gmail.com> wrote:
>
>> Hello, ghc-devs! My name is Daniel Trujillo, I'm a Haskell enthusiast
>> from Spain and I'm trying to make my Master's thesis about accelerating
>> Haskell programs with a custom ISA extension.
>>
>>
>> Right now, my focus is in executing software written in Haskell within
>> QEMU in order to get traces that tells me, basically, how many times
>> each block (not exactly basic blocks, but sort of) of assembly code has
>> been executed, with the hope of finding some patterns of RISCV
>> instructions that I could implement together into 1 instruction.
>>
>>
>> As you can see, my method is a bit crude, and I was wondering if the
>> people involved with any of the different internal representations (STG,
>> Cmm...) and/or native code generators (particularly RISCV) could provide
>> me hints about assembly instructions that would have made the work
>> easier, by removing the need of "massaging" the Cmm code to make CodeGen
>> easier, or the need of particular optimizations, or in general, dirty
>> tricks because of lacking of proper support of the standard RISCV ISA.
>>
>>
>> And of course, I would also appreciate very much other hints from people
>> involved in general performance (as oppossed to, for example, libraries
>> for SIMD and parallel execution, or Haskell wrappers to lower-level code
>> for performance reasons).
>>
>>
>> P.D. I'm sorry if I broke any netiquette rule, but I'm very new to the
>> email list, and haven't received yet any email from it.
>>
>>
>> Looking forward to hear from you!
>>
>> Cheers,
>>
>> Dani.
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250417/c5a5fcdf/attachment.html>

From danihacker.viedma at gmail.com  Fri Apr 18 10:13:44 2025
From: danihacker.viedma at gmail.com (Daniel Trujillo Viedma)
Date: Fri, 18 Apr 2025 12:13:44 +0200
Subject: Questions regarding the RISCV native codegen & performance
In-Reply-To: <CALgrwrYHdOkQ_5+7D-UqLHrkaR_G3-iC-RxKu4DTmaurYME3Xw@mail.gmail.com>
References: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
 <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>
 <CALgrwrYHdOkQ_5+7D-UqLHrkaR_G3-iC-RxKu4DTmaurYME3Xw@mail.gmail.com>
Message-ID: <22850366-f63a-4158-a2b4-9d01e78173b6@gmail.com>

Thank you so much for all the information and the help.


Seriously, this is much more than I was hoping to get, even the 
suggestion for generating commented assembly code (which is, I assume, 
the method that Compiler Explorer uses to relate the high-level Haskell 
code with the assembly output of the compiler, which is really nice). 
And four your RISC-V NGC, which I found easy to understand.


I guess this is the kind of professionals that Haskell attracts, which 
is a big part of why I love it.


I will send here an executive summary of my findings, including 
statistics about a couple of programs that I try. I don't know if I'll 
be able to do a very statistical significant analysis, because I'll 
still have a lot of things to do (to extend QEMU, maybe also gem5, and 
implement it in a Clash microprocessor design, probably VELDT), but 
maybe in the future I can automate more of it and running a more 
comprehensive analysis. FYI, I have found that the RISC-V specs mention 
Haskell among other languages in a still empty J extension section, 
which will be aimed at helping dynamically translated languages as well 
as garbage-collected, but I guess RISC-V people is still more focused on 
other things and it will take some time to start work on that extension.


I find also very interesting your suggestion for far-jumping, but I'm 
afraid that will be very unpopular among hardware designers because it 
messes with their highly appreciated and scarce L1 cache. But funnily 
enough, I had the impresion before starting this project that some kind 
of simple mechanism for complex jumping would be a good idea. I will 
keep this in mind when looking for patterns in the assembly code.


Once more, thank you so much for your work and the help, and I hope I 
can deliver soon information that you all could find interesting.


Have a very nice weekend!

Cheers,

Dani.


On 17/4/25 9:19, Sven Tennie wrote:
> Hey Daniel 👋
>
> That's really an interesting topic, because we never analyzed the 
> emitted RISC-V assembly with statistical measures.
>
> So, if I may ask for a favour: If you spot anything that could be 
> better expressed with the current ISA, please open a ticket and label 
> it as RISC-V: https://gitlab.haskell.org/ghc/ghc/-/issues
> (We haven't decided which RISC-V profile to require. I.e. requiring 
> the very latest extensions would frustrate people with older 
> hardware... However, it's in anycase good to have possible 
> improvements documented in tickets.)
>
> I'm wondering if you really have to go through QEMU. Or, if feeding 
> assembly code to a parser and then doing the math on that wouldn't be 
> sufficient? (Of course, tracing the execution is more accurate. 
> However, it's much more complicated as well.)
>
> To account Assembly instructions to Cmm statements you may use the GHC 
> parameters -ddump-cmm and -dppr-debug (and to stream this into files 
> instead of stdout -ddump-to-file.) This will add comments for most Cmm 
> statements into the dumped assembly code.
>
> At first, I thought that sign-extension / truncation might be a good 
> candidate. However, it turned out that this is already covered by the 
> RISC-V B-extension. Which led to this new ticket: 
> https://gitlab.haskell.org/ghc/ghc/-/issues/25966
>
> Skimming over the NCG code and watching out for longer or repeating 
> instruction lists might be a good strategy to make educated guesses.
>
> From a developer's perspective, I found the immediate sizes (usually 
> 12bit) rather limiting. E.g. the Note [RISCV64 far jumps] 
> (https://gitlab.haskell.org/ghc/ghc/-/blob/395e0ad17c0d309637f079a05dbdc23e0d4188f6/compiler/GHC/CmmToAsm/RV64/CodeGen.hs?page=2#L1996) 
> tells a story how we had to work around this limit for addresses in 
> conditional jumps.
>
> So, you could raise the question if - analog to compressed expressions 
> - it wouldn't make sense to have extended expressions that cover two 
> words. Such that the first word is the instruction and the second it's 
> immediate(s). (Hardware designers would probably hate that, because it 
> means a bigger change to the instruction decoding unit. However, I got 
> asked as a software developer ;) )
>
> Other than that, I've unfortunately got no great ideas.
>
> Please feel free to keep us in the loop (especially regarding the 
> results of your analyses.) And, if you've got any questions regarding 
> the RISC-V NCG, please feel free to reach out either here or directly 
> to me. There's also a #GHC "room" on Matrix where you can quickly drop 
> smaller scoped questions.
>
> I hope that was of any help. Best regards,
>
> Sven
>
> Am Mi., 16. Apr. 2025 um 10:34 Uhr schrieb Matthew Pickering 
> <matthewtpickering at gmail.com>:
>
>     Hi Daniel. I think Sven Tennie and the LoongArch contributors are
>     the experts in NCG for these kinds of instruction sets. I have
>     cced them.
>
>     Cheers,
>
>     Matt
>
>     On Tue, Apr 15, 2025 at 5:40 PM Daniel Trujillo Viedma
>     <danihacker.viedma at gmail.com> wrote:
>
>         Hello, ghc-devs! My name is Daniel Trujillo, I'm a Haskell
>         enthusiast
>         from Spain and I'm trying to make my Master's thesis about
>         accelerating
>         Haskell programs with a custom ISA extension.
>
>
>         Right now, my focus is in executing software written in
>         Haskell within
>         QEMU in order to get traces that tells me, basically, how many
>         times
>         each block (not exactly basic blocks, but sort of) of assembly
>         code has
>         been executed, with the hope of finding some patterns of RISCV
>         instructions that I could implement together into 1 instruction.
>
>
>         As you can see, my method is a bit crude, and I was wondering
>         if the
>         people involved with any of the different internal
>         representations (STG,
>         Cmm...) and/or native code generators (particularly RISCV)
>         could provide
>         me hints about assembly instructions that would have made the
>         work
>         easier, by removing the need of "massaging" the Cmm code to
>         make CodeGen
>         easier, or the need of particular optimizations, or in
>         general, dirty
>         tricks because of lacking of proper support of the standard
>         RISCV ISA.
>
>
>         And of course, I would also appreciate very much other hints
>         from people
>         involved in general performance (as oppossed to, for example,
>         libraries
>         for SIMD and parallel execution, or Haskell wrappers to
>         lower-level code
>         for performance reasons).
>
>
>         P.D. I'm sorry if I broke any netiquette rule, but I'm very
>         new to the
>         email list, and haven't received yet any email from it.
>
>
>         Looking forward to hear from you!
>
>         Cheers,
>
>         Dani.
>
>         _______________________________________________
>         ghc-devs mailing list
>         ghc-devs at haskell.org
>         http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250418/85e5e5fb/attachment.html>

From sven.tennie at gmail.com  Fri Apr 18 12:41:28 2025
From: sven.tennie at gmail.com (Sven Tennie)
Date: Fri, 18 Apr 2025 14:41:28 +0200
Subject: Questions regarding the RISCV native codegen & performance
In-Reply-To: <22850366-f63a-4158-a2b4-9d01e78173b6@gmail.com>
References: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
 <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>
 <CALgrwrYHdOkQ_5+7D-UqLHrkaR_G3-iC-RxKu4DTmaurYME3Xw@mail.gmail.com>
 <22850366-f63a-4158-a2b4-9d01e78173b6@gmail.com>
Message-ID: <CALgrwrb+tHz9c0rH9G45ydmNQX5PV6rVBCyLf8BCSf_t4ib2oA@mail.gmail.com>

Hey Daniel 👋

Thanks a lot for your kind words.

The AArch64 ISA might also be some source of inspiration. AArch64 has some
combined instructions which RISC-V hasn't. E.g ADD of two registers with an
included shift. Though, we don't seem to use many of them and I haven't
found any usage that wouldn't be well covered in RISC-V NCG. Probably,
that's because MachOp (
https://hackage.haskell.org/package/ghc-9.12.1/docs/GHC-Cmm-MachOp.html#t:MachOp)
is pretty fine grained.

A good candidate for investigations could be the CSET pseudo-instruction. I
stumbled over it while looking for pseudo-ops which lead to combined
instructions in AArch64 NCG. The CSET pseudo-op leads to two instructions
in RISC-V NCG and to one in AArch64 NCG:
-
https://gitlab.haskell.org/ghc/ghc/-/blob/386f18548e3c66d04f648a9d34f167a086c1328b/compiler/GHC/CmmToAsm/AArch64/Ppr.hs#L443
-
https://gitlab.haskell.org/ghc/ghc/-/blob/386f18548e3c66d04f648a9d34f167a086c1328b/compiler/GHC/CmmToAsm/RV64/Ppr.hs#L554

Though, this might be a sub-optimal implementation (in this case we'd be
happy to get a ticket ;) ). As CSET is used for comparisons, it should
appear pretty frequently.

A bit off-topic, but for the sake of completeness: The Compiler Explorer
seems to use DWARF symbols to map assembly instructions to Haskell code
lines. At least, it compiles with -g.

VELDT's profile is stated as "RV32I (no FENCE, ECALL, EBREAK)" on their
Github page. But, we target RV64G with both, the NCG and LLVM backends.
(The main reason to not support simpler profiles is that all hardware on
the market that is powerful enough to reasonably run Haskell supports at
least RV64G.)

Thanks for the hint about the J-extension. I will take a look at it.

Enjoy your weekend & best regards,

Sven

Am Fr., 18. Apr. 2025 um 12:13 Uhr schrieb Daniel Trujillo Viedma <
danihacker.viedma at gmail.com>:

> Thank you so much for all the information and the help.
>
>
> Seriously, this is much more than I was hoping to get, even the suggestion
> for generating commented assembly code (which is, I assume, the method that
> Compiler Explorer uses to relate the high-level Haskell code with the
> assembly output of the compiler, which is really nice). And four your
> RISC-V NGC, which I found easy to understand.
>
>
> I guess this is the kind of professionals that Haskell attracts, which is
> a big part of why I love it.
>
>
> I will send here an executive summary of my findings, including statistics
> about a couple of programs that I try. I don't know if I'll be able to do a
> very statistical significant analysis, because I'll still have a lot of
> things to do (to extend QEMU, maybe also gem5, and implement it in a Clash
> microprocessor design, probably VELDT), but maybe in the future I can
> automate more of it and running a more comprehensive analysis. FYI, I have
> found that the RISC-V specs mention Haskell among other languages in a
> still empty J extension section, which will be aimed at helping dynamically
> translated languages as well as garbage-collected, but I guess RISC-V
> people is still more focused on other things and it will take some time to
> start work on that extension.
>
>
> I find also very interesting your suggestion for far-jumping, but I'm
> afraid that will be very unpopular among hardware designers because it
> messes with their highly appreciated and scarce L1 cache. But funnily
> enough, I had the impresion before starting this project that some kind of
> simple mechanism for complex jumping would be a good idea. I will keep this
> in mind when looking for patterns in the assembly code.
>
>
> Once more, thank you so much for your work and the help, and I hope I can
> deliver soon information that you all could find interesting.
>
>
> Have a very nice weekend!
>
> Cheers,
>
> Dani.
>
>
> On 17/4/25 9:19, Sven Tennie wrote:
>
> Hey Daniel 👋
>
> That's really an interesting topic, because we never analyzed the emitted
> RISC-V assembly with statistical measures.
>
> So, if I may ask for a favour: If you spot anything that could be better
> expressed with the current ISA, please open a ticket and label it as
> RISC-V: https://gitlab.haskell.org/ghc/ghc/-/issues
> (We haven't decided which RISC-V profile to require. I.e. requiring the
> very latest extensions would frustrate people with older hardware...
> However, it's in anycase good to have possible improvements documented in
> tickets.)
>
> I'm wondering if you really have to go through QEMU. Or, if feeding
> assembly code to a parser and then doing the math on that wouldn't be
> sufficient? (Of course, tracing the execution is more accurate. However,
> it's much more complicated as well.)
>
> To account Assembly instructions to Cmm statements you may use the GHC
> parameters -ddump-cmm and -dppr-debug (and to stream this into files
> instead of stdout -ddump-to-file.) This will add comments for most Cmm
> statements into the dumped assembly code.
>
> At first, I thought that sign-extension / truncation might be a good
> candidate. However, it turned out that this is already covered by the
> RISC-V B-extension. Which led to this new ticket:
> https://gitlab.haskell.org/ghc/ghc/-/issues/25966
>
> Skimming over the NCG code and watching out for longer or repeating
> instruction lists might be a good strategy to make educated guesses.
>
> From a developer's perspective, I found the immediate sizes (usually
> 12bit) rather limiting. E.g. the Note [RISCV64 far jumps] (
> https://gitlab.haskell.org/ghc/ghc/-/blob/395e0ad17c0d309637f079a05dbdc23e0d4188f6/compiler/GHC/CmmToAsm/RV64/CodeGen.hs?page=2#L1996)
> tells a story how we had to work around this limit for addresses in
> conditional jumps.
>
> So, you could raise the question if - analog to compressed expressions -
> it wouldn't make sense to have extended expressions that cover two words.
> Such that the first word is the instruction and the second it's
> immediate(s). (Hardware designers would probably hate that, because it
> means a bigger change to the instruction decoding unit. However, I got
> asked as a software developer ;) )
>
> Other than that, I've unfortunately got no great ideas.
>
> Please feel free to keep us in the loop (especially regarding the results
> of your analyses.) And, if you've got any questions regarding the RISC-V
> NCG, please feel free to reach out either here or directly to me. There's
> also a #GHC "room" on Matrix where you can quickly drop smaller scoped
> questions.
>
> I hope that was of any help. Best regards,
>
> Sven
>
> Am Mi., 16. Apr. 2025 um 10:34 Uhr schrieb Matthew Pickering <
> matthewtpickering at gmail.com>:
>
>> Hi Daniel. I think Sven Tennie and the LoongArch contributors are the
>> experts in NCG for these kinds of instruction sets. I have cced them.
>>
>> Cheers,
>>
>> Matt
>>
>> On Tue, Apr 15, 2025 at 5:40 PM Daniel Trujillo Viedma <
>> danihacker.viedma at gmail.com> wrote:
>>
>>> Hello, ghc-devs! My name is Daniel Trujillo, I'm a Haskell enthusiast
>>> from Spain and I'm trying to make my Master's thesis about accelerating
>>> Haskell programs with a custom ISA extension.
>>>
>>>
>>> Right now, my focus is in executing software written in Haskell within
>>> QEMU in order to get traces that tells me, basically, how many times
>>> each block (not exactly basic blocks, but sort of) of assembly code has
>>> been executed, with the hope of finding some patterns of RISCV
>>> instructions that I could implement together into 1 instruction.
>>>
>>>
>>> As you can see, my method is a bit crude, and I was wondering if the
>>> people involved with any of the different internal representations (STG,
>>> Cmm...) and/or native code generators (particularly RISCV) could provide
>>> me hints about assembly instructions that would have made the work
>>> easier, by removing the need of "massaging" the Cmm code to make CodeGen
>>> easier, or the need of particular optimizations, or in general, dirty
>>> tricks because of lacking of proper support of the standard RISCV ISA.
>>>
>>>
>>> And of course, I would also appreciate very much other hints from people
>>> involved in general performance (as oppossed to, for example, libraries
>>> for SIMD and parallel execution, or Haskell wrappers to lower-level code
>>> for performance reasons).
>>>
>>>
>>> P.D. I'm sorry if I broke any netiquette rule, but I'm very new to the
>>> email list, and haven't received yet any email from it.
>>>
>>>
>>> Looking forward to hear from you!
>>>
>>> Cheers,
>>>
>>> Dani.
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250418/c2e4430b/attachment.html>

From ben at well-typed.com  Fri Apr 18 23:31:12 2025
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 18 Apr 2025 19:31:12 -0400
Subject: GHC 9.10.2-rc1 is now available
Message-ID: <87sem5ujwi.fsf@smart-cactus.org>


The GHC developers are very pleased to announce the availability
of the release candidate for GHC 9.10.2. Binary distributions, source
distributions, and documentation are available at [downloads.haskell.org][] and
via [GHCup](https://www.haskell.org/ghcup/).

GHC 9.10.2 is a bug-fix release fixing over 50 issues of a variety of
severities and scopes. A full accounting of these fixes can be found in
the [release notes][]. As always, GHC's release status, including
planned future releases, can be found on the GHC Wiki [status][].

This release candidate will have a two-week testing period. If all goes well
the final release will be available the week of 1 May 2025.

We would like to thank Well-Typed, Tweag I/O, Juspay, QBayLogic, Channable,
Serokell, SimSpace, the Haskell Foundation, and other anonymous contributors
whose on-going financial and in-kind support has facilitated GHC maintenance
and release management over the years. Finally, this release would not have
been possible without the hundreds of open-source contributors whose work
comprise this release.

As always, do give this release a try and open a [ticket][] if you see
anything amiss.

Cheers,

- Ben


[release notes]: https://gitlab.haskell.org/ghc/ghc/-/blob/ghc-9.10/docs/users_guide/9.10.2-notes.rst?ref_type=heads&plain=1
[status]: https://gitlab.haskell.org/ghc/ghc/-/wikis/GHC-status
[downloads.haskell.org]: https://downloads.haskell.org/ghc/9.10.2-rc1
[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 255 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250418/7872b790/attachment.sig>

From danihacker.viedma at gmail.com  Sat Apr 19 19:23:52 2025
From: danihacker.viedma at gmail.com (Daniel Trujillo Viedma)
Date: Sat, 19 Apr 2025 21:23:52 +0200
Subject: Questions regarding the RISCV native codegen & performance
In-Reply-To: <CALgrwrb+tHz9c0rH9G45ydmNQX5PV6rVBCyLf8BCSf_t4ib2oA@mail.gmail.com>
References: <896e1326-02cf-415c-bdab-7c5150911a1c@gmail.com>
 <CALuQ0m8qtUne=NCohX0oh64_gxPu4gWmp88Ee-cSgiHVX5RDCQ@mail.gmail.com>
 <CALgrwrYHdOkQ_5+7D-UqLHrkaR_G3-iC-RxKu4DTmaurYME3Xw@mail.gmail.com>
 <22850366-f63a-4158-a2b4-9d01e78173b6@gmail.com>
 <CALgrwrb+tHz9c0rH9G45ydmNQX5PV6rVBCyLf8BCSf_t4ib2oA@mail.gmail.com>
Message-ID: <c59eac4a-27d2-449f-a216-a970d18cdd9e@gmail.com>

Thank you so much, again!


I can't overstate how useful all these recommendations are, and I'm 
looking forward to see how these play out in simulations. So I guess 
it's time for me to work!


Sincerely yours,

Dani.


On 18/4/25 14:41, Sven Tennie wrote:
> Hey Daniel 👋
>
> Thanks a lot for your kind words.
>
> The AArch64 ISA might also be some source of inspiration. AArch64 has 
> some combined instructions which RISC-V hasn't. E.g ADD of two 
> registers with an included shift. Though, we don't seem to use many of 
> them and I haven't found any usage that wouldn't be well covered in 
> RISC-V NCG. Probably, that's because MachOp 
> (https://hackage.haskell.org/package/ghc-9.12.1/docs/GHC-Cmm-MachOp.html#t:MachOp) 
> is pretty fine grained.
>
> A good candidate for investigations could be the CSET 
> pseudo-instruction. I stumbled over it while looking for pseudo-ops 
> which lead to combined instructions in AArch64 NCG. The CSET pseudo-op 
> leads to two instructions in RISC-V NCG and to one in AArch64 NCG:
> - 
> https://gitlab.haskell.org/ghc/ghc/-/blob/386f18548e3c66d04f648a9d34f167a086c1328b/compiler/GHC/CmmToAsm/AArch64/Ppr.hs#L443
> - 
> https://gitlab.haskell.org/ghc/ghc/-/blob/386f18548e3c66d04f648a9d34f167a086c1328b/compiler/GHC/CmmToAsm/RV64/Ppr.hs#L554
>
> Though, this might be a sub-optimal implementation (in this case we'd 
> be happy to get a ticket ;) ). As CSET is used for comparisons, it 
> should appear pretty frequently.
>
> A bit off-topic, but for the sake of completeness: The Compiler 
> Explorer seems to use DWARF symbols to map assembly instructions to 
> Haskell code lines. At least, it compiles with -g.
>
> VELDT's profile is stated as "RV32I (no FENCE, ECALL, EBREAK)" on 
> their Github page. But, we target RV64G with both, the NCG and LLVM 
> backends. (The main reason to not support simpler profiles is that all 
> hardware on the market that is powerful enough to reasonably run 
> Haskell supports at least RV64G.)
>
> Thanks for the hint about the J-extension. I will take a look at it.
>
> Enjoy your weekend & best regards,
>
> Sven
>
> Am Fr., 18. Apr. 2025 um 12:13 Uhr schrieb Daniel Trujillo Viedma 
> <danihacker.viedma at gmail.com>:
>
>     Thank you so much for all the information and the help.
>
>
>     Seriously, this is much more than I was hoping to get, even the
>     suggestion for generating commented assembly code (which is, I
>     assume, the method that Compiler Explorer uses to relate the
>     high-level Haskell code with the assembly output of the compiler,
>     which is really nice). And four your RISC-V NGC, which I found
>     easy to understand.
>
>
>     I guess this is the kind of professionals that Haskell attracts,
>     which is a big part of why I love it.
>
>
>     I will send here an executive summary of my findings, including
>     statistics about a couple of programs that I try. I don't know if
>     I'll be able to do a very statistical significant analysis,
>     because I'll still have a lot of things to do (to extend QEMU,
>     maybe also gem5, and implement it in a Clash microprocessor
>     design, probably VELDT), but maybe in the future I can automate
>     more of it and running a more comprehensive analysis. FYI, I have
>     found that the RISC-V specs mention Haskell among other languages
>     in a still empty J extension section, which will be aimed at
>     helping dynamically translated languages as well as
>     garbage-collected, but I guess RISC-V people is still more focused
>     on other things and it will take some time to start work on that
>     extension.
>
>
>     I find also very interesting your suggestion for far-jumping, but
>     I'm afraid that will be very unpopular among hardware designers
>     because it messes with their highly appreciated and scarce L1
>     cache. But funnily enough, I had the impresion before starting
>     this project that some kind of simple mechanism for complex
>     jumping would be a good idea. I will keep this in mind when
>     looking for patterns in the assembly code.
>
>
>     Once more, thank you so much for your work and the help, and I
>     hope I can deliver soon information that you all could find
>     interesting.
>
>
>     Have a very nice weekend!
>
>     Cheers,
>
>     Dani.
>
>
>     On 17/4/25 9:19, Sven Tennie wrote:
>>     Hey Daniel 👋
>>
>>     That's really an interesting topic, because we never analyzed the
>>     emitted RISC-V assembly with statistical measures.
>>
>>     So, if I may ask for a favour: If you spot anything that could be
>>     better expressed with the current ISA, please open a ticket and
>>     label it as RISC-V: https://gitlab.haskell.org/ghc/ghc/-/issues
>>     (We haven't decided which RISC-V profile to require. I.e.
>>     requiring the very latest extensions would frustrate people with
>>     older hardware... However, it's in anycase good to have possible
>>     improvements documented in tickets.)
>>
>>     I'm wondering if you really have to go through QEMU. Or, if
>>     feeding assembly code to a parser and then doing the math on that
>>     wouldn't be sufficient? (Of course, tracing the execution is more
>>     accurate. However, it's much more complicated as well.)
>>
>>     To account Assembly instructions to Cmm statements you may use
>>     the GHC parameters -ddump-cmm and -dppr-debug (and to stream this
>>     into files instead of stdout -ddump-to-file.) This will add
>>     comments for most Cmm statements into the dumped assembly code.
>>
>>     At first, I thought that sign-extension / truncation might be a
>>     good candidate. However, it turned out that this is already
>>     covered by the RISC-V B-extension. Which led to this new ticket:
>>     https://gitlab.haskell.org/ghc/ghc/-/issues/25966
>>
>>     Skimming over the NCG code and watching out for longer or
>>     repeating instruction lists might be a good strategy to make
>>     educated guesses.
>>
>>     From a developer's perspective, I found the immediate sizes
>>     (usually 12bit) rather limiting. E.g. the Note [RISCV64 far
>>     jumps]
>>     (https://gitlab.haskell.org/ghc/ghc/-/blob/395e0ad17c0d309637f079a05dbdc23e0d4188f6/compiler/GHC/CmmToAsm/RV64/CodeGen.hs?page=2#L1996)
>>     tells a story how we had to work around this limit for addresses
>>     in conditional jumps.
>>
>>     So, you could raise the question if - analog to compressed
>>     expressions - it wouldn't make sense to have extended expressions
>>     that cover two words. Such that the first word is the instruction
>>     and the second it's immediate(s). (Hardware designers would
>>     probably hate that, because it means a bigger change to the
>>     instruction decoding unit. However, I got asked as a software
>>     developer ;) )
>>
>>     Other than that, I've unfortunately got no great ideas.
>>
>>     Please feel free to keep us in the loop (especially regarding the
>>     results of your analyses.) And, if you've got any questions
>>     regarding the RISC-V NCG, please feel free to reach out either
>>     here or directly to me. There's also a #GHC "room" on Matrix
>>     where you can quickly drop smaller scoped questions.
>>
>>     I hope that was of any help. Best regards,
>>
>>     Sven
>>
>>     Am Mi., 16. Apr. 2025 um 10:34 Uhr schrieb Matthew Pickering
>>     <matthewtpickering at gmail.com>:
>>
>>         Hi Daniel. I think Sven Tennie and the LoongArch contributors
>>         are the experts in NCG for these kinds of instruction sets. I
>>         have cced them.
>>
>>         Cheers,
>>
>>         Matt
>>
>>         On Tue, Apr 15, 2025 at 5:40 PM Daniel Trujillo Viedma
>>         <danihacker.viedma at gmail.com> wrote:
>>
>>             Hello, ghc-devs! My name is Daniel Trujillo, I'm a
>>             Haskell enthusiast
>>             from Spain and I'm trying to make my Master's thesis
>>             about accelerating
>>             Haskell programs with a custom ISA extension.
>>
>>
>>             Right now, my focus is in executing software written in
>>             Haskell within
>>             QEMU in order to get traces that tells me, basically, how
>>             many times
>>             each block (not exactly basic blocks, but sort of) of
>>             assembly code has
>>             been executed, with the hope of finding some patterns of
>>             RISCV
>>             instructions that I could implement together into 1
>>             instruction.
>>
>>
>>             As you can see, my method is a bit crude, and I was
>>             wondering if the
>>             people involved with any of the different internal
>>             representations (STG,
>>             Cmm...) and/or native code generators (particularly
>>             RISCV) could provide
>>             me hints about assembly instructions that would have made
>>             the work
>>             easier, by removing the need of "massaging" the Cmm code
>>             to make CodeGen
>>             easier, or the need of particular optimizations, or in
>>             general, dirty
>>             tricks because of lacking of proper support of the
>>             standard RISCV ISA.
>>
>>
>>             And of course, I would also appreciate very much other
>>             hints from people
>>             involved in general performance (as oppossed to, for
>>             example, libraries
>>             for SIMD and parallel execution, or Haskell wrappers to
>>             lower-level code
>>             for performance reasons).
>>
>>
>>             P.D. I'm sorry if I broke any netiquette rule, but I'm
>>             very new to the
>>             email list, and haven't received yet any email from it.
>>
>>
>>             Looking forward to hear from you!
>>
>>             Cheers,
>>
>>             Dani.
>>
>>             _______________________________________________
>>             ghc-devs mailing list
>>             ghc-devs at haskell.org
>>             http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250419/8d628191/attachment.html>

From simon.peytonjones at gmail.com  Mon Apr 28 16:18:20 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 28 Apr 2025 17:18:20 +0100
Subject: Marge
Message-ID: <CAJKmMz8zZ-aZjZrZww21pgbh9adUaCmTOKvn66X0RZBPOF1Asg@mail.gmail.com>

Colleagues

Marge has failed to pick up !14242 for nearly 24 hrs.  Is something stuck?


Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250428/3b6290f2/attachment.html>

From allbery.b at gmail.com  Mon Apr 28 16:24:39 2025
From: allbery.b at gmail.com (Brandon Allbery)
Date: Mon, 28 Apr 2025 12:24:39 -0400
Subject: Marge
In-Reply-To: <CAJKmMz8zZ-aZjZrZww21pgbh9adUaCmTOKvn66X0RZBPOF1Asg@mail.gmail.com>
References: <CAJKmMz8zZ-aZjZrZww21pgbh9adUaCmTOKvn66X0RZBPOF1Asg@mail.gmail.com>
Message-ID: <CAKFCL4Vg=hJz85rmaYxNyRku48F2iWLhbejS=Zb44Jq77QoqHg@mail.gmail.com>

The dashboard (
https://grafana.gitlab.haskell.org/d/iiCppweMz/marge-bot?orgId=2&from=now-24h&to=now&timezone=browser&refresh=30m)
says it's the only mergeable MR, and Marge is configured to require at
least two MRs.

On Mon, Apr 28, 2025 at 12:18 PM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> Colleagues
>
> Marge has failed to pick up !14242 for nearly 24 hrs.  Is something stuck?
>
>
> Thanks
>
> Simon
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250428/0838a74e/attachment.html>

From simon.peytonjones at gmail.com  Mon Apr 28 20:02:47 2025
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 28 Apr 2025 21:02:47 +0100
Subject: Marge
In-Reply-To: <CAKFCL4Vg=hJz85rmaYxNyRku48F2iWLhbejS=Zb44Jq77QoqHg@mail.gmail.com>
References: <CAJKmMz8zZ-aZjZrZww21pgbh9adUaCmTOKvn66X0RZBPOF1Asg@mail.gmail.com>
 <CAKFCL4Vg=hJz85rmaYxNyRku48F2iWLhbejS=Zb44Jq77QoqHg@mail.gmail.com>
Message-ID: <CAJKmMz9EZE_SXOeMRhH3F=iW0v=Sa=6fVr_qRcCBZNm4zs9sOg@mail.gmail.com>

> Marge is configured to require at least two MRs.

Interesting, thank you.   I didn't know that.   Maybe after a while, if
there is only one, it should do it anyway.

Simon

On Mon, 28 Apr 2025 at 17:24, Brandon Allbery <allbery.b at gmail.com> wrote:

> The dashboard (
> https://grafana.gitlab.haskell.org/d/iiCppweMz/marge-bot?orgId=2&from=now-24h&to=now&timezone=browser&refresh=30m)
> says it's the only mergeable MR, and Marge is configured to require at
> least two MRs.
>
> On Mon, Apr 28, 2025 at 12:18 PM Simon Peyton Jones <
> simon.peytonjones at gmail.com> wrote:
>
>> Colleagues
>>
>> Marge has failed to pick up !14242 for nearly 24 hrs.  Is something stuck?
>>
>>
>> Thanks
>>
>> Simon
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> brandon s allbery kf8nh
> allbery.b at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250428/4b6782a6/attachment.html>

From allbery.b at gmail.com  Mon Apr 28 20:04:20 2025
From: allbery.b at gmail.com (Brandon Allbery)
Date: Mon, 28 Apr 2025 16:04:20 -0400
Subject: Marge
In-Reply-To: <CAJKmMz9EZE_SXOeMRhH3F=iW0v=Sa=6fVr_qRcCBZNm4zs9sOg@mail.gmail.com>
References: <CAJKmMz8zZ-aZjZrZww21pgbh9adUaCmTOKvn66X0RZBPOF1Asg@mail.gmail.com>
 <CAKFCL4Vg=hJz85rmaYxNyRku48F2iWLhbejS=Zb44Jq77QoqHg@mail.gmail.com>
 <CAJKmMz9EZE_SXOeMRhH3F=iW0v=Sa=6fVr_qRcCBZNm4zs9sOg@mail.gmail.com>
Message-ID: <CAKFCL4XUHSnc_B7Z8=fPb0WxV=-twCt32jvDRi-ygK6j8sOJpA@mail.gmail.com>

IIRC it's been discussed but that part of the bot is hard to change.

On Mon, Apr 28, 2025 at 4:03 PM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> > Marge is configured to require at least two MRs.
>
> Interesting, thank you.   I didn't know that.   Maybe after a while, if
> there is only one, it should do it anyway.
>
> Simon
>
> On Mon, 28 Apr 2025 at 17:24, Brandon Allbery <allbery.b at gmail.com> wrote:
>
>> The dashboard (
>> https://grafana.gitlab.haskell.org/d/iiCppweMz/marge-bot?orgId=2&from=now-24h&to=now&timezone=browser&refresh=30m)
>> says it's the only mergeable MR, and Marge is configured to require at
>> least two MRs.
>>
>> On Mon, Apr 28, 2025 at 12:18 PM Simon Peyton Jones <
>> simon.peytonjones at gmail.com> wrote:
>>
>>> Colleagues
>>>
>>> Marge has failed to pick up !14242 for nearly 24 hrs.  Is something
>>> stuck?
>>>
>>>
>>> Thanks
>>>
>>> Simon
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> brandon s allbery kf8nh
>> allbery.b at gmail.com
>>
>

-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20250428/864c74f1/attachment.html>