Trying to speedup GHC compile times...Help!

[Simon Peyton Jones]

Jeff

Great stuff!  Welcome.

I strongly urge you to keep a constantly-update status wiki page, which lists the ideas you are working on, and points to relevant resources and tickets.  An email thread like this is a good way to gather ideas, but NOT a good way to organise and track them.

Looking carefully at profiles is a good plan.  That's the hard data!

I think that some careful investigation of IntMap (at least the bits that GHC uses heavily) would be a good idea.  Clearly we spend a lot of time in these maps, so speedups here will yield a lot of benefit.  Look at the heavy hitters from the profile, stare at the Core code and see if it's s good as it can be.

For example, Sebastian discovered a strange infelicity in IntMap.lookup, which I've documented in a new ticket
https://gitlab.haskell.org/ghc/ghc/-/issues/20069

I think it'd also be worth measuring how unbalanced our IntMap trees get.  See
   https://gitlab.haskell.org/ghc/ghc/-/issues/19820
The speculation there is that we are getting very unbalanced trees.  So measure it!  If it's true, we could improve matters by using a different IntMap; or maybe by scrambling the key a bit --- see the ticket.

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Young, Jeff
Sent: 02 July 2021 02:36
To: ghc-devs at haskell.org
Subject: Trying to speedup GHC compile times...Help!

Hi ghc devs,

I'm a long-time Haskeller but am just getting into GHC development. I started a 12 week internship at Tweag I/O under Richard Eisenberg this week with the singular goal to speedup GHC compile times. I'm specifically looking to contribute to ghc issues 18541<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F18541&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394069068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zh5dzhUSjwsZdmr8B3G5T49CssX%2Bv8IGcHILRp8Ekjc%3D&reserved=0> and 18535<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F18535&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394079062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2I%2BQXmd%2Bn1nGz%2Bcr0qvhq9BLkKCucJkedPNxu7RpKSw%3D&reserved=0>. So I thought I would reach out to the community to get some direction on issues/features/problems to tackle in the pursuit of faster compilation times. This is a full time internship and so I think there is a real opportunity to nail down a deliverable for the community, but I want to get some guidance from the experts (you fine people!) before going down a rabbit hole.

To be specific I'm looking for lingering items such as:
  1. It would be great if we had <thing-here> but no one has time.
  2. Primop foo is half complete but is the right type for <common-use-case-which-is-currently-just-a-list>.
  3. Swap <some-type> to an array-like type non-incrementally, that is, establish a patch that rips out the previous type and replaces it with the array-like across the entire compiler, rather than module-by-module.

Point 2 is a specific reference to MR 3571<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fmerge_requests%2F3571&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394079062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5Sr1EjmeYN8ttZHFc4FS9%2BAXH1KoRSaNoozORhwwQYA%3D&reserved=0> but I'm unsure of the status and etiquette around MRs, and I'm unsure exactly how fulfilling the todos at the end of that MR would aid in faster compilation times (and if there is evidence to that effect somewhere).

Thanks for the help!

- Jeff



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210702/c9243693/attachment.html>