<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">One piece I'm curious about, reading this thread: why do we have so many IntMaps and operations on them? Name lookup is a fundamental operation a compiler must do, and that would use an IntMap: good. But maybe there are other IntMaps used that are less necessary. A key example: whenever we do substitution, we track an InScopeSet, which is really just an IntMap. This InScopeSet remembers the name of all variables in scope, useful when we need to create a new variable name (this is done by uniqAway). Yet perhaps the tracking of these in-scope variables is very expensive and comprises much of the IntMap time. Might it be better just to always work in a monad capable of giving fresh names? We actually don't even need a monad, if that's too annoying. Instead, we could just pass around an infinite list of fresh uniques. This would still be clutterful, but if it grants us a big speed improvement, the clutter might be worth it.<div class=""><br class=""></div><div class="">The high-level piece here is that there may be good things that come from understanding where these IntMaps arise.</div><div class=""><br class=""></div><div class="">Richard<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 2, 2021, at 4:08 AM, Simon Peyton Jones via ghc-devs <<a href="mailto:ghc-devs@haskell.org" class="">ghc-devs@haskell.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">Jeff<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">Great stuff! Welcome.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">I strongly urge you to keep a constantly-update status wiki page, which lists the ideas you are working on, and points to relevant resources and tickets. An email thread like this is a good way to gather ideas, but NOT a good way to organise and track them.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">Looking carefully at profiles is a good plan. That’s the hard data!<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">I think that some careful investigation of IntMap (at least the bits that GHC uses heavily) would be a good idea. Clearly we spend a lot of time in these maps, so speedups here will yield a lot of benefit. Look at the heavy hitters from the profile, stare at the Core code and see if it’s s good as it can be.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">For example, Sebastian discovered a strange infelicity in IntMap.lookup, which I’ve documented in a new ticket<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 5.25pt;" class=""><span class=""><a href="https://gitlab.haskell.org/ghc/ghc/-/issues/20069" style="color: blue; text-decoration: underline;" class="">https://gitlab.haskell.org/ghc/ghc/-/issues/20069</a><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">I think it’d also be worth measuring how unbalanced our IntMap trees get. See<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""> <span class="Apple-converted-space"> </span><a href="https://gitlab.haskell.org/ghc/ghc/-/issues/19820" style="color: blue; text-decoration: underline;" class="">https://gitlab.haskell.org/ghc/ghc/-/issues/19820</a><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">The speculation there is that we are getting very unbalanced trees. So measure it! If it’s true, we could improve matters by using a different IntMap; or maybe by scrambling the key a bit --- see the ticket.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="">Simon<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="border-style: none none none solid; border-left-width: 1.5pt; border-left-color: blue; padding: 0cm 0cm 0cm 4pt;" class=""><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span lang="EN-US" class="">From:</span></b><span lang="EN-US" class=""><span class="Apple-converted-space"> </span>ghc-devs <<a href="mailto:ghc-devs-bounces@haskell.org" class="">ghc-devs-bounces@haskell.org</a>><span class="Apple-converted-space"> </span><b class="">On Behalf Of<span class="Apple-converted-space"> </span></b>Young, Jeff<br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>02 July 2021 02:36<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span><a href="mailto:ghc-devs@haskell.org" class="">ghc-devs@haskell.org</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Trying to speedup GHC compile times...Help!<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">Hi ghc devs,<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">I'm a long-time Haskeller but am just getting into GHC development. I started a 12 week internship at Tweag I/O under Richard Eisenberg this week with the singular goal to speedup GHC compile times. I'm specifically looking to contribute to ghc issues<span class="Apple-converted-space"> </span><a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F18541&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394069068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zh5dzhUSjwsZdmr8B3G5T49CssX%2Bv8IGcHILRp8Ekjc%3D&reserved=0" style="color: blue; text-decoration: underline;" class="">18541</a>and<span class="Apple-converted-space"> </span><a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F18535&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394079062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2I%2BQXmd%2Bn1nGz%2Bcr0qvhq9BLkKCucJkedPNxu7RpKSw%3D&reserved=0" style="color: blue; text-decoration: underline;" class="">18535</a>. So I thought I would reach out to the community to get some direction on issues/features/problems to tackle in the pursuit of faster compilation times. This is a full time internship and so I think there is a real opportunity to nail down a deliverable for the community, but I want to get some guidance from the experts (you fine people!) before going down a rabbit hole.<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">To be specific I'm looking for lingering items such as:<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"> 1. It would be great if we had <thing-here> but no one has time.<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"> 2. Primop foo is half complete but is the right type for <common-use-case-which-is-currently-just-a-list>.<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"> 3. Swap <some-type> to an array-like type<span class="Apple-converted-space"> </span><i class="">non-incrementally</i>, that is, establish a patch that rips out the previous type and replaces it with the array-like across the entire compiler, rather than module-by-module.<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">Point 2 is a specific reference to<span class="Apple-converted-space"> </span><a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fmerge_requests%2F3571&data=04%7C01%7Csimonpj%40microsoft.com%7C8a3369ac28654df6f08008d93cf9e28b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637607867394079062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5Sr1EjmeYN8ttZHFc4FS9%2BAXH1KoRSaNoozORhwwQYA%3D&reserved=0" style="color: blue; text-decoration: underline;" class="">MR 3571</a><span class="Apple-converted-space"> </span>but I'm unsure of the status and etiquette around MRs, and I'm unsure exactly how fulfilling the todos at the end of that MR would aid in faster compilation times (and if there is evidence to that effect somewhere).<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">Thanks for the help!<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;">- Jeff<o:p class=""></o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></p></div><div class=""><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <span class="Apple-converted-space"> </span><o:p class=""></o:p></p></div></div></div></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">ghc-devs mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""><a href="mailto:ghc-devs@haskell.org" class="">ghc-devs@haskell.org</a></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" class="">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a></span></div></blockquote></div><br class=""></div></body></html>