[Git][ghc/ghc][wip/romes/graph-compact-easy] driver: Store the HomePackageTable in a mutable reference

[Rodrigo Mesquita (@alt-romes)]

Rodrigo Mesquita pushed to branch wip/romes/graph-compact-easy at Glasgow Haskell Compiler / GHC


Commits:
4b2d116f by Rodrigo Mesquita at 2025-01-08T19:15:32+00:00
driver: Store the HomePackageTable in a mutable reference

This commit refactors the HomePackageTable and HomeUnitGraph:

(1) It fixes a quadratic-in-the-number-of-modules space leak in upsweep (#25511)

(2) And it reworks these structures into their own modules to simplify
    the driver. The refactor is driven by the introduction of IO in the HPT
    interface, but is a bit more aggressive in simplifying the
    interfaces to enforce correct usage (ie to avoid performance
    pitfalls).

Specifically:

- The `HomeUnitGraph` (HUG) is now in `GHC.Unit.Home.Graph`
- The `HomePackageTable` (HPT) is now in `GHC.Unit.Home.PackageTable`
    - The HPT now stores an `IORef` with the table of loaded home package modules.
    - The interface to the HPT now requires IO
    - The interface now enforces that the HPT is a datastructure that
      only grows
    - This is not enforced in the interface, but, clients of the HPT
      should never care about there being more or less entries in the
      HPT when these additional entries are not relevant to their result.
    - The exception to the invariant that the HPT is monotonically
      increasing is `restrictHpt`, a function which is called at a
      "barrier point" (during which there are no other threads
      inspecting or inserting in the HPT). The invariant can be
      temporarily broken at this point (currently, after upsweep).
      This is safe because a single thread holds control over the
      structure (thus the invariant being broken is never observed).

The hug_var and associated structures in the driver, which aimed to
improve memory usage in the driver by updating in place a HUG during
upsweep, are no longer required as the HPT entries in the HUG are now
themselves mutable by construction. This was previously explained in
Note [ModuleNameSet, efficiency and space leaks], which is no longer
relevant and was deleted.

Fixes #25511

Co-authored-by: Matthew Pickering <matthewtpickering at gmail.com>

-------------------------
Metric Decrease:
    MultiComponentModulesRecomp
    MultiLayerModulesRecomp
-------------------------

- - - - -


30 changed files:

- compiler/GHC.hs
- compiler/GHC/Core/Opt/Pipeline.hs
- compiler/GHC/Data/IOEnv.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Hooks.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/HsToCore/Monad.hs
- compiler/GHC/HsToCore/Usage.hs
- compiler/GHC/Iface/Errors.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/IfaceToCore.hs
- compiler/GHC/Linker/Deps.hs
- compiler/GHC/Linker/ExtraObj.hs
- compiler/GHC/Linker/Loader.hs
- compiler/GHC/Linker/Static.hs
- compiler/GHC/Rename/Names.hs
- compiler/GHC/Runtime/Eval.hs
- compiler/GHC/Runtime/Interpreter.hs
- compiler/GHC/Runtime/Interpreter/JS.hs
- compiler/GHC/Runtime/Loader.hs
- compiler/GHC/StgToByteCode.hs
- compiler/GHC/StgToJS/Linker/Linker.hs
- compiler/GHC/SysTools/Cpp.hs
- compiler/GHC/Tc/Instance/Family.hs
- compiler/GHC/Tc/Module.hs
- compiler/GHC/Tc/Utils/Monad.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4b2d116fb5e8472ca08c7e41b7d1858cb53de6ff

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4b2d116fb5e8472ca08c7e41b7d1858cb53de6ff
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250108/3a5afe82/attachment.html>