[Git][ghc/ghc][wip/fendor/ghc-iface-sharing-avoid-reserialisation] 6 commits: Make flip representation polymorphic, similar to ($) and (&)

Hannes Siebenhandl (@fendor) gitlab at gitlab.haskell.org
Mon Jun 17 15:32:30 UTC 2024 


Hannes Siebenhandl pushed to branch wip/fendor/ghc-iface-sharing-avoid-reserialisation at Glasgow Haskell Compiler / GHC


Commits:
e0099721 by Andrew Lelechenko at 2024-06-16T17:57:38-04:00
Make flip representation polymorphic, similar to ($) and (&)

CLC proposal: https://github.com/haskell/core-libraries-committee/issues/245

- - - - -
118a1292 by Alan Zimmerman at 2024-06-16T17:58:15-04:00
EPA: Add location to Match Pats list

So we can freely modify the pats and the following item spacing will
still be valid when exact printing.

Closes #24862

- - - - -
db343324 by Fabricio de Sousa Nascimento at 2024-06-17T10:01:51-04:00
compiler: Rejects RULES whose LHS immediately fails to type-check

Fixes GHC crashing on `decomposeRuleLhs` due to ignoring coercion values. This
happens when we have a RULE that does not type check, and enable
`-fdefer-type-errors`. We prevent this to happen by rejecting RULES with an
immediately LHS type error.

Fixes #24026

- - - - -
e7a95662 by Dylan Thinnes at 2024-06-17T10:02:35-04:00
Add hscTypecheckRenameWithDiagnostics, for HLS (#24996)

Use runHsc' in runHsc so that both functions can't fall out of sync

We're currently copying parts of GHC code to get structured warnings
in HLS, so that we can recreate `hscTypecheckRenameWithDiagnostics`
locally. Once we get this function into GHC we can drop the copied code
in future versions of HLS.

- - - - -
13320ff2 by Fendor at 2024-06-17T17:31:34+02:00
Improve sharing of duplicated values in `ModIface`, fixes #24723

As a `ModIface` often contains duplicated values that are not
necessarily shared, we improve sharing by serialising the `ModIface`
to an in-memory byte array. Serialisation uses deduplication tables, and
deserialisation implicitly shares duplicated values.

This helps reducing the peak memory usage while compiling in
`--make` mode. The peak memory usage is especially smaller when
generating interface files with core expressions
(`-fwrite-if-simplified-core`).

On agda, this reduces the peak memory usage:

* `2.2 GB` to `1.9 GB` for a ghci session.

On `lib:Cabal`, we report:

* `570 MB` to `500 MB` for a ghci session
* `790 MB` to `667 MB` for compiling `lib:Cabal` with ghc

There is a small impact on execution time, around 2% on the agda code
base.

- - - - -
52e2537a by Fendor at 2024-06-17T17:31:37+02:00
Avoid unneccessarily re-serialising the `ModIface`

To reduce memory usage of `ModIface`, we serialise `ModIface` to an
in-memory byte array, which implicitly shares duplicated values.

This serialised byte array can be reused to avoid work when we actually
write the `ModIface` to disk.
We introduce a new field to `ModIface` which allows us to save the byte
array, and write it direclty to disk if the `ModIface` wasn't changed
after the initial serialisation.

This requires us to change absolute offsets, for example to jump to the
deduplication table for `Name` or `FastString` with relative offsets, as
the deduplication byte array doesn't contain header information, such as
fingerprints.
To allow us to dump the binary blob to disk, we need to replace all
absolute offsets with relative ones.

We introduce additional helpers for `ModIface` binary serialisation, which
construct relocatable binary blobs. We say the binary blob is relocatable,
if the binary representation can be moved and does not contain any
absolute offsets.

Further, we introduce new primitives for `Binary` that allow to create
relocatable binaries, such as `forwardGetRel` and `forwardPutRel`.

-------------------------
Metric Decrease:
    MultiLayerModulesDefsGhcWithCore
Metric Increase:
    MultiComponentModules
    MultiLayerModules
    T10421
    T12150
    T12234
    T12425
    T13035
    T13253-spj
    T13701
    T13719
    T14697
    T15703
    T16875
    T18698b
    T18140
    T18304
    T18698a
    T18730
    T18923
    T20049
    T24582
    T5837
    T6048
    T9198
    T9961
    mhu-perf
-------------------------

These metric increases may look bad, but they are all completely benign,
we simply allocate 1 MB per module for `shareIface`. As this allocation
is quite quick, it has a negligible impact on run-time performance.
In fact, the performance difference wasn't measurable on my local
machine. Reducing the size of the pre-allocated 1 MB buffer avoids these
test failures, but also requires us to reallocate the buffer if the
interface file is too big. These reallocations *did* have an impact on
performance, which is why I have opted to accept all these metric
increases, as the number of allocated bytes is merely a guidance.

This 1MB allocation increase causes a lot of tests to fail that
generally have a low allocation number. E.g., increasing from 40MB to
41MB is a 2.5% increase.
In particular, the tests T12150, T13253-spj, T18140, T18304, T18698a,
T18923, T20049, T24582, T5837, T6048, and T9961 only fail on i386-darwin
job, where the number of allocated bytes seems to be lower than in other
jobs.
The tests T16875 and T18698b fail on i386-linux for the same reason.

- - - - -


30 changed files:

- compiler/GHC.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/Hs/Utils.hs
- compiler/GHC/HsToCore/Arrows.hs
- compiler/GHC/HsToCore/Binds.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/HsToCore/Match.hs
- compiler/GHC/HsToCore/Pmc/Desugar.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/HsToCore/Ticks.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Iface/Ext/Binary.hs
- compiler/GHC/Iface/Ext/Fields.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Rename.hs
- compiler/GHC/Parser.y
- compiler/GHC/Parser/Annotation.hs
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Rename/Bind.hs
- compiler/GHC/Rename/Expr.hs
- compiler/GHC/Rename/HsType.hs
- compiler/GHC/Rename/Module.hs
- compiler/GHC/Rename/Utils.hs
- compiler/GHC/Tc/Deriv/Functor.hs
- compiler/GHC/Tc/Deriv/Generate.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/146d298ecaa3b343b35b2a56713d1b022ce77728...52e2537a0c24ff3a75a808403a63313c809c355d

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/146d298ecaa3b343b35b2a56713d1b022ce77728...52e2537a0c24ff3a75a808403a63313c809c355d
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240617/a2481d7f/attachment.html>

More information about the ghc-commits mailing list