[Git][ghc/ghc][wip/t25571] 12 commits: perf: Speed up the bytecode assembler
Matthew Pickering (@mpickering)
gitlab at gitlab.haskell.org
Thu Mar 6 11:40:20 UTC 2025
Matthew Pickering pushed to branch wip/t25571 at Glasgow Haskell Compiler / GHC
Commits:
8f6cc90c by Matthew Pickering at 2025-03-05T04:48:02-05:00
perf: Speed up the bytecode assembler
This commit contains a number of optimisations to the bytecode
assembler. In programs which generate a large amount of bytecode, the
assembler is called a lot of times on many instructions.
1. Specialise the assembleI function for the two intepreters to avoid
having to materialise the intermediate free-monad like structure.
2. Directly compute the UArray and SmallArray needed rather than going
via the intermediate SizedSeq
3. Use optimised monads
4. Define unrolled "any" and "mapM6" functions which can be inlined
and avoid calling recursive functions.
The resulting generated code is much more direct.
Before:
./ByteCodeAsm /home/matt/ghc-profiling-light/_build/stage1/lib/ +RTS -s
48,923,125,664 bytes allocated in the heap
678,221,152 bytes copied during GC
395,648 bytes maximum residency (2 sample(s))
50,040 bytes maximum slop
6 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 11731 colls, 0 par 0.419s 0.425s 0.0000s 0.0004s
Gen 1 2 colls, 0 par 0.001s 0.001s 0.0007s 0.0012s
INIT time 0.000s ( 0.000s elapsed)
MUT time 6.466s ( 6.484s elapsed)
GC time 0.421s ( 0.426s elapsed)
EXIT time 0.000s ( 0.000s elapsed)
Total time 6.887s ( 6.910s elapsed)
After:
1,518,321,200 bytes allocated in the heap
4,299,552 bytes copied during GC
322,288 bytes maximum residency (2 sample(s))
50,280 bytes maximum slop
6 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 369 colls, 0 par 0.003s 0.003s 0.0000s 0.0002s
Gen 1 2 colls, 0 par 0.001s 0.001s 0.0007s 0.0012s
INIT time 0.001s ( 0.001s elapsed)
MUT time 0.465s ( 0.466s elapsed)
GC time 0.004s ( 0.004s elapsed)
EXIT time 0.000s ( 0.000s elapsed)
Total time 0.470s ( 0.471s elapsed)
- - - - -
f2d43e11 by Teo Camarasu at 2025-03-05T04:48:40-05:00
ghc-boot-th: expose all TH packages from proper GHC.Boot.* modules
Previously we defined some modules here in the GHC.Internal namespace.
Others were merely re-exposed from GHC.Internal.
Re-exposed modules weren't handled correctly by Haddock, so the
Haddocks for the `template-haskell` library couldn't see them.
This change also makes the home package of these modules a bit clearer.
Work towards #25705
- - - - -
91ef82df by Teo Camarasu at 2025-03-05T04:48:40-05:00
ghc-boot-th: fix synopsis formatting
`@...@` syntax doesn't seem to work in synposes and is just kept by
Haddock verbatim.
- - - - -
eb9fe1ec by Brandon Chinn at 2025-03-05T04:49:17-05:00
Collapse string gaps as \& (#25784)
In 9.10, "\65\ \0" would result in "A0", but in 9.12, it results in
"\650", due to the string refactoring I did in !13128. Previously, we
were resolving escape codes and collapsing string gaps as we come across
them, but after the refactor, string processing is broken out into
phases, which is both more readable and useful for multiline strings.
- - - - -
8037f487 by Cheng Shao at 2025-03-05T04:49:54-05:00
ghc-experimental: make JSVal abstract in GHC.Wasm.Prim
This commit makes JSVal an abstract type in the export list of
GHC.Wasm.Prim. JSVal's internal representation is supposed to be a non
user facing implementation detail subject to change at any time. We
should only expose things that are newtypes of JSVal, not JSVal
itself.
- - - - -
4f342431 by Cheng Shao at 2025-03-05T04:49:54-05:00
wasm: make JSVal internal Weak# point to lifted JSVal
JSVal has an internal Weak# with the unlifted JSVal# object as key to
arrange its builtin finalization logic. The Weak# used to designate
Unit_closure as a dummy value; now this commit designates the lifted
JSVal closure as the Weak# value. This allows the implementation of
mkWeakJSVal which can be used to observe the liveliness of a JSVal and
attach a user-specified finalizer.
- - - - -
55af20e6 by Cheng Shao at 2025-03-05T04:49:54-05:00
ghc-experimental: add mkWeakJSVal
This commit adds a mkWeakJSVal function that can be used to set up a
Weak pointer with a JSVal key to observe the key's lifetime and
optionally attach a finalizer.
- - - - -
8273d7d1 by Matthew Pickering at 2025-03-05T04:50:30-05:00
simplifier: Zap Id unfoldings before constructing InScopeSet in simpleOptExpr
Care must be taken to remove unfoldings from `Var`s collected by exprFreeVars
before using them to construct an in-scope set hence `zapIdUnfolding` in `init_subst`.
Consider calling `simpleOptExpr` on an expression like
```
case x of (a,b) -> (x,a)
```
* One of those two occurrences of x has an unfolding (the one in (x,a), with
unfolding x = (a,b)) and the other does not. (Inside a case GHC adds
unfolding-info to the scrutinee's Id.)
* But exprFreeVars just builds a set, so it's a bit random which occurrence is collected.
* Then simpleOptExpr replaces each occurrence of x with the one in the in-scope set.
* Bad bad bad: then the x in case x of ... may be replaced with a version that has an unfolding.
Fixes #25790
- - - - -
07fe6d1d by Rodrigo Mesquita at 2025-03-05T04:51:07-05:00
docs: Fix ghci :doc documentation
Fixes #25799
- - - - -
c5fa0226 by Matthew Pickering at 2025-03-06T11:34:02+00:00
Add flag to control whether self-recompilation information is written to interface
This patch adds the flag -fwrite-if-self-recomp which controls whether
interface files contain the information necessary to answer the
question:
Do I need to recompile myself or is this current interface file
suitable?
Why? Most packages are only built once either by a distribution or cabal
and then placed into an immutable store, after which we will never ask
this question. Therefore we can derive two benefits from omitting this
information.
* Primary motivation: It vastly reduces the surface area for creating
non-deterministic interface files. See issue #10424 which motivated a
proper fix to that issue. Distributions have long contained versions
of GHC which just have broken self-recompilation checking (in order to
get deterministic interface files).
* Secondary motivation: This reduces the size of interface files
slightly.. the `mi_usages` field can be quite big but probably this
isn't such a great benefit.
* Third motivation: Conceptually clarity about which parts of an
interface file are used in order to **communicate** with subsequent
packages about the **interface** for a module. And which parts are
used to self-communicate during recompilation checking.
The main tracking issue is #22188 but fixes issues such as #10424 in a
proper way.
- - - - -
def66aef by Matthew Pickering at 2025-03-06T11:34:02+00:00
Disable self recomp in release flavour
The interface files that we distribute should not contain any
information which is used by the recompilation checking logic since
source file will never be compiled again.
I am not 100% sure this won't cause unexpected issues, there many be
downstream consumers which are incorrectly using the information from
interfaces, but this commit can be reverted if we detect issues.
- - - - -
fbe973e4 by Matthew Pickering at 2025-03-06T11:34:14+00:00
iface: Store flags in interface files
When reporting the reason why a module is recompiled (using
`-dump-hi-diffs`), it is much more informative to inform the user about
which flag exactly has changed, rather than just an opaque reference to
a hash.
Now, when the user enables `-fwrite-if-self-recomp-flags`
there is a difference the precise part of the flags is
reported:
```
codegen flags changed:
before: [Opt_NoTypeableBinds, Opt_OmitYields]
after: [Opt_NoTypeableBinds, Opt_OmitYields, Opt_DictsStrict]
```
Fixes #25571
- - - - -
90 changed files:
- .gitlab/rel_eng/upload_ghc_libs.py
- compiler/GHC.hs
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/ByteCode/Instr.hs
- compiler/GHC/ByteCode/Types.hs
- compiler/GHC/Core/Opt/CallerCC/Types.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Data/FlatBag.hs
- compiler/GHC/Data/SmallArray.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Flags.hs
- + compiler/GHC/Driver/IncludeSpecs.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/HsToCore.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Env.hs
- + compiler/GHC/Iface/Flags.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Recomp/Flags.hs
- + compiler/GHC/Iface/Recomp/Types.hs
- compiler/GHC/Parser/String.hs
- compiler/GHC/Plugins.hs
- compiler/GHC/Rename/Splice.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Tc/Gen/Splice.hs-boot
- compiler/GHC/Tc/Plugin.hs
- compiler/GHC/Tc/Types/ErrCtxt.hs
- compiler/GHC/Tc/Types/TH.hs
- compiler/GHC/ThToHs.hs
- compiler/GHC/Types/ProfAuto.hs
- compiler/GHC/Types/SafeHaskell.hs
- compiler/GHC/Unit/Module/ModGuts.hs
- compiler/GHC/Unit/Module/ModIface.hs
- compiler/ghc.cabal.in
- docs/users_guide/ghci.rst
- docs/users_guide/phases.rst
- hadrian/doc/flavours.md
- hadrian/src/Flavour.hs
- hadrian/src/Settings/Flavours/Release.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Lib.hs
- libraries/ghc-boot-th/GHC/Internal/TH/Lib/Map.hs → libraries/ghc-boot-th/GHC/Boot/TH/Lib/Map.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Lift.hs
- libraries/ghc-boot-th/GHC/Internal/TH/Ppr.hs → libraries/ghc-boot-th/GHC/Boot/TH/Ppr.hs
- libraries/ghc-boot-th/GHC/Internal/TH/PprLib.hs → libraries/ghc-boot-th/GHC/Boot/TH/PprLib.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Quote.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Syntax.hs
- libraries/ghc-boot-th/ghc-boot-th.cabal.in
- libraries/ghc-experimental/src/GHC/Wasm/Prim.hs
- libraries/ghc-internal/src/GHC/Internal/Wasm/Prim.hs
- libraries/ghc-internal/src/GHC/Internal/Wasm/Prim/Types.hs
- libraries/ghci/GHCi/Message.hs
- libraries/ghci/GHCi/TH.hs
- libraries/ghci/GHCi/TH/Binary.hs
- libraries/template-haskell/Language/Haskell/TH/Lib.hs
- libraries/template-haskell/Language/Haskell/TH/Lib/Internal.hs
- libraries/template-haskell/Language/Haskell/TH/Ppr.hs
- libraries/template-haskell/Language/Haskell/TH/PprLib.hs
- libraries/template-haskell/Language/Haskell/TH/Quote.hs
- libraries/template-haskell/Language/Haskell/TH/Syntax.hs
- rts/wasm/JSFFI.c
- testsuite/tests/count-deps/CountDepsAst.stdout
- testsuite/tests/count-deps/CountDepsParser.stdout
- + testsuite/tests/driver/self-recomp/Makefile
- + testsuite/tests/driver/self-recomp/SelfRecomp01.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp02.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp03.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.stdout
- + testsuite/tests/driver/self-recomp/all.T
- + testsuite/tests/ghci/should_run/T25790.hs
- + testsuite/tests/ghci/should_run/T25790.script
- testsuite/tests/ghci/should_run/all.T
- testsuite/tests/interface-stability/template-haskell-exports.stdout
- testsuite/tests/jsffi/jsffigc.hs
- + testsuite/tests/parser/should_run/T25784.hs
- + testsuite/tests/parser/should_run/T25784.stdout
- testsuite/tests/parser/should_run/all.T
- + testsuite/tests/perf/should_run/ByteCodeAsm.hs
- testsuite/tests/perf/should_run/all.T
- utils/haddock/html-test/ref/QuasiExpr.html
- utils/haddock/html-test/ref/TH.html
- utils/haddock/html-test/ref/Threaded_TH.html
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/70b5f1da951de72f23f57c2da99e8fabc0fb2ded...fbe973e4aff63e953415337cba57aad42fb9fc75
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/70b5f1da951de72f23f57c2da99e8fabc0fb2ded...fbe973e4aff63e953415337cba57aad42fb9fc75
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250306/39d0dace/attachment-0001.html>
More information about the ghc-commits
mailing list