[Git][ghc/ghc][wip/22188] 11 commits: perf: Speed up the bytecode assembler

Matthew Pickering (@mpickering) gitlab at gitlab.haskell.org
Thu Mar 6 11:40:40 UTC 2025 


Matthew Pickering pushed to branch wip/22188 at Glasgow Haskell Compiler / GHC


Commits:
8f6cc90c by Matthew Pickering at 2025-03-05T04:48:02-05:00
perf: Speed up the bytecode assembler

This commit contains a number of optimisations to the bytecode
assembler. In programs which generate a large amount of bytecode, the
assembler is called a lot of times on many instructions.

1. Specialise the assembleI function for the two intepreters to avoid
having to materialise the intermediate free-monad like structure.
2. Directly compute the UArray and SmallArray needed rather than going
   via the intermediate SizedSeq
3. Use optimised monads
4. Define unrolled "any" and "mapM6" functions which can be inlined
   and avoid calling recursive functions.

The resulting generated code is much more direct.

Before:

./ByteCodeAsm /home/matt/ghc-profiling-light/_build/stage1/lib/ +RTS  -s
  48,923,125,664 bytes allocated in the heap
     678,221,152 bytes copied during GC
         395,648 bytes maximum residency (2 sample(s))
          50,040 bytes maximum slop
               6 MiB total memory in use (0 MiB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0     11731 colls,     0 par    0.419s   0.425s     0.0000s    0.0004s
  Gen  1         2 colls,     0 par    0.001s   0.001s     0.0007s    0.0012s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    6.466s  (  6.484s elapsed)
  GC      time    0.421s  (  0.426s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    6.887s  (  6.910s elapsed)

After:

   1,518,321,200 bytes allocated in the heap
       4,299,552 bytes copied during GC
         322,288 bytes maximum residency (2 sample(s))
          50,280 bytes maximum slop
               6 MiB total memory in use (0 MiB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0       369 colls,     0 par    0.003s   0.003s     0.0000s    0.0002s
  Gen  1         2 colls,     0 par    0.001s   0.001s     0.0007s    0.0012s

  INIT    time    0.001s  (  0.001s elapsed)
  MUT     time    0.465s  (  0.466s elapsed)
  GC      time    0.004s  (  0.004s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.470s  (  0.471s elapsed)

- - - - -
f2d43e11 by Teo Camarasu at 2025-03-05T04:48:40-05:00
ghc-boot-th: expose all TH packages from proper GHC.Boot.* modules

Previously we defined some modules here in the GHC.Internal namespace.
Others were merely re-exposed from GHC.Internal.

Re-exposed modules weren't handled correctly by Haddock, so the
Haddocks for the `template-haskell` library couldn't see them.
This change also makes the home package of these modules a bit clearer.

Work towards #25705

- - - - -
91ef82df by Teo Camarasu at 2025-03-05T04:48:40-05:00
ghc-boot-th: fix synopsis formatting

`@...@` syntax doesn't seem to work in synposes and is just kept by
Haddock verbatim.

- - - - -
eb9fe1ec by Brandon Chinn at 2025-03-05T04:49:17-05:00
Collapse string gaps as \& (#25784)

In 9.10, "\65\ \0" would result in "A0", but in 9.12, it results in
"\650", due to the string refactoring I did in !13128. Previously, we
were resolving escape codes and collapsing string gaps as we come across
them, but after the refactor, string processing is broken out into
phases, which is both more readable and useful for multiline strings.

- - - - -
8037f487 by Cheng Shao at 2025-03-05T04:49:54-05:00
ghc-experimental: make JSVal abstract in GHC.Wasm.Prim

This commit makes JSVal an abstract type in the export list of
GHC.Wasm.Prim. JSVal's internal representation is supposed to be a non
user facing implementation detail subject to change at any time. We
should only expose things that are newtypes of JSVal, not JSVal
itself.

- - - - -
4f342431 by Cheng Shao at 2025-03-05T04:49:54-05:00
wasm: make JSVal internal Weak# point to lifted JSVal

JSVal has an internal Weak# with the unlifted JSVal# object as key to
arrange its builtin finalization logic. The Weak# used to designate
Unit_closure as a dummy value; now this commit designates the lifted
JSVal closure as the Weak# value. This allows the implementation of
mkWeakJSVal which can be used to observe the liveliness of a JSVal and
attach a user-specified finalizer.

- - - - -
55af20e6 by Cheng Shao at 2025-03-05T04:49:54-05:00
ghc-experimental: add mkWeakJSVal

This commit adds a mkWeakJSVal function that can be used to set up a
Weak pointer with a JSVal key to observe the key's lifetime and
optionally attach a finalizer.

- - - - -
8273d7d1 by Matthew Pickering at 2025-03-05T04:50:30-05:00
simplifier: Zap Id unfoldings before constructing InScopeSet in simpleOptExpr

Care must be taken to remove unfoldings from `Var`s collected by exprFreeVars
before using them to construct an in-scope set hence `zapIdUnfolding` in `init_subst`.
Consider calling `simpleOptExpr` on an expression like

```
 case x of (a,b) -> (x,a)
```

* One of those two occurrences of x has an unfolding (the one in (x,a), with
unfolding x = (a,b)) and the other does not. (Inside a case GHC adds
unfolding-info to the scrutinee's Id.)
* But exprFreeVars just builds a set, so it's a bit random which occurrence is collected.
* Then simpleOptExpr replaces each occurrence of x with the one in the in-scope set.
* Bad bad bad: then the x in  case x of ... may be replaced with a version that has an unfolding.

Fixes #25790

- - - - -
07fe6d1d by Rodrigo Mesquita at 2025-03-05T04:51:07-05:00
docs: Fix ghci :doc documentation

Fixes #25799

- - - - -
c5fa0226 by Matthew Pickering at 2025-03-06T11:34:02+00:00
Add flag to control whether self-recompilation information is written to interface

This patch adds the flag -fwrite-if-self-recomp which controls whether
interface files contain the information necessary to answer the
question:

  Do I need to recompile myself or is this current interface file
  suitable?

Why? Most packages are only built once either by a distribution or cabal
and then placed into an immutable store, after which we will never ask
this question. Therefore we can derive two benefits from omitting this
information.

* Primary motivation: It vastly reduces the surface area for creating
  non-deterministic interface files. See issue #10424 which motivated a
  proper fix to that issue. Distributions have long contained versions
  of GHC which just have broken self-recompilation checking (in order to
  get deterministic interface files).

* Secondary motivation: This reduces the size of interface files
  slightly.. the `mi_usages` field can be quite big but probably this
  isn't such a great benefit.

* Third motivation: Conceptually clarity about which parts of an
  interface file are used in order to **communicate** with subsequent
  packages about the **interface** for a module. And which parts are
  used to self-communicate during recompilation checking.

The main tracking issue is #22188 but fixes issues such as #10424 in a
proper way.

- - - - -
def66aef by Matthew Pickering at 2025-03-06T11:34:02+00:00
Disable self recomp in release flavour

The interface files that we distribute should not contain any
information which is used by the recompilation checking logic since
source file will never be compiled again.

I am not 100% sure this won't cause unexpected issues, there many be
downstream consumers which are incorrectly using the information from
interfaces, but this commit can be reverted if we detect issues.

- - - - -


83 changed files:

- .gitlab/rel_eng/upload_ghc_libs.py
- compiler/GHC.hs
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/ByteCode/Instr.hs
- compiler/GHC/ByteCode/Types.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Data/FlatBag.hs
- compiler/GHC/Data/SmallArray.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/HsToCore.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- + compiler/GHC/Iface/Recomp/Types.hs
- compiler/GHC/Parser/String.hs
- compiler/GHC/Plugins.hs
- compiler/GHC/Rename/Splice.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Tc/Gen/Splice.hs-boot
- compiler/GHC/Tc/Plugin.hs
- compiler/GHC/Tc/Types/ErrCtxt.hs
- compiler/GHC/Tc/Types/TH.hs
- compiler/GHC/ThToHs.hs
- compiler/GHC/Unit/Module/ModGuts.hs
- compiler/GHC/Unit/Module/ModIface.hs
- compiler/ghc.cabal.in
- docs/users_guide/ghci.rst
- docs/users_guide/phases.rst
- hadrian/doc/flavours.md
- hadrian/src/Flavour.hs
- hadrian/src/Settings/Flavours/Release.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Lib.hs
- libraries/ghc-boot-th/GHC/Internal/TH/Lib/Map.hs → libraries/ghc-boot-th/GHC/Boot/TH/Lib/Map.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Lift.hs
- libraries/ghc-boot-th/GHC/Internal/TH/Ppr.hs → libraries/ghc-boot-th/GHC/Boot/TH/Ppr.hs
- libraries/ghc-boot-th/GHC/Internal/TH/PprLib.hs → libraries/ghc-boot-th/GHC/Boot/TH/PprLib.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Quote.hs
- + libraries/ghc-boot-th/GHC/Boot/TH/Syntax.hs
- libraries/ghc-boot-th/ghc-boot-th.cabal.in
- libraries/ghc-experimental/src/GHC/Wasm/Prim.hs
- libraries/ghc-internal/src/GHC/Internal/Wasm/Prim.hs
- libraries/ghc-internal/src/GHC/Internal/Wasm/Prim/Types.hs
- libraries/ghci/GHCi/Message.hs
- libraries/ghci/GHCi/TH.hs
- libraries/ghci/GHCi/TH/Binary.hs
- libraries/template-haskell/Language/Haskell/TH/Lib.hs
- libraries/template-haskell/Language/Haskell/TH/Lib/Internal.hs
- libraries/template-haskell/Language/Haskell/TH/Ppr.hs
- libraries/template-haskell/Language/Haskell/TH/PprLib.hs
- libraries/template-haskell/Language/Haskell/TH/Quote.hs
- libraries/template-haskell/Language/Haskell/TH/Syntax.hs
- rts/wasm/JSFFI.c
- testsuite/tests/count-deps/CountDepsAst.stdout
- testsuite/tests/count-deps/CountDepsParser.stdout
- + testsuite/tests/driver/self-recomp/Makefile
- + testsuite/tests/driver/self-recomp/SelfRecomp01.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp02.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp03.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.stdout
- + testsuite/tests/driver/self-recomp/all.T
- + testsuite/tests/ghci/should_run/T25790.hs
- + testsuite/tests/ghci/should_run/T25790.script
- testsuite/tests/ghci/should_run/all.T
- testsuite/tests/interface-stability/template-haskell-exports.stdout
- testsuite/tests/jsffi/jsffigc.hs
- + testsuite/tests/parser/should_run/T25784.hs
- + testsuite/tests/parser/should_run/T25784.stdout
- testsuite/tests/parser/should_run/all.T
- + testsuite/tests/perf/should_run/ByteCodeAsm.hs
- testsuite/tests/perf/should_run/all.T
- utils/haddock/html-test/ref/QuasiExpr.html
- utils/haddock/html-test/ref/TH.html
- utils/haddock/html-test/ref/Threaded_TH.html


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cf4972430f38429b07ca978faef21d3d4a17d12c...def66aef4d6b511532f5af92864f8724ff3e93ff

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cf4972430f38429b07ca978faef21d3d4a17d12c...def66aef4d6b511532f5af92864f8724ff3e93ff
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250306/31a69665/attachment-0001.html>

More information about the ghc-commits mailing list