[Git][ghc/ghc][wip/fendor/ghc-iface-sharing-avoid-reserialisation] 40 commits: ghcup-metadata: Various fixes from 9.10.1

Hannes Siebenhandl (@fendor) gitlab at gitlab.haskell.org
Fri May 24 11:58:20 UTC 2024 


Hannes Siebenhandl pushed to branch wip/fendor/ghc-iface-sharing-avoid-reserialisation at Glasgow Haskell Compiler / GHC


Commits:
20b0136a by Ben Gamari at 2024-05-22T00:31:39-04:00
ghcup-metadata: Various fixes from 9.10.1

Use Debian 12/x86-64, Debian 10/aarch64, and Debian 11/aarch64 bindists
where possible.

- - - - -
6838a7c3 by Sylvain Henry at 2024-05-22T00:32:23-04:00
Reverse arguments to stgCallocBytes (fix #24828)

- - - - -
f50f46c3 by Fendor at 2024-05-22T00:32:59-04:00
Add log messages for Iface serialisation compression level

Fix the label of the number of 'IfaceType' entries in the log message.
Add log message for the compression level that is used to serialise a an
interface file.

Adds `Outputable` instance for 'CompressionIFace'.

- - - - -
3bad5d55 by Hécate Moonlight at 2024-05-22T00:33:40-04:00
base: Update doctests outputs

ghc-internal: Update doctests outputs

- - - - -
9317c6fb by David Binder at 2024-05-22T00:34:21-04:00
haddock: Fix the testsuites of the haddock-library

- Apply all the metadata revisions from Hackage
  to the cabal file.
- Fix the `ParserSpec.hs` file in the `spec`
  testsuite of haddock-library.
- Make `CHANGES.md` an extra-doc-file instead of
  an extra-source-file.

- - - - -
54073b02 by David Binder at 2024-05-22T00:34:21-04:00
haddock: Fix parser of @since pragma

The testsuite contained tests for annotations of
the form `@since foo-bar-0.5.0`, but the parser was
written incorrectly.

- - - - -
ede6ede3 by Matthew Pickering at 2024-05-22T00:34:57-04:00
Fix nightly pages job

It seems likely broken by 9f99126a which moved `index.html` from the
root folder into `docs/` folder.

Fixes #24840

- - - - -
b7bcf729 by Cheng Shao at 2024-05-22T00:35:32-04:00
autoconf: remove unused context diff check

This patch removes redundant autoconf check for the context diff
program given it isn't actually been used anywhere, especially since
make removal.

- - - - -
ea2fe66e by Hécate Moonlight at 2024-05-22T00:36:13-04:00
haddock: Rework the contributing guide

- - - - -
0f302a94 by Hécate Moonlight at 2024-05-22T00:36:52-04:00
haddock: Add module relationships diagrams of haddock-api and haddock-library

- - - - -
d1a9f34f by Hécate Moonlight at 2024-05-22T00:36:52-04:00
Add instructions

- - - - -
b880ee80 by Hécate Moonlight at 2024-05-22T00:36:52-04:00
Add SVG outputs

- - - - -
6d7e6ad8 by Ben Gamari at 2024-05-22T13:40:05-04:00
rts: Fix size of StgOrigThunkInfo frames

Previously the entry code of the `stg_orig_thunk` frame failed to
account for the size of the profiling header as it hard-coded the frame
size. Fix this.

Fixes #24809.

- - - - -
c645fe40 by Fendor at 2024-05-22T13:40:05-04:00
Add regression test T24809 for stg_orig_thunk_info_frame size

- - - - -
4181aa40 by Andreas Klebinger at 2024-05-22T13:40:42-04:00
bindists: Check for existence of share folder before trying to copy it.

This folder isn't distributed in windows bindists

A lack of doing so resulted us copying loads of files twice.

- - - - -
d216510e by Matthew Pickering at 2024-05-22T13:40:42-04:00
Remove ad-hoc installation of mingw toolchain in relocatable bindists

This reverts 616ac30026e8dd7d2ebb98d92dde071eedf5d951

The choice about whether to install mingw is taken in the installation
makefile.

This is also broken on non-windows systems.

The actual issue was the EnableDistroToolchain variable wasn't declared
in mk/config.mk and therefore the check to install mingw was failing.

- - - - -
7b4c1998 by Cheng Shao at 2024-05-22T21:52:52-04:00
testsuite: fix T17920 for wasm backend

T17920 was marked as fragile on wasm before; it can be trivially fixed
by avoiding calling variadic printf() in cmm.

- - - - -
c739383b by Cheng Shao at 2024-05-22T21:53:29-04:00
testsuite: bump T22744 timeout to 5x

- - - - -
c4c6d714 by Cheng Shao at 2024-05-22T21:54:06-04:00
testsuite: don't attempt to detect host cpu features when testing cross ghc

The testsuite driver CPU feature detection logic only detects host CPU
and only makes sense when we are not testing a cross GHC.

- - - - -
3d9e4ce6 by Simon Peyton Jones at 2024-05-22T21:54:43-04:00
Better skolemisation

As #24810 showed, it is (a little) better to skolemise en-bloc,
so that Note [Let-bound skolems] fires more often.

See Note [Skolemisation en bloc] in GHC.Tc.Utils.Instantiate.

- - - - -
a3cd3a1d by Ryan Scott at 2024-05-22T21:55:19-04:00
Add missing parenthesizePat in cvtp

We need to ensure that the output of `cvtp` is parenthesized (at precedence
`sigPrec`) so that any pattern signatures with a surrounding pattern signature
can parse correctly.

Fixes #24837.

- - - - -
4bb2a7cc by Hécate Moonlight at 2024-05-22T21:55:59-04:00
[base] Document the memory overhead of ByteArray

Add a diagram that shows the constituent parts of a ByteArray and their
memory overhead.

- - - - -
8b2a016a by Hécate Moonlight at 2024-05-22T21:56:38-04:00
Haddock: Add MR template for Haddock

- - - - -
ead75532 by Peter Trommler at 2024-05-23T02:28:05-04:00
PPC: Support ELF v2 on powerpc64 big-endian

Detect ELF v2 on PowerPC 64-bit systems. Check for `_CALL_ELF`
preprocessor macro.

Fixes #21191

- - - - -
9d4c10f2 by Hécate Kleidukos at 2024-05-23T02:28:44-04:00
gitlab: Add @Kleidukos to CODEOWNERS for utils/haddock

- - - - -
28e64170 by Preetham Gujjula at 2024-05-23T07:20:48-04:00
haddock: Add cabal-fmt to tools for `make style`

- - - - -
00126a89 by Andrei Borzenkov at 2024-05-23T07:21:24-04:00
haddock: fix verbosity option parsing

- - - - -
a3e0b68b by Ryan Hendrickson at 2024-05-23T15:52:03-04:00
base: specify tie-breaking behavior of min, max, and related list/Foldable functions

- - - - -
bdcc0f37 by doyougnu at 2024-05-24T07:51:18-04:00
cmm: add word <-> double/float bitcast

- closes: #25331

This is the last step in the project plan described in #25331. This
commit:

- adds bitcast operands for x86_64, LLVM, aarch64
- For PPC and i386 we resort to using the cmm implementations
- renames conversion MachOps from Conv to Round|Truncate

- - - - -
f0d257f7 by Krzysztof Gogolewski at 2024-05-24T07:51:55-04:00
StgToByteCode: minor refactor

Some functions in StgToByteCode were filtering out void arguments.
However, StgToByteCode is called after unarisation: the void arguments
should have been removed earlier.
Instead of filtering out, we assert that the args are non-void.

- - - - -
03137fd2 by Krzysztof Gogolewski at 2024-05-24T07:51:55-04:00
StgToByteCode: minor refactor

`layoutNativeCall` was always called with a `primRepCmmType platform`
callback. Hence we can put it inside of `layoutNativeCall` rather than
repeat it.

- - - - -
27c430f3 by David Binder at 2024-05-24T07:52:38-04:00
haddock: Remove compatibility shims for GHC < 8.4 from haddock-library

- - - - -
8dd8a076 by Cheng Shao at 2024-05-24T07:53:14-04:00
compiler: avoid saving foreign call target to local when there are no caller-save GlobalRegs

This patch makes the STG->Cmm backend avoid saving foreign call target
to local when there are no caller-save GlobalRegs.

Since 321941a8ebe25192cdeece723e1058f2f47809ea, when we lower a
foreign call, we unconditionally save the foreign call target to a
temporary local first, then rely on cmmSink to clean it up later,
which only happens with -fcmm-sink (implied by -O) and not in
unoptimized code.

And this is troublesome for the wasm backend NCG, which needs to infer
a foreign call target symbol's type signature from the Cmm call site.
Previously, the NCG has been emitting incorrect type signatures for
unoptimized code, which happens to work with `wasm-ld` most of the
time, but this is never future-proof against upstream toolchain
updates, and it causes horrible breakages when LTO objects are
included in linker input. Hence this patch.

- - - - -
986df1ab by Cheng Shao at 2024-05-24T07:53:14-04:00
testsuite: add callee-no-local regression test

- - - - -
52d62e2a by Sylvain Henry at 2024-05-24T07:53:57-04:00
Fix HasCallStack leftovers from !12514 / #24726

- - - - -
c5e00c35 by crumbtoo at 2024-05-24T07:54:38-04:00
user_guide: Fix typo in MultiWayIf chapter

Close #24829

- - - - -
bd323b0e by Ben Gamari at 2024-05-24T07:55:15-04:00
base: Ensure that CHANGELOG is included in extra-source-files

This was missed in the `ghc-internal` split.

Closes #24831.

- - - - -
1bfd32e8 by Ben Gamari at 2024-05-24T07:55:15-04:00
base: Fix changelog reference to setBacktraceMechanismState

(cherry picked from commit b63f7ba01fdfd98a01d2f0dec8d9262b3e595c5d)

- - - - -
f3a30893 by Fendor at 2024-05-24T13:56:21+02:00
Improve sharing of duplicated values in `ModIface`

As a `ModIface` contains often duplicated values that are not
necessarily shared, we improve sharing by serialising the `ModIface`
to an in-memory byte array. Serialisation uses deduplication tables, and
deserialisation implicitly shares duplicated values.

This helps reducing the peak memory usage while compiling in
`--make` mode. The peak memory usage is especially reduced when
generating interface files with core expressions
(`-fwrite-if-simplified-core`).

On agda, this reduces the peak memory usage:

* `2.2 GB` to `1.9 GB` for a ghci session.

On `lib:Cabal`, we report:

* `570 MB` to `500 MB` for a ghci session
* `790 MB` to `667 MB` for compiling `lib:Cabal` with ghc

There is a small impact on execution time, around 2% on the agda code
base.

- - - - -
25428e61 by Fendor at 2024-05-24T13:56:21+02:00
Avoid unneccessarily re-serialising the `ModIface`

To reduce memory usage of `ModIface`, we serialise `ModIface` to an
in-memory byte array, which implicitly shares duplicated values.

This serailised byte array can be reused to avoid work when we actually
write the `ModIface` to disk.
We introduce a new field to `ModIface` which allows us to save the byte
array, and write it to disk if the `ModIface` wasn't changed after the
initial serialisation.

This requires us to change absolute offsets, for example to jump to the
deduplication table for `Name` or `FastString` with relative offsets, as
the deduplication byte array doesn't contain header information, such as
fingerprints.
To allow us to dump the binary blob to disk, we need to replace all
absolute offsets with relative ones.

This leads to new primitives for `ModIface`, which help to construct
relative offsets.

Bump Haddock submodule, to account for interface file changes.

-------------------------
Metric Decrease:
    MultiLayerModulesDefsGhcWithCore
Metric Increase:
    MultiComponentModules
    MultiLayerModules
    T10421
    T12234
    T12425
    T13035
    T13701
    T13719
    T14697
    T15703
    T16875
    T18730
    T9198
    mhu-perf
-------------------------
These metric increases may look bad, but they are all completely benign,
we simply allocate 1 MB per module for `shareIface`. As this allocation
is quite quick, it has a neglible impact on run-time performance.

- - - - -


30 changed files:

- .gitlab-ci.yml
- + .gitlab/merge_request_templates/Haddock.md
- .gitlab/rel_eng/mk-ghcup-metadata/mk_ghcup_metadata.py
- CODEOWNERS
- compiler/GHC.hs
- compiler/GHC/Cmm/MachOp.hs
- compiler/GHC/Cmm/Opt.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Instr.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToAsm/X86/Instr.hs
- compiler/GHC/CmmToAsm/X86/Ppr.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/TyCo/Rep.hs
- compiler/GHC/Core/Utils.hs
- compiler/GHC/Driver/Config/StgToCmm.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Ext/Binary.hs
- compiler/GHC/Iface/Ext/Fields.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Rename.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2c0a6b7043fb5d49e4bb3476d71bbed53c79cde2...25428e61cc5e9bbe89b70d9b6a65e6134cc584c2

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2c0a6b7043fb5d49e4bb3476d71bbed53c79cde2...25428e61cc5e9bbe89b70d9b6a65e6134cc584c2
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240524/9aa405bb/attachment-0001.html>

More information about the ghc-commits mailing list