[Git][ghc/ghc][wip/fendor/ghc-iface-sharing-avoid-reserialisation] 28 commits: template-haskell: Move wired-ins to ghc-internal
Hannes Siebenhandl (@fendor)
gitlab at gitlab.haskell.org
Fri May 31 13:16:45 UTC 2024
Hannes Siebenhandl pushed to branch wip/fendor/ghc-iface-sharing-avoid-reserialisation at Glasgow Haskell Compiler / GHC
Commits:
228dcae6 by Teo Camarasu at 2024-05-28T13:12:24+00:00
template-haskell: Move wired-ins to ghc-internal
Thus we make `template-haskell` reinstallable and keep it as the public
API for Template Haskell.
All of the wired-in identifiers are moved to `ghc-internal`.
This necessitates also moving much of `ghc-boot-th` into `ghc-internal`.
These modules are then re-exported from `ghc-boot-th` and
`template-haskell`.
To avoid a dependency on `template-haskell` from `lib:ghc`, we instead
depend on the TH ASTs via `ghc-boot-th`.
As `template-haskell` no longer has special status, we can drop the
logic adding an implicit dependency on `template-haskell` when using TH.
We can also drop the `template-haskell-next` package, which was
previously used when bootstrapping.
When bootstrapping, we need to vendor the TH AST modules from
`ghc-internal` into `ghc-boot-th`. This is controlled by the `bootstrap`
cabal flag as before. See Note [Bootstrapping Template Haskell].
We split out a GHC.Internal.TH.Lift module resolving #24752.
This module is only built when not bootstrapping.
Resolves #24703
-------------------------
Metric Increase:
ghc_boot_th_dir
ghc_boot_th_so
-------------------------
- - - - -
62dded28 by Teo Camarasu at 2024-05-28T13:12:24+00:00
testsuite: mark tests broken by #24886
Now that `template-haskell` is no longer wired-in.
These tests are triggering #24886, and so need to be marked broken.
- - - - -
3ca72ad9 by Cheng Shao at 2024-05-30T02:57:06-04:00
rts: fix missing function prototypes in ClosureMacros.h
- - - - -
e0029e3d by Andreas Klebinger at 2024-05-30T02:57:43-04:00
UnliftedFFITypes: Allow `(# #)` as argument when it's the only argument.
This allows representing functions like:
int foo(void);
to be imported like this:
foreign import ccall "a_number_c"
c_number :: (# #) -> Int64#
Which can be useful when the imported function isn't implicitly
stateful.
- - - - -
d0401335 by Matthew Pickering at 2024-05-30T02:58:19-04:00
ci: Update ci-images commit for fedora38 image
The fedora38 nightly job has been failing for quite a while because
`diff` was no longer installed. The ci-images bump explicitly installs
`diffutils` into these images so hopefully they now pass again.
- - - - -
3c97c74a by Jan Hrček at 2024-05-30T02:58:58-04:00
Update exactprint docs
- - - - -
77760cd7 by Jan Hrček at 2024-05-30T02:58:58-04:00
Incorporate review feedback
- - - - -
87591368 by Jan Hrček at 2024-05-30T02:58:58-04:00
Remove no longer relevant reference to comments
- - - - -
05f4f142 by Jan Hrček at 2024-05-30T02:58:59-04:00
Replace outdated code example
- - - - -
45a4a5f3 by Andreas Klebinger at 2024-05-30T02:59:34-04:00
Reword error resulting from missing -XBangPatterns.
It can be the result of either a bang pattern or strict binding,
so now we say so instead of claiming it must be a bang pattern.
Fixes #21032
- - - - -
e17f2df9 by Cheng Shao at 2024-05-30T03:00:10-04:00
testsuite: bump MultiLayerModulesDefsGhciReload timeout to 10x
- - - - -
7a660042 by Cheng Shao at 2024-05-30T14:42:29-04:00
rts: ensure gc_thread/gen_workspace is allocated with proper alignment
gc_thread/gen_workspace are required to be aligned by 64 bytes.
However, this property has not been properly enforced before, and
numerous alignment violations at runtime has been caught by
UndefinedBehaviorSanitizer that look like:
```
rts/sm/GC.c:1167:8: runtime error: member access within misaligned address 0x0000027a3390 for type 'gc_thread' (aka 'struct gc_thread_'), which requires 64 byte alignment
0x0000027a3390: note: pointer points here
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/sm/GC.c:1167:8
rts/sm/GC.c:1184:13: runtime error: member access within misaligned address 0x0000027a3450 for type 'gen_workspace' (aka 'struct gen_workspace_'), which requires 64 byte alignment
0x0000027a3450: note: pointer points here
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/sm/GC.c:1184:13
```
This patch fixes the gc_thread/gen_workspace misalignment issue by
explicitly allocating them with alignment constraint.
- - - - -
c77a48af by Cheng Shao at 2024-05-30T14:42:29-04:00
rts: fix an unaligned load in nonmoving gc
This patch fixes an unaligned load in nonmoving gc by ensuring the
closure address is properly untagged first before attempting to
prefetch its header. The unaligned load is reported by
UndefinedBehaviorSanitizer:
```
rts/sm/NonMovingMark.c:921:9: runtime error: member access within misaligned address 0x0042005f3a71 for type 'StgClosure' (aka 'struct StgClosure_'), which requires 8 byte alignment
0x0042005f3a71: note: pointer points here
00 00 00 98 43 13 8e 12 7f 00 00 50 3c 5f 00 42 00 00 00 58 17 b7 92 12 7f 00 00 89 cb 5e 00 42
^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/sm/NonMovingMark.c:921:9
```
This issue had previously gone unnoticed since it didn't really harm
runtime correctness, the invalid header address directly loaded from a
tagged pointer is only used as prefetch address and will not cause
segfaults. However, it still should be corrected because the prefetch
would be rendered useless by this issue, and untagging only involves a
single bitwise operation without memory access so it's cheap enough to
add.
- - - - -
05c4fafb by Cheng Shao at 2024-05-30T14:42:29-04:00
rts: use __builtin_offsetof to implement STG_FIELD_OFFSET
This patch fixes the STG_FIELD_OFFSET macro definition by using
__builtin_offsetof, which is what gcc/clang uses to implement offsetof
in standard C. The previous definition that uses NULL pointer involves
subtle undefined behavior in C and thus reported by
UndefinedBehaviorSanitizer as well:
```
rts/Capability.h:243:58: runtime error: member access within null pointer of type 'Capability' (aka 'struct Capability_')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/Capability.h:243:58
```
- - - - -
5ff83bfc by Sylvain Henry at 2024-05-30T14:43:10-04:00
JS: remove useless h$CLOCK_REALTIME (#23202)
- - - - -
95ef2d58 by Matthew Pickering at 2024-05-30T14:43:47-04:00
ghcup-metadata: Fix metadata generation
There were some syntax errors in the generation script which were
preventing it from running.
I have tested this with:
```
nix shell --extra-experimental-features nix-command -f .gitlab/rel_eng -c ghcup-metadata --metadata ghcup-0.0.7.yaml --date="2024-05-27" --pipeline-id=95534 --version=9.11.20240525
```
which completed successfully.
- - - - -
1bc66ee4 by Jakob Bruenker at 2024-05-30T14:44:22-04:00
Add diagrams to Arrows documentation
This adds diagrams to the documentation of Arrows, similar to the ones found on
https://www.haskell.org/arrows/.
It does not add diagrams for ArrowChoice for the time being, mainly because it's
not clear to me how to visually distinguish them from the ones for Arrow. Ideally,
you might want to do something like highlight the arrows belonging to the same
tuple or same Either in common colors, but that's not really possible with unicode.
- - - - -
d10a1c65 by Matthew Craven at 2024-05-30T23:35:48-04:00
Make UnsafeSNat et al. into pattern synonyms
...so that they do not cause coerce to bypass the nominal
role on the corresponding singleton types when they are imported.
See Note [Preventing unsafe coercions for singleton types] and
the discussion at #23478.
This also introduces unsafeWithSNatCo (and analogues for Char
and Symbol) so that users can still access the dangerous coercions
that importing the real constructors would allow, but only in a
very localized way.
- - - - -
0958937e by Cheng Shao at 2024-05-30T23:36:25-04:00
hadrian: build C/C++ with split sections when enabled
When split sections is enabled, ensure -fsplit-sections is passed to
GHC as well when invoking GHC to compile C/C++; and pass
-ffunction-sections -fdata-sections to gcc/clang when compiling C/C++
with the hadrian Cc builder. Fixes #23381.
- - - - -
02b1f91e by Cheng Shao at 2024-05-30T23:36:25-04:00
driver: build C/C++ with -ffunction-sections -fdata-sections when split sections is enabled
When -fsplit-sections is passed to GHC, pass -ffunction-sections
-fdata-sections to gcc/clang when building C/C++. Previously,
-fsplit-sections was only respected by the NCG/LLVM backends, but not
the unregisterised backend; the GHC driver did not pass
-fdata-sections and -ffunction-sections to the C compiler, which
resulted in excessive executable sizes.
Fixes #23381.
-------------------------
Metric Decrease:
size_hello_artifact
size_hello_unicode
-------------------------
- - - - -
fd47e2e3 by Cheng Shao at 2024-05-30T23:37:00-04:00
testsuite: mark process005 as fragile on JS
- - - - -
34a04ea1 by Matthew Pickering at 2024-05-31T06:08:36-04:00
Add -Wderiving-typeable to -Wall
Deriving `Typeable` does nothing, and it hasn't done for a long while.
There has also been a warning for a long while which warns you about
uselessly deriving it but it wasn't enabled in -Wall.
Fixes #24784
- - - - -
75fa7b0b by Matthew Pickering at 2024-05-31T06:08:36-04:00
docs: Fix formatting of changelog entries
- - - - -
303c4b33 by Preetham Gujjula at 2024-05-31T06:09:21-04:00
docs: Fix link to injective type families paper
Closes #24863
- - - - -
df97e9a6 by Ben Gamari at 2024-05-31T06:09:57-04:00
ghc-internal: Fix package description
The previous description was inherited from `base` and was inappropriate
for `ghc-internal`. Also fix the maintainer and bug reporting fields.
Closes #24906.
- - - - -
bf0737c0 by Cheng Shao at 2024-05-31T06:10:33-04:00
compiler: remove ArchWasm32 special case in cmmDoCmmSwitchPlans
This patch removes special consideration for ArchWasm32 in
cmmDoCmmSwitchPlans, which means the compiler will now disable
cmmImplementSwitchPlans for wasm unreg backend, just like unreg
backend of other targets. We enabled it in the past to workaround some
compile-time panic in older versions of LLVM, but those panics are no
longer present, hence no need to keep this workaround.
- - - - -
91a3fe53 by Fendor at 2024-05-31T15:16:36+02:00
Improve sharing of duplicated values in `ModIface`, fixes #24723
As a `ModIface` often contains duplicated values that are not
necessarily shared, we improve sharing by serialising the `ModIface`
to an in-memory byte array. Serialisation uses deduplication tables, and
deserialisation implicitly shares duplicated values.
This helps reducing the peak memory usage while compiling in
`--make` mode. The peak memory usage is especially smaller when
generating interface files with core expressions
(`-fwrite-if-simplified-core`).
On agda, this reduces the peak memory usage:
* `2.2 GB` to `1.9 GB` for a ghci session.
On `lib:Cabal`, we report:
* `570 MB` to `500 MB` for a ghci session
* `790 MB` to `667 MB` for compiling `lib:Cabal` with ghc
There is a small impact on execution time, around 2% on the agda code
base.
- - - - -
2136be60 by Fendor at 2024-05-31T15:16:36+02:00
Avoid unneccessarily re-serialising the `ModIface`
To reduce memory usage of `ModIface`, we serialise `ModIface` to an
in-memory byte array, which implicitly shares duplicated values.
This serialised byte array can be reused to avoid work when we actually
write the `ModIface` to disk.
We introduce a new field to `ModIface` which allows us to save the byte
array, and write it direclty to disk if the `ModIface` wasn't changed
after the initial serialisation.
This requires us to change absolute offsets, for example to jump to the
deduplication table for `Name` or `FastString` with relative offsets, as
the deduplication byte array doesn't contain header information, such as
fingerprints.
To allow us to dump the binary blob to disk, we need to replace all
absolute offsets with relative ones.
We introduce additional helpers for `ModIface` binary serialisation, which
construct relocatable binary blobs. We say the binary blob is relocatable,
if the binary representation can be moved and does not contain any
absolute offsets.
Further, we introduce new primitives for `Binary` that allow to create
relocatable binaries, such as `forwardGetRel` and `forwardPutRel`.
-------------------------
Metric Decrease:
MultiLayerModulesDefsGhcWithCore
Metric Increase:
MultiComponentModules
MultiLayerModules
T10421
T12234
T12425
T13035
T13701
T13719
T14697
T15703
T16875
T18730
T9198
mhu-perf
-------------------------
These metric increases may look bad, but they are all completely benign,
we simply allocate 1 MB per module for `shareIface`. As this allocation
is quite quick, it has a negligible impact on run-time performance.
- - - - -
30 changed files:
- .gitignore
- .gitlab-ci.yml
- .gitlab/rel_eng/mk-ghcup-metadata/mk_ghcup_metadata.py
- compiler/GHC.hs
- compiler/GHC/Builtin/Names/TH.hs
- compiler/GHC/Core/TyCon.hs
- compiler/GHC/Driver/Config/Cmm.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/HsToCore/Foreign/Call.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Iface/Ext/Binary.hs
- compiler/GHC/Iface/Ext/Fields.hs
- compiler/GHC/Iface/Ext/Types.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Rename.hs
- compiler/GHC/JS/JStg/Syntax.hs
- compiler/GHC/JS/Syntax.hs
- compiler/GHC/Parser/Annotation.hs
- compiler/GHC/Parser/Errors/Ppr.hs
- compiler/GHC/Plugins.hs
- compiler/GHC/Rename/Names.hs
- compiler/GHC/Rename/Splice.hs
- compiler/GHC/Runtime/Interpreter/JS.hs
- compiler/GHC/Tc/Errors/Hole.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/655444cae6c6af15aa6ca7985cbae9027e538b3c...2136be608496f8cfb27c91fcb0d27c49576830c8
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/655444cae6c6af15aa6ca7985cbae9027e538b3c...2136be608496f8cfb27c91fcb0d27c49576830c8
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240531/5de02be5/attachment-0001.html>
More information about the ghc-commits
mailing list