[Git][ghc/ghc][wip/fendor/ifacetype-deduplication] 47 commits: utils: remove obsolete vagrant scripts
Hannes Siebenhandl (@fendor)
gitlab at gitlab.haskell.org
Tue May 7 11:36:17 UTC 2024
Hannes Siebenhandl pushed to branch wip/fendor/ifacetype-deduplication at Glasgow Haskell Compiler / GHC
Commits:
728af21e by Cheng Shao at 2024-04-30T05:30:23-04:00
utils: remove obsolete vagrant scripts
Vagrantfile has long been removed in !5288. This commit further
removes the obsolete vagrant scripts in the tree.
- - - - -
36f2c342 by Cheng Shao at 2024-04-30T05:31:00-04:00
Update autoconf scripts
Scripts taken from autoconf 948ae97ca5703224bd3eada06b7a69f40dd15a02
- - - - -
ecbf22a6 by Ben Gamari at 2024-04-30T05:31:36-04:00
ghcup-metadata: Drop output_name field
This is entirely redundant to the filename of the URL. There is no
compelling reason to name the downloaded file differently from its
source.
- - - - -
c56d728e by Zubin Duggal at 2024-04-30T22:45:09-04:00
testsuite: Handle exceptions in framework_fail when testdir is not initialised
When `framework_fail` is called before initialising testdir, it would fail with
an exception reporting the testdir not being initialised instead of the actual failure.
Ensure we report the actual reason for the failure instead of failing in this way.
One way this can manifest is when trying to run a test that doesn't exist using `--only`
- - - - -
d5bea4d6 by Alan Zimmerman at 2024-04-30T22:45:45-04:00
EPA: Fix range for GADT decl with sig only
Closes #24714
- - - - -
4d78c53c by Sylvain Henry at 2024-05-01T17:23:06-04:00
Fix TH dependencies (#22229)
Add a dependency between Syntax and Internal (via module reexport).
- - - - -
37e38db4 by Sylvain Henry at 2024-05-01T17:23:06-04:00
Bump haddock submodule
- - - - -
ca13075c by Sylvain Henry at 2024-05-01T17:23:47-04:00
JS: cleanup to prepare for #24743
- - - - -
40026ac3 by Alan Zimmerman at 2024-05-01T22:45:07-04:00
EPA: Preserve comments for PrefixCon
Preserve comments in
fun (Con {- c1 -} a b)
= undefined
Closes #24736
- - - - -
92134789 by Hécate Moonlight at 2024-05-01T22:45:42-04:00
Correct `@since` metadata in HpcFlags
It was introduced in base-4.20, not 4.22.
Fix #24721
- - - - -
a580722e by Cheng Shao at 2024-05-02T08:18:45-04:00
testsuite: fix req_target_smp predicate
- - - - -
ac9c5f84 by Andreas Klebinger at 2024-05-02T08:18:45-04:00
STM: Remove (unused)coarse grained locking.
The STM code had a coarse grained locking mode guarded by #defines that was unused.
This commit removes the code.
- - - - -
917ef81b by Andreas Klebinger at 2024-05-02T08:18:45-04:00
STM: Be more optimistic when validating in-flight transactions.
* Don't lock tvars when performing non-committal validation.
* If we encounter a locked tvar don't consider it a failure.
This means in-flight validation will only fail if committing at the
moment of validation is *guaranteed* to fail.
This prevents in-flight validation from failing spuriously if it happens in
parallel on multiple threads or parallel to thread comitting.
- - - - -
167a56a0 by Alan Zimmerman at 2024-05-02T08:19:22-04:00
EPA: fix span for empty \case(s)
In
instance SDecide Nat where
SZero %~ (SSucc _) = Disproved (\case)
Ensure the span for the HsLam covers the full construct.
Closes #24748
- - - - -
9bae34d8 by doyougnu at 2024-05-02T15:41:08-04:00
testsuite: expand size testing infrastructure
- closes #24191
- adds windows_skip, wasm_skip, wasm_arch, find_so, _find_so
- path_from_ghcPkg, collect_size_ghc_pkg, collect_object_size, find_non_inplace functions to testsuite
- adds on_windows and req_dynamic_ghc predicate to testsuite
The design is to not make the testsuite too smart and simply offload to
ghc-pkg for locations of object files and directories.
- - - - -
b85b1199 by Sylvain Henry at 2024-05-02T15:41:49-04:00
GHCi: support inlining breakpoints (#24712)
When a breakpoint is inlined, its context may change (e.g. tyvars in
scope). We must take this into account and not used the breakpoint tick
index as its sole identifier. Each instance of a breakpoint (even with
the same tick index) now gets a different "info" index.
We also need to distinguish modules:
- tick module: module with the break array (tick counters, status, etc.)
- info module: module having the CgBreakInfo (info at occurrence site)
- - - - -
649c24b9 by Oleg Grenrus at 2024-05-03T20:45:42-04:00
Expose constructors of SNat, SChar and SSymbol in ghc-internal
- - - - -
d603f199 by Mikolaj Konarski at 2024-05-03T20:46:19-04:00
Add DCoVarSet to PluginProv (!12037)
- - - - -
ba480026 by Serge S. Gulin at 2024-05-03T20:47:01-04:00
JS: Enable more efficient packing of string data (fixes #24706)
- - - - -
be1e60ee by Simon Peyton Jones at 2024-05-03T20:47:37-04:00
Track in-scope variables in ruleCheckProgram
This small patch fixes #24726, by tracking in-scope variables
properly in -drule-check. Not hard to do!
- - - - -
58408c77 by Simon Peyton Jones at 2024-05-03T20:47:37-04:00
Add a couple more HasCallStack constraints in SimpleOpt
Just for debugging, no effect on normal code
- - - - -
70e245e8 by Simon Peyton Jones at 2024-05-03T20:47:37-04:00
Add comments to Prep.hs
This documentation patch fixes a TODO left over from !12364
- - - - -
e5687186 by Simon Peyton Jones at 2024-05-03T20:47:37-04:00
Use HasDebugCallStack, rather than HasCallStack
- - - - -
631cefec by Cheng Shao at 2024-05-03T20:48:17-04:00
driver: always merge objects when possible
This patch makes the driver always merge objects with `ld -r` when
possible, and only fall back to calling `ar -L` when merge objects
command is unavailable. This completely reverts !8887 and !12313,
given more fixes in Cabal seems to be needed to avoid breaking certain
configurations and the maintainence cost is exceeding the behefits in
this case :/
- - - - -
1dacb506 by Ben Gamari at 2024-05-03T20:48:53-04:00
Bump time submodule to 1.14
As requested in #24528.
-------------------------
Metric Decrease:
ghc_bignum_so
rts_so
Metric Increase:
cabal_syntax_dir
rts_so
time_dir
time_so
-------------------------
- - - - -
4941b90e by Ben Gamari at 2024-05-03T20:48:53-04:00
Bump terminfo submodule to current master
- - - - -
43d48b44 by Cheng Shao at 2024-05-03T20:49:30-04:00
wasm: use scheduler.postTask() for context switch when available
This patch makes use of scheduler.postTask() for JSFFI context switch
when it's available. It's a more principled approach than our
MessageChannel based setImmediate() implementation, and it's available
in latest version of Chromium based browsers.
- - - - -
08207501 by Cheng Shao at 2024-05-03T20:50:08-04:00
testsuite: give pre_cmd for mhu-perf 5x time
- - - - -
bf3d4db0 by Alan Zimmerman at 2024-05-03T20:50:43-04:00
EPA: Preserve comments for pattern synonym sig
Closes #24749
- - - - -
c49493f2 by Matthew Pickering at 2024-05-04T06:02:57-04:00
tests: Widen acceptance window for dir and so size tests
These are testing things which are sometimes out the control of a GHC
developer. Therefore we shouldn't fail CI if something about these
dependencies change because we can't do anything about it.
It is still useful to have these statistics for visualisation in grafana
though.
Ticket #24759
- - - - -
9562808d by Matthew Pickering at 2024-05-04T06:02:57-04:00
Disable rts_so test
It has already manifested large fluctuations and destabilising CI
Fixes #24762
- - - - -
fc24c5cf by Ryan Scott at 2024-05-04T06:03:33-04:00
unboxedSum{Type,Data}Name: Use GHC.Types as the module
Unboxed sum constructors are now defined in the `GHC.Types` module, so if you
manually quote an unboxed sum (e.g., `''Sum2#`), you will get a `Name` like:
```hs
GHC.Types.Sum2#
```
The `unboxedSumTypeName` function in `template-haskell`, however, mistakenly
believes that unboxed sum constructors are defined in `GHC.Prim`, so
`unboxedSumTypeName 2` would return an entirely different `Name`:
```hs
GHC.Prim.(#|#)
```
This is a problem for Template Haskell users, as it means that they can't be
sure which `Name` is the correct one. (Similarly for `unboxedSumDataName`.)
This patch fixes the implementations of `unboxedSum{Type,Data}Name` to use
`GHC.Types` as the module. For consistency with `unboxedTupleTypeName`, the
`unboxedSumTypeName` function now uses the non-punned syntax for unboxed sums
(`Sum<N>#`) as the `OccName`.
Fixes #24750.
- - - - -
7eab4e01 by Alan Zimmerman at 2024-05-04T16:14:55+01:00
EPA: Widen stmtslist to include last semicolon
Closes #24754
- - - - -
06f7db40 by Teo Camarasu at 2024-05-05T00:19:38-04:00
doc: Fix type error in hs_try_putmvar example
- - - - -
af000532 by Moritz Schuler at 2024-05-05T06:30:58-04:00
Fix parsing of module names in CLI arguments
closes issue #24732
- - - - -
da74e9c9 by Ben Gamari at 2024-05-05T06:31:34-04:00
ghc-platform: Add Setup.hs
The Hadrian bootstrapping script relies upon `Setup.hs` to drive its
build.
Addresses #24761.
- - - - -
35d34fde by Alan Zimmerman at 2024-05-05T12:52:40-04:00
EPA: preserve comments in class and data decls
Fix checkTyClHdr which was discarding comments.
Closes #24755
- - - - -
03c5dfbf by Simon Peyton Jones at 2024-05-05T12:53:15-04:00
Fix a float-out error
Ticket #24768 showed that the Simplifier was accidentally destroying
a join point. It turned out to be that we were sending a bottoming
join point to the top, accidentally abstracting over /other/ join
points.
Easily fixed.
- - - - -
adba68e7 by John Ericson at 2024-05-05T19:35:56-04:00
Substitute bindist files with Hadrian not configure
The `ghc-toolchain` overhaul will eventually replace all this stuff with
something much more cleaned up, but I think it is still worth making
this sort of cleanup in the meantime so other untanglings and dead code
cleaning can procede.
I was able to delete a fair amount of dead code doing this too.
`LLVMTarget_CPP` is renamed to / merged with `LLVMTarget` because it
wasn't actually turned into a valid CPP identifier. (Original to
1345c7cc42c45e63ab1726a8fd24a7e4d4222467, actually.)
Progress on #23966
Co-Authored-By: Sylvain Henry <hsyl20 at gmail.com>
- - - - -
18f4ff84 by Alan Zimmerman at 2024-05-05T19:36:32-04:00
EPA: fix mkHsOpTyPV duplicating comments
Closes #24753
- - - - -
a19201d4 by Matthew Craven at 2024-05-06T19:54:29-04:00
Add test cases for #24664
...since none are present in the original MR !12463 fixing this issue.
- - - - -
46328a49 by Alan Zimmerman at 2024-05-06T19:55:05-04:00
EPA: preserve comments in data decls
Closes #24771
- - - - -
ae0330ac by Fendor at 2024-05-07T13:15:43+02:00
Add Eq and Ord instance to `IfaceType`
We add an `Ord` instance so that we can store `IfaceType` in a
`Data.Map` container.
This is required to deduplicate `IfaceType` while writing `.hi` files to
disk. Deduplication has many beneficial consequences to both file size
and memory usage, as the deduplication enables implicit sharing of
values.
See issue #24540 for more motivation.
The `Ord` instance would be unnecessary if we used a `TrieMap` instead
of `Data.Map` for the deduplication process. While in theory this is
clerarly the better option, experiments on the agda code base showed
that a `TrieMap` implementation has worse run-time performance
characteristics.
To the change itself, we mostly derive `Eq` and `Ord`. This requires us
to change occurrences of `FastString` with `LexicalFastString`, since
`FastString` has no `Ord` instance.
We change the definition of `IfLclName` to a newtype of
`LexicalFastString`, to make such changes in the future easier.
Bump haddock submodule for IfLclName changes
- - - - -
dd3cc486 by Fendor at 2024-05-07T13:17:13+02:00
Move out LiteralMap to avoid cyclic module dependencies
- - - - -
0276bbd6 by Fendor at 2024-05-07T13:36:05+02:00
Add deduplication table for `IfaceType`
The type `IfaceType` is a highly redundant, tree-like data structure.
While benchmarking, we realised that the high redundancy of `IfaceType`
causes high memory consumption in GHCi sessions when byte code is
embedded into the `.hi` file via `-fwrite-if-simplified-core` or
`-fbyte-code-and-object-code`.
Loading such `.hi` files from disk introduces many duplicates of
memory expensive values in `IfaceType`, such as `IfaceTyCon`,
`IfaceTyConApp`, `IA_Arg` and many more.
We improve the memory behaviour of GHCi by adding an additional
deduplication table for `IfaceType` to the serialisation of `ModIface`,
similar to how we deduplicate `Name`s and `FastString`s.
When reading the interface file back, the table allows us to automatically
share identical values of `IfaceType`.
To provide some numbers, we evaluated this patch on the agda code base.
We loaded the full library from the `.hi` files, which contained the
embedded core expressions (`-fwrite-if-simplified-core`).
Before this patch:
* Load time: 11.7 s, 2.5 GB maximum residency.
After this patch:
* Load time: 7.3 s, 1.7 GB maximum residency.
This deduplication has the beneficial side effect to additionally reduce
the size of the on-disk interface files tremendously.
For example, on agda, we reduce the size of `.hi` files (with
`-fwrite-if-simplified-core`):
* Before: 101 MB on disk
* Now: 24 MB on disk
This has even a beneficial side effect on the cabal store. We reduce the
size of the store on disk:
* Before: 341 MB on disk
* Now: 310 MB on disk
Note, none of the dependencies have been compiled with
`-fwrite-if-simplified-core`, but `IfaceType` occurs in multiple
locations in a `ModIface`.
We also add IfaceType deduplication table to .hie serialisation and
refactor .hie file serialisation to use the same infrastrucutre as
`putWithTables`.
Bump haddock submodule to accomodate for changes to the deduplication
table layout and binary interface.
- - - - -
da5a7fff by Matthew Pickering at 2024-05-07T13:36:05+02:00
Add run-time configurability of .hi file compression
Introduce the flag `-fwrite-if-compression=<n>` which allows to
configure the compression level of writing .hi files.
The motivation is that some deduplication operations are too expensive
for the average use case. Hence, we introduce multiple compression
levels that have a minimal impact on performance, but still reduce the
memory residency and `.hi` file size on disk considerably.
We introduce three compression levels:
* `1`: `Normal` mode. This is the least amount of compression.
It deduplicates only `Name` and `FastString`s, and is naturally the
fastest compression mode.
* `2`: `Safe` mode. It has a noticeable impact on .hi file size and is
marginally slower than `Normal` mode. In general, it should be safe to
always use `Safe` mode.
* `3`: `Full` deduplication mode. Deduplicate as much as we can,
resulting in minimal .hi files, but at the cost of additional
compilation time.
Reading .hi files doesn't need to know the initial compression level,
and can always deserialise a `ModIface`.
This allows users to experiment with different compression levels for
packages, without recompilation of dependencies.
Note, the deduplication also has an additional side effect of reduced
memory consumption to implicit sharing of deduplicated elements.
See https://gitlab.haskell.org/ghc/ghc/-/issues/24540 for example where
that matters.
-------------------------
Metric Decrease:
MultiLayerModulesDefsGhciWithCore
T16875
T21839c
T24471
hard_hole_fits
-------------------------
- - - - -
1ac9d688 by Matthew Pickering at 2024-05-07T13:36:05+02:00
Introduce regression tests for `.hi` file sizes
Add regression tests to track how `-fwrite-if-compression` levels affect
the size of `.hi` files.
- - - - -
30 changed files:
- .gitlab/rel_eng/mk-ghcup-metadata/mk_ghcup_metadata.py
- compiler/GHC.hs
- compiler/GHC/Builtin/Names/TH.hs
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/ByteCode/Instr.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/Reg/Linear/AArch64.hs
- compiler/GHC/Core/Coercion.hs
- compiler/GHC/Core/Coercion/Opt.hs
- compiler/GHC/Core/FVs.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Map/Expr.hs
- compiler/GHC/Core/Opt/DmdAnal.hs
- compiler/GHC/Core/Opt/SetLevels.hs
- compiler/GHC/Core/Rules.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/TyCo/FVs.hs
- compiler/GHC/Core/TyCo/Rep.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/Core/TyCo/Tidy.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/CoreToIface.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Data/FastString.hs
- compiler/GHC/Data/Maybe.hs
- compiler/GHC/Data/TrieMap.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Session.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/5388078182762a2912f2feaae024ec465975f912...1ac9d688e176ade5e309b273fdd298a8ded1d4be
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/5388078182762a2912f2feaae024ec465975f912...1ac9d688e176ade5e309b273fdd298a8ded1d4be
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240507/dc8bf80e/attachment-0001.html>
More information about the ghc-commits
mailing list