[Git][ghc/ghc][wip/fendor/ghc-iface-sharing-avoid-reserialisation] 53 commits: Migrate `Finder` component to `OsPath`, fixed #24616
Hannes Siebenhandl (@fendor)
gitlab at gitlab.haskell.org
Wed Jun 12 13:04:32 UTC 2024
Hannes Siebenhandl pushed to branch wip/fendor/ghc-iface-sharing-avoid-reserialisation at Glasgow Haskell Compiler / GHC
Commits:
c8ece0df by Fendor at 2024-06-03T19:43:22-04:00
Migrate `Finder` component to `OsPath`, fixed #24616
For each module in a GHCi session, we keep alive one `ModLocation`.
A `ModLocation` is fairly inefficiently packed, as `String`s are
expensive in memory usage.
While benchmarking the agda codebase, we concluded that we keep alive
around 11MB of `FilePath`'s, solely retained by `ModLocation`.
We provide a more densely packed encoding of `ModLocation`, by moving
from `FilePath` to `OsPath`. Further, we migrate the full `Finder`
component to `OsPath` to avoid unnecessary transformations.
As the `Finder` component is well-encapsulated, this requires only a
minimal amount of changes in other modules.
We introduce pattern synonym for 'ModLocation' which maintains backwards
compatibility and avoids breaking consumers of 'ModLocation'.
- - - - -
0cff083a by Cheng Shao at 2024-06-03T19:43:58-04:00
compiler: emit NaturallyAligned when element type & index type are the same width
This commit fixes a subtle mistake in alignmentFromTypes that used to
generate Unaligned when element type & index type are the same width.
Fixes #24930.
- - - - -
18f63970 by Sebastian Graf at 2024-06-04T05:05:27-04:00
Parser: Remove unused `apats` rule
- - - - -
38757c30 by David Knothe at 2024-06-04T05:05:27-04:00
Implement Or Patterns (#22596)
This commit introduces a new language extension, `-XOrPatterns`, as described in
GHC Proposal 522.
An or-pattern `pat1; ...; patk` succeeds iff one of the patterns `pat1`, ...,
`patk` succeed, in this order.
See also the summary `Note [Implmentation of OrPatterns]`.
Co-Authored-By: Sebastian Graf <sgraf1337 at gmail.com>
- - - - -
395412e8 by Cheng Shao at 2024-06-04T05:06:04-04:00
compiler/ghci/rts: remove stdcall support completely
We have formally dropped i386 windows support (#18487) a long time
ago. The stdcall foreign call convention is only used by i386 windows,
and the legacy logic around it is a significant maintenance burden for
future work that adds arm64 windows support (#24603). Therefore, this
patch removes stdcall support completely from the compiler as well as
the RTS (#24883):
- stdcall is still recognized as a FFI calling convention in Haskell
syntax. GHC will now unconditionally emit a warning
(-Wunsupported-calling-conventions) and treat it as ccall.
- Apart from minimum logic to support the parsing and warning logic,
all other code paths related to stdcall has been completely stripped
from the compiler.
- ghci only supports FFI_DEFAULT_ABI and ccall convention from now on.
- FFI foreign export adjustor code on all platforms no longer handles
the stdcall case and only handles ccall from now on.
- The Win32 specific parts of RTS no longer has special code paths for
stdcall.
This commit is the final nail on the coffin for i386 windows support.
Further commits will perform more housecleaning to strip the legacy
code paths and pave way for future arm64 windows support.
- - - - -
d1fe9ab6 by Cheng Shao at 2024-06-04T05:06:04-04:00
rts: remove legacy i386 windows code paths
This commit removes some legacy i386 windows related code paths in the
RTS, given this target is no longer supported.
- - - - -
a605e4b2 by Cheng Shao at 2024-06-04T05:06:04-04:00
autoconf: remove i386 windows related logic
This commit removes legacy i386 windows logic in autoconf scripts.
- - - - -
91e5ac5e by Cheng Shao at 2024-06-04T05:06:04-04:00
llvm-targets: remove i386 windows support
This commit removes i386 windows from llvm-targets and the script to
generate it.
- - - - -
65fe75a4 by Cheng Shao at 2024-06-04T05:06:04-04:00
libraries/utils: remove stdcall related legacy logic
This commit removes stdcall related legacy logic in libraries and
utils. ccall should be used uniformly for all supported windows hosts
from now on.
- - - - -
d2a83302 by Cheng Shao at 2024-06-04T05:06:04-04:00
testsuite: adapt the testsuite for stdcall removal
This patch adjusts test cases to handle the stdcall removal:
- Some stdcall usages are replaced with ccall since stdcall doesn't
make sense anymore.
- We also preserve some stdcall usages, and check in the expected
warning messages to ensure GHC always warn about stdcall usages
(-Wunsupported-calling-conventions) as expected.
- Error code testsuite coverage is slightly improved,
-Wunsupported-calling-conventions is now tested.
- Obsolete code paths related to i386 windows are also removed.
- - - - -
cef8f47a by Cheng Shao at 2024-06-04T05:06:04-04:00
docs: minor adjustments for stdcall removal
This commit include minor adjustments of documentation related to
stdcall removal.
- - - - -
54332437 by Cheng Shao at 2024-06-04T05:06:04-04:00
docs: mention i386 Windows removal in 9.12 changelog
This commit mentions removal of i386 Windows support and stdcall
related change in the 9.12 changelog.
- - - - -
2aaea8a1 by Cheng Shao at 2024-06-04T05:06:40-04:00
hadrian: improve user settings documentation
This patch adds minor improvements to hadrian user settings documentation:
- Add missing `ghc.cpp.opts` case
- Remove non-existent `cxx` case
- Clarify `cc.c.opts` also works for C++, while `cc.deps.opts` doesn't
- Add example of passing configure argument to autoconf packages
- - - - -
71010381 by Alex Mason at 2024-06-04T12:09:07-04:00
Add AArch64 CLZ, CTZ, RBIT primop implementations.
Adds support for emitting the clz and rbit instructions, which are
used by GHC.Prim.clz*#, GHC.Prim.ctz*# and GHC.Prim.bitReverse*#.
- - - - -
44e2abfb by Cheng Shao at 2024-06-04T12:09:43-04:00
hadrian: add +text_simdutf flavour transformer to allow building text with simdutf
This patch adds a +text_simdutf flavour transformer to hadrian to
allow downstream packagers and users that build from source to opt-in
simdutf support for text, in order to benefit from SIMD speedup at
run-time. It's still disabled by default for the time being.
- - - - -
077cb2e1 by Cheng Shao at 2024-06-04T12:09:43-04:00
ci: enable +text_simdutf flavour transformer for wasm jobs
This commit enables +text_simdutf flavour transformer for wasm jobs,
so text is now built with simdutf support for wasm.
- - - - -
b23746ad by Teo Camarasu at 2024-06-04T22:50:50-04:00
base: Use TemplateHaskellQuotes in instance Lift ByteArray
Resolves #24852
- - - - -
3fd25743 by Teo Camarasu at 2024-06-04T22:50:50-04:00
base: Mark addrToByteArray as NOINLINE
This function should never be inlined in order to keep code size small.
- - - - -
98ad1ea5 by Cheng Shao at 2024-06-04T22:51:26-04:00
compiler: remove unused CompilerInfo/LinkerInfo types
This patch removes CompilerInfo/LinkerInfo types from the compiler
since they aren't actually used anywhere.
- - - - -
11795244 by Cheng Shao at 2024-06-05T06:33:17-04:00
rts: remove unused PowerPC/IA64 native adjustor code
This commit removes unused PowerPC/IA64 native adjustor code which is
never actually enabled by autoconf/hadrian. Fixes #24920.
- - - - -
5132754b by Sylvain Henry at 2024-06-05T06:33:57-04:00
RTS: fix warnings with doing*Profiling (#24918)
- - - - -
accc8c33 by Cheng Shao at 2024-06-05T11:35:36-04:00
hadrian: don't depend on inplace/mingw when --enable-distro-toolchain on Windows
- - - - -
6ffbd678 by Cheng Shao at 2024-06-05T11:35:37-04:00
autoconf: normalize paths of some build-time dependencies on Windows
This commit applies path normalization via cygpath -m to some
build-time dependencies on Windows. Without this logic, the
/clang64/bin prefixed msys2-style paths cause the build to fail with
--enable-distro-toolchain.
- - - - -
075dc6d4 by Cheng Shao at 2024-06-05T11:36:12-04:00
hadrian: remove OSDarwin mention from speedHack
This commit removes mentioning of OSDarwin from speedHack, since
speedHack is purely for i386 and we no longer support i386 darwin
(#24921).
- - - - -
83235c4c by Cheng Shao at 2024-06-05T11:36:12-04:00
compiler: remove 32-bit darwin logic
This commit removes all 32-bit darwin logic from the compiler, given
we no longer support 32-bit apple systems (#24921). Also contains a
bit more cleanup of obsolete i386 windows logic.
- - - - -
1eb99bc3 by Cheng Shao at 2024-06-05T11:36:12-04:00
rts: remove 32-bit darwin/ios logic
This commit removes 32-bit darwin/ios related logic from the rts,
given we no longer support them (#24921).
- - - - -
24f65892 by Cheng Shao at 2024-06-05T11:36:12-04:00
llvm-targets: remove 32-bit darwin/ios targets
This commit removes 32-bit darwin/ios targets from llvm-targets given
we no longer support them (#24921).
- - - - -
ccdbd689 by Cheng Shao at 2024-06-05T11:36:12-04:00
testsuite: remove 32-bit darwin logic
This commit removes 32-bit darwin logic from the testsuite given it's
no longer supported (#24921). Also contains more cleanup of obsolete
i386 windows logic.
- - - - -
11d661c4 by Cheng Shao at 2024-06-05T11:36:13-04:00
docs: mention 32-bit darwin/ios removal in 9.12 changelog
This commit mentions removal of 32-bit darwin/ios support (#24921) in
the 9.12 changelog.
- - - - -
7c173310 by Georgi Lyubenov at 2024-06-05T15:17:22-04:00
Add firstA and secondA to Data.Bitraversable
Please see https://github.com/haskell/core-libraries-committee/issues/172
for related discussion
- - - - -
3b6f9fd1 by Ben Gamari at 2024-06-05T15:17:59-04:00
base: Fix name of changelog
Fixes #24899. Also place it under `extra-doc-files` to better reflect
its nature and avoid triggering unnecessary recompilation if it
changes.
- - - - -
1f4d2ef7 by Sebastian Graf at 2024-06-05T15:18:34-04:00
Announce Or-patterns in the release notes for GHC 9.12 (#22596)
Leftover from !9229.
- - - - -
8650338d by Jan Hrček at 2024-06-06T10:39:24-04:00
Improve haddocks of Language.Haskell.Syntax.Pat.Pat
- - - - -
2eee65e1 by Cheng Shao at 2024-06-06T10:40:00-04:00
testsuite: bump T7653 timeout for wasm
- - - - -
990fed60 by Sylvain Henry at 2024-06-07T14:45:23-04:00
StgToCmm: refactor opTranslate and friends
- Change arguments order to avoid `\args -> ...` lambdas
- Fix documentation
- Rename StgToCmm options ("big" doesn't mean anything)
- - - - -
1afad514 by Sylvain Henry at 2024-06-07T14:45:23-04:00
NCG x86: remove dead code (#5444)
Since 6755d833af8c21bbad6585144b10e20ac4a0a1ab this code is dead.
- - - - -
595c0894 by Cheng Shao at 2024-06-07T14:45:58-04:00
testsuite: skip objc-hi/objcxx-hi when cross compiling
objc-hi/objcxx-hi should be skipped when cross compiling. The existing
opsys('darwin') predicate only asserts the host system is darwin but
tells us nothing about the target, hence the oversight.
- - - - -
edfe6140 by qqwy at 2024-06-08T11:23:54-04:00
Replace '?callStack' implicit param with HasCallStack in GHC.Internal.Exception.throw
- - - - -
35a64220 by Cheng Shao at 2024-06-08T11:24:30-04:00
rts: cleanup inlining logic
This patch removes pre-C11 legacy code paths related to
INLINE_HEADER/STATIC_INLINE/EXTERN_INLINE macros, ensure EXTERN_INLINE
is treated as static inline in most cases (fixes #24945), and also
corrects the comments accordingly.
- - - - -
9ea90ed2 by Andrew Lelechenko at 2024-06-08T11:25:06-04:00
CODEOWNERS: add @core-libraries to track base interface changes
A low-tech tactical solution for #24919
- - - - -
580fef7b by Ben Gamari at 2024-06-09T01:27:21-04:00
ghc-internal: Update CHANGELOG to reflect current version
- - - - -
391ecff5 by Ben Gamari at 2024-06-09T01:27:21-04:00
ghc-internal: Update prologue.txt to reflect package description
- - - - -
3dca3b7d by Ben Gamari at 2024-06-09T01:27:57-04:00
compiler: Clarify comment regarding need for MOVABS
The comment wasn't clear in stating that it was only applicable to
immediate source and memory target operands.
- - - - -
6bd850e8 by doyougnu at 2024-06-09T21:02:14-04:00
JS: establish single source of truth for symbols
In pursuit of: #22736.
This MR moves ad-hoc symbols used throughout the js backend into a
single symbols file. Why? First, this cleans up the code by removing
ad-hoc strings created on the fly and therefore makes the code more
maintainable. Second, it makes it much easier to eventually type these
identifiers.
- - - - -
f3017dd3 by Cheng Shao at 2024-06-09T21:02:49-04:00
rts: replace ad-hoc MYTASK_USE_TLV with proper CC_SUPPORTS_TLS
This patch replaces the ad-hoc `MYTASK_USE_TLV` with the
`CC_SUPPORTS_TLS` macro. If TLS support is detected by autoconf, then
we should use that for managing `myTask` in the threaded RTS.
- - - - -
e17d7e8c by Ben Gamari at 2024-06-11T05:25:21-04:00
users-guide: Fix stylistic issues in 9.12 release notes
- - - - -
8a8a982a by Hugo Peters at 2024-06-11T05:25:57-04:00
fix typo in the simplifier debug output:
baling -> bailing
- - - - -
16475bb8 by Hécate Moonlight at 2024-06-12T03:07:55-04:00
haddock: Correct the Makefile to take into account Darwin systems
- - - - -
a2f60da5 by Hécate Kleidukos at 2024-06-12T03:08:35-04:00
haddock: Remove obsolete links to github.com/haskell/haddock in the docs
- - - - -
de4395cd by qqwy at 2024-06-12T03:09:12-04:00
Add `__GLASGOW_HASKELL_ASSERTS_IGNORED__` as CPP macro name if `-fasserts-ignored is set.
This allows users to create their own Control.Exception.assert-like functionality that
does something other than raising an `AssertFailed` exception.
Fixes #24967
- - - - -
0e9c4dee by Ryan Hendrickson at 2024-06-12T03:09:53-04:00
compiler: add hint to TcRnBadlyStaged message
- - - - -
c5e7b418 by Fendor at 2024-06-12T15:04:12+02:00
Improve sharing of duplicated values in `ModIface`, fixes #24723
As a `ModIface` often contains duplicated values that are not
necessarily shared, we improve sharing by serialising the `ModIface`
to an in-memory byte array. Serialisation uses deduplication tables, and
deserialisation implicitly shares duplicated values.
This helps reducing the peak memory usage while compiling in
`--make` mode. The peak memory usage is especially smaller when
generating interface files with core expressions
(`-fwrite-if-simplified-core`).
On agda, this reduces the peak memory usage:
* `2.2 GB` to `1.9 GB` for a ghci session.
On `lib:Cabal`, we report:
* `570 MB` to `500 MB` for a ghci session
* `790 MB` to `667 MB` for compiling `lib:Cabal` with ghc
There is a small impact on execution time, around 2% on the agda code
base.
- - - - -
c3d98201 by Fendor at 2024-06-12T15:04:12+02:00
Avoid unneccessarily re-serialising the `ModIface`
To reduce memory usage of `ModIface`, we serialise `ModIface` to an
in-memory byte array, which implicitly shares duplicated values.
This serialised byte array can be reused to avoid work when we actually
write the `ModIface` to disk.
We introduce a new field to `ModIface` which allows us to save the byte
array, and write it direclty to disk if the `ModIface` wasn't changed
after the initial serialisation.
This requires us to change absolute offsets, for example to jump to the
deduplication table for `Name` or `FastString` with relative offsets, as
the deduplication byte array doesn't contain header information, such as
fingerprints.
To allow us to dump the binary blob to disk, we need to replace all
absolute offsets with relative ones.
We introduce additional helpers for `ModIface` binary serialisation, which
construct relocatable binary blobs. We say the binary blob is relocatable,
if the binary representation can be moved and does not contain any
absolute offsets.
Further, we introduce new primitives for `Binary` that allow to create
relocatable binaries, such as `forwardGetRel` and `forwardPutRel`.
-------------------------
Metric Decrease:
MultiLayerModulesDefsGhcWithCore
Metric Increase:
MultiComponentModules
MultiLayerModules
T10421
T12150
T12234
T12425
T13035
T13253-spj
T13701
T13719
T14697
T15703
T18140
T18304
T18698a
T18730
T18923
T20049
T24582
T5837
T6048
T9198
T9961
mhu-perf
-------------------------
These metric increases may look bad, but they are all completely benign,
we simply allocate 1 MB per module for `shareIface`. As this allocation
is quite quick, it has a negligible impact on run-time performance.
In fact, the performance difference wasn't measurable on my local
machine. Reducing the size of the pre-allocated 1 MB buffer avoids these
test failures, but also requires us to reallocate the buffer if the
interface file is too big. These reallocations *did* have an impact on
performance, which is why I have opted to accept all these metric
increases, as the number of allocated bytes is merely a guidance.
This 1MB allocation increase causes a lot of tests to fail that
generally have a low allocation number. E.g., increasing from 40MB to
41MB is a 2.5% increase.
In particular, the tests T12150, T13253-spj, T18140, T18304, T18698a,
T18923, T20049, T24582, T5837, T6048, and T9961 only fail on i386-linux
job, where the number of allocated bytes seems to be lower than in other
jobs.
- - - - -
30 changed files:
- .gitlab/generate-ci/gen_ci.hs
- .gitlab/jobs.yaml
- CODEOWNERS
- compiler/GHC.hs
- compiler/GHC/Builtin/Names/TH.hs
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/Cmm/CLabel.hs
- compiler/GHC/Cmm/Node.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/ThreadSanitizer.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Instr.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/CmmToAsm/PIC.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToAsm/X86/Instr.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Core/Opt/Simplify.hs
- + compiler/GHC/Data/OsPath.hs
- compiler/GHC/Data/Strict.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/CodeOutput.hs
- compiler/GHC/Driver/Config/Cmm.hs
- compiler/GHC/Driver/Config/Finder.hs
- compiler/GHC/Driver/Config/StgToCmm.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/12d59b3b208ca8a88eecc88e9b4fa5fc651e55bc...c3d98201a9e63a7a178090678806636f7b5ce230
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/12d59b3b208ca8a88eecc88e9b4fa5fc651e55bc...c3d98201a9e63a7a178090678806636f7b5ce230
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240612/bdffa010/attachment-0001.html>
More information about the ghc-commits
mailing list