[Git][ghc/ghc][wip/t21766] 68 commits: rts: Drop redundant prototype
Finley McIlwaine (@FinleyMcIlwaine)
gitlab at gitlab.haskell.org
Fri Mar 10 16:39:19 UTC 2023
Finley McIlwaine pushed to branch wip/t21766 at Glasgow Haskell Compiler / GHC
Commits:
321d46d9 by Ben Gamari at 2023-03-08T15:02:30-05:00
rts: Drop redundant prototype
- - - - -
abb6070f by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Fix style
- - - - -
be278901 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Deduplicate assertion
- - - - -
b9034639 by Ben Gamari at 2023-03-08T15:02:30-05:00
rts: Fix type issues in Sparks.h
Adds explicit casts to satisfy a C++ compiler.
- - - - -
da7b2b94 by Ben Gamari at 2023-03-08T15:02:30-05:00
rts: Use release ordering when storing thread labels
Since this makes the ByteArray# visible from other cores.
- - - - -
5b7f6576 by Ben Gamari at 2023-03-08T15:02:30-05:00
rts/BlockAlloc: Allow disabling of internal assertions
These can be quite expensive and it is sometimes useful to compile a
DEBUG RTS without them.
- - - - -
6283144f by Ben Gamari at 2023-03-08T15:02:30-05:00
rts/Sanity: Mark pinned_object_blocks
- - - - -
9b528404 by Ben Gamari at 2023-03-08T15:02:30-05:00
rts/Sanity: Look at nonmoving saved_filled lists
- - - - -
0edc5438 by Ben Gamari at 2023-03-08T15:02:30-05:00
Evac: Squash data race in eval_selector_chain
- - - - -
7eab831a by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Clarify implementation
This makes the intent of this implementation a bit clearer.
- - - - -
532262b9 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Clarify comment
- - - - -
bd9cd84b by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Add missing no-op in busy-wait loop
- - - - -
c4e6bfc8 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Don't push empty arrays to update remembered set
Previously the write barrier of resizeSmallArray# incorrectly handled
resizing of zero-sized arrays, pushing an invalid pointer to the update
remembered set.
Fixes #22931.
- - - - -
92227b60 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Fix handling of weak pointers
This fixes an interaction between aging and weak pointer handling which
prevented the finalization of some weak pointers. In particular, weak
pointers could have their keys incorrectly marked by the preparatory
collector, preventing their finalization by the subsequent concurrent
collection.
While in the area, we also significantly improve the assertions
regarding weak pointers.
Fixes #22327.
- - - - -
ba7e7972 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Sanity check nonmoving large objects and compacts
- - - - -
71b038a1 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Sanity check mutable list
Assert that entries in the nonmoving generation's generational
remembered set (a.k.a. mutable list) live in nonmoving generation.
- - - - -
99d144d5 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Don't show occupancy if we didn't collect live words
- - - - -
81d6cc55 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Fix tracking of FILLED_SWEEPING segments
Previously we only updated the state of the segment at the head of each
allocator's filled list.
- - - - -
58e53bc4 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Assert state of swept segments
- - - - -
2db92e01 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Handle new closures in nonmovingIsNowAlive
We must conservatively assume that new closures are reachable since we
are not guaranteed to mark such blocks.
- - - - -
e4c3249f by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Don't clobber update rem sets of old capabilities
Previously `storageAddCapabilities` (called by `setNumCapabilities`) would
clobber the update remembered sets of existing capabilities when
increasing the capability count. Fix this by only initializing the
update remembered sets of the newly-created capabilities.
Fixes #22927.
- - - - -
1b069671 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Add missing write barriers in selector optimisation
This fixes the selector optimisation, adding a few write barriers which
are necessary for soundness. See the inline comments for details.
Fixes #22930.
- - - - -
d4032690 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Post-sweep sanity checking
- - - - -
0baa8752 by Ben Gamari at 2023-03-08T15:02:30-05:00
nonmoving: Avoid n_caps race
- - - - -
5d3232ba by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Don't push if nonmoving collector isn't enabled
- - - - -
0a7eb0aa by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Be more paranoid in segment tracking
Previously we left various segment link pointers dangling. None of this
wrong per se, but it did make it harder than necessary to debug.
- - - - -
7c817c0a by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Sync-phase mark budgeting
Here we significantly improve the bound on sync phase pause times by
imposing a limit on the amount of work that we can perform during the
sync. If we find that we have exceeded our marking budget then we allow
the mutators to resume, return to concurrent marking, and try
synchronizing again later.
Fixes #22929.
- - - - -
ce22a3e2 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Allow pinned gen0 objects to be WEAK keys
- - - - -
78746906 by Ben Gamari at 2023-03-08T15:02:31-05:00
rts: Reenable assertion
- - - - -
b500867a by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Move current segment array into Capability
The current segments are conceptually owned by the mutator, not the
collector. Consequently, it was quite tricky to prove that the mutator
would not race with the collect due to this shared state. It turns out
that such races are possible: when resizing the current segment array
we may concurrently try to take a heap census. This will attempt to walk
the current segment array, causing a data race.
Fix this by moving the current segment array into `Capability`, where it
belongs.
Fixes #22926.
- - - - -
56e669c1 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Fix Note references
Some references to Note [Deadlock detection under the non-moving
collector] were missing an article.
- - - - -
4a7650d7 by Ben Gamari at 2023-03-08T15:02:31-05:00
rts/Sanity: Fix block count assertion with non-moving collector
The nonmoving collector does not use `oldest_gen->blocks` to track its
block list. However, it nevertheless updates `oldest_gen->n_blocks` to
ensure that its size is accounted for by the storage manager.
Consequently, we must not attempt to assert consistency between the two.
- - - - -
96a5aaed by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Don't call prepareUnloadCheck
When the nonmoving GC is in use we do not call `checkUnload` (since we
don't unload code) and therefore should not call `prepareUnloadCheck`,
lest we run into assertions.
- - - - -
6c6674ca by Ben Gamari at 2023-03-08T15:02:31-05:00
rts: Encapsulate block allocator spinlock
This makes it a bit easier to add instrumentation on this spinlock
while debugging.
- - - - -
e84f7167 by Ben Gamari at 2023-03-08T15:02:31-05:00
testsuite: Skip some tests when sanity checking is enabled
- - - - -
3ae0f368 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Fix unregisterised build
- - - - -
4eb9d06b by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Ensure that sanity checker accounts for saved_filled segments
- - - - -
f0cf384d by Ben Gamari at 2023-03-08T15:02:31-05:00
hadrian: Add +boot_nonmoving_gc flavour transformer
For using GHC bootstrapping to validate the non-moving GC.
- - - - -
581e58ac by Ben Gamari at 2023-03-08T15:02:31-05:00
gitlab-ci: Add job bootstrapping with nonmoving GC
- - - - -
487a8b58 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Move allocator into new source file
- - - - -
8f374139 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Split out nonmovingAllocateGC
- - - - -
662b6166 by Ben Gamari at 2023-03-08T15:02:31-05:00
testsuite: Only run T22795* in the normal way
It doesn't make sense to run these in multiple ways as they merely test
whether `-threaded`/`-single-threaded` flags.
- - - - -
0af21dfa by Ben Gamari at 2023-03-08T15:02:31-05:00
rts: Rename clear_segment(_free_blocks)?
To reflect the fact that these are to do with the nonmoving collector,
now since they are exposed no longer static.
- - - - -
7bcb192b by Ben Gamari at 2023-03-08T15:02:31-05:00
rts: Fix incorrect STATIC_INLINE
This should be INLINE_HEADER lest we get unused declaration warnings.
- - - - -
f1fd3ffb by Ben Gamari at 2023-03-08T15:02:31-05:00
testsuite: Mark ffi023 as broken due to #23089
- - - - -
a57f12b3 by Ben Gamari at 2023-03-08T15:02:31-05:00
testsuite: Skip T7160 in the nonmoving way
Finalization order is different under the nonmoving collector.
- - - - -
f6f12a36 by Ben Gamari at 2023-03-08T15:02:31-05:00
rts: Capture GC configuration in a struct
The number of distinct arguments passed to GarbageCollect was getting a
bit out of hand.
- - - - -
ba73a807 by Ben Gamari at 2023-03-08T15:02:31-05:00
nonmoving: Non-concurrent collection
- - - - -
7c813d06 by Alexis King at 2023-03-08T15:03:10-05:00
hadrian: Fix flavour compiler stage options off-by-one error
!9193 pointed out that ghcDebugAssertions was supposed to be a predicate
on the stage of the built compiler, but in practice it was a predicate
on the stage of the compiler used to build. Unfortunately, while it
fixed that issue for ghcDebugAssertions, it documented every other
similar option as behaving the same way when in fact they all used the
old behavior.
The new behavior of ghcDebugAssertions seems more intuitive, so this
commit changes the interpretation of every other option to match. It
also improves the enableProfiledGhc and debugGhc flavour transformers by
making them more selective about which stages in which they build
additional library/RTS ways.
- - - - -
f97c7f6d by Luite Stegeman at 2023-03-09T09:52:09-05:00
Delete created temporary subdirectories at end of session.
This patch adds temporary subdirectories to the list of
paths do clean up at the end of the GHC session. This
fixes warnings about non-empty temporary directories.
Fixes #22952
- - - - -
9ea719f2 by Apoorv Ingle at 2023-03-09T09:52:45-05:00
Fixes #19627.
Previously the solver failed with an unhelpful "solver reached too may iterations" error.
With the fix for #21909 in place we no longer have the possibility of generating such an error if we have `-fconstraint-solver-iteration` > `-fgivens-fuel > `-fwanteds-fuel`. This is true by default, and the said fix also gives programmers a knob to control how hard the solver should try before giving up.
This commit adds:
* Reference to ticket #19627 in the Note [Expanding Recursive Superclasses and ExpansionFuel]
* Test `typecheck/should_fail/T19627.hs` for regression purposes
- - - - -
ec2d93eb by Sebastian Graf at 2023-03-10T10:18:54-05:00
DmdAnal: Fix a panic on OPAQUE and trivial/PAP RHS (#22997)
We should not panic in `add_demands` (now `set_lam_dmds`), because that code
path is legimitely taken for OPAQUE PAP bindings, as in T22997.
Fixes #22997.
- - - - -
5b4628ae by Sylvain Henry at 2023-03-10T10:19:34-05:00
JS: remove dead code for old integer-gmp
- - - - -
9f2af97a by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Restructure IPE buffer layout
Reference ticket #21766
This commit restructures IPE buffer list entries to not contain
references to their corresponding info tables. IPE buffer list nodes now
point to two lists of equal length, one holding the list of info table
pointers and one holding the corresponding entries for each info table.
This will allow the entry data to be compressed without losing the
references to the info tables.
- - - - -
9ad2b569 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Add IPE compression to configure
Reference ticket #21766
Adds an `--enable-ipe-data-compreesion` flag to the configure script
which will check for libzstd and set the appropriate flags to allow for
IPE data compression in the compiler
- - - - -
246c81d1 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
IPE data compression
Reference ticket #21766
When IPE data compression is enabled, compress the emitted IPE buffer
entries and decompress them in the RTS.
- - - - -
95192e05 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Fix libzstd detection in configure and RTS
Ensure that `HAVE_LIBZSTD` gets defined to either 0 or 1 in all cases
and properly check that before IPE data decompression in the RTS. See
ticket #21766.
- - - - -
38376a48 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Add note describing IPE data compression
See ticket #21766
- - - - -
7d73cad2 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Fix byte order of IPE data, fix IPE tests
Make sure byte order of written IPE buffer entries matches target.
Make sure the IPE-related tests properly access the fields of IPE buffer
entry nodes with the new IPE layout.
This commit also introduces checks to avoid importing modules if IPE
compression is not enabled.
See ticket #21766.
- - - - -
5137000d by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Fix IPE data decompression buffer allocation
Capacity of buffers allocated for decompressed IPE data was
incorrect due to a misuse of the `ZSTD_findFrameCompressedSize`
function. Fix by always storing decompressed size of IPE data in IPE
buffer list nodes and using `ZSTD_findFrameCompressedSize` to determine
the size of the compressed data.
See ticket #21766
- - - - -
54593f47 by Finley McIlwaine at 2023-03-10T09:34:12-07:00
Add optional dependencies to ./configure output
Changes the configure script to indicate whether libnuma, libzstd, or
libdw are being used as dependencies due to their optional features
being enabled.
- - - - -
6378f84b by Finley McIlwaine at 2023-03-10T09:36:56-07:00
Add IPE-enabled builds to CI
- Adds an IPE job to the CI pipeline which is triggered by the ~IPE label
- Introduces CI logic to enable IPE data compression
- Enables uncompressed IPE data on debug CI job
- Regenerates jobs.yaml
MR https://gitlab.haskell.org/ghc/ci-images/-/merge_requests/112 on the
images repository is meant to ensure that the proper images have
libzstd-dev installed.
- - - - -
894685c2 by Finley McIlwaine at 2023-03-10T09:36:59-07:00
Update user's guide and release notes
Add mention of IPE data compression to user's guide and the release
notes for 9.8.1. Also note the impact compression has on binary size in
both places.
See ticket #21766
- - - - -
770beeea by Finley McIlwaine at 2023-03-10T09:37:00-07:00
Fix multiline string in `IPE.c`
- - - - -
ac20c0ee by Finley McIlwaine at 2023-03-10T09:37:00-07:00
Optional static linking of libzstd
Allow for libzstd to be statically linked with a
`--enable-static-libzstd` configure flag. Not supported on darwin due to
incompatibility with `:x.a` linker flags.
- - - - -
ba2c1d8b by Finley McIlwaine at 2023-03-10T09:37:00-07:00
Detect darwin for `--enable-static-libzstd`
Update users guide to note the optional static linking of libzstd, and
update ci-images rev to get libzstd in debian images on CI
- - - - -
9948f26a by Finley McIlwaine at 2023-03-10T09:38:07-07:00
Revert `+ipe` enabled CI jobs for ~IPE label
- - - - -
2ac6b704 by Finley McIlwaine at 2023-03-10T09:39:00-07:00
Use correct image for wasm jobs in CI
- - - - -
30 changed files:
- .gitlab-ci.yml
- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC/Core/Opt/DmdAnal.hs
- compiler/GHC/Linker/Static.hs
- compiler/GHC/StgToCmm/InfoTableProv.hs
- compiler/GHC/Tc/Solver.hs
- compiler/GHC/Utils/TmpFs.hs
- compiler/ghc.cabal.in
- configure.ac
- docs/users_guide/9.8.1-notes.rst
- docs/users_guide/debug-info.rst
- hadrian/cfg/system.config.in
- hadrian/doc/user-settings.md
- hadrian/src/Expression.hs
- hadrian/src/Flavour.hs
- hadrian/src/Oracles/Flag.hs
- hadrian/src/Oracles/Flavour.hs
- hadrian/src/Oracles/Setting.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Builders/Ghc.hs
- hadrian/src/Settings/Builders/RunTest.hs
- hadrian/src/Settings/Flavours/Development.hs
- hadrian/src/Settings/Flavours/Quick.hs
- hadrian/src/Settings/Packages.hs
- libraries/ghc-heap/tests/all.T
- m4/fp_find_libnuma.m4
- + m4/fp_find_libzstd.m4
- rts/Capability.c
- rts/Capability.h
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/4a8095948fdfb562f6802869a506eb4cd572ab43...2ac6b704de7b7120af3ebc5aa45636b493f26fd6
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/4a8095948fdfb562f6802869a506eb4cd572ab43...2ac6b704de7b7120af3ebc5aa45636b493f26fd6
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230310/6035f433/attachment-0001.html>
More information about the ghc-commits
mailing list