[Git][ghc/ghc][wip/T22264] 70 commits: hadrian: add hi_core flavour transformer

Wed Jan 25 16:02:16 UTC 2023

Ben Gamari pushed to branch wip/T22264 at Glasgow Haskell Compiler / GHC

Commits:
1fe806d3 by Cheng Shao at 2023-01-23T04:48:47-05:00
hadrian: add hi_core flavour transformer

The hi_core flavour transformer enables -fwrite-if-simplified-core for
stage1 libraries, which emit core into interface files to make it
possible to restart code generation. Building boot libs with it makes
it easier to use GHC API to prototype experimental backends that needs
core/stg at link time.

- - - - -
317cad26 by Cheng Shao at 2023-01-23T04:48:47-05:00
hadrian: add missing docs for recently added flavour transformers

- - - - -
658f4446 by Ben Gamari at 2023-01-23T04:49:23-05:00
gitlab-ci: Add Rocky8 jobs

Addresses #22268.

- - - - -
a83ec778 by Vladislav Zavialov at 2023-01-23T04:49:58-05:00
Set "since: 9.8" for TypeAbstractions and -Wterm-variable-capture

These flags did not make it into the 9.6 release series,
so the "since" annotations must be corrected.

- - - - -
fec7c2ea by Alan Zimmerman at 2023-01-23T04:50:33-05:00
EPA: Add SourceText to HsOverLabel

To be able to capture string literals with possible escape codes as labels.

Close #22771

- - - - -
3efd1e99 by Ben Gamari at 2023-01-23T04:51:08-05:00
template-haskell: Bump version to 2.20.0.0

Updates `text` and `exceptions` submodules for bounds bumps.

Addresses #22767.

- - - - -
0900b584 by Cheng Shao at 2023-01-23T04:51:45-05:00
hadrian: disable alloca for in-tree GMP on wasm32

When building in-tree GMP for wasm32, disable its alloca usage, since
it may potentially cause stack overflow (e.g. #22602).

- - - - -
db0f1bfd by Cheng Shao at 2023-01-23T04:52:21-05:00
Bump process submodule

Includes a critical fix for wasm32, see
https://github.com/haskell/process/pull/272 for details. Also changes
the existing cross test to include process stuff and avoid future
regression here.

- - - - -
9222b167 by Matthew Pickering at 2023-01-23T04:52:57-05:00
ghcup metadata: Fix subdir for windows bindist

- - - - -
9a9bec57 by Matthew Pickering at 2023-01-23T04:52:57-05:00
ghcup metadata: Remove viPostRemove field from generated metadata

This has been removed from the downstream metadata.

- - - - -
82884ce0 by Simon Peyton Jones at 2023-01-23T04:53:32-05:00
Fix #22742

runtimeRepLevity_maybe was panicing unnecessarily; and
the error printing code made use of the case when it should
return Nothing rather than panicing.

For some bizarre reason perf/compiler/T21839r shows a 10% bump in runtime
peak-megagbytes-used, on a single architecture (alpine). See !9753 for
commentary, but I'm going to accept it.

Metric Increase:
    T21839r

- - - - -
2c6deb18 by Bryan Richter at 2023-01-23T14:12:22+02:00
codeowners: Add Ben, Matt, and Bryan to CI

- - - - -
eee3bf05 by Matthew Craven at 2023-01-23T21:46:41-05:00
Do not collect compile-time metrics for T21839r

...the testsuite doesn't handle this properly since it
also collects run-time metrics.  Compile-time metrics
for this test are already tracked via T21839c.

Metric Decrease:
    T21839r

- - - - -
1d1dd3fb by Matthew Pickering at 2023-01-24T05:37:52-05:00
Fix recompilation checking for multiple home units

The key part of this change is to store a UnitId in the
`UsageHomeModule` and `UsageHomeModuleInterface`.

* Fine-grained dependency tracking is used if the dependency comes from
  any home unit.
* We actually look up the right module when checking whether we need to
  recompile in the `UsageHomeModuleInterface` case.

These scenarios are both checked by the new tests (
multipleHomeUnits_recomp and multipleHomeUnits_recomp_th )

Fixes #22675

- - - - -
7bfb30f9 by Matthew Pickering at 2023-01-24T05:37:52-05:00
Augment target filepath by working directory when checking if module satisfies target

This fixes a spurious warning in -Wmissing-home-modules.

This is a simple oversight where when looking for the target in the
first place we augment the search by the -working-directory flag but
then fail to do so when checking this warning.

Fixes #22676

- - - - -
69500dd4 by Matthew Pickering at 2023-01-24T05:37:52-05:00
Use NodeKey rather than ModuleName in pruneCache

The `pruneCache` function assumes that the list of `CachedInfo` all have unique `ModuleName`, this is not true:

* In normal compilation, the same module name can appear for a file and it's boot file.
* In multiple home unit compilation the same ModuleName can appear in different units

The fix is to use a `NodeKey` as the actual key for the interfaces which includes `ModuleName`, `IsBoot` and `UnitId`.

Fixes #22677

- - - - -
336b2b1c by Matthew Pickering at 2023-01-24T05:37:52-05:00
Recompilation checking: Don't try to find artefacts for Interactive & hs-boot combo

In interactive mode we don't produce any linkables for hs-boot files. So
we also need to not going looking for them when we check to see if we
have all the right objects needed for recompilation.

Ticket #22669

- - - - -
6469fea7 by Matthew Pickering at 2023-01-24T05:37:52-05:00
Don't write o-boot files in Interactive mode

We should not be producing object files when in interactive mode but we
still produced the dummy o-boot files. These never made it into a
`Linkable` but then confused the recompilation checker.

Fixes #22669

- - - - -
06cc0a95 by Matthew Pickering at 2023-01-24T05:37:52-05:00
Improve driver diagnostic messages by including UnitId in message

Currently the driver diagnostics don't give any indication about which unit they correspond to.

For example `-Wmissing-home-modules` can fire multiple times for each different home unit and gives no indication about which unit it's actually reporting about.

Perhaps a longer term fix is to generalise the providence information away from a SrcSpan so that these kind of whole project errors can be reported with an accurate provenance. For now we can just include the `UnitId` in the error message.

Fixes #22678

- - - - -
4fe9eaff by Matthew Pickering at 2023-01-24T05:37:52-05:00
Key ModSummary cache by UnitId as well as FilePath

Multiple units can refer to the same files without any problem. Just
another assumption which needs to be updated when we may have multiple
home units.

However, there is the invariant that within each unit each file only
maps to one module, so as long as we also key the cache by UnitId then
we are all good.

This led to some confusing behaviour in GHCi when reloading,
multipleHomeUnits_shared distils the essence of what can go wrong.

Fixes #22679

- - - - -
ada29f5c by Matthew Pickering at 2023-01-24T05:37:52-05:00
Finder: Look in current unit before looking in any home package dependencies

In order to preserve existing behaviour it's important to look within the current component before consideirng a module might come from an external component.

This already happened by accident in `downsweep`, (because roots are used to repopulated the cache) but in the `Finder` the logic was the wrong way around.

Fixes #22680

-------------------------
Metric Decrease:
    MultiComponentModules
    MultiComponentModulesRecomp
-------------------------p

- - - - -
be701cc6 by Matthew Pickering at 2023-01-24T05:37:52-05:00
Debug: Print full NodeKey when pretty printing ModuleGraphNode

This is helpful when debugging multiple component issues.

- - - - -
34d2d463 by Krzysztof Gogolewski at 2023-01-24T05:38:32-05:00
Fix Lint check for duplicate external names

Lint was checking for duplicate external names by calling removeDups,
which needs a comparison function that is passed to Data.List.sortBy.
But the comparison was not a valid ordering - it returned LT
if one of the names was not external.
For example, the previous implementation won't find a duplicate in
[M.x, y, M.x].
Instead, we filter out non-external names before looking for duplicates.

- - - - -
1c050ed2 by Matthew Pickering at 2023-01-24T05:39:08-05:00
Add test for T22671

This was fixed by b13c6ea5

Closes #22671

- - - - -
05e6a2d9 by Tom Ellis at 2023-01-24T12:10:52-05:00
Clarify where `f` is defined

- - - - -
d151546e by Cheng Shao at 2023-01-24T12:11:29-05:00
CmmToC: fix CmmRegOff for 64-bit register on a 32-bit target

We used to print the offset value to a platform word sized integer.
This is incorrect when the offset is negative (e.g. output of cmm
constant folding) and the register is 64-bit but on a 32-bit target,
and may lead to incorrect runtime result (e.g. #22607).

The fix is simple: just treat it as a proper MO_Add, with the correct
width info inferred from the register itself.

Metric Increase:
    T12707
    T13379
    T4801
    T5321FD
    T5321Fun

- - - - -
e5383a29 by Wander Hillen at 2023-01-24T20:02:26-05:00
Allow waiting for timerfd to be interrupted during rts shutdown

- - - - -
1957eda1 by Ryan Scott at 2023-01-24T20:03:01-05:00
Restore Compose's Read/Show behavior to match Read1/Show1 instances

Fixes #22816.

- - - - -
30972827 by Matthew Pickering at 2023-01-25T03:54:14-05:00
docs: Update INSTALL.md

Removes references to make.

Fixes #22480

- - - - -
bc038c3b by Cheng Shao at 2023-01-25T03:54:50-05:00
compiler: fix handling of MO_F_Neg in wasm NCG

In the wasm NCG, we used to compile MO_F_Neg to 0.0-x. It was an
oversight, there actually exists f32.neg/f64.neg opcodes in the wasm
spec and those should be used instead! The old behavior almost works,
expect when GHC compiles the -0.0 literal, which will incorrectly
become 0.0.

- - - - -
0cd532e1 by Ben Gamari at 2023-01-25T11:01:25-05:00
nonmoving: Fix style

- - - - -
412662f4 by Ben Gamari at 2023-01-25T11:01:25-05:00
nonmoving: Deduplicate assertion

- - - - -
4d0f0aef by Ben Gamari at 2023-01-25T11:01:25-05:00
rts: Fix type issues in Sparks.h

Adds explicit casts to satisfy a C++ compiler.

- - - - -
9f772ef5 by Ben Gamari at 2023-01-25T11:01:25-05:00
rts/BlockAlloc: Allow disabling of internal assertions

These can be quite expensive and it is sometimes useful to compile a
DEBUG RTS without them.

- - - - -
ae15e69d by Ben Gamari at 2023-01-25T11:01:25-05:00
rts/Sanity: Mark pinned_object_blocks

- - - - -
321d758a by Ben Gamari at 2023-01-25T11:01:26-05:00
rts/Sanity: Look at nonmoving saved_filled lists

- - - - -
da793084 by Ben Gamari at 2023-01-25T11:01:26-05:00
Evac: Squash data race in eval_selector_chain

- - - - -
f24ee701 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Clarify implementation

This makes the intent of this implementation a bit clearer.

- - - - -
85e903ad by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Clarify comment

- - - - -
ff5acc2e by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Add missing no-op in busy-wait loop

- - - - -
3c8af87e by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Don't push empty arrays to update remembered set

Previously the write barrier of resizeSmallArray# incorrectly handled
resizing of zero-sized arrays, pushing an invalid pointer to the update
remembered set.

Likely fixes the cause of #22264.

- - - - -
fbc51408 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Fix handling of weak pointers

This fixes an interaction between aging and weak pointer handling which
prevented the finalization of some weak pointers. In particular, weak
pointers could have their keys incorrectly marked by the preparatory
collector, preventing their finalization by the subsequent concurrent
collection.

While in the area, we also significantly improve the assertions
regarding weak pointers.

Fixes #22327.

- - - - -
4c5daac6 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Sanity check nonmoving large objects and compacts

- - - - -
328b683e by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Sanity check mutable list

Assert that entries in the nonmoving generation's generational
remembered set (a.k.a. mutable list) live in nonmoving generation.

- - - - -
dd99d945 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Don't show occupancy if we didn't collect live words

- - - - -
a78cb59c by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Fix tracking of FILLED_SWEEPING segments

Previously we only updated the state of the segment at the head of each
allocator's filled list.

- - - - -
fbef885a by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Assert state of swept segments

- - - - -
a5c69dbe by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Handle new closures in nonmovingIsNowAlive

- - - - -
816336e3 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Don't clobber update rem sets of old capabilities

Previously `storageAddCapabilities` (called by `setNumCapabilities`) would
clobber the update remembered sets of existing capabilities when
increasing the capability count. Fix this by only initializing the
update remembered sets of the newly-created capabilities.

- - - - -
cdb75f21 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Add missing write barriers in selector optimisation

This fixes the selector optimisation, adding a few write barriers which
are necessary for soundness.

- - - - -
817d8785 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Post-sweep sanity checking

- - - - -
6a5dca4a by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Avoid n_caps race

- - - - -
bfc5f704 by Ben Gamari at 2023-01-25T11:01:26-05:00
nonmoving: Don't push if nonmoving collector isn't enabled

- - - - -
313c0b30 by Ben Gamari at 2023-01-25T11:01:27-05:00
nonmoving: Be more paranoid in segment tracking

Previously we left various segment link pointers dangling. None of this
wrong per se, but it did make it harder than necessary to debug.

- - - - -
52f04b5f by Ben Gamari at 2023-01-25T11:01:27-05:00
nonmoving: Sync-phase mark budgeting

Here we significantly improve the bound on sync phase pause times by
imposing a limit on the amount of work that we can perform during the
sync. If we find that we have exceeded our marking budget then we allow
the mutators to resume, return to concurrent marking, and try
synchronizing again later.

- - - - -
a2e2df8c by Ben Gamari at 2023-01-25T11:01:27-05:00
nonmoving: Allow pinned gen0 objects to be WEAK keys

- - - - -
c0cab8f1 by Ben Gamari at 2023-01-25T11:01:27-05:00
rts: Fix typo

"tracingAddCapabilities" was mis-named

- - - - -
b08bced8 by Ben Gamari at 2023-01-25T11:01:27-05:00
rts: Reenable assertion

- - - - -
9ce64147 by Ben Gamari at 2023-01-25T11:01:27-05:00
rts: C++ typing issues

Make the RTS compilable with a C++ compiler by inserting necessary
casts.

- - - - -
11e9da04 by Ben Gamari at 2023-01-25T11:01:27-05:00
rts: Drop SM spinlock

- - - - -
eff4f3ab by Ben Gamari at 2023-01-25T11:01:27-05:00
rts: Fix non-atomic mutation of enabled_capabilities

- - - - -
b31a233b by Ben Gamari at 2023-01-25T11:01:27-05:00
nonmoving: Move current segment array into Capability

- - - - -
c3b895bb by Ben Gamari at 2023-01-25T11:01:59-05:00
hadrian: Add +boot_nonmoving_gc flavour transformer

For using GHC bootstrapping to validate the non-moving GC.

- - - - -
410af41c by Ben Gamari at 2023-01-25T11:02:00-05:00
testsuite: Make listThreads more robust

Previously it was sensitive to the labels of threads which it did not
create (e.g. the IO manager event loop threads). Fix this.

- - - - -
24244f43 by Ben Gamari at 2023-01-25T11:02:01-05:00
rts/Sanity: Fix block count assertion with non-moving collector

The nonmoving collector does not use `oldest_gen->blocks` to track its
block list. However, it nevertheless updates `oldest_gen->n_blocks` to
ensure that its size is accounted for by the storage manager.
Consequently, we must not attempt to assert consistency between the two.

- - - - -
6f6b8cff by Ben Gamari at 2023-01-25T11:02:01-05:00
nonmoving: Fix Note references

Some references to Note [Deadlock detection under the non-moving
collector] were missing an article.

- - - - -
6756fddb by Ben Gamari at 2023-01-25T11:02:01-05:00
testsuite: Fix race in UnliftedTVar2

Previously UnliftedTVar2 would fail when run with multiple capabilities
(and possibly even with one capability) as it would assume that
`killThread#` would immediately kill the "increment" thread.

Also, refactor the the executable to now succeed with no output and
fails with an exit code.

- - - - -
507d468b by Ben Gamari at 2023-01-25T11:02:01-05:00
setNumCapabilities

- - - - -
49987153 by Ben Gamari at 2023-01-25T11:02:01-05:00
nonmoving: Disable shortcutting

- - - - -
fc0b7f9d by Ben Gamari at 2023-01-25T11:02:01-05:00
CheckGC

- - - - -

30 changed files:

- .gitlab/gen_ci.hs
- .gitlab/hello.hs
- .gitlab/jobs.yaml
- .gitlab/rel_eng/mk-ghcup-metadata/mk_ghcup_metadata.py
- CODEOWNERS
- INSTALL.md
- compiler/GHC/CmmToAsm/Wasm/Asm.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/Wasm/Types.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/Driver/Backend.hs
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/Hs/Syn/Type.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/HsToCore/Usage.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Parser.y
- compiler/GHC/Parser/Lexer.x
- compiler/GHC/Rename/Expr.hs
- compiler/GHC/Tc/Types/Origin.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/21b425ddf11eb9f1155e8961804734359d99b207...fc0b7f9de4336c28c54429de10fab2a4d5495eea

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/21b425ddf11eb9f1155e8961804734359d99b207...fc0b7f9de4336c28c54429de10fab2a4d5495eea
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230125/f7f3a216/attachment-0001.html>