[Git][ghc/ghc][wip/tsan/fix-races] 45 commits: ghc-toolchain: Initial commit
Ben Gamari (@bgamari)
gitlab at gitlab.haskell.org
Wed Jul 26 23:29:16 UTC 2023
Ben Gamari pushed to branch wip/tsan/fix-races at Glasgow Haskell Compiler / GHC
Commits:
83cbc672 by Ben Gamari at 2023-07-24T07:40:49+00:00
ghc-toolchain: Initial commit
- - - - -
31dcd26c by Rodrigo Mesquita at 2023-07-24T07:40:49+00:00
ghc-toolchain: Toolchain Selection
This commit integrates ghc-toolchain, the brand new way of configuring
toolchains for GHC, with the Hadrian build system, with configure, and
extends and improves the first iteration of ghc-toolchain.
The general overview is
* We introduce a program invoked `ghc-toolchain --triple=...` which, when run,
produces a file with a `Target`. A `GHC.Toolchain.Target.Target`
describes the properties of a target and the toolchain (executables
and configured flags) to produce code for that target
* Hadrian was modified to read Target files, and will both
* Invoke the toolchain configured in the Target file as needed
* Produce a `settings` file for GHC based on the Target file for that stage
* `./configure` will invoke ghc-toolchain to generate target files, but
it will also generate target files based on the flags configure itself
configured (through `.in` files that are substituted)
* By default, the Targets generated by configure are still (for now) the ones used by Hadrian
* But we additionally validate the Target files generated by
ghc-toolchain against the ones generated by configure, to get a head
start on catching configuration bugs before we transition
completely.
* When we make that transition, we will want to drop a lot of the
toolchain configuration logic from configure, but keep it otherwise.
* For each compiler stage we should have 1 target file (up to a stage compiler we can't run in our machine)
* We just have a HOST target file, which we use as the target for stage0
* And a TARGET target file, which we use for stage1 (and later stages, if not cross compiling)
* Note there is no BUILD target file, because we only support cross compilation where BUILD=HOST
* (for more details on cross-compilation see discussion on !9263)
See also
* Note [How we configure the bundled windows toolchain]
* Note [ghc-toolchain consistency checking]
* Note [ghc-toolchain overview]
Ticket: #19877
MR: !9263
- - - - -
a732b6d3 by Rodrigo Mesquita at 2023-07-24T07:40:49+00:00
Add flag to enable/disable ghc-toolchain based configurations
This flag is disabled by default, and we'll use the
configure-generated-toolchains by default until we remove the toolchain
configuration logic from configure.
- - - - -
61eea240 by Rodrigo Mesquita at 2023-07-24T07:40:49+00:00
Split ghc-toolchain executable to new packge
In light of #23690, we split the ghc-toolchain executable out of the
library package to be able to ship it in the bindist using Hadrian.
Ideally, we eventually revert this commit.
- - - - -
38e795ff by Rodrigo Mesquita at 2023-07-24T07:40:49+00:00
Ship ghc-toolchain in the bindist
Add the ghc-toolchain binary to the binary distribution we ship to
users, and teach the bindist configure to use the existing ghc-toolchain.
- - - - -
32cae784 by Matthew Craven at 2023-07-24T16:48:24-04:00
Kill off gen_bytearray_addr_access_ops.py
The relevant primop descriptions are now
generated directly by genprimopcode.
This makes progress toward fixing #23490, but it is not
a complete fix since there is more than one way in which
cabal-reinstall (hadrian/build build-cabal) is broken.
- - - - -
02e6a6ce by Matthew Pickering at 2023-07-24T16:49:00-04:00
compiler: Remove unused `containers.h` include
Fixes #23712
- - - - -
822ef66b by Matthew Pickering at 2023-07-25T08:44:50-04:00
Fix pretty printing of WARNING pragmas
There is still something quite unsavoury going on with WARNING pragma
printing because the printing relies on the fact that for decl
deprecations the SourceText of WarningTxt is empty. However, I let that
lion sleep and just fixed things directly.
Fixes #23465
- - - - -
e7b38ede by Matthew Pickering at 2023-07-25T08:45:28-04:00
ci-images: Bump to commit which has 9.6 image
The test-bootstrap job has been failing for 9.6 because we accidentally
used a non-master commit.
- - - - -
bb408936 by Matthew Pickering at 2023-07-25T08:45:28-04:00
Update bootstrap plans for 9.6.2 and 9.4.5
- - - - -
5166e3cc by Ben Gamari at 2023-07-26T14:38:18-04:00
compiler: Style fixes
- - - - -
1175632b by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Fix implicit cast
This ensures that Task.h can be built with a C++ compiler.
- - - - -
471eb500 by Ben Gamari at 2023-07-26T14:38:18-04:00
testsuite: Fix warning in hs_try_putmvar001
- - - - -
ae5888d0 by Ben Gamari at 2023-07-26T14:38:18-04:00
testsuite: Add AtomicModifyIORef test
- - - - -
dae5a29e by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Introduce NO_WARN macro
This allows fine-grained ignoring of warnings.
- - - - -
cdc1a61d by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Simplify atomicModifyMutVar2# implementation
Previously we would perform a redundant load in the non-threaded RTS in
atomicModifyMutVar2# implementation for the benefit of the non-moving
GC's write barrier. Eliminate this.
- - - - -
2d8f2ac1 by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Introduce more principled fence operations
- - - - -
5b0aab1e by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Introduce SET_INFO_RELAXED
- - - - -
65bd7826 by Ben Gamari at 2023-07-26T14:38:18-04:00
rts: Style fixes
- - - - -
ecf789cb by Ben Gamari at 2023-07-26T14:38:43-04:00
codeGen/tsan: Rework handling of spilling
- - - - -
8280e720 by Ben Gamari at 2023-07-26T14:38:43-04:00
hadrian: More debug information
- - - - -
86d7e5dc by Ben Gamari at 2023-07-26T14:38:43-04:00
Improve TSAN documentation
- - - - -
dbc86a80 by Ben Gamari at 2023-07-26T14:38:43-04:00
hadrian: More selective TSAN instrumentation
- - - - -
3218c34c by Ben Gamari at 2023-07-26T15:36:27-04:00
rts: Fix data race in threadPaused
This only affects an assertion in the debug RTS and only needs relaxed
ordering.
- - - - -
dc923619 by Ben Gamari at 2023-07-26T18:54:19-04:00
cmm: Introduce MO_RelaxedRead
In hand-written Cmm it can sometimes be necessary to atomically load
from memory deep within an expression (e.g. see the `CHECK_GC` macro).
This MachOp provides a convenient way to do so without breaking the
expression into multiple statements.
- - - - -
fc0c301e by Ben Gamari at 2023-07-26T18:54:19-04:00
rts: Silence spurious data races in ticky counters
Previously we would use non-atomic accesses when bumping ticky counters,
which would result in spurious data race reports from ThreadSanitizer
when the threaded RTS was in use.
- - - - -
0f91f5b9 by Ben Gamari at 2023-07-26T18:54:19-04:00
codeGen: Use relaxed accesses in ticky bumping
- - - - -
40a460c8 by Ben Gamari at 2023-07-26T18:54:20-04:00
rts: Fix data race in Interpreter's preemption check
- - - - -
a2dbb2bc by Ben Gamari at 2023-07-26T18:54:22-04:00
rts: Fix data race in threadStatus#
- - - - -
e6e60bff by Ben Gamari at 2023-07-26T18:54:22-04:00
rts: Fix data race in CHECK_GC
- - - - -
5ffd69ec by Ben Gamari at 2023-07-26T18:54:22-04:00
base: use atomic write when updating timer manager
- - - - -
2b6e78f1 by Ben Gamari at 2023-07-26T18:54:22-04:00
Use relaxed atomics to manipulate TSO status fields
- - - - -
ea553e5b by Ben Gamari at 2023-07-26T18:54:22-04:00
rts: Add necessary barriers when manipulating TSO owner
- - - - -
c0b2dcca by Ben Gamari at 2023-07-26T19:28:56-04:00
rts: Fix synchronization on thread blocking state
- - - - -
7c21614f by Ben Gamari at 2023-07-26T19:28:56-04:00
rts: Use relaxed ordering on dirty/clean info tables updates
When changing the dirty/clean state of a mutable object we needn't have
any particular ordering.
- - - - -
2dd66b52 by Ben Gamari at 2023-07-26T19:28:56-04:00
Wordsmith TSAN Note
- - - - -
779544fe by Ben Gamari at 2023-07-26T19:28:56-04:00
codeGen: Use relaxed-read in closureInfoPtr
- - - - -
8e961f1d by Ben Gamari at 2023-07-26T19:28:56-04:00
Fix thunk update ordering
Previously we attempted to ensure soundness of concurrent thunk update
by synchronizing on the access of the thunk's info table pointer field.
This was believed to be sufficient since the indirectee (which may
expose a closure allocated by another core) would not be examined
until the info table pointer update is complete.
However, it turns out that this can result in data races in the presence
of multiple threads racing a update a single thunk. For instance,
consider this interleaving under the old scheme:
Thread A Thread B
--------- ---------
t=0 Enter t
1 Push update frame
2 Begin evaluation
4 Pause thread
5 t.indirectee=tso
6 Release t.info=BLACKHOLE
7 ... (e.g. GC)
8 Resume thread
9 Finish evaluation
10 Relaxed t.indirectee=x
11 Load t.info
12 Acquire fence
13 Inspect t.indirectee
14 Release t.info=BLACKHOLE
Here Thread A enters thunk `t` but is soon paused, resulting in `t`
being lazily blackholed at t=6. Then, at t=10 Thread A finishes
evaluation and updates `t.indirectee` with a relaxed store.
Meanwhile, Thread B enters the blackhole. Under the old scheme this
would introduce an acquire-fence but this would only synchronize with
Thread A at t=6. Consequently, the result of the evaluation, `x`, is not
visible to Thread B, introducing a data race.
We fix this by treating the `indirectee` field as we do all other
mutable fields. This means we must always access this field with
acquire-loads and release-stores.
See #23185.
- - - - -
4e67b2ec by Ben Gamari at 2023-07-26T19:28:56-04:00
STM: Use acquire loads when possible
Full sequential consistency is not needed here.
- - - - -
8603f75e by Ben Gamari at 2023-07-26T19:28:56-04:00
rts/Interpreter: Fix data race
- - - - -
35450d87 by Ben Gamari at 2023-07-26T19:28:57-04:00
rts/Messages: Fix data race
- - - - -
15540e63 by Ben Gamari at 2023-07-26T19:28:57-04:00
rts/Prof: Fix data race
- - - - -
e1ae6715 by Ben Gamari at 2023-07-26T19:28:57-04:00
rts: Use fence rather than redundant load
Previously we would use an atomic load to ensure acquire ordering.
However, we now have `ACQUIRE_FENCE_ON`, which allows us to express this
more directly.
- - - - -
640453cb by Ben Gamari at 2023-07-26T19:28:57-04:00
rts: Fix data races in profiling timer
- - - - -
d001bd84 by Ben Gamari at 2023-07-26T19:28:57-04:00
rts/RaiseAsync: Drop redundant release fence
- - - - -
30 changed files:
- .gitlab-ci.yml
- − compiler/GHC/Builtin/gen_bytearray_addr_access_ops.py
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Cmm/Expr.hs
- compiler/GHC/Cmm/Info.hs
- compiler/GHC/Cmm/MachOp.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/ThreadSanitizer.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Data/Word64Map.hs
- compiler/GHC/Data/Word64Map/Internal.hs
- compiler/GHC/Data/Word64Map/Lazy.hs
- compiler/GHC/Data/Word64Map/Strict.hs
- compiler/GHC/Data/Word64Map/Strict/Internal.hs
- compiler/GHC/Data/Word64Set.hs
- compiler/GHC/Data/Word64Set/Internal.hs
- compiler/GHC/Hs/Decls.hs
- compiler/GHC/Linker/Static.hs
- compiler/GHC/Platform.hs
- compiler/GHC/StgToCmm/Bind.hs
- compiler/GHC/StgToCmm/Ticky.hs
- compiler/GHC/StgToCmm/Utils.hs
- compiler/GHC/Unit/Module/Warnings.hs
- compiler/GHC/Utils/Containers/Internal/BitUtil.hs
- compiler/GHC/Utils/Containers/Internal/StrictPair.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/690a4e4fac47008933e58836b6f3d28d772d7d68...d001bd8446b1e539016064db84c1adc7776d83e1
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/690a4e4fac47008933e58836b6f3d28d772d7d68...d001bd8446b1e539016064db84c1adc7776d83e1
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230726/0817ae63/attachment-0001.html>
More information about the ghc-commits
mailing list