[Git][ghc/ghc][wip/tsan/fixes] 71 commits: Convert interface file loading errors into proper diagnostics
Ben Gamari (@bgamari)
gitlab at gitlab.haskell.org
Mon Apr 24 14:44:12 UTC 2023
Ben Gamari pushed to branch wip/tsan/fixes at Glasgow Haskell Compiler / GHC
Commits:
5e1d33d7 by Matthew Pickering at 2023-04-18T10:31:02-04:00
Convert interface file loading errors into proper diagnostics
This patch converts all the errors to do with loading interface files
into proper structured diagnostics.
* DriverMessage: Sometimes in the driver we attempt to load an interface
file so we embed the IfaceMessage into the DriverMessage.
* TcRnMessage: Most the time we are loading interface files during
typechecking, so we embed the IfaceMessage
This patch also removes the TcRnInterfaceLookupError constructor which
is superceded by the IfaceMessage, which is now structured compared to
just storing an SDoc before.
- - - - -
df1a5811 by sheaf at 2023-04-18T10:31:43-04:00
Don't panic in ltPatersonSize
The function GHC.Tc.Utils.TcType.ltPatersonSize would panic when it
encountered a type family on the RHS, as usually these are not allowed
(type families are not allowed on the RHS of class instances or of
quantified constraints). However, it is possible to still encounter
type families on the RHS after doing a bit of constraint solving, as
seen in test case T23171. This could trigger the panic in the call to
ltPatersonSize in GHC.Tc.Solver.Canonical.mk_strict_superclasses, which
is involved in avoiding loopy superclass constraints.
This patch simply changes ltPatersonSize to return "I don't know, because
there's a type family involved" in these cases.
Fixes #23171
- - - - -
d442ac05 by Sylvain Henry at 2023-04-19T20:04:35-04:00
JS: fix thread-related primops
- - - - -
7a96f90b by Bryan Richter at 2023-04-19T20:05:11-04:00
CI: Disable abi-test-nightly
See #23269
- - - - -
ab6c1d29 by Sylvain Henry at 2023-04-19T20:05:50-04:00
Testsuite: don't use obsolescent egrep (#22351)
Recent egrep displays the following message, breaking golden tests:
egrep: warning: egrep is obsolescent; using grep -E
Switch to using "grep -E" instead
- - - - -
f15b0ce5 by Matthew Pickering at 2023-04-20T11:01:06-04:00
hadrian: Pass haddock file arguments in a response file
In !10119 CI was failing on windows because the command line was too
long. We can mitigate this by passing the file arguments to haddock in a
response file.
We can't easily pass all the arguments in a response file because the
`+RTS` arguments can't be placed in the response file.
Fixes #23273
- - - - -
7012ec2f by tocic at 2023-04-20T11:01:42-04:00
Fix doc typo in GHC.Read.readList
- - - - -
5c873124 by sheaf at 2023-04-20T18:33:34-04:00
Implement -jsem: parallelism controlled by semaphores
See https://github.com/ghc-proposals/ghc-proposals/pull/540/ for a
complete description for the motivation for this feature.
The `-jsem` option allows a build tool to pass a semaphore to GHC which
GHC can use in order to control how much parallelism it requests.
GHC itself acts as a client in the GHC jobserver protocol.
```
GHC Jobserver Protocol
~~~~~~~~~~~~~~~~~~~~~~
This proposal introduces the GHC Jobserver Protocol. This protocol allows
a server to dynamically invoke many instances of a client process,
while restricting all of those instances to use no more than <n> capabilities.
This is achieved by coordination over a system semaphore (either a POSIX
semaphore [6]_ in the case of Linux and Darwin, or a Win32 semaphore [7]_
in the case of Windows platforms).
There are two kinds of participants in the GHC Jobserver protocol:
- The *jobserver* creates a system semaphore with a certain number of
available tokens.
Each time the jobserver wants to spawn a new jobclient subprocess, it **must**
first acquire a single token from the semaphore, before spawning
the subprocess. This token **must** be released once the subprocess terminates.
Once work is finished, the jobserver **must** destroy the semaphore it created.
- A *jobclient* is a subprocess spawned by the jobserver or another jobclient.
Each jobclient starts with one available token (its *implicit token*,
which was acquired by the parent which spawned it), and can request more
tokens through the Jobserver Protocol by waiting on the semaphore.
Each time a jobclient wants to spawn a new jobclient subprocess, it **must**
pass on a single token to the child jobclient. This token can either be the
jobclient's implicit token, or another token which the jobclient acquired
from the semaphore.
Each jobclient **must** release exactly as many tokens as it has acquired from
the semaphore (this does not include the implicit tokens).
```
Build tools such as cabal act as jobservers in the protocol and are
responsibile for correctly creating, cleaning up and managing the
semaphore.
Adds a new submodule (semaphore-compat) for managing and interacting
with semaphores in a cross-platform way.
Fixes #19349
- - - - -
52d3e9b4 by Ben Gamari at 2023-04-20T18:34:11-04:00
rts: Initialize Array# header in listThreads#
Previously the implementation of listThreads# failed to initialize the
header of the created array, leading to various nastiness.
Fixes #23071
- - - - -
1db30fe1 by Ben Gamari at 2023-04-20T18:34:11-04:00
testsuite: Add test for #23071
- - - - -
dae514f9 by tocic at 2023-04-21T13:31:21-04:00
Fix doc typos in libraries/base/GHC
- - - - -
113e21d7 by Sylvain Henry at 2023-04-21T13:32:01-04:00
Testsuite: replace some js_broken/js_skip predicates with req_c
Using req_c is more precise.
- - - - -
038bb031 by Krzysztof Gogolewski at 2023-04-21T18:03:04-04:00
Minor doc fixes
- Add docs/index.html to .gitignore.
It is created by ./hadrian/build docs, and it was the only file
in Hadrian's templateRules not present in .gitignore.
- Mention that MultiWayIf supports non-boolean guards
- Remove documentation of optdll - removed in 2007, 763daed95
- Fix markdown syntax
- - - - -
e826cdb2 by amesgen at 2023-04-21T18:03:44-04:00
User's guide: DeepSubsumption is implied by Haskell{98,2010}
- - - - -
499a1c20 by PHO at 2023-04-23T13:39:32-04:00
Implement executablePath for Solaris and make getBaseDir less platform-dependent
Use base-4.17 executablePath when possible, and fall back on
getExecutablePath when it's not available. The sole reason why getBaseDir
had #ifdef's was apparently that getExecutablePath wasn't reliable, and we
could reduce the number of CPP conditionals by making use of
executablePath instead.
Also export executablePath on js_HOST_ARCH.
- - - - -
97a6f7bc by tocic at 2023-04-23T13:40:08-04:00
Fix doc typos in libraries/base
- - - - -
fa2c82e4 by Ben Gamari at 2023-04-24T06:00:39-04:00
compiler: Style fixes
- - - - -
add3ee5d by Ben Gamari at 2023-04-24T06:00:39-04:00
rts: Fix data race in threadPaused
This only affects an assertion in the debug RTS, but it's a data race
nevertheless.
- - - - -
c9642199 by Ben Gamari at 2023-04-24T06:00:39-04:00
rts: Silence spurious data races in ticky counters
Previously we would use non-atomic accesses when bumping ticky counters,
which would result in spurious data race reports from ThreadSanitizer
when the threaded RTS was in use.
- - - - -
d29a3679 by Ben Gamari at 2023-04-24T06:00:40-04:00
Improve TSAN documentation
- - - - -
5165a358 by Ben Gamari at 2023-04-24T06:00:40-04:00
compiler/cmm: Ensure that dump output has proc name
Previously dump output from the early Cmm passes would not be labelled
with a proc label.
- - - - -
47d0ad2e by Ben Gamari at 2023-04-24T06:00:40-04:00
cmm: Introduce MO_RelaxedRead
In hand-written Cmm it can sometimes be necessary to atomically load
from memory deep within an expression (e.g. see the `CHECK_GC` macro).
This MachOp provides a convenient way to do so without breaking the
expression into multiple statements.
- - - - -
4eb1aeb0 by Ben Gamari at 2023-04-24T06:00:40-04:00
rts: Fix various data races
- - - - -
e1986113 by Ben Gamari at 2023-04-24T06:00:40-04:00
base: use atomic write when updating timer manager
- - - - -
e5a4c2e4 by Ben Gamari at 2023-04-24T06:00:40-04:00
Use relaxed atomics to manipulate TSO status fields
- - - - -
d0f300cf by Ben Gamari at 2023-04-24T06:00:40-04:00
rts: Drop unnecessary atomic
- - - - -
5946bdf1 by Ben Gamari at 2023-04-24T06:03:12-04:00
codeGen: Ensure that TSAN is aware of writeArray# write barriers
- - - - -
3c1e4a69 by Ben Gamari at 2023-04-24T06:03:49-04:00
rts: Fix synchronization on thread blocking state
- - - - -
c3b59aec by Ben Gamari at 2023-04-24T06:03:49-04:00
rts: Relaxed load MutVar info table
- - - - -
48512cb5 by Ben Gamari at 2023-04-24T06:03:49-04:00
More principled treatment of acquire fences
- - - - -
7b06478a by Ben Gamari at 2023-04-24T06:03:49-04:00
IND
- - - - -
a9abd9e6 by Ben Gamari at 2023-04-24T06:03:49-04:00
Wordsmith Note
- - - - -
6bfb2ac0 by Ben Gamari at 2023-04-24T06:03:49-04:00
Use relaxed accesses in ticky bumping
- - - - -
cd8e6a32 by Ben Gamari at 2023-04-24T06:03:49-04:00
Fix thunk update ordering
Previously we attempted to ensure soundness of concurrent thunk update
by synchronizing on the access of the thunk's info table pointer field.
This was believed to be sufficient since the indirectee (which may
expose a closure allocated by another core) would not be examined
until the info table pointer update is complete.
However, it turns out that this can result in data races in the presence
of multiple threads racing a update a single thunk. For instance,
consider this interleaving under the old scheme:
Thread A Thread B
--------- ---------
t=0 Enter t
1 Push update frame
2 Begin evaluation
4 Pause thread
5 t.indirectee=tso
6 Release t.info=BLACKHOLE
7 ... (e.g. GC)
8 Resume thread
9 Finish evaluation
10 Relaxed t.indirectee=x
11 Load t.info
12 Acquire fence
13 Inspect t.indirectee
14 Release t.info=BLACKHOLE
Here Thread A enters thunk `t` but is soon paused, resulting in `t`
being lazily blackholed at t=6. Then, at t=10 Thread A finishes
evaluation and updates `t.indirectee` with a relaxed store.
Meanwhile, Thread B enters the blackhole. Under the old scheme this
would introduce an acquire-fence but this would only synchronize with
Thread A at t=6. Consequently, the result of the evaluation, `x`, is not
visible to Thread B, introducing a data race.
We fix this by treating the `indirectee` field as we do all other
mutable fields. This means we must always access this field with
acquire-loads and release-stores.
See #23185.
- - - - -
c4a93bd7 by Ben Gamari at 2023-04-24T06:03:49-04:00
Things
- - - - -
e968aa19 by Ben Gamari at 2023-04-24T06:03:49-04:00
rts/IPE: Fix unused mutex warning
- - - - -
edb1a8f1 by Ben Gamari at 2023-04-24T06:03:49-04:00
rts: Relax info pointer stores
- - - - -
da2ef6fc by Ben Gamari at 2023-04-24T06:03:49-04:00
TSAN: Rework handling of spilling
- - - - -
aada555b by Ben Gamari at 2023-04-24T06:03:49-04:00
hadrian: More debug information
- - - - -
3f8b3216 by Ben Gamari at 2023-04-24T06:03:49-04:00
hadrian: More selective TSAN instrumentation
- - - - -
5090582c by Ben Gamari at 2023-04-24T06:03:49-04:00
rts: Drop benign race
- - - - -
53451434 by Ubuntu at 2023-04-24T06:03:49-04:00
rts: C++ fixes
- - - - -
4fa414c0 by Ben Gamari at 2023-04-24T06:04:16-04:00
codeGen: Ensure that TSAN is aware of writeArray# write barriers
- - - - -
82fcb0d3 by Ben Gamari at 2023-04-24T06:04:18-04:00
More principled treatment of acquire fences
- - - - -
69fcb273 by Ubuntu at 2023-04-24T06:04:18-04:00
Work around #22451
- - - - -
ffce10ef by Ubuntu at 2023-04-24T06:04:18-04:00
llvm: Sequential consistency
- - - - -
6f92a601 by Ubuntu at 2023-04-24T06:04:18-04:00
rts: sequential consistency
- - - - -
96adc99c by Ubuntu at 2023-04-24T06:04:18-04:00
ghc-prim: Use C11 atomics
- - - - -
59793d31 by Ubuntu at 2023-04-24T06:04:18-04:00
Fix thunk update further
- - - - -
b20f33cf by Ubuntu at 2023-04-24T06:04:18-04:00
Add acquire fence in WHITEHOLE
- - - - -
76973b5c by Ubuntu at 2023-04-24T06:04:18-04:00
whitespace
- - - - -
b381a3b1 by Ubuntu at 2023-04-24T06:04:18-04:00
Note
- - - - -
1a98c968 by Ben Gamari at 2023-04-24T06:04:18-04:00
STM: relaxed
- - - - -
1079e23f by Ubuntu at 2023-04-24T06:04:18-04:00
Run script
- - - - -
4bced947 by Ubuntu at 2023-04-24T06:04:18-04:00
Refine run script
- - - - -
022d958b by Ubuntu at 2023-04-24T06:04:18-04:00
run
- - - - -
a419561b by Ubuntu at 2023-04-24T06:04:18-04:00
Disable selector optimisation
- - - - -
213fe43e by Ben Gamari at 2023-04-24T06:04:18-04:00
CheckGC
- - - - -
8d07f07c by Ubuntu at 2023-04-24T06:04:18-04:00
check-gc: fix large objects
- - - - -
c1102ada by Ubuntu at 2023-04-24T06:04:18-04:00
check-gc on GC
- - - - -
e0ec4666 by Ben Gamari at 2023-04-24T06:04:18-04:00
ACQUIRE_FENCE
- - - - -
d215f65d by Ben Gamari at 2023-04-24T06:04:18-04:00
Add Note
- - - - -
eb524511 by Ben Gamari at 2023-04-24T06:04:18-04:00
STM: Acquire instead of seq-cst
- - - - -
dda36120 by Ben Gamari at 2023-04-24T06:04:18-04:00
Comment
- - - - -
bd27d186 by Ubuntu at 2023-04-24T06:04:18-04:00
Fix acquire fences on indirections
- - - - -
7fb7f719 by Ubuntu at 2023-04-24T06:04:18-04:00
STM
- - - - -
b702af6e by Ubuntu at 2023-04-24T06:04:18-04:00
comment fixes
- - - - -
756d7141 by Ben Gamari at 2023-04-24T08:45:48-04:00
hadrian: Fix
- - - - -
166b7062 by Ben Gamari at 2023-04-24T10:33:39-04:00
rts: BLACKHOLE fix
- - - - -
8094c785 by Ben Gamari at 2023-04-24T10:34:07-04:00
rts: TSO owner
- - - - -
db98923f by Ben Gamari at 2023-04-24T10:34:31-04:00
updateThunk
- - - - -
30 changed files:
- .gitignore
- .gitlab-ci.yml
- .gitmodules
- cabal.project-reinstall
- compiler/GHC/Cmm/Expr.hs
- compiler/GHC/Cmm/Info.hs
- compiler/GHC/Cmm/MachOp.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/Pipeline.hs
- compiler/GHC/Cmm/ThreadSanitizer.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Driver/Config/Diagnostic.hs
- compiler/GHC/Driver/Config/Tidy.hs
- compiler/GHC/Driver/Errors.hs
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/MakeFile.hs
- + compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Pipeline/LogQueue.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Iface/Errors.hs
- + compiler/GHC/Iface/Errors/Ppr.hs
- + compiler/GHC/Iface/Errors/Types.hs
- compiler/GHC/Iface/Load.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/3caadbd50211ef5fddbcefcba17b4c5baaa1793e...db98923f70606b156f3cf82147ff6ca4172aed14
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/3caadbd50211ef5fddbcefcba17b4c5baaa1793e...db98923f70606b156f3cf82147ff6ca4172aed14
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230424/4a130651/attachment-0001.html>
More information about the ghc-commits
mailing list