[Git][ghc/ghc][wip/tsan/fixes-2] 75 commits: Avoid desugaring non-recursive lets into recursive lets

Ben Gamari (@bgamari) gitlab at gitlab.haskell.org
Mon Jun 26 13:13:46 UTC 2023 


Ben Gamari pushed to branch wip/tsan/fixes-2 at Glasgow Haskell Compiler / GHC


Commits:
3e80c2b4 by Arnaud Spiwack at 2023-06-20T03:19:41-04:00
Avoid desugaring non-recursive lets into recursive lets

This prepares for having linear let expressions in the frontend.

When desugaring lets, SPECIALISE statements create more copies of a
let binding. Because of the rewrite rules attached to the bindings,
there are dependencies between the generated binds.

Before this commit, we simply wrapped all these in a mutually
recursive let block, and left it to the simplified to sort it out.

With this commit: we are careful to generate the bindings in
dependency order, so that we can wrap them in consecutive lets (if the
source is non-recursive).

- - - - -
9fad49e0 by Ben Gamari at 2023-06-20T03:20:19-04:00
rts: Do not call exit() from SIGINT handler

Previously `shutdown_handler` would call `stg_exit` if the scheduler was
Oalready found to be in `SCHED_INTERRUPTING` state (or higher). However,
`stg_exit` is not signal-safe as it calls `exit` (which calls `atexit`
handlers). The only safe thing to do in this situation is to call
`_exit`, which terminates with minimal cleanup.

Fixes #23417.

- - - - -
7485f848 by Bodigrim at 2023-06-20T03:20:57-04:00
Bump Cabal submodule

This requires changing the recomp007 test because now cabal passes
`this-unit-id` to executable components, and that unit-id contains a
hash which includes the ABI of the dependencies. Therefore changing the
dependencies means that -this-unit-id changes and recompilation is
triggered.

The spririt of the test is to test GHC's recompilation logic assuming
that `-this-unit-id` is constant, so we explicitly pass `-ipid` to
`./configure` rather than letting `Cabal` work it out.

- - - - -
1464a2a8 by mangoiv at 2023-06-20T03:21:34-04:00
[feat] add a hint to `HasField` error message
- add a hint that indicates that the record that the record dot is used
  on might just be missing a field
- as the intention of the programmer is not entirely clear, it is only
  shown if the type is known
- This addresses in part issue #22382

- - - - -
b65e78dd by Ben Gamari at 2023-06-20T16:56:43-04:00
rts/ipe: Fix unused lock warning

- - - - -
6086effd by Ben Gamari at 2023-06-20T16:56:44-04:00
rts/ProfilerReportJson: Fix memory leak

- - - - -
1e48c434 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Various warnings fixes

- - - - -
471486b9 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Fix printf format mismatch

- - - - -
80603fb3 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Fix incorrect #include <sys/poll.h>

According to Alpine's warnings and poll(2), <poll.h> should be
preferred.

- - - - -
ff18e6fd by Ben Gamari at 2023-06-20T16:56:44-04:00
nonmoving: Fix unused definition warrnings

- - - - -
6e7fe8ee by Ben Gamari at 2023-06-20T16:56:44-04:00
Disable futimens on Darwin.

See #22938

- - - - -
b7706508 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Fix incorrect CPP guard

- - - - -
94f00e9b by Ben Gamari at 2023-06-20T16:56:44-04:00
hadrian: Ensure that -Werror is passed when compiling the RTS.

Previously the `+werror` transformer would only pass `-Werror` to GHC,
which does not ensure that the same is passed to the C compiler when
building the RTS. Arguably this is itself a bug but for now we will just
work around this by passing `-optc-Werror` to GHC.

I tried to enable `-Werror` in all C compilations but the boot libraries
are something of a portability nightmare.

- - - - -
5fb54bf8 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Disable `#pragma GCC`s on clang compilers

Otherwise the build fails due to warnings. See #23530.

- - - - -
cf87f380 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Fix capitalization of prototype

- - - - -
17f250d7 by Ben Gamari at 2023-06-20T16:56:44-04:00
rts: Fix incorrect format specifier

- - - - -
0ff1c501 by Josh Meredith at 2023-06-20T16:57:20-04:00
JS: remove js_broken(22576) in favour of the pre-existing wordsize(32) condition (#22576)

- - - - -
3d1d42b7 by Finley McIlwaine at 2023-06-21T12:04:58-04:00
Memory usage fixes for Haddock

- Do not include `mi_globals` in the `NoBackend` backend. It was only included
  for Haddock, but Haddock does not actually need it. This causes a 200MB
  reduction in max residency when generating haddocks on the Agda codebase
  (roughly 1GB to 800MB).

- Make haddock_{parser,renamer}_perf tests more accurate by forcing docs to
  be written to interface files using `-fwrite-interface`

Bumps haddock submodule.

Metric Decrease:
    haddock.base

- - - - -
8185b1c2 by Finley McIlwaine at 2023-06-21T12:04:58-04:00
Fix associated data family doc structure items

Associated data families were being given their own export DocStructureItems,
which resulted in them being documented separately from their classes in
haddocks. This commit fixes it.

- - - - -
4d356ea3 by Sylvain Henry at 2023-06-21T12:04:59-04:00
JS: implement TH support

- Add ghc-interp.js bootstrap script for the JS interpreter
- Interactively link and execute iserv code from the ghci package
- Incrementally load and run JS code for splices into the running iserv

Co-authored-by: Luite Stegeman <stegeman at gmail.com>

- - - - -
3249cf12 by Sylvain Henry at 2023-06-21T12:04:59-04:00
Don't use getKey

- - - - -
f84ff161 by Sylvain Henry at 2023-06-21T12:04:59-04:00
Stg: return imported FVs

This is used to determine what to link when using the interpreter.
For now it's only used by the JS interpreter but it could easily be used
by the native interpreter too (instead of extracting names from compiled BCOs).

- - - - -
fab2ad23 by Sylvain Henry at 2023-06-21T12:04:59-04:00
Fix some recompilation avoidance tests

- - - - -
a897dc13 by Sylvain Henry at 2023-06-21T12:04:59-04:00
TH_import_loop is now broken as expected

- - - - -
dbb4ad51 by Sylvain Henry at 2023-06-21T12:04:59-04:00
JS: always recompile when TH is enabled (cf #23013)

- - - - -
711b1d24 by Bartłomiej Cieślar at 2023-06-21T12:59:27-04:00
Add support for deprecating exported items (proposal #134)

This is an implementation of the deprecated exports proposal #134.
The proposal introduces an ability to introduce warnings to exports.
This allows for deprecating a name only when it is exported from a specific
module, rather than always depreacting its usage. In this example:

    module A ({-# DEPRECATED "do not use" #-} x) where
    x = undefined
    ---
    module B where
    import A(x)

`x` will emit a warning when it is explicitly imported.

Like the declaration warnings, export warnings are first accumulated within
the `Warnings` struct, then passed into the ModIface, from which they are
then looked up and warned about in the importing module in the `lookup_ie`
helpers of the `filterImports` function (for the explicitly imported names)
and in the `addUsedGRE(s)` functions where they warn about regular usages
of the imported name.

In terms of the AST information, the custom warning is stored in the
extension field of the variants of the `IE` type (see Trees that Grow for
more information).

The commit includes a bump to the haddock submodule added in MR #28

Signed-off-by: Bartłomiej Cieślar <bcieslar2001 at gmail.com>

- - - - -
c1865854 by Ben Gamari at 2023-06-21T12:59:30-04:00
configure: Bump version to 9.8

Bumps Haddock submodule

- - - - -
4e1de71c by Ben Gamari at 2023-06-21T21:07:48-04:00
configure: Bump version to 9.9

Bumps haddock submodule.

- - - - -
5b6612bc by Ben Gamari at 2023-06-23T03:56:49-04:00
rts: Work around missing prototypes errors

Darwin's toolchain inexpliciably claims that `write_barrier` and friends
have declarations without prototypes, despite the fact that (a) they are
definitions, and (b) the prototypes appear only a few lines above. Work
around this by making the definitions proper prototypes.

- - - - -
43b66a13 by Matthew Pickering at 2023-06-23T03:57:26-04:00
ghcup-metadata: Fix date modifier (M = minutes, m = month)

Fixes #23552

- - - - -
17a2feb2 by Ben Gamari at 2023-06-26T09:11:01-04:00
compiler: Introduce MO_{ACQUIRE,RELEASE}_FENCE

- - - - -
6be3da22 by Ben Gamari at 2023-06-26T09:11:01-04:00
compiler: Drop MO_WriteBarrier

rts: Drop write_barrier

- - - - -
76c19a69 by Ben Gamari at 2023-06-26T09:11:44-04:00
rts: Drop load_store_barrier()

This is no longer used.

- - - - -
4aafa3b4 by Ben Gamari at 2023-06-26T09:11:45-04:00
rts: Drop last instances of prim_{write,read}_barrier

- - - - -
2f9ba321 by Ben Gamari at 2023-06-26T09:11:45-04:00
rts: Eliminate remaining uses of load_load_barrier

- - - - -
7e2b9860 by Sven Tennie at 2023-06-26T09:11:45-04:00
compiler: Drop MO_ReadBarrier

- - - - -
cae88d65 by Ben Gamari at 2023-06-26T09:12:43-04:00
rts: Drop load_load_barrier

This is no longer used.

- - - - -
c798dde6 by Sven Tennie at 2023-06-26T09:13:10-04:00
Delete write_barrier function

- - - - -
cb514e59 by Ben Gamari at 2023-06-26T09:13:11-04:00
compiler: Style fixes

- - - - -
8b6915e5 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix implicit cast

This ensures that Task.h can be built with a C++ compiler.

- - - - -
b38d96f1 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix data race in threadPaused

This only affects an assertion in the debug RTS, but it's a data race
nevertheless.

- - - - -
66716432 by Ben Gamari at 2023-06-26T09:13:39-04:00
cmm: Introduce MO_RelaxedRead

In hand-written Cmm it can sometimes be necessary to atomically load
from memory deep within an expression (e.g. see the `CHECK_GC` macro).
This MachOp provides a convenient way to do so without breaking the
expression into multiple statements.

- - - - -
6328d4cf by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Silence spurious data races in ticky counters

Previously we would use non-atomic accesses when bumping ticky counters,
which would result in spurious data race reports from ThreadSanitizer
when the threaded RTS was in use.

- - - - -
4d4f8c39 by Ben Gamari at 2023-06-26T09:13:39-04:00
Improve TSAN documentation

- - - - -
ad231ac6 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix data race in Interpreter's preemption check

- - - - -
a2628de0 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix data race in threadStatus#

- - - - -
b1fd8183 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix data race in CHECK_GC

- - - - -
13f9cef1 by Ben Gamari at 2023-06-26T09:13:39-04:00
base: use atomic write when updating timer manager

- - - - -
28043a78 by Ben Gamari at 2023-06-26T09:13:39-04:00
Use relaxed atomics to manipulate TSO status fields

- - - - -
fcffa3b2 by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Add necessary barriers when manipulating TSO owner

- - - - -
f7c7a28f by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Fix synchronization on thread blocking state

- - - - -
0089491a by Ben Gamari at 2023-06-26T09:13:39-04:00
rts: Relaxed load MutVar info table

- - - - -
3d5b0991 by Ben Gamari at 2023-06-26T09:13:39-04:00
hadrian: More debug information

- - - - -
57027e21 by Ben Gamari at 2023-06-26T09:13:39-04:00
hadrian: More selective TSAN instrumentation

- - - - -
586b29a8 by Ben Gamari at 2023-06-26T09:13:39-04:00
codeGen/tsan: Rework handling of spilling

- - - - -
c821034e by Ben Gamari at 2023-06-26T09:13:39-04:00
codeGen: Ensure that TSAN is aware of writeArray# write barriers

- - - - -
72841699 by Ben Gamari at 2023-06-26T09:13:39-04:00
codeGen: Ensure that array reads have necessary barriers

This was the cause of #23541.

- - - - -
4cb38828 by Ben Gamari at 2023-06-26T09:13:39-04:00
Wordsmith TSAN Note

- - - - -
16d60720 by Ben Gamari at 2023-06-26T09:13:39-04:00
codeGen: Use relaxed accesses in ticky bumping

- - - - -
d3374bdd by Ben Gamari at 2023-06-26T09:13:39-04:00
codeGen: Use relaxed-read in closureInfoPtr

- - - - -
98e332b5 by Ben Gamari at 2023-06-26T09:13:39-04:00
Fix thunk update ordering

Previously we attempted to ensure soundness of concurrent thunk update
by synchronizing on the access of the thunk's info table pointer field.
This was believed to be sufficient since the indirectee (which may
expose a closure allocated by another core) would not be examined
until the info table pointer update is complete.

However, it turns out that this can result in data races in the presence
of multiple threads racing a update a single thunk. For instance,
consider this interleaving under the old scheme:

            Thread A                             Thread B
            ---------                            ---------
    t=0     Enter t
      1     Push update frame
      2     Begin evaluation

      4     Pause thread
      5     t.indirectee=tso
      6     Release t.info=BLACKHOLE

      7     ... (e.g. GC)

      8     Resume thread
      9     Finish evaluation
      10    Relaxed t.indirectee=x

      11                                         Load t.info
      12                                         Acquire fence
      13                                         Inspect t.indirectee

      14    Release t.info=BLACKHOLE

Here Thread A enters thunk `t` but is soon paused, resulting in `t`
being lazily blackholed at t=6. Then, at t=10 Thread A finishes
evaluation and updates `t.indirectee` with a relaxed store.

Meanwhile, Thread B enters the blackhole. Under the old scheme this
would introduce an acquire-fence but this would only synchronize with
Thread A at t=6. Consequently, the result of the evaluation, `x`, is not
visible to Thread B, introducing a data race.

We fix this by treating the `indirectee` field as we do all other
mutable fields. This means we must always access this field with
acquire-loads and release-stores.

See #23185.

- - - - -
4f2d04d8 by Ben Gamari at 2023-06-26T09:13:39-04:00
STM: Use acquire loads when possible

Full sequential consistency is not needed here.

- - - - -
b55a494a by Ubuntu at 2023-06-26T09:13:39-04:00
ghc-prim: Use C11 atomics

- - - - -
80f9d775 by Ubuntu at 2023-06-26T09:13:39-04:00
Run script

- - - - -
86a2119e by Ben Gamari at 2023-06-26T09:13:39-04:00
hadrian: Ensure that way-flags are passed to CC

Previously the way-specific compilation flags (e.g. `-DDEBUG`,
`-DTHREADED_RTS`) would not be passed to the CC invocations. This meant
that C dependency files would not correctly reflect
dependencies predicated on the way, resulting in the rather
painful #23554.

Closes #23554.

- - - - -
568b514d by Ben Gamari at 2023-06-26T09:13:40-04:00
rts/Interpreter: Fix data race

- - - - -
efc9df57 by Ben Gamari at 2023-06-26T09:13:40-04:00
rts/Messages: Fix data race

- - - - -
529fef1b by Ben Gamari at 2023-06-26T09:13:40-04:00
rts/Prof: Fix data race

- - - - -
b1d8443d by Ben Gamari at 2023-06-26T09:13:40-04:00
rts: Fix various data races

- - - - -
b06cb3bf by Ben Gamari at 2023-06-26T09:13:40-04:00
rts: Use fence rather than redundant load

- - - - -
acc89bfa by Ben Gamari at 2023-06-26T09:13:40-04:00
codeGen: More precise barriers for eager blackholing

- - - - -
2e36d1c5 by Ben Gamari at 2023-06-26T09:13:40-04:00
testsuite: Add AtomicModifyIORef test

- - - - -
7739ae27 by Ben Gamari at 2023-06-26T09:13:40-04:00
MutVar

- - - - -
4fdd316f by Ben Gamari at 2023-06-26T09:13:40-04:00
Tighten up thunk update barriers

- - - - -
73b1da45 by Ben Gamari at 2023-06-26T09:13:40-04:00
testsuite: Fix warning in hs_try_putmvar001

- - - - -


30 changed files:

- .gitlab-ci.yml
- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC.hs
- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Cmm/Expr.hs
- compiler/GHC/Cmm/Info.hs
- compiler/GHC/Cmm/MachOp.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/ThreadSanitizer.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Instr.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Driver/Backend.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Doc.hs
- compiler/GHC/Hs/ImpExp.hs
- compiler/GHC/HsToCore/Binds.hs
- compiler/GHC/HsToCore/Docs.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/Iface/Make.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6093adb74d77096b8053910cd81d1f2116bd47c3...73b1da45be8171ac78c6c14b1e7edfe08896b0c1

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6093adb74d77096b8053910cd81d1f2116bd47c3...73b1da45be8171ac78c6c14b1e7edfe08896b0c1
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230626/4e1a4da2/attachment-0001.html>

More information about the ghc-commits mailing list