[Git][ghc/ghc][wip/tsan/codegen] 26 commits: Introduce CapIOManager as the per-cap I/O mangager state

Wed Nov 23 13:01:17 UTC 2022

Ben Gamari pushed to branch wip/tsan/codegen at Glasgow Haskell Compiler / GHC

Commits:
8d6aaa49 by Duncan Coutts at 2022-11-22T02:06:16-05:00
Introduce CapIOManager as the per-cap I/O mangager state

Rather than each I/O manager adding things into the Capability structure
ad-hoc, we should have a common CapIOManager iomgr member of the
Capability structure, with a common interface to initialise etc.

The content of the CapIOManager struct will be defined differently for
each I/O manager implementation. Eventually we should be able to have
the CapIOManager be opaque to the rest of the RTS, and known just to the
I/O manager implementation. We plan for that by making the Capability
contain a pointer to the CapIOManager rather than containing the
structure directly.

Initially just move the Unix threaded I/O manager's control FD.

- - - - -
8901285e by Duncan Coutts at 2022-11-22T02:06:17-05:00
Add hook markCapabilityIOManager

To allow I/O managers to have GC roots in the Capability, within the
CapIOManager structure.

Not yet used in this patch.

- - - - -
5cf709c5 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Move APPEND_TO_BLOCKED_QUEUE from cmm to C

The I/O and delay blocking primitives for the non-threaded way
currently access the blocked_queue and sleeping_queue directly.

We want to move where those queues are to make their ownership clearer:
to have them clearly belong to the I/O manager impls rather than to the
scheduler. Ultimately we will want to change their representation too.

It's inconvenient to do that if these queues are accessed directly from
cmm code. So as a first step, replace the APPEND_TO_BLOCKED_QUEUE with a
C version appendToIOBlockedQueue(), and replace the open-coded
sleeping_queue insertion with insertIntoSleepingQueue().

- - - - -
ced9acdb by Duncan Coutts at 2022-11-22T02:06:17-05:00
Move {blocked,sleeping}_queue from scheduler global vars to CapIOManager

The blocked_queue_{hd,tl} and the sleeping_queue are currently
cooperatively managed between the scheduler and (some but not all of)
the non-threaded I/O manager implementations.

They lived as global vars with the scheduler, but are poked by I/O
primops and the I/O manager backends.

This patch is a step on the path towards making the management of I/O or
timer blocking belong to the I/O managers and not the scheduler.

Specifically, this patch moves the {blocked,sleeping}_queue from being
global vars in the scheduler to being members of the CapIOManager struct
within each Capability. They are not yet exclusively used by the I/O
managers: they are still poked from a couple other places, notably in
the scheduler before calling awaitEvent.

- - - - -
0f68919e by Duncan Coutts at 2022-11-22T02:06:17-05:00
Remove the now-unused markScheduler

The global vars {blocked,sleeping}_queue are now in the Capability and
so get marked there via markCapabilityIOManager.

- - - - -
39a91f60 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Move macros for checking for pending IO or timers

from Schedule.h to Schedule.c and IOManager.h

This is just moving, the next step will be to rejig them slightly.

For the non-threaded RTS the scheduler needs to be able to test for
there being pending I/O operation or pending timers. The implementation
of these tests should really be considered to be part of the I/O
managers and not part of the scheduler.

- - - - -
664b034b by Duncan Coutts at 2022-11-22T02:06:17-05:00
Replace EMPTY_{BLOCKED,SLEEPING}_QUEUE macros by function

These are the macros originaly from Scheduler.h, previously moved to
IOManager.h, and now replaced with a single inline function
anyPendingTimeoutsOrIO(). We can use a single function since the two
macros were always checked together.

Note that since anyPendingTimeoutsOrIO is defined for all IO manager
cases, including threaded, we do not need to guard its use by cpp
 #if !defined(THREADED_RTS)

- - - - -
32946220 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Expand emptyThreadQueues inline for clarity

It was not really adding anything. The name no longer meant anything
since those I/O and timeout queues do not belong to the scheuler.

In one of the two places it was used, the comments already had to
explain what it did, whereas now the code matches the comment nicely.

- - - - -
9943baf9 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Move the awaitEvent declaration into IOManager.h

And add or adjust comments at the use sites of awaitEvent.

- - - - -
054dcc9d by Duncan Coutts at 2022-11-22T02:06:17-05:00
Pass the Capability *cap explicitly to awaitEvent

It is currently only used in the non-threaded RTS so it works to use
MainCapability, but it's a bit nicer to pass the cap anyway. It's
certainly shorter.

- - - - -
667fe5a4 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Pass the Capability *cap explicitly to appendToIOBlockedQueue

And to insertIntoSleepingQueue. Again, it's a bit cleaner and simpler
though not strictly necessary given that these primops are currently
only used in the non-threaded RTS.

- - - - -
7181b074 by Duncan Coutts at 2022-11-22T02:06:17-05:00
Reveiew feedback: improve one of the TODO comments

The one about the nonsense (const False) test on WinIO for there being any IO
or timers pending, leading to unnecessary complication later in the
scheduler.

- - - - -
e5b68183 by Andreas Klebinger at 2022-11-22T02:06:52-05:00
Optimize getLevity.

Avoid the intermediate data structures allocated by splitTyConApp.
This avoids ~0.5% of allocations for a build using -O2.

Fixes #22254

- - - - -
de5fb348 by Andreas Klebinger at 2022-11-22T02:07:28-05:00
hadrian:Set TNTC when running testsuite.

- - - - -
9d166368 by Ben Gamari at 2022-11-23T07:57:38-05:00
hadrian: Don't enable TSAN in stage0 build

- - - - -
6010727d by Ben Gamari at 2022-11-23T07:57:38-05:00
cmm: Introduce blockConcat

- - - - -
cb0f6a50 by Ben Gamari at 2022-11-23T07:57:38-05:00
cmm: Introduce MemoryOrderings

- - - - -
bd421925 by Ben Gamari at 2022-11-23T07:57:38-05:00
llvm: Respect memory specified orderings

- - - - -
7f25acd2 by Ben Gamari at 2022-11-23T07:57:38-05:00
Codegen/x86: Eliminate barrier for relaxed accesses

- - - - -
fcc1b188 by Ben Gamari at 2022-11-23T07:57:38-05:00
cmm/Parser: Reduce some repetition

- - - - -
d6db535f by Ben Gamari at 2022-11-23T07:57:38-05:00
cmm/Parser: Add syntax for ordered loads and stores

- - - - -
797b2637 by Ben Gamari at 2022-11-23T07:57:38-05:00
cmm/Parser: Atomic load syntax

Originally I had thought I would just use the `prim` call syntax instead
of introducing new syntax for atomic loads. However, it turns out that
`prim` call syntax tends to make things quite unreadable. This new
syntax seems quite natural.

- - - - -
8f35951a by Ben Gamari at 2022-11-23T07:57:38-05:00
codeGen: Introduce ThreadSanitizer instrumentation

This introduces a new Cmm pass which instruments the program with
ThreadSanitizer annotations, allowing full tracking of mutator memory
accesses via TSAN.

- - - - -
a8b5318d by Ben Gamari at 2022-11-23T07:57:38-05:00
Hadrian: Drop TSAN_ENABLED define from flavour

This is redundant since the TSANUtils.h already defines it.

- - - - -
40b52e3c by Ben Gamari at 2022-11-23T07:57:38-05:00
hadrian: Enable Cmm instrumentation in TSAN flavour

- - - - -
fe44f275 by Ben Gamari at 2022-11-23T07:59:07-05:00
rts: Ensure that global regs are never passed as fun call args

This is in general unsafe as they may be clobbered if they are mapped to
caller-saved machine registers. See Note [Register parameter passing].

- - - - -

30 changed files:

- compiler/GHC/Cmm/Config.hs
- compiler/GHC/Cmm/ContFlowOpt.hs
- compiler/GHC/Cmm/Dataflow/Block.hs
- compiler/GHC/Cmm/Lexer.x
- compiler/GHC/Cmm/MachOp.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/Pipeline.hs
- + compiler/GHC/Cmm/ThreadSanitizer.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
- compiler/GHC/CmmToAsm/Wasm/FromCmm.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/Driver/Config/Cmm.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/StgToCmm/ExtCode.hs
- compiler/GHC/StgToCmm/Prim.hs
- compiler/ghc.cabal.in
- docs/users_guide/debugging.rst
- hadrian/src/Flavour.hs
- hadrian/src/Settings/Builders/RunTest.hs
- − rts/AwaitEvent.h
- rts/Capability.c
- rts/Capability.h
- rts/HeapStackCheck.cmm
- rts/IOManager.c
- rts/IOManager.h

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/18993c8c8c5c49976d83edfc7914f27e09ba5b10...fe44f27593eed7fb00115df0d30ac93481201208

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/18993c8c8c5c49976d83edfc7914f27e09ba5b10...fe44f27593eed7fb00115df0d30ac93481201208
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20221123/5bda2b8f/attachment-0001.html>