[Git][ghc/ghc][wip/ghc-9.4.5-backports] 68 commits: Bump submodules: text, parsec, bytestring, containers

Fri Apr 14 07:38:02 UTC 2023

Zubin pushed to branch wip/ghc-9.4.5-backports at Glasgow Haskell Compiler / GHC

Commits:
0bc34bd0 by Zubin Duggal at 2023-04-14T13:07:37+05:30
Bump submodules: text, parsec, bytestring, containers

- - - - -
40bf9669 by Zubin Duggal at 2023-04-14T13:07:37+05:30
Bump base to 4.17.1.0 and add release notes

- - - - -
8244cdcf by Zubin Duggal at 2023-04-14T13:07:37+05:30
Allow LLVM 14

- - - - -
083017b9 by Ben Gamari at 2023-04-14T13:07:37+05:30
rts: Always use atomics for context_switch and interrupt

Since these are modified by the timer handler.

(cherry picked from commit 605d954722a314c0da59ea07efc26d8a7cb59296)

- - - - -
c192ecd0 by Ben Gamari at 2023-04-14T13:07:37+05:30
rts/Timer: Always use atomic operations

As noted in #22447, the existence of the pthread-based ITimer
implementation means that we cannot assume that the program is
single-threaded.

(cherry picked from commit 86f20258ab7dbfb56e323ee811e9eaef80b077d3)

- - - - -
509c30d2 by Ben Gamari at 2023-04-14T13:07:37+05:30
rts: Encapsulate recent_activity access

This makes it easier to ensure that it is accessed using the necessary
atomic operations.

(cherry picked from commit f8e901dcc090ef81723fb9f3d8ea0a1baac4cbc3)

- - - - -
1717f212 by Ben Gamari at 2023-04-14T13:07:37+05:30
rts: Encapsulate access to capabilities array

(cherry picked from commit e0affaa9fc3e6dc0e65808afa383426b7fe9420a)

- - - - -
2d6d6fdd by Ben Gamari at 2023-04-14T13:07:38+05:30
rts: Encapsulate sched_state

(cherry picked from commit 7ca683e44f9f7a9a7984bbed4f49712838638fc8)

- - - - -
76508973 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix race in marking of blackholes

We must use an acquire-fence when marking to ensure that the indirectee
is visible.

(cherry picked from commit 2d71481ad481e6887e2ee342910b1495d53e6e4a)

- - - - -
de27e933 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix segment list races

(cherry picked from commit d92b5bf5ca90fb126321224f28096224bce87539)

- - - - -
1405f97a by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Use atomic when looking at bd->gen

Since it may have been mutated by a moving GC.

(cherry picked from commit 0b0d924edcf7631ed1afc88d614849558fa8fa7f)

- - - - -
d01a6ea7 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Eliminate race in bump_static_flag

To ensure that we don't race with a mutator entering a new CAF we take
the SM mutex before touching static_flag. The other option here would be
to instead modify newCAF to use a CAS but the present approach is a bit
safer.

(cherry picked from commit 9a9874d2d96acdcd9daddb7aceaa7951153f6ac0)

- - - - -
ec088ed3 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Ensure that mutable fields have acquire barrier

(cherry picked from commit d6990e6789efeb091c51c50580b68ea1bd997d00)

- - - - -
fe5c048b by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix races in collector status tracking

Mark a number of accesses to do with tracking of the status of the
concurrent collection thread as atomic. No interesting races here,
merely necessary to satisfy TSAN.

(cherry picked from commit 83c2876d3783ac2f4ea969c9e01d15198aea03a3)

- - - - -
6326b966 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Make segment state updates atomic

(cherry picked from commit 377697ea4d44023e0c690d722e1e27f03629227f)

- - - - -
d3404b5a by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Refactor update remembered set initialization

This avoids a lock inversion between the storage manager mutex and
the stable pointer table mutex by not dropping the SM_MUTEX in
nonmovingCollect. This requires quite a bit of rejiggering but it
does seem like a better strategy.

(cherry picked from commit c96fc9d52e138b89a85f8df8f722140f874bcede)

- - - - -
9c77c4b9 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Ensure that we aren't holding locks when closing them

TSAN complains about this sort of thing.

(cherry picked from commit 71aea6d3160e1389c38bb94c281bd7c30a030147)

- - - - -
b96e007c by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Make bitmap accesses atomic

This is a benign race on any sensible hard since these are byte
accesses. Nevertheless, atomic accesses are necessary to satisfy
TSAN.

(cherry picked from commit 2d10d8dfdff1212286490c2c19b8f725ce501b10)

- - - - -
d29d4837 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix benign race in update remembered set check

Relaxed load is fine here since we will take the lock before looking at
the list.

(cherry picked from commit 7570ae43218fbdee30f783af5a3f49fa3d8301e3)

- - - - -
6d1d5e84 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix race in shortcutting

We must use an acquire load to read the info table pointer since if we
find an indirection we must be certain that we see the indirectee.

(cherry picked from commit b6a3ab00712ca40acd1d91ba910e18c972a345a8)

- - - - -
e07c917c by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Make free list counter accesses atomic

Since these may race with the allocator(s).

(cherry picked from commit 5a40353566516d7ba8b1db6877f8c3aba06e64aa)

- - - - -
d8c99faa by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Fix style

(cherry picked from commit 1ca730ce410007781489821188ea8c1ca85f78f9)

- - - - -
f521c304 by Ben Gamari at 2023-04-14T13:07:38+05:30
nonmoving: Deduplicate assertion

(cherry picked from commit 1cc4bd7a169c6d813330d3613836889568515000)

- - - - -
f59ee743 by Ben Gamari at 2023-04-14T13:07:38+05:30
rts: Fix type issues in Sparks.h

Adds explicit casts to satisfy a C++ compiler.

(cherry picked from commit 8623ab4bd7fb1326cebb3191fdee841af74bb345)

- - - - -
66c32a26 by Ben Gamari at 2023-04-14T13:07:38+05:30
rts/BlockAlloc: Allow disabling of internal assertions

These can be quite expensive and it is sometimes useful to compile a
DEBUG RTS without them.

(cherry picked from commit 3072bef55dc6e123e7eb84bb9154a13db4d043da)

- - - - -
a7be6c65 by Ben Gamari at 2023-04-14T13:07:38+05:30
rts/Sanity: Mark pinned_object_blocks

(cherry picked from commit 3223048a1ee73384010aa20be548bad95ef1744a)

- - - - -
70a8ca3f by Ben Gamari at 2023-04-14T13:07:38+05:30
rts/Sanity: Look at nonmoving saved_filled lists

(cherry picked from commit ce4cf9d59552b52ae3d4c1c1e71a83296f022c27)

- - - - -
19db4bf7 by Ben Gamari at 2023-04-14T13:07:38+05:30
Evac: Squash data race in eval_selector_chain

(cherry picked from commit 9ce9fba46bf3fe1af4ce39dba4276bb241f3ea29)

- - - - -
1a6716be by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Clarify implementation

This makes the intent of this implementation a bit clearer.

(cherry picked from commit ba3c7b67bd74906de63b3988b18a14b01861d3c9)

- - - - -
fb9caa54 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Clarify comment

(cherry picked from commit 883d00f9cfffe3bd451f7302116eb29ce836c600)

- - - - -
ecf80b17 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Add missing no-op in busy-wait loop

(cherry picked from commit ed5443ec9195f27522abf4b2882dbc6b6a9f17f5)

- - - - -
64f05017 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Don't push empty arrays to update remembered set

Previously the write barrier of resizeSmallArray# incorrectly handled
resizing of zero-sized arrays, pushing an invalid pointer to the update
remembered set.

Fixes #22931.

(cherry picked from commit 244640a067c0158db5f14f133212caa1a830030e)

- - - - -
dbf32a81 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Fix handling of weak pointers

This fixes an interaction between aging and weak pointer handling which
prevented the finalization of some weak pointers. In particular, weak
pointers could have their keys incorrectly marked by the preparatory
collector, preventing their finalization by the subsequent concurrent
collection.

While in the area, we also significantly improve the assertions
regarding weak pointers.

Fixes #22327.

(cherry picked from commit 683e0c7aef2bbfedbee02c8fa32789e32c89a4db)

- - - - -
77ddce65 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Sanity check nonmoving large objects and compacts

(cherry picked from commit 90e240045c1ac5047f4038bbeed6d2b71b20bc77)

- - - - -
162c3509 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Sanity check mutable list

Assert that entries in the nonmoving generation's generational
remembered set (a.k.a. mutable list) live in nonmoving generation.

(cherry picked from commit 2f08265751c13f7bd0df1c46928fc09a064a8add)

- - - - -
a5fb1176 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Don't show occupancy if we didn't collect live words

(cherry picked from commit 3288e96d4ffe8c5f70929c4fedbfef918148def1)

- - - - -
986028e8 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Fix tracking of FILLED_SWEEPING segments

Previously we only updated the state of the segment at the head of each
allocator's filled list.

(cherry picked from commit 4392965d14920117db2cf4ce460dec6f094f6dfe)

- - - - -
2c0ed71d by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Assert state of swept segments

(cherry picked from commit d478ac18395962d33e14fc378ad9a424bdcd1496)

- - - - -
a6e08eff by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Handle new closures in nonmovingIsNowAlive

We must conservatively assume that new closures are reachable since we
are not guaranteed to mark such blocks.

(cherry picked from commit 3e47be7887b2df48d2ad50791a98131634d7e8ec)

- - - - -
ec8ae75e by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Don't clobber update rem sets of old capabilities

Previously `storageAddCapabilities` (called by `setNumCapabilities`) would
clobber the update remembered sets of existing capabilities when
increasing the capability count. Fix this by only initializing the
update remembered sets of the newly-created capabilities.

Fixes #22927.

(cherry picked from commit 7dc1679b133672029dd7afae834a7fc67265035c)

- - - - -
bdf67900 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Add missing write barriers in selector optimisation

This fixes the selector optimisation, adding a few write barriers which
are necessary for soundness. See the inline comments for details.

Fixes #22930.

(cherry picked from commit bd80fcd18f7be5b9d445d80c828b1a734c0e5a3b)

- - - - -
61e6b54d by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Post-sweep sanity checking

(cherry picked from commit e941801a267a3b9d898b48d90737924a7b7e4da2)

- - - - -
1f9376bc by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Avoid n_caps race

(cherry picked from commit 68fa47f3f5d1c207bd9edd7206bcceb150377647)

- - - - -
146530f0 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Don't push if nonmoving collector isn't enabled

(cherry picked from commit b15c813717ab9797df7733575904a7131bc4044c)

- - - - -
ee3a6e37 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Be more paranoid in segment tracking

Previously we left various segment link pointers dangling. None of this
wrong per se, but it did make it harder than necessary to debug.

(cherry picked from commit ca3bc40255ffadfd0df31ec3a6ec5df7ca7f663e)

- - - - -
c2b57cc8 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Sync-phase mark budgeting

Here we significantly improve the bound on sync phase pause times by
imposing a limit on the amount of work that we can perform during the
sync. If we find that we have exceeded our marking budget then we allow
the mutators to resume, return to concurrent marking, and try
synchronizing again later.

Fixes #22929.

(cherry picked from commit 023998885b16bea51c0c1cdceb3f0ef96dccd5e4)

- - - - -
4f86e5d6 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Allow pinned gen0 objects to be WEAK keys

(cherry picked from commit 0c6f15762d14ade2666d5ad7f665718b4a10f89e)

- - - - -
f0e75bc3 by Ben Gamari at 2023-04-14T13:07:39+05:30
rts: Reenable assertion

(cherry picked from commit 5b60acba8d76ecc4d733043460972633b13eedb0)

- - - - -
4fd491c1 by Ben Gamari at 2023-04-14T13:07:39+05:30
nonmoving: Move current segment array into Capability

The current segments are conceptually owned by the mutator, not the
collector. Consequently, it was quite tricky to prove that the mutator
would not race with the collect due to this shared state. It turns out
that such races are possible: when resizing the current segment array
we may concurrently try to take a heap census. This will attempt to walk
the current segment array, causing a data race.

Fix this by moving the current segment array into `Capability`, where it
belongs.

Fixes #22926.

(cherry picked from commit f849374508432ce07c25db8848fbea0d71040038)

- - - - -
890d0df6 by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Fix Note references

Some references to Note [Deadlock detection under the non-moving
collector] were missing an article.

(cherry picked from commit 43fda6487f729f8c627dbb2abbfc027444d54e52)

- - - - -
d616fef8 by Ben Gamari at 2023-04-14T13:07:40+05:30
rts/Sanity: Fix block count assertion with non-moving collector

The nonmoving collector does not use `oldest_gen->blocks` to track its
block list. However, it nevertheless updates `oldest_gen->n_blocks` to
ensure that its size is accounted for by the storage manager.
Consequently, we must not attempt to assert consistency between the two.

(cherry picked from commit a9062eaa577b3f1edf07908e57832524b5525d7d)

- - - - -
ca9068c0 by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Don't call prepareUnloadCheck

When the nonmoving GC is in use we do not call `checkUnload` (since we
don't unload code) and therefore should not call `prepareUnloadCheck`,
lest we run into assertions.

(cherry picked from commit b284e1c1d1e8e2513a286e3bbc5f46d635b33dd8)

- - - - -
4b6acb26 by Ben Gamari at 2023-04-14T13:07:40+05:30
rts: Encapsulate block allocator spinlock

This makes it a bit easier to add instrumentation on this spinlock
while debugging.

(cherry picked from commit 9c9899d5c99f09312a093b5cf99e5a0a9579bf3a)

- - - - -
f9b47bde by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Fix unregisterised build

(cherry picked from commit f2ef2f5e7a01cf5ec97b751f344233a6a8b781ca)

- - - - -
a79e52cf by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Ensure that sanity checker accounts for saved_filled segments

(cherry picked from commit 7cc7461c74bf78aca0987c8e8e0ec48297c28f2c)

- - - - -
8a031eb8 by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Move allocator into new source file

(cherry picked from commit e6f7b809885609de9e647e7f96cc95929fde7a4a)

- - - - -
1b294f15 by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Split out nonmovingAllocateGC

(cherry picked from commit cfc68b5c9ee9000010ccb2b7f1d346542df3b79f)

- - - - -
4c855ce5 by Ben Gamari at 2023-04-14T13:07:40+05:30
testsuite: Mark ffi023 as broken due to #23089

(cherry picked from commit 600fdd58332e52ac1620178f4b4a296e131465a1)

- - - - -
6a18fc31 by Ben Gamari at 2023-04-14T13:07:40+05:30
rts: Capture GC configuration in a struct

The number of distinct arguments passed to GarbageCollect was getting a
bit out of hand.

(cherry picked from commit ac0240af44026b9fcc08b62beef696ce7ea50e53)

- - - - -
fa9c8e61 by Ben Gamari at 2023-04-14T13:07:40+05:30
nonmoving: Non-concurrent collection

(cherry picked from commit be9b4ca4da78b10bc065957472ecf2c8ce7599a4)

- - - - -
b7a380d9 by Ben Gamari at 2023-04-14T13:07:40+05:30
hadrian: Add +boot_nonmoving_gc flavour transformer

For using GHC bootstrapping to validate the non-moving GC.

(cherry picked from commit ec382ccc087015db3d8782d3079c757ef8f10e9b)

- - - - -
70d9353f by Ben Gamari at 2023-04-14T13:07:40+05:30
contextswitch

- - - - -
b82c9613 by Ben Gamari at 2023-04-14T13:07:40+05:30
rts: Fix incorrect format specifier warnings

- - - - -
11ac9861 by Zubin Duggal at 2023-04-14T13:07:40+05:30
compiler: Fix performance regression in backport of "Make FloatIn robust to shadowing" (6206cb9287f3f6e70c669660a646a65274870d2b)

In 9.4, we have

noFloatIntoArg :: CoreExprWithFVs' -> Type -> Bool
noFloatIntoArg expr expr_ty = ...

But in master when 6206cb92 landed, after "Drop the app invariant" (dcf30da8) we had

noFloatIntoArg :: CoreExprWithFVs' -> Bool
noFloatIntoArg expr = ...

When deciding whether to float things into the argument of a function, in 9.4 we
must know the type of the argument. This was previously done by extracting the
type of the argument from the function type, computed as we walked through all
the arguments.

However, this backport regressed compile time performance due to allocations by
`exprType` particularly in T16577 and T5642, where it turns out that computing
the type of the arguments to a function is quite expensive.

Instead, we can compute the type of the argument by looking at the argument
term directly, which turns out to be much faster and eliminates the performance
regression.

- - - - -
6fd97ab1 by Zubin Duggal at 2023-04-14T13:07:40+05:30
Prepare release 9.4.5

- - - - -
82b1c20b by Teo Camarasu at 2023-04-14T13:07:40+05:30
Add regression test for #17574

This test currently fails in the nonmoving way

(cherry picked from commit a56141a69842a78d56ec11be85a775eb703219bf)

- - - - -
90da06d5 by Teo Camarasu at 2023-04-14T13:07:40+05:30
fix: account for large and compact object stats with nonmoving gc

Make sure that we keep track of the size of large and compact objects that have been moved onto the nonmoving heap.
We keep track of their size and add it to the amount of live bytes in nonmoving segments to get the total size of the live nonmoving heap.

Resolves #17574

(cherry picked from commit 20c6669fc46c567e00d3cdf22aa84479b6d8dc17)

- - - - -
0ac1c43d by Teo Camarasu at 2023-04-14T13:07:40+05:30
Allow running memInventory when the concurrent nonmoving gc is enabled

If the nonmoving gc is enabled and we are using a threaded RTS,
we now try to grab the collector mutex to avoid memInventory and
the collection racing.

Before memInventory was disabled.

(cherry picked from commit 62c3f7ee4199305cde009ededeae6ece1bcde7f0)
(cherry picked from commit fabc5a1c9aa468e97429ca5f8e501ec4fbd1084f)

- - - - -

30 changed files:

- compiler/GHC/Core/Opt/FloatIn.hs
- configure.ac
- + docs/users_guide/9.4.5-notes.rst
- docs/users_guide/release-notes.rst
- hadrian/src/Flavour.hs
- libraries/base/base.cabal
- libraries/base/changelog.md
- libraries/bytestring
- libraries/containers
- libraries/parsec
- libraries/text
- rts/Capability.c
- rts/Capability.h
- rts/HeapStackCheck.cmm
- rts/IOManager.c
- rts/PrimOps.cmm
- rts/Printer.c
- rts/Profiling.c
- rts/Proftimer.c
- rts/RetainerProfile.c
- rts/RtsAPI.c
- rts/RtsStartup.c
- rts/Schedule.c
- rts/Schedule.h
- rts/Sparks.h
- rts/Stats.c
- rts/Threads.c
- rts/Timer.c
- rts/TraverseHeap.c
- rts/eventlog/EventLog.c

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/19c8dde2a7d1f404d1c92ab0a053c085f6c76ced...0ac1c43dd13a6b515ee8fd0158690949e862b75d

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/19c8dde2a7d1f404d1c92ab0a053c085f6c76ced...0ac1c43dd13a6b515ee8fd0158690949e862b75d
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230414/b5ce420f/attachment-0001.html>