[Git][ghc/ghc][wip/andreask/keep-auto-rules-note] 33 commits: Add a note about about W/W for unlifting strict arguments
Andreas Klebinger (@AndreasK)
gitlab at gitlab.haskell.org
Mon Aug 8 12:04:55 UTC 2022
Andreas Klebinger pushed to branch wip/andreask/keep-auto-rules-note at Glasgow Haskell Compiler / GHC
Commits:
fb529cae by Andreas Klebinger at 2022-08-04T13:57:25-04:00
Add a note about about W/W for unlifting strict arguments
This fixes #21236.
- - - - -
fffc75a9 by Matthew Pickering at 2022-08-04T13:58:01-04:00
Force safeInferred to avoid retaining extra copy of DynFlags
This will only have a (very) modest impact on memory but we don't want
to retain old copies of DynFlags hanging around so best to force this
value.
- - - - -
0f43837f by Matthew Pickering at 2022-08-04T13:58:01-04:00
Force name selectors to ensure no reference to Ids enter the NameCache
I observed some unforced thunks in the NameCache which were retaining a
whole Id, which ends up retaining a Type.. which ends up retaining old
copies of HscEnv containing stale HomeModInfo.
- - - - -
0b1f5fd1 by Matthew Pickering at 2022-08-04T13:58:01-04:00
Fix leaks in --make mode when there are module loops
This patch fixes quite a tricky leak where we would end up retaining
stale ModDetails due to rehydrating modules against non-finalised
interfaces.
== Loops with multiple boot files
It is possible for a module graph to have a loop (SCC, when ignoring boot files)
which requires multiple boot files to break. In this case we must perform the
necessary hydration steps before and after compiling modules which have boot files
which are described above for corectness but also perform an additional hydration step
at the end of the SCC to remove space leaks.
Consider the following example:
┌───────┐ ┌───────┐
│ │ │ │
│ A │ │ B │
│ │ │ │
└─────┬─┘ └───┬───┘
│ │
┌────▼─────────▼──┐
│ │
│ C │
└────┬─────────┬──┘
│ │
┌────▼──┐ ┌───▼───┐
│ │ │ │
│ A-boot│ │ B-boot│
│ │ │ │
└───────┘ └───────┘
A, B and C live together in a SCC. Say we compile the modules in order
A-boot, B-boot, C, A, B then when we compile A we will perform the hydration steps
(because A has a boot file). Therefore C will be hydrated relative to A, and the
ModDetails for A will reference C/A. Then when B is compiled C will be rehydrated again,
and so B will reference C/A,B, its interface will be hydrated relative to both A and B.
Now there is a space leak because say C is a very big module, there are now two different copies of
ModDetails kept alive by modules A and B.
The way to avoid this space leak is to rehydrate an entire SCC together at the
end of compilation so that all the ModDetails point to interfaces for .hs files.
In this example, when we hydrate A, B and C together then both A and B will refer to
C/A,B.
See #21900 for some more discussion.
-------------------------------------------------------
In addition to this simple case, there is also the potential for a leak
during parallel upsweep which is also fixed by this patch. Transcibed is
Note [ModuleNameSet, efficiency and space leaks]
Note [ModuleNameSet, efficiency and space leaks]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
During unsweep the results of compiling modules are placed into a MVar, to find
the environment the module needs to compile itself in the MVar is consulted and
the HomeUnitGraph is set accordingly. The reason we do this is that precisely tracking
module dependencies and recreating the HUG from scratch each time is very expensive.
In serial mode (-j1), this all works out fine because a module can only be compiled after
its dependencies have finished compiling and not interleaved with compiling module loops.
Therefore when we create the finalised or no loop interfaces, the HUG only contains
finalised interfaces.
In parallel mode, we have to be more careful because the HUG variable can contain
non-finalised interfaces which have been started by another thread. In order to avoid
a space leak where a finalised interface is compiled against a HPT which contains a
non-finalised interface we have to restrict the HUG to only the visible modules.
The visible modules is recording in the ModuleNameSet, this is propagated upwards
whilst compiling and explains which transitive modules are visible from a certain point.
This set is then used to restrict the HUG before the module is compiled to only
the visible modules and thus avoiding this tricky space leak.
Efficiency of the ModuleNameSet is of utmost importance because a union occurs for
each edge in the module graph. Therefore the set is represented directly as an IntSet
which provides suitable performance, even using a UniqSet (which is backed by an IntMap) is
too slow. The crucial test of performance here is the time taken to a do a no-op build in --make mode.
See test "jspace" for an example which used to trigger this problem.
Fixes #21900
- - - - -
1d94a59f by Matthew Pickering at 2022-08-04T13:58:01-04:00
Store interfaces in ModIfaceCache more directly
I realised hydration was completely irrelavant for this cache because
the ModDetails are pruned from the result. So now it simplifies things a
lot to just store the ModIface and Linkable, which we can put into the
cache straight away rather than wait for the final version of a
HomeModInfo to appear.
- - - - -
6c7cd50f by Cheng Shao at 2022-08-04T23:01:45-04:00
cmm: Remove unused ReadOnlyData16
We don't actually emit rodata16 sections anywhere.
- - - - -
16333ad7 by Andreas Klebinger at 2022-08-04T23:02:20-04:00
findExternalRules: Don't needlessly traverse the list of rules.
- - - - -
52c15674 by Krzysztof Gogolewski at 2022-08-05T12:47:05-04:00
Remove backported items from 9.6 release notes
They have been backported to 9.4 in commits 5423d84bd9a28f,
13c81cb6be95c5, 67ccbd6b2d4b9b.
- - - - -
78d232f5 by Matthew Pickering at 2022-08-05T12:47:40-04:00
ci: Fix pages job
The job has been failing because we don't bundle haddock docs anymore in
the docs dist created by hadrian.
Fixes #21789
- - - - -
037bc9c9 by Ben Gamari at 2022-08-05T22:00:29-04:00
codeGen/X86: Don't clobber switch variable in switch generation
Previously ce8745952f99174ad9d3bdc7697fd086b47cdfb5 assumed that it was
safe to clobber the switch variable when generating code for a jump
table since we were at the end of a block. However, this assumption is
wrong; the register could be live in the jump target.
Fixes #21968.
- - - - -
50c8e1c5 by Matthew Pickering at 2022-08-05T22:01:04-04:00
Fix equality operator in jspace test
- - - - -
e9c77a22 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Improve BUILD_PAP comments
- - - - -
41234147 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Make dropTail comment a haddock comment
- - - - -
ff11d579 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Add one more sanity check in stg_restore_cccs
- - - - -
1f6c56ae by Andreas Klebinger at 2022-08-06T06:13:17-04:00
StgToCmm: Fix isSimpleScrut when profiling is enabled.
When profiling is enabled we must enter functions that might represent
thunks in order for their sccs to show up in the profile.
We might allocate even if the function is already evaluated in this
case. So we can't consider any potential function thunk to be a simple
scrut when profiling.
Not doing so caused profiled binaries to segfault.
- - - - -
fab0ee93 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Change `-fprof-late` to insert cost centres after unfolding creation.
The former behaviour of adding cost centres after optimization but
before unfoldings are created is not available via the flag
`prof-late-inline` instead.
I also reduced the overhead of -fprof-late* by pushing the cost centres
into lambdas. This means the cost centres will only account for
execution of functions and not their partial application.
Further I made LATE_CC cost centres it's own CC flavour so they now
won't clash with user defined ones if a user uses the same string for
a custom scc.
LateCC: Don't put cost centres inside constructor workers.
With -fprof-late they are rarely useful as the worker is usually
inlined. Even if the worker is not inlined or we use -fprof-late-linline
they are generally not helpful but bloat compile and run time
significantly. So we just don't add sccs inside constructor workers.
-------------------------
Metric Decrease:
T13701
-------------------------
- - - - -
f8bec4e3 by Ben Gamari at 2022-08-06T06:13:53-04:00
gitlab-ci: Fix hadrian bootstrapping of release pipelines
Previously we would attempt to test hadrian bootstrapping in the
`validate` build flavour. However, `ci.sh` refuses to run validation
builds during release pipelines, resulting in job failures. Fix this by
testing bootstrapping in the `release` flavour during release pipelines.
We also attempted to record perf notes for these builds, which is
redundant work and undesirable now since we no longer build in a
consistent flavour.
- - - - -
c0348865 by Ben Gamari at 2022-08-06T11:45:17-04:00
compiler: Eliminate two uses of foldr in favor of foldl'
These two uses constructed maps, which is a case where foldl' is
generally more efficient since we avoid constructing an intermediate
O(n)-depth stack.
- - - - -
d2e4e123 by Ben Gamari at 2022-08-06T11:45:17-04:00
rts: Fix code style
- - - - -
57f530d3 by Ben Gamari at 2022-08-06T11:45:17-04:00
genprimopcode: Drop ArrayArray# references
As ArrayArray# no longer exists
- - - - -
7267cd52 by Ben Gamari at 2022-08-06T11:45:17-04:00
base: Organize Haddocks in GHC.Conc.Sync
- - - - -
aa818a9f by Ben Gamari at 2022-08-06T11:48:50-04:00
Add primop to list threads
A user came to #ghc yesterday wondering how best to check whether they
were leaking threads. We ended up using the eventlog but it seems to me
like it would be generally useful if Haskell programs could query their
own threads.
- - - - -
6d1700b6 by Ben Gamari at 2022-08-06T11:51:35-04:00
rts: Move thread labels into TSO
This eliminates the thread label HashTable and instead tracks this
information in the TSO, allowing us to use proper StgArrBytes arrays for
backing the label and greatly simplifying management of object lifetimes
when we expose them to the user with the coming `threadLabel#` primop.
- - - - -
1472044b by Ben Gamari at 2022-08-06T11:54:52-04:00
Add a primop to query the label of a thread
- - - - -
43f2b271 by Ben Gamari at 2022-08-06T11:55:14-04:00
base: Share finalization thread label
For efficiency's sake we float the thread label assigned to the
finalization thread to the top-level, ensuring that we only need to
encode the label once.
- - - - -
1d63b4fb by Ben Gamari at 2022-08-06T11:57:11-04:00
users-guide: Add release notes entry for thread introspection support
- - - - -
09bca1de by Ben Gamari at 2022-08-07T01:19:35-04:00
hadrian: Fix binary distribution install attributes
Previously we would use plain `cp` to install various parts of the
binary distribution. However, `cp`'s behavior w.r.t. file attributes is
quite unclear; for this reason it is much better to rather use
`install`.
Fixes #21965.
- - - - -
2b8ea16d by Ben Gamari at 2022-08-07T01:19:35-04:00
hadrian: Fix installation of system-cxx-std-lib package conf
- - - - -
7b514848 by Ben Gamari at 2022-08-07T01:20:10-04:00
gitlab-ci: Bump Docker images
To give the ARMv7 job access to lld, fixing #21875.
- - - - -
afa584a3 by Ben Gamari at 2022-08-07T05:08:52-04:00
hadrian: Don't use mk/config.mk.in
Ultimately we want to drop mk/config.mk so here I extract the bits
needed by the Hadrian bindist installation logic into a Hadrian-specific
file. While doing this I fixed binary distribution installation, #21901.
- - - - -
b9bb45d7 by Ben Gamari at 2022-08-07T05:08:52-04:00
hadrian: Fix naming of cross-compiler wrappers
- - - - -
78d04cfa by Ben Gamari at 2022-08-07T11:44:58-04:00
hadrian: Extend xattr Darwin hack to cover /lib
As noted in #21506, it is now necessary to remove extended attributes
from `/lib` as well as `/bin` to avoid SIP issues on Darwin.
Fixes #21506.
- - - - -
de465b03 by Andreas Klebinger at 2022-08-08T12:04:53+00:00
Note [Trimming auto-rules]: State that this improves compiler perf.
- - - - -
30 changed files:
- .gitlab-ci.yml
- .gitlab/ci.sh
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Cmm.hs
- compiler/GHC/Cmm/DebugBlock.hs
- compiler/GHC/Cmm/Info/Build.hs
- compiler/GHC/Cmm/Sink.hs
- compiler/GHC/CmmToAsm/PPC/Ppr.hs
- compiler/GHC/CmmToAsm/Ppr.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToAsm/X86/Ppr.hs
- compiler/GHC/CmmToLlvm/Data.hs
- compiler/GHC/Core/LateCC.hs
- compiler/GHC/Core/Opt/Pipeline.hs
- compiler/GHC/Core/Opt/WorkWrap/Utils.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/CodeOutput.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Iface/Tidy.hs
- compiler/GHC/Stg/Pipeline.hs
- compiler/GHC/StgToCmm/Closure.hs
- compiler/GHC/StgToCmm/Expr.hs
- compiler/GHC/StgToCmm/Prim.hs
- compiler/GHC/Types/CostCentre.hs
- compiler/GHC/Types/RepType.hs
- compiler/GHC/Types/Unique/DFM.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6ee2d509455f01645819f06dbe2c1ba69d8b6e16...de465b0365537d525344214893f0b77ca4db758f
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6ee2d509455f01645819f06dbe2c1ba69d8b6e16...de465b0365537d525344214893f0b77ca4db758f
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20220808/2ec2a19d/attachment-0001.html>
More information about the ghc-commits
mailing list