[Git][ghc/ghc][wip/T21286] 129 commits: Add a note about about W/W for unlifting strict arguments
Simon Peyton Jones (@simonpj)
gitlab at gitlab.haskell.org
Tue Aug 30 10:47:49 UTC 2022
Simon Peyton Jones pushed to branch wip/T21286 at Glasgow Haskell Compiler / GHC
Commits:
fb529cae by Andreas Klebinger at 2022-08-04T13:57:25-04:00
Add a note about about W/W for unlifting strict arguments
This fixes #21236.
- - - - -
fffc75a9 by Matthew Pickering at 2022-08-04T13:58:01-04:00
Force safeInferred to avoid retaining extra copy of DynFlags
This will only have a (very) modest impact on memory but we don't want
to retain old copies of DynFlags hanging around so best to force this
value.
- - - - -
0f43837f by Matthew Pickering at 2022-08-04T13:58:01-04:00
Force name selectors to ensure no reference to Ids enter the NameCache
I observed some unforced thunks in the NameCache which were retaining a
whole Id, which ends up retaining a Type.. which ends up retaining old
copies of HscEnv containing stale HomeModInfo.
- - - - -
0b1f5fd1 by Matthew Pickering at 2022-08-04T13:58:01-04:00
Fix leaks in --make mode when there are module loops
This patch fixes quite a tricky leak where we would end up retaining
stale ModDetails due to rehydrating modules against non-finalised
interfaces.
== Loops with multiple boot files
It is possible for a module graph to have a loop (SCC, when ignoring boot files)
which requires multiple boot files to break. In this case we must perform the
necessary hydration steps before and after compiling modules which have boot files
which are described above for corectness but also perform an additional hydration step
at the end of the SCC to remove space leaks.
Consider the following example:
┌───────┐ ┌───────┐
│ │ │ │
│ A │ │ B │
│ │ │ │
└─────┬─┘ └───┬───┘
│ │
┌────▼─────────▼──┐
│ │
│ C │
└────┬─────────┬──┘
│ │
┌────▼──┐ ┌───▼───┐
│ │ │ │
│ A-boot│ │ B-boot│
│ │ │ │
└───────┘ └───────┘
A, B and C live together in a SCC. Say we compile the modules in order
A-boot, B-boot, C, A, B then when we compile A we will perform the hydration steps
(because A has a boot file). Therefore C will be hydrated relative to A, and the
ModDetails for A will reference C/A. Then when B is compiled C will be rehydrated again,
and so B will reference C/A,B, its interface will be hydrated relative to both A and B.
Now there is a space leak because say C is a very big module, there are now two different copies of
ModDetails kept alive by modules A and B.
The way to avoid this space leak is to rehydrate an entire SCC together at the
end of compilation so that all the ModDetails point to interfaces for .hs files.
In this example, when we hydrate A, B and C together then both A and B will refer to
C/A,B.
See #21900 for some more discussion.
-------------------------------------------------------
In addition to this simple case, there is also the potential for a leak
during parallel upsweep which is also fixed by this patch. Transcibed is
Note [ModuleNameSet, efficiency and space leaks]
Note [ModuleNameSet, efficiency and space leaks]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
During unsweep the results of compiling modules are placed into a MVar, to find
the environment the module needs to compile itself in the MVar is consulted and
the HomeUnitGraph is set accordingly. The reason we do this is that precisely tracking
module dependencies and recreating the HUG from scratch each time is very expensive.
In serial mode (-j1), this all works out fine because a module can only be compiled after
its dependencies have finished compiling and not interleaved with compiling module loops.
Therefore when we create the finalised or no loop interfaces, the HUG only contains
finalised interfaces.
In parallel mode, we have to be more careful because the HUG variable can contain
non-finalised interfaces which have been started by another thread. In order to avoid
a space leak where a finalised interface is compiled against a HPT which contains a
non-finalised interface we have to restrict the HUG to only the visible modules.
The visible modules is recording in the ModuleNameSet, this is propagated upwards
whilst compiling and explains which transitive modules are visible from a certain point.
This set is then used to restrict the HUG before the module is compiled to only
the visible modules and thus avoiding this tricky space leak.
Efficiency of the ModuleNameSet is of utmost importance because a union occurs for
each edge in the module graph. Therefore the set is represented directly as an IntSet
which provides suitable performance, even using a UniqSet (which is backed by an IntMap) is
too slow. The crucial test of performance here is the time taken to a do a no-op build in --make mode.
See test "jspace" for an example which used to trigger this problem.
Fixes #21900
- - - - -
1d94a59f by Matthew Pickering at 2022-08-04T13:58:01-04:00
Store interfaces in ModIfaceCache more directly
I realised hydration was completely irrelavant for this cache because
the ModDetails are pruned from the result. So now it simplifies things a
lot to just store the ModIface and Linkable, which we can put into the
cache straight away rather than wait for the final version of a
HomeModInfo to appear.
- - - - -
6c7cd50f by Cheng Shao at 2022-08-04T23:01:45-04:00
cmm: Remove unused ReadOnlyData16
We don't actually emit rodata16 sections anywhere.
- - - - -
16333ad7 by Andreas Klebinger at 2022-08-04T23:02:20-04:00
findExternalRules: Don't needlessly traverse the list of rules.
- - - - -
52c15674 by Krzysztof Gogolewski at 2022-08-05T12:47:05-04:00
Remove backported items from 9.6 release notes
They have been backported to 9.4 in commits 5423d84bd9a28f,
13c81cb6be95c5, 67ccbd6b2d4b9b.
- - - - -
78d232f5 by Matthew Pickering at 2022-08-05T12:47:40-04:00
ci: Fix pages job
The job has been failing because we don't bundle haddock docs anymore in
the docs dist created by hadrian.
Fixes #21789
- - - - -
037bc9c9 by Ben Gamari at 2022-08-05T22:00:29-04:00
codeGen/X86: Don't clobber switch variable in switch generation
Previously ce8745952f99174ad9d3bdc7697fd086b47cdfb5 assumed that it was
safe to clobber the switch variable when generating code for a jump
table since we were at the end of a block. However, this assumption is
wrong; the register could be live in the jump target.
Fixes #21968.
- - - - -
50c8e1c5 by Matthew Pickering at 2022-08-05T22:01:04-04:00
Fix equality operator in jspace test
- - - - -
e9c77a22 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Improve BUILD_PAP comments
- - - - -
41234147 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Make dropTail comment a haddock comment
- - - - -
ff11d579 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Add one more sanity check in stg_restore_cccs
- - - - -
1f6c56ae by Andreas Klebinger at 2022-08-06T06:13:17-04:00
StgToCmm: Fix isSimpleScrut when profiling is enabled.
When profiling is enabled we must enter functions that might represent
thunks in order for their sccs to show up in the profile.
We might allocate even if the function is already evaluated in this
case. So we can't consider any potential function thunk to be a simple
scrut when profiling.
Not doing so caused profiled binaries to segfault.
- - - - -
fab0ee93 by Andreas Klebinger at 2022-08-06T06:13:17-04:00
Change `-fprof-late` to insert cost centres after unfolding creation.
The former behaviour of adding cost centres after optimization but
before unfoldings are created is not available via the flag
`prof-late-inline` instead.
I also reduced the overhead of -fprof-late* by pushing the cost centres
into lambdas. This means the cost centres will only account for
execution of functions and not their partial application.
Further I made LATE_CC cost centres it's own CC flavour so they now
won't clash with user defined ones if a user uses the same string for
a custom scc.
LateCC: Don't put cost centres inside constructor workers.
With -fprof-late they are rarely useful as the worker is usually
inlined. Even if the worker is not inlined or we use -fprof-late-linline
they are generally not helpful but bloat compile and run time
significantly. So we just don't add sccs inside constructor workers.
-------------------------
Metric Decrease:
T13701
-------------------------
- - - - -
f8bec4e3 by Ben Gamari at 2022-08-06T06:13:53-04:00
gitlab-ci: Fix hadrian bootstrapping of release pipelines
Previously we would attempt to test hadrian bootstrapping in the
`validate` build flavour. However, `ci.sh` refuses to run validation
builds during release pipelines, resulting in job failures. Fix this by
testing bootstrapping in the `release` flavour during release pipelines.
We also attempted to record perf notes for these builds, which is
redundant work and undesirable now since we no longer build in a
consistent flavour.
- - - - -
c0348865 by Ben Gamari at 2022-08-06T11:45:17-04:00
compiler: Eliminate two uses of foldr in favor of foldl'
These two uses constructed maps, which is a case where foldl' is
generally more efficient since we avoid constructing an intermediate
O(n)-depth stack.
- - - - -
d2e4e123 by Ben Gamari at 2022-08-06T11:45:17-04:00
rts: Fix code style
- - - - -
57f530d3 by Ben Gamari at 2022-08-06T11:45:17-04:00
genprimopcode: Drop ArrayArray# references
As ArrayArray# no longer exists
- - - - -
7267cd52 by Ben Gamari at 2022-08-06T11:45:17-04:00
base: Organize Haddocks in GHC.Conc.Sync
- - - - -
aa818a9f by Ben Gamari at 2022-08-06T11:48:50-04:00
Add primop to list threads
A user came to #ghc yesterday wondering how best to check whether they
were leaking threads. We ended up using the eventlog but it seems to me
like it would be generally useful if Haskell programs could query their
own threads.
- - - - -
6d1700b6 by Ben Gamari at 2022-08-06T11:51:35-04:00
rts: Move thread labels into TSO
This eliminates the thread label HashTable and instead tracks this
information in the TSO, allowing us to use proper StgArrBytes arrays for
backing the label and greatly simplifying management of object lifetimes
when we expose them to the user with the coming `threadLabel#` primop.
- - - - -
1472044b by Ben Gamari at 2022-08-06T11:54:52-04:00
Add a primop to query the label of a thread
- - - - -
43f2b271 by Ben Gamari at 2022-08-06T11:55:14-04:00
base: Share finalization thread label
For efficiency's sake we float the thread label assigned to the
finalization thread to the top-level, ensuring that we only need to
encode the label once.
- - - - -
1d63b4fb by Ben Gamari at 2022-08-06T11:57:11-04:00
users-guide: Add release notes entry for thread introspection support
- - - - -
09bca1de by Ben Gamari at 2022-08-07T01:19:35-04:00
hadrian: Fix binary distribution install attributes
Previously we would use plain `cp` to install various parts of the
binary distribution. However, `cp`'s behavior w.r.t. file attributes is
quite unclear; for this reason it is much better to rather use
`install`.
Fixes #21965.
- - - - -
2b8ea16d by Ben Gamari at 2022-08-07T01:19:35-04:00
hadrian: Fix installation of system-cxx-std-lib package conf
- - - - -
7b514848 by Ben Gamari at 2022-08-07T01:20:10-04:00
gitlab-ci: Bump Docker images
To give the ARMv7 job access to lld, fixing #21875.
- - - - -
afa584a3 by Ben Gamari at 2022-08-07T05:08:52-04:00
hadrian: Don't use mk/config.mk.in
Ultimately we want to drop mk/config.mk so here I extract the bits
needed by the Hadrian bindist installation logic into a Hadrian-specific
file. While doing this I fixed binary distribution installation, #21901.
- - - - -
b9bb45d7 by Ben Gamari at 2022-08-07T05:08:52-04:00
hadrian: Fix naming of cross-compiler wrappers
- - - - -
78d04cfa by Ben Gamari at 2022-08-07T11:44:58-04:00
hadrian: Extend xattr Darwin hack to cover /lib
As noted in #21506, it is now necessary to remove extended attributes
from `/lib` as well as `/bin` to avoid SIP issues on Darwin.
Fixes #21506.
- - - - -
20457d77 by Andreas Klebinger at 2022-08-08T14:42:26+02:00
NCG(x86): Compile add+shift as lea if possible.
- - - - -
742292e4 by Andreas Klebinger at 2022-08-08T16:46:37-04:00
dataToTag#: Skip runtime tag check if argument is infered tagged
This addresses one part of #21710.
- - - - -
1504a93e by Cheng Shao at 2022-08-08T16:47:14-04:00
rts: remove redundant stg_traceCcszh
This out-of-line primop has no Haskell wrapper and hasn't been used
anywhere in the tree. Furthermore, the code gets in the way of !7632, so
it should be garbage collected.
- - - - -
a52de3cb by Andreas Klebinger at 2022-08-08T16:47:50-04:00
Document a divergence from the report in parsing function lhss.
GHC is happy to parse `(f) x y = x + y` when it should be a parse error
based on the Haskell report. Seems harmless enough so we won't fix it
but it's documented now.
Fixes #19788
- - - - -
5765e133 by Ben Gamari at 2022-08-08T16:48:25-04:00
gitlab-ci: Add release job for aarch64/debian 11
- - - - -
5b26f324 by Ben Gamari at 2022-08-08T19:39:20-04:00
gitlab-ci: Introduce validation job for aarch64 cross-compilation
Begins to address #11958.
- - - - -
e866625c by Ben Gamari at 2022-08-08T19:39:20-04:00
Bump process submodule
- - - - -
ae707762 by Ben Gamari at 2022-08-08T19:39:20-04:00
gitlab-ci: Add basic support for cross-compiler testiing
Here we add a simple qemu-based test for cross-compilers.
- - - - -
50912d68 by Ben Gamari at 2022-08-08T19:39:57-04:00
rts: Ensure that Array# card arrays are initialized
In #19143 I noticed that newArray# failed to initialize the card table
of newly-allocated arrays. However, embarrassingly, I then only fixed
the issue in newArrayArray# and, in so doing, introduced the potential
for an integer underflow on zero-length arrays (#21962).
Here I fix the issue in newArray#, this time ensuring that we do not
underflow in pathological cases.
Fixes #19143.
- - - - -
e5ceff56 by Ben Gamari at 2022-08-08T19:39:57-04:00
testsuite: Add test for #21962
- - - - -
c1c08bd8 by Ben Gamari at 2022-08-09T02:31:14-04:00
gitlab-ci: Don't use coreutils on Darwin
In general we want to ensure that the tested environment is as similar
as possible to the environment the user will use. In the case of Darwin,
this means we want to use the system's BSD command-line utilities, not
coreutils.
This would have caught #21974.
- - - - -
1c582f44 by Ben Gamari at 2022-08-09T02:31:14-04:00
hadrian: Fix bindist installation on Darwin
It turns out that `cp -P` on Darwin does not always copy a symlink as
a symlink. In order to get these semantics one must pass `-RP`. It's not
entirely clear whether this is valid under POSIX, but it is nevertheless
what Apple does.
- - - - -
681aa076 by Ben Gamari at 2022-08-09T02:31:49-04:00
hadrian: Fix access mode of installed package registration files
Previously hadrian's bindist Makefile would modify package
registrations placed by `install` via a shell pipeline and `mv`.
However, the use of `mv` means that if umask is set then the user may
otherwise end up with package registrations which are inaccessible.
Fix this by ensuring that the mode is 0644.
- - - - -
e9dfd26a by Krzysztof Gogolewski at 2022-08-09T02:32:24-04:00
Cleanups around pretty-printing
* Remove hack when printing OccNames. No longer needed since e3dcc0d5
* Remove unused `pprCmms` and `instance Outputable Instr`
* Simplify `pprCLabel` (no need to pass platform)
* Remove evil `Show`/`Eq` instances for `SDoc`. They were needed by
ImmLit, but that can take just a String instead.
* Remove instance `Outputable CLabel` - proper output of labels
needs a platform, and is done by the `OutputableP` instance
- - - - -
66d2e927 by Ben Gamari at 2022-08-09T13:46:48-04:00
rts/linker: Resolve iconv_* on FreeBSD
FreeBSD's libiconv includes an implementation of the
iconv_* functions in libc. Unfortunately these can
only be resolved using dlvsym, which is how the RTS linker
usually resolves such functions. To fix this we include an ad-hoc
special case for iconv_*.
Fixes #20354.
- - - - -
5d66a0ce by Ben Gamari at 2022-08-09T13:46:48-04:00
system-cxx-std-lib: Add support for FreeBSD libcxxrt
- - - - -
ea90e61d by Ben Gamari at 2022-08-09T13:46:48-04:00
gitlab-ci: Bump to use freebsd13 runners
- - - - -
d71a2051 by sheaf at 2022-08-09T13:47:28-04:00
Fix size_up_alloc to account for UnliftedDatatypes
The size_up_alloc function mistakenly considered any type that isn't
lifted to not allocate anything, which is wrong. What we want instead
is to check the type isn't boxed. This accounts for (BoxedRep Unlifted).
Fixes #21939
- - - - -
76b52cf0 by Douglas Wilson at 2022-08-10T06:01:53-04:00
testsuite: 21651 add test for closeFdWith + setNumCapabilities
This bug does not affect windows, which does not use the
base module GHC.Event.Thread.
- - - - -
7589ee72 by Douglas Wilson at 2022-08-10T06:01:53-04:00
base: Fix races in IOManager (setNumCapabilities,closeFdWith)
Fix for #21651
Fixes three bugs:
- writes to eventManager should be atomic. It is accessed concurrently by ioManagerCapabilitiesChanged and closeFdWith.
- The race in closeFdWith described in the ticket.
- A race in getSystemEventManager where it accesses the 'IOArray' in
'eventManager' before 'ioManagerCapabilitiesChanged' has written to
'eventManager', causing an Array Index exception. The fix here is to
'yield' and retry.
- - - - -
dc76439d by Trevis Elser at 2022-08-10T06:02:28-04:00
Updates language extension documentation
Adding a 'Status' field with a few values:
- Deprecated
- Experimental
- InternalUseOnly
- Noting if included in 'GHC2021', 'Haskell2010' or 'Haskell98'
Those values are pulled from the existing descriptions or elsewhere in
the documentation.
While at it, include the :implied by: where appropriate, to provide
more detail.
Fixes #21475
- - - - -
823fe5b5 by Jens Petersen at 2022-08-10T06:03:07-04:00
hadrian RunRest: add type signature for stageNumber
avoids warning seen on 9.4.1:
src/Settings/Builders/RunTest.hs:264:53: warning: [-Wtype-defaults]
• Defaulting the following constraints to type ‘Integer’
(Show a0)
arising from a use of ‘show’
at src/Settings/Builders/RunTest.hs:264:53-84
(Num a0)
arising from a use of ‘stageNumber’
at src/Settings/Builders/RunTest.hs:264:59-83
• In the second argument of ‘(++)’, namely
‘show (stageNumber (C.stage ctx))’
In the second argument of ‘($)’, namely
‘"config.stage=" ++ show (stageNumber (C.stage ctx))’
In the expression:
arg $ "config.stage=" ++ show (stageNumber (C.stage ctx))
|
264 | , arg "-e", arg $ "config.stage=" ++ show (stageNumber (C.stage ctx))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
compilation tested locally
- - - - -
f95bbdca by Sylvain Henry at 2022-08-10T09:44:46-04:00
Add support for external static plugins (#20964)
This patch adds a new command-line flag:
-fplugin-library=<file-path>;<unit-id>;<module>;<args>
used like this:
-fplugin-library=path/to/plugin.so;package-123;Plugin.Module;["Argument","List"]
It allows a plugin to be loaded directly from a shared library. With
this approach, GHC doesn't compile anything for the plugin and doesn't
load any .hi file for the plugin and its dependencies. As such GHC
doesn't need to support two environments (one for plugins, one for
target code), which was the more ambitious approach tracked in #14335.
Fix #20964
Co-authored-by: Josh Meredith <joshmeredith2008 at gmail.com>
- - - - -
5bc489ca by Ben Gamari at 2022-08-10T09:45:22-04:00
gitlab-ci: Fix ARMv7 build
It appears that the CI refactoring carried out in
5ff690b8474c74e9c968ef31e568c1ad0fe719a1 failed to carry over some
critical configuration: setting the build/host/target platforms and
forcing use of a non-broken linker.
- - - - -
596db9a5 by Ben Gamari at 2022-08-10T09:45:22-04:00
gitlab-ci: Run ARMv7 jobs when ~ARM label is used
- - - - -
7cabea7c by Ben Gamari at 2022-08-10T15:37:58-04:00
hadrian: Don't attempt to install documentation if doc/ doesn't exist
Previously we would attempt to install documentation even if the `doc`
directory doesn't exist (e.g. due to `--docs=none`). This would result
in the surprising side-effect of the entire contents of the bindist
being installed in the destination documentation directory. Fix this.
Fixes #21976.
- - - - -
67575f20 by normalcoder at 2022-08-10T15:38:34-04:00
ncg/aarch64: Don't use x18 register on AArch64/Darwin
Apple's ABI documentation [1] says: "The platforms reserve register x18.
Don’t use this register." While this wasn't problematic in previous
Darwin releases, macOS 13 appears to start zeroing this register
periodically. See #21964.
[1] https://developer.apple.com/documentation/xcode/writing-arm64-code-for-apple-platforms
- - - - -
45eb4cbe by Andreas Klebinger at 2022-08-10T22:41:12-04:00
Note [Trimming auto-rules]: State that this improves compiler perf.
- - - - -
5c24b1b3 by Bodigrim at 2022-08-10T22:41:50-04:00
Document that threadDelay / timeout are susceptible to overflows on 32-bit machines
- - - - -
ff67c79e by Alan Zimmerman at 2022-08-11T16:19:57-04:00
EPA: DotFieldOcc does not have exact print annotations
For the code
{-# LANGUAGE OverloadedRecordUpdate #-}
operatorUpdate f = f{(+) = 1}
There are no exact print annotations for the parens around the +
symbol, nor does normal ppr print them.
This MR fixes that.
Closes #21805
Updates haddock submodule
- - - - -
dca43a04 by Matthew Pickering at 2022-08-11T16:20:33-04:00
Revert "gitlab-ci: Add release job for aarch64/debian 11"
This reverts commit 5765e13370634979eb6a0d9f67aa9afa797bee46.
The job was not tested before being merged and fails CI
(https://gitlab.haskell.org/ghc/ghc/-/jobs/1139392)
Ticket #22005
- - - - -
ffc9116e by Eric Lindblad at 2022-08-16T09:01:26-04:00
typo
- - - - -
cd6f5bfd by Ben Gamari at 2022-08-16T09:02:02-04:00
CmmToLlvm: Don't aliasify builtin LLVM variables
Our aliasification logic would previously turn builtin LLVM variables
into aliases, which apparently confuses LLVM. This manifested in
initializers failing to be emitted, resulting in many profiling failures
with the LLVM backend.
Fixes #22019.
- - - - -
dc7da356 by Bryan Richter at 2022-08-16T09:02:38-04:00
run_ci: remove monoidal-containers
Fixes #21492
MonoidalMap is inlined and used to implement Variables, as before.
The top-level value "jobs" is reimplemented as a regular Map, since it
doesn't use the monoidal union anyway.
- - - - -
64110544 by Cheng Shao at 2022-08-16T09:03:15-04:00
CmmToAsm/AArch64: correct a typo
- - - - -
f6a5524a by Andreas Klebinger at 2022-08-16T14:34:11-04:00
Fix #21979 - compact-share failing with -O
I don't have good reason to believe the optimization level should affect
if sharing works or not here. So limit the test to the normal way.
- - - - -
68154a9d by Ben Gamari at 2022-08-16T14:34:47-04:00
users-guide: Fix reference to dead llvm-version substitution
Fixes #22052.
- - - - -
28c60d26 by Ben Gamari at 2022-08-16T14:34:47-04:00
users-guide: Fix incorrect reference to `:extension: role
- - - - -
71102c8f by Ben Gamari at 2022-08-16T14:34:47-04:00
users-guide: Add :ghc-flag: reference
- - - - -
385f420b by Ben Gamari at 2022-08-16T14:34:47-04:00
hadrian: Place manpage in docroot
This relocates it from docs/ to doc/
- - - - -
84598f2e by Ben Gamari at 2022-08-16T14:34:47-04:00
Bump haddock submodule
Includes merge of `main` into `ghc-head` as well as some Haddock users
guide fixes.
- - - - -
59ce787c by Ben Gamari at 2022-08-16T14:34:47-04:00
base: Add changelog entries from ghc-9.2
Closes #21922.
- - - - -
a14e6ae3 by Ben Gamari at 2022-08-16T14:34:47-04:00
relnotes: Add "included libraries" section
As noted in #21988, some users rely on this.
- - - - -
a4212edc by Ben Gamari at 2022-08-16T14:34:47-04:00
users-guide: Rephrase the rewrite rule documentation
Previously the wording was a tad unclear. Fix this.
Closes #21114.
- - - - -
3e493dfd by Peter Becich at 2022-08-17T08:43:21+01:00
Implement Response File support for HPC
This is an improvement to HPC authored by Richard Wallace
(https://github.com/purefn) and myself. I have received permission from
him to attempt to upstream it. This improvement was originally
implemented as a patch to HPC via input-output-hk/haskell.nix:
https://github.com/input-output-hk/haskell.nix/pull/1464
Paraphrasing Richard, HPC currently requires all inputs as command line arguments.
With large projects this can result in an argument list too long error.
I have only seen this error in Nix, but I assume it can occur is a plain Unix environment.
This MR adds the standard response file syntax support to HPC. For
example you can now pass a file to the command line which contains the
arguments.
```
hpc @response_file_1 @response_file_2 ...
The contents of a Response File must have this format:
COMMAND ...
example:
report my_library.tix --include=ModuleA --include=ModuleB
```
Updates hpc submodule
Co-authored-by: Richard Wallace <rwallace at thewallacepack.net>
Fixes #22050
- - - - -
436867d6 by Matthew Pickering at 2022-08-18T09:24:08-04:00
ghc-heap: Fix decoding of TSO closures
An extra field was added to the TSO structure in 6d1700b6 but the
decoding logic in ghc-heap was not updated for this new field.
Fixes #22046
- - - - -
a740a4c5 by Matthew Pickering at 2022-08-18T09:24:44-04:00
driver: Honour -x option
The -x option is used to manually specify which phase a file should be
started to be compiled from (even if it lacks the correct extension). I
just failed to implement this when refactoring the driver.
In particular Cabal calls GHC with `-E -cpp -x hs Foo.cpphs` to
preprocess source files using GHC.
I added a test to exercise this case.
Fixes #22044
- - - - -
e293029d by Simon Peyton Jones at 2022-08-18T09:25:19-04:00
Be more careful in chooseInferredQuantifiers
This fixes #22065. We were failing to retain a quantifier that
was mentioned in the kind of another retained quantifier.
Easy to fix.
- - - - -
714c936f by Bryan Richter at 2022-08-18T18:37:21-04:00
testsuite: Add test for #21583
- - - - -
989b844d by Ben Gamari at 2022-08-18T18:37:57-04:00
compiler: Drop --build-id=none hack
Since 2011 the object-joining implementation has had a hack to pass
`--build-id=none` to `ld` when supported, seemingly to work around a
linker bug. This hack is now unnecessary and may break downstream users
who expect objects to have valid build-ids. Remove it.
Closes #22060.
- - - - -
519c712e by Matthew Pickering at 2022-08-19T00:09:11-04:00
Make ru_fn field strict to avoid retaining Ids
It's better to perform this projection from Id to Name strictly so we
don't retain an old Id (hence IdInfo, hence Unfolding, hence everything
etc)
- - - - -
7dda04b0 by Matthew Pickering at 2022-08-19T00:09:11-04:00
Force `getOccFS bndr` to avoid retaining reference to Bndr.
This is another symptom of #19619
- - - - -
4303acba by Matthew Pickering at 2022-08-19T00:09:11-04:00
Force unfoldings when they are cleaned-up in Tidy and CorePrep
If these thunks are not forced then the entire unfolding for the binding
is live throughout the whole of CodeGen despite the fact it should have
been discarded.
Fixes #22071
- - - - -
2361b3bc by Matthew Pickering at 2022-08-19T00:09:47-04:00
haddock docs: Fix links from identifiers to dependent packages
When implementing the base_url changes I made the pretty bad mistake of
zipping together two lists which were in different orders. The simpler
thing to do is just modify `haddockDependencies` to also return the
package identifier so that everything stays in sync.
Fixes #22001
- - - - -
9a7e2ea1 by Matthew Pickering at 2022-08-19T00:10:23-04:00
Revert "Refactor SpecConstr to use treat bindings uniformly"
This reverts commit 415468fef8a3e9181b7eca86de0e05c0cce31729.
This refactoring introduced quite a severe residency regression (900MB
live from 650MB live when compiling mmark), see #21993 for a reproducer
and more discussion.
Ticket #21993
- - - - -
9789e845 by Zachary Wood at 2022-08-19T14:17:28-04:00
tc: warn about lazy annotations on unlifted arguments (fixes #21951)
- - - - -
e5567289 by Andreas Klebinger at 2022-08-19T14:18:03-04:00
Fix #22048 where we failed to drop rules for -fomit-interface-pragmas.
Now we also filter the local rules (again) which fixes the issue.
- - - - -
51ffd009 by Swann Moreau at 2022-08-19T18:29:21-04:00
Print constraints in quotes (#21167)
This patch improves the uniformity of error message formatting by
printing constraints in quotes, as we do for types.
Fix #21167
- - - - -
ab3e0f5a by Sasha Bogicevic at 2022-08-19T18:29:57-04:00
19217 Implicitly quantify type variables in :kind command
- - - - -
9939e95f by MorrowM at 2022-08-21T16:51:38-04:00
Recognize file-header pragmas in GHCi (#21507)
- - - - -
fb7c2d99 by Matthew Pickering at 2022-08-21T16:52:13-04:00
hadrian: Fix bootstrapping with ghc-9.4
The error was that we were trying to link together
containers from boot package library (which depends template-haskell in boot package library)
template-haskell from in-tree package database
So the fix is to build containers in stage0 (and link against template-haskell built in stage0).
Fixes #21981
- - - - -
b946232c by Mario Blažević at 2022-08-22T22:06:21-04:00
Added pprType with precedence argument, as a prerequisite to fix issues #21723 and #21942.
* refines the precedence levels, adding `qualPrec` and `funPrec` to better control parenthesization
* `pprParendType`, `pprFunArgType`, and `instance Ppr Type` all just call `pprType` with proper precedence
* `ParensT` constructor is now always printed parenthesized
* adds the precedence argument to `pprTyApp` as well, as it needs to keep track and pass it down
* using `>=` instead of former `>` to match the Core type printing logic
* some test outputs have changed, losing extraneous parentheses
- - - - -
fe4ff0f7 by Mario Blažević at 2022-08-22T22:06:21-04:00
Fix and test for issue #21723
- - - - -
33968354 by Mario Blažević at 2022-08-22T22:06:21-04:00
Test for issue #21942
- - - - -
c9655251 by Mario Blažević at 2022-08-22T22:06:21-04:00
Updated the changelog
- - - - -
80102356 by Ben Gamari at 2022-08-22T22:06:57-04:00
hadrian: Don't duplicate binaries on installation
Previously we used `install` on symbolic links, which ended up
copying the target file rather than installing a symbolic link.
Fixes #22062.
- - - - -
b929063e by M Farkas-Dyck at 2022-08-24T02:37:01-04:00
Unbreak Haddock comments in `GHC.Core.Opt.WorkWrap.Utils`.
Closes #22092.
- - - - -
112e4f9c by Cheng Shao at 2022-08-24T02:37:38-04:00
driver: don't actually merge objects when ar -L works
- - - - -
a9f0e68e by Ben Gamari at 2022-08-24T02:38:13-04:00
rts: Consistently use MiB in stats output
Previously we would say `MB` even where we
meant `MiB`.
- - - - -
a90298cc by Simon Peyton Jones at 2022-08-25T08:38:16+01:00
Fix arityType: -fpedantic-bottoms, join points, etc
This MR fixes #21694, #21755. It also makes sure that #21948 and
fix to #21694.
* For #21694 the underlying problem was that we were calling arityType
on an expression that had free join points. This is a Bad Bad Idea.
See Note [No free join points in arityType].
* To make "no free join points in arityType" work out I had to avoid
trying to use eta-expansion for runRW#. This entailed a few changes
in the Simplifier's treatment of runRW#. See
GHC.Core.Opt.Simplify.Iteration Note [No eta-expansion in runRW#]
* I also made andArityType work correctly with -fpedantic-bottoms;
see Note [Combining case branches: andWithTail].
* Rewrote Note [Combining case branches: optimistic one-shot-ness]
* arityType previously treated join points differently to other
let-bindings. This patch makes them unform; arityType analyses
the RHS of all bindings to get its ArityType, and extends am_sigs.
I realised that, now we have am_sigs giving the ArityType for
let-bound Ids, we don't need the (pre-dating) special code in
arityType for join points. But instead we need to extend the env for
Rec bindings, which weren't doing before. More uniform now. See
Note [arityType for let-bindings].
This meant we could get rid of ae_joins, and in fact get rid of
EtaExpandArity altogether. Simpler.
* And finally, it was the strange treatment of join-point Ids in
arityType (involving a fake ABot type) that led to a serious bug:
#21755. Fixed by this refactoring, which treats them uniformly;
but without breaking #18328.
In fact, the arity for recursive join bindings is pretty tricky;
see the long Note [Arity for recursive join bindings]
in GHC.Core.Opt.Simplify.Utils. That led to more refactoring,
including deciding that an Id could have an Arity that is bigger
than its JoinArity; see Note [Invariants on join points], item
2(b) in GHC.Core
* Make sure that the "demand threshold" for join points in DmdAnal
is no bigger than the join-arity. In GHC.Core.Opt.DmdAnal see
Note [Demand signatures are computed for a threshold arity based on idArity]
* I moved GHC.Core.Utils.exprIsDeadEnd into GHC.Core.Opt.Arity,
where it more properly belongs.
* Remove an old, redundant hack in FloatOut. The old Note was
Note [Bottoming floats: eta expansion] in GHC.Core.Opt.SetLevels.
Compile time improves very slightly on average:
Metrics: compile_time/bytes allocated
---------------------------------------------------------------------------------------
T18223(normal) ghc/alloc 725,808,720 747,839,216 +3.0% BAD
T6048(optasm) ghc/alloc 105,006,104 101,599,472 -3.2% GOOD
geo. mean -0.2%
minimum -3.2%
maximum +3.0%
For some reason Windows was better
T10421(normal) ghc/alloc 125,888,360 124,129,168 -1.4% GOOD
T18140(normal) ghc/alloc 85,974,520 83,884,224 -2.4% GOOD
T18698b(normal) ghc/alloc 236,764,568 234,077,288 -1.1% GOOD
T18923(normal) ghc/alloc 75,660,528 73,994,512 -2.2% GOOD
T6048(optasm) ghc/alloc 112,232,512 108,182,520 -3.6% GOOD
geo. mean -0.6%
I had a quick look at T18223 but it is knee deep in coercions and
the size of everything looks similar before and after. I decided
to accept that 3% increase in exchange for goodness elsewhere.
Metric Decrease:
T10421
T18140
T18698b
T18923
T6048
Metric Increase:
T18223
- - - - -
909edcfc by Ben Gamari at 2022-08-25T10:03:34-04:00
upload_ghc_libs: Add means of passing Hackage credentials
- - - - -
28402eed by M Farkas-Dyck at 2022-08-25T10:04:17-04:00
Scrub some partiality in `CommonBlockElim`.
- - - - -
54affbfa by Ben Gamari at 2022-08-25T20:05:31-04:00
hadrian: Fix whitespace
Previously this region of Settings.Packages was incorrectly indented.
- - - - -
c4bba0f0 by Ben Gamari at 2022-08-25T20:05:31-04:00
validate: Drop --legacy flag
In preparation for removal of the legacy `make`-based build system.
- - - - -
822b0302 by Ben Gamari at 2022-08-25T20:05:31-04:00
gitlab-ci: Drop make build validation jobs
In preparation for removal of the `make`-based build system
- - - - -
6fd9b0a1 by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop make build system
Here we at long last remove the `make`-based build system, it having
been replaced with the Shake-based Hadrian build system. Users are
encouraged to refer to the documentation in `hadrian/doc` and this [1]
blog post for details on using Hadrian.
Closes #17527.
[1] https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html
- - - - -
dbb004b0 by Ben Gamari at 2022-08-25T20:05:31-04:00
Remove testsuite/tests/perf/haddock/.gitignore
As noted in #16802, this is no longer needed.
Closes #16802.
- - - - -
fe9d824d by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop hc-build script
This has not worked for many, many years and relied on the now-removed
`make`-based build system.
- - - - -
659502bc by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop mkdirhier
This is only used by nofib's dead `dist` target
- - - - -
4a426924 by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop mk/{build,install,config}.mk.in
- - - - -
46924b75 by Ben Gamari at 2022-08-25T20:05:31-04:00
compiler: Drop comment references to make
- - - - -
d387f687 by Harry Garrood at 2022-08-25T20:06:10-04:00
Add inits1 and tails1 to Data.List.NonEmpty
See https://github.com/haskell/core-libraries-committee/issues/67
- - - - -
8603c921 by Harry Garrood at 2022-08-25T20:06:10-04:00
Add since annotations and changelog entries
- - - - -
6b47aa1c by Krzysztof Gogolewski at 2022-08-25T20:06:46-04:00
Fix redundant import
This fixes a build error on x86_64-linux-alpine3_12-validate.
See the function 'loadExternalPlugins' defined in this file.
- - - - -
4786acf7 by sheaf at 2022-08-26T15:05:23-04:00
Pmc: consider any 2 dicts of the same type equal
This patch massages the keys used in the `TmOracle` `CoreMap` to ensure
that dictionaries of coherent classes give the same key.
That is, whenever we have an expression we want to insert or lookup in
the `TmOracle` `CoreMap`, we first replace any dictionary
`$dict_abcd :: ct` with a value of the form `error @ct`.
This allows us to common-up view pattern functions with required
constraints whose arguments differed only in the uniques of the
dictionaries they were provided, thus fixing #21662.
This is a rather ad-hoc change to the keys used in the
`TmOracle` `CoreMap`. In the long run, we would probably want to use
a different representation for the keys instead of simply using
`CoreExpr` as-is. This more ambitious plan is outlined in #19272.
Fixes #21662
Updates unix submodule
- - - - -
f5e0f086 by Krzysztof Gogolewski at 2022-08-26T15:06:01-04:00
Remove label style from printing context
Previously, the SDocContext used for code generation contained
information whether the labels should use Asm or C style.
However, at every individual call site, this is known statically.
This removes the parameter to 'PprCode' and replaces every 'pdoc'
used to print a label in code style with 'pprCLabel' or 'pprAsmLabel'.
The OutputableP instance is now used only for dumps.
The output of T15155 changes, it now uses the Asm style
(which is faithful to what actually happens).
- - - - -
1007829b by Cheng Shao at 2022-08-26T15:06:40-04:00
boot: cleanup legacy args
Cleanup legacy boot script args, following removal of the legacy make
build system.
- - - - -
95fe09da by Simon Peyton Jones at 2022-08-27T00:29:02-04:00
Improve SpecConstr for evals
As #21763 showed, we were over-specialising in some cases, when
the function involved was doing a simple 'eval', but not taking
the value apart, or branching on it.
This MR fixes the problem. See Note [Do not specialise evals].
Nofib barely budges, except that spectral/cichelli allocates about
3% less.
Compiler bytes-allocated improves a bit
geo. mean -0.1%
minimum -0.5%
maximum +0.0%
The -0.5% is on T11303b, for what it's worth.
- - - - -
565a8ec8 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Revert "Revert "Refactor SpecConstr to use treat bindings uniformly""
This reverts commit 851d8dd89a7955864b66a3da8b25f1dd88a503f8.
This commit was originally reverted due to an increase in space usage.
This was diagnosed as because the SCE increased in size and that was
being retained by another leak. See #22102
- - - - -
82ce1654 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Avoid retaining bindings via ModGuts held on the stack
It's better to overwrite the bindings fields of the ModGuts before
starting an iteration as then all the old bindings can be collected as
soon as the simplifier has processed them. Otherwise we end up with the
old bindings being alive until right at the end of the simplifier pass
as the mg_binds field is only modified right at the end.
- - - - -
64779dcd by Matthew Pickering at 2022-08-27T00:29:39-04:00
Force imposs_deflt_cons in filterAlts
This fixes a pretty serious space leak as the forced thunk would retain
`Alt b` values which would then contain reference to a lot of old
bindings and other simplifier gunk.
The OtherCon unfolding was not forced on subsequent simplifier runs so
more and more old stuff would be retained until the end of
simplification.
Fixing this has a drastic effect on maximum residency for the mmark
package which goes from
```
45,005,401,056 bytes allocated in the heap
17,227,721,856 bytes copied during GC
818,281,720 bytes maximum residency (33 sample(s))
9,659,144 bytes maximum slop
2245 MiB total memory in use (0 MB lost due to fragmentation)
```
to
```
45,039,453,304 bytes allocated in the heap
13,128,181,400 bytes copied during GC
331,546,608 bytes maximum residency (40 sample(s))
7,471,120 bytes maximum slop
916 MiB total memory in use (0 MB lost due to fragmentation)
```
See #21993 for some more discussion.
- - - - -
a3b23a33 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Use Solo to avoid retaining the SCE but to avoid performing the substitution
The use of Solo here allows us to force the selection into the SCE to obtain
the Subst but without forcing the substitution to be applied. The resulting thunk
is placed into a lazy field which is rarely forced, so forcing it regresses
peformance.
- - - - -
161a6f1f by Simon Peyton Jones at 2022-08-27T00:30:14-04:00
Fix a nasty loop in Tidy
As the remarkably-simple #22112 showed, we were making a black hole
in the unfolding of a self-recursive binding. Boo!
It's a bit tricky. Documented in GHC.Iface.Tidy,
Note [tidyTopUnfolding: avoiding black holes]
- - - - -
68e6786f by Giles Anderson at 2022-08-29T00:01:35+02:00
Use TcRnDiagnostic in GHC.Tc.TyCl.Class (#20117)
The following `TcRnDiagnostic` messages have been introduced:
TcRnIllegalHsigDefaultMethods
TcRnBadGenericMethod
TcRnWarningMinimalDefIncomplete
TcRnDefaultMethodForPragmaLacksBinding
TcRnIgnoreSpecialisePragmaOnDefMethod
TcRnBadMethodErr
TcRnNoExplicitAssocTypeOrDefaultDeclaration
- - - - -
cbe51ac5 by Simon Peyton Jones at 2022-08-29T04:18:57-04:00
Fix a bug in anyInRnEnvR
This bug was a subtle error in anyInRnEnvR, introduced by
commit d4d3fe6e02c0eb2117dbbc9df72ae394edf50f06
Author: Andreas Klebinger <klebinger.andreas at gmx.at>
Date: Sat Jul 9 01:19:52 2022 +0200
Rule matching: Don't compute the FVs if we don't look at them.
The net result was #22028, where a rewrite rule would wrongly
match on a lambda.
The fix to that function is easy.
- - - - -
85579ac5 by Simon Peyton Jones at 2022-08-30T11:18:54+01:00
Improve aggressive specialisation
This patch fixes #21286, by not unboxing dictionaries in
worker/wrapper (ever). The main payload is tiny:
* In `GHC.Core.Opt.DmdAnal.finaliseArgBoxities`, do not unbox
dictionaries in `get_dmd`. See Note [Do not unbox class dictionaries]
in that modules
* I also found that imported wrappers were being fruitlessly
specialised, so I fixed that too, in canSpecImport.
See Note [Specialising imported functions] point (2).
In doing due diligence in the testsuite I fixed a number of
other things:
* Improve Note [Specialising unfoldings] in GHC.Core.Unfold.Make,
and Note [Inline specialisations] in GHC.Core.Opt.Specialise,
and remove duplication between the two. The new Note describes
how we specialise functions with an INLINABLE pragma.
And simplify the defn of `spec_unf` in `GHC.Core.Opt.Specialise.specCalls`.
* Improve Note [Worker/wrapper for INLINABLE functions] in
GHC.Core.Opt.WorkWrap.
And (critially) make an actual change which is to propagate the
user-written pragma from the original function to the wrapper; see
`mkStrWrapperInlinePrag`.
* Write new Note [Specialising imported functions] in
GHC.Core.Opt.Specialise
All this has a big effect on some compile times:
Metrics: compile_time/bytes allocated
--------------------------------------------------------
LargeRecord(normal) -50.2% GOOD
MultiLayerModulesTH_OneShot(normal) +2.0%
T10547(normal) +1.3%
T12545(normal) -3.9%
T13056(optasm) -8.4% GOOD
T14052(ghci) +1.9%
T15164(normal) -3.5% GOOD
T16577(normal) -2.6% GOOD
T18223(normal) -33.4% GOOD
T3064(normal) +1.5%
T8095(normal) +1.3%
T9630(normal) 32.9% GOOD
WWRec(normal) -9.8% GOOD
hie002(normal) +1.8%
geo. mean -1.5%
I diligently investigated all these big drops.
* Caused by not doing w/w for dictionaries:
T13056, T15164, WWRec, T18223
* Caused by not fruitlesslly specialising wrappers
LargeRecord, T9630
Metric Decrease:
LargeRecord
T13056
T15164
T16577
T18223
T9630
WWRec
- - - - -
564b4fda by Simon Peyton Jones at 2022-08-30T11:21:29+01:00
Refactor UnfoldingSource and IfaceUnfolding
- - - - -
30 changed files:
- .gitlab-ci.yml
- .gitlab/ci.sh
- .gitlab/darwin/toolchain.nix
- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- .gitlab/upload_ghc_libs.py
- − MAKEHELP.md
- − Makefile
- − bindisttest/ghc.mk
- boot
- compiler/CodeGen.Platform.h
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Cmm.hs
- compiler/GHC/Cmm/CLabel.hs
- compiler/GHC/Cmm/CommonBlockElim.hs
- compiler/GHC/Cmm/DebugBlock.hs
- compiler/GHC/Cmm/Info/Build.hs
- compiler/GHC/Cmm/Lint.hs
- compiler/GHC/Cmm/Node.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/Sink.hs
- compiler/GHC/CmmToAsm.hs
- compiler/GHC/CmmToAsm/AArch64.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/CmmToAsm/AArch64/Regs.hs
- compiler/GHC/CmmToAsm/Dwarf.hs
- compiler/GHC/CmmToAsm/Dwarf/Types.hs
- compiler/GHC/CmmToAsm/PIC.hs
- compiler/GHC/CmmToAsm/PPC/CodeGen.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a19e0c8c946a2901fbdb23b722e6d07dcd6cf51f...564b4fda0f88e768b70d44c50da692852570bdb8
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a19e0c8c946a2901fbdb23b722e6d07dcd6cf51f...564b4fda0f88e768b70d44c50da692852570bdb8
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20220830/462dce45/attachment-0001.html>
More information about the ghc-commits
mailing list