[Git][ghc/ghc][wip/inplace-final] 31 commits: Fix arityType: -fpedantic-bottoms, join points, etc

Matthew Pickering (@mpickering) gitlab at gitlab.haskell.org
Mon Aug 29 10:22:25 UTC 2022



Matthew Pickering pushed to branch wip/inplace-final at Glasgow Haskell Compiler / GHC


Commits:
a90298cc by Simon Peyton Jones at 2022-08-25T08:38:16+01:00
Fix arityType: -fpedantic-bottoms, join points, etc

This MR fixes #21694, #21755.  It also makes sure that #21948 and
fix to #21694.

* For #21694 the underlying problem was that we were calling arityType
  on an expression that had free join points.  This is a Bad Bad Idea.
  See Note [No free join points in arityType].

* To make "no free join points in arityType" work out I had to avoid
  trying to use eta-expansion for runRW#. This entailed a few changes
  in the Simplifier's treatment of runRW#.  See
  GHC.Core.Opt.Simplify.Iteration Note [No eta-expansion in runRW#]

* I also made andArityType work correctly with -fpedantic-bottoms;
  see Note [Combining case branches: andWithTail].

* Rewrote Note [Combining case branches: optimistic one-shot-ness]

* arityType previously treated join points differently to other
  let-bindings. This patch makes them unform; arityType analyses
  the RHS of all bindings to get its ArityType, and extends am_sigs.

  I realised that, now we have am_sigs giving the ArityType for
  let-bound Ids, we don't need the (pre-dating) special code in
  arityType for join points. But instead we need to extend the env for
  Rec bindings, which weren't doing before.  More uniform now.  See
  Note [arityType for let-bindings].

  This meant we could get rid of ae_joins, and in fact get rid of
  EtaExpandArity altogether.  Simpler.

* And finally, it was the strange treatment of join-point Ids in
  arityType (involving a fake ABot type) that led to a serious bug:
  #21755.  Fixed by this refactoring, which treats them uniformly;
  but without breaking #18328.

  In fact, the arity for recursive join bindings is pretty tricky;
  see the long Note [Arity for recursive join bindings]
  in GHC.Core.Opt.Simplify.Utils.  That led to more refactoring,
  including deciding that an Id could have an Arity that is bigger
  than its JoinArity; see Note [Invariants on join points], item
  2(b) in GHC.Core

* Make sure that the "demand threshold" for join points in DmdAnal
  is no bigger than the join-arity.  In GHC.Core.Opt.DmdAnal see
  Note [Demand signatures are computed for a threshold arity based on idArity]

* I moved GHC.Core.Utils.exprIsDeadEnd into GHC.Core.Opt.Arity,
  where it more properly belongs.

* Remove an old, redundant hack in FloatOut.  The old Note was
  Note [Bottoming floats: eta expansion] in GHC.Core.Opt.SetLevels.

Compile time improves very slightly on average:

Metrics: compile_time/bytes allocated
---------------------------------------------------------------------------------------
  T18223(normal) ghc/alloc    725,808,720    747,839,216  +3.0%  BAD
  T6048(optasm)  ghc/alloc    105,006,104    101,599,472  -3.2% GOOD
  geo. mean                                          -0.2%
  minimum                                            -3.2%
  maximum                                            +3.0%

For some reason Windows was better

   T10421(normal) ghc/alloc    125,888,360    124,129,168  -1.4% GOOD
   T18140(normal) ghc/alloc     85,974,520     83,884,224  -2.4% GOOD
  T18698b(normal) ghc/alloc    236,764,568    234,077,288  -1.1% GOOD
   T18923(normal) ghc/alloc     75,660,528     73,994,512  -2.2% GOOD
    T6048(optasm) ghc/alloc    112,232,512    108,182,520  -3.6% GOOD
  geo. mean                                          -0.6%

I had a quick look at T18223 but it is knee deep in coercions and
the size of everything looks similar before and after.  I decided
to accept that 3% increase in exchange for goodness elsewhere.

Metric Decrease:
    T10421
    T18140
    T18698b
    T18923
    T6048

Metric Increase:
    T18223

- - - - -
909edcfc by Ben Gamari at 2022-08-25T10:03:34-04:00
upload_ghc_libs: Add means of passing Hackage credentials

- - - - -
28402eed by M Farkas-Dyck at 2022-08-25T10:04:17-04:00
Scrub some partiality in `CommonBlockElim`.

- - - - -
54affbfa by Ben Gamari at 2022-08-25T20:05:31-04:00
hadrian: Fix whitespace

Previously this region of Settings.Packages was incorrectly indented.

- - - - -
c4bba0f0 by Ben Gamari at 2022-08-25T20:05:31-04:00
validate: Drop --legacy flag

In preparation for removal of the legacy `make`-based build system.

- - - - -
822b0302 by Ben Gamari at 2022-08-25T20:05:31-04:00
gitlab-ci: Drop make build validation jobs

In preparation for removal of the `make`-based build system

- - - - -
6fd9b0a1 by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop make build system

Here we at long last remove the `make`-based build system, it having
been replaced with the Shake-based Hadrian build system. Users are
encouraged to refer to the documentation in `hadrian/doc` and this [1]
blog post for details on using Hadrian.

Closes #17527.

[1] https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html

- - - - -
dbb004b0 by Ben Gamari at 2022-08-25T20:05:31-04:00
Remove testsuite/tests/perf/haddock/.gitignore

As noted in #16802, this is no longer needed.

Closes #16802.

- - - - -
fe9d824d by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop hc-build script

This has not worked for many, many years and relied on the now-removed
`make`-based build system.

- - - - -
659502bc by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop mkdirhier

This is only used by nofib's dead `dist` target

- - - - -
4a426924 by Ben Gamari at 2022-08-25T20:05:31-04:00
Drop mk/{build,install,config}.mk.in

- - - - -
46924b75 by Ben Gamari at 2022-08-25T20:05:31-04:00
compiler: Drop comment references to make

- - - - -
d387f687 by Harry Garrood at 2022-08-25T20:06:10-04:00
Add inits1 and tails1 to Data.List.NonEmpty

See https://github.com/haskell/core-libraries-committee/issues/67

- - - - -
8603c921 by Harry Garrood at 2022-08-25T20:06:10-04:00
Add since annotations and changelog entries

- - - - -
6b47aa1c by Krzysztof Gogolewski at 2022-08-25T20:06:46-04:00
Fix redundant import

This fixes a build error on x86_64-linux-alpine3_12-validate.
See the function 'loadExternalPlugins' defined in this file.

- - - - -
4786acf7 by sheaf at 2022-08-26T15:05:23-04:00
Pmc: consider any 2 dicts of the same type equal

This patch massages the keys used in the `TmOracle` `CoreMap` to ensure
that dictionaries of coherent classes give the same key.
That is, whenever we have an expression we want to insert or lookup in
the `TmOracle` `CoreMap`, we first replace any dictionary
`$dict_abcd :: ct` with a value of the form `error @ct`.

This allows us to common-up view pattern functions with required
constraints whose arguments differed only in the uniques of the
dictionaries they were provided, thus fixing #21662.

This is a rather ad-hoc change to the keys used in the
`TmOracle` `CoreMap`. In the long run, we would probably want to use
a different representation for the keys instead of simply using
`CoreExpr` as-is. This more ambitious plan is outlined in #19272.

Fixes #21662
Updates unix submodule

- - - - -
f5e0f086 by Krzysztof Gogolewski at 2022-08-26T15:06:01-04:00
Remove label style from printing context

Previously, the SDocContext used for code generation contained
information whether the labels should use Asm or C style.
However, at every individual call site, this is known statically.
This removes the parameter to 'PprCode' and replaces every 'pdoc'
used to print a label in code style with 'pprCLabel' or 'pprAsmLabel'.
The OutputableP instance is now used only for dumps.

The output of T15155 changes, it now uses the Asm style
(which is faithful to what actually happens).

- - - - -
1007829b by Cheng Shao at 2022-08-26T15:06:40-04:00
boot: cleanup legacy args

Cleanup legacy boot script args, following removal of the legacy make
build system.

- - - - -
95fe09da by Simon Peyton Jones at 2022-08-27T00:29:02-04:00
Improve SpecConstr for evals

As #21763 showed, we were over-specialising in some cases, when
the function involved was doing a simple 'eval', but not taking
the value apart, or branching on it.

This MR fixes the problem.  See Note [Do not specialise evals].

Nofib barely budges, except that spectral/cichelli allocates about
3% less.

Compiler bytes-allocated improves a bit
   geo. mean                                          -0.1%
   minimum                                            -0.5%
   maximum                                            +0.0%

The -0.5% is on T11303b, for what it's worth.

- - - - -
565a8ec8 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Revert "Revert "Refactor SpecConstr to use treat bindings uniformly""

This reverts commit 851d8dd89a7955864b66a3da8b25f1dd88a503f8.

This commit was originally reverted due to an increase in space usage.
This was diagnosed as because the SCE increased in size and that was
being retained by another leak. See #22102

- - - - -
82ce1654 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Avoid retaining bindings via ModGuts held on the stack

It's better to overwrite the bindings fields of the ModGuts before
starting an iteration as then all the old bindings can be collected as
soon as the simplifier has processed them. Otherwise we end up with the
old bindings being alive until right at the end of the simplifier pass
as the mg_binds field is only modified right at the end.

- - - - -
64779dcd by Matthew Pickering at 2022-08-27T00:29:39-04:00
Force imposs_deflt_cons in filterAlts

This fixes a pretty serious space leak as the forced thunk would retain
`Alt b` values which would then contain reference to a lot of old
bindings and other simplifier gunk.

The OtherCon unfolding was not forced on subsequent simplifier runs so
more and more old stuff would be retained until the end of
simplification.

Fixing this has a drastic effect on maximum residency for the mmark
package which goes from

```
  45,005,401,056 bytes allocated in the heap
  17,227,721,856 bytes copied during GC
     818,281,720 bytes maximum residency (33 sample(s))
       9,659,144 bytes maximum slop
            2245 MiB total memory in use (0 MB lost due to fragmentation)
```

to

```
  45,039,453,304 bytes allocated in the heap
  13,128,181,400 bytes copied during GC
     331,546,608 bytes maximum residency (40 sample(s))
       7,471,120 bytes maximum slop
             916 MiB total memory in use (0 MB lost due to fragmentation)
```

See #21993 for some more discussion.

- - - - -
a3b23a33 by Matthew Pickering at 2022-08-27T00:29:39-04:00
Use Solo to avoid retaining the SCE but to avoid performing the substitution

The use of Solo here allows us to force the selection into the SCE to obtain
the Subst but without forcing the substitution to be applied. The resulting thunk
is placed into a lazy field which is rarely forced, so forcing it regresses
peformance.

- - - - -
161a6f1f by Simon Peyton Jones at 2022-08-27T00:30:14-04:00
Fix a nasty loop in Tidy

As the remarkably-simple #22112 showed, we were making a black hole
in the unfolding of a self-recursive binding.  Boo!

It's a bit tricky.  Documented in GHC.Iface.Tidy,
   Note [tidyTopUnfolding: avoiding black holes]

- - - - -
68e6786f by Giles Anderson at 2022-08-29T00:01:35+02:00
Use TcRnDiagnostic in GHC.Tc.TyCl.Class (#20117)

The following `TcRnDiagnostic` messages have been introduced:

TcRnIllegalHsigDefaultMethods
TcRnBadGenericMethod
TcRnWarningMinimalDefIncomplete
TcRnDefaultMethodForPragmaLacksBinding
TcRnIgnoreSpecialisePragmaOnDefMethod
TcRnBadMethodErr
TcRnNoExplicitAssocTypeOrDefaultDeclaration

- - - - -
cbe51ac5 by Simon Peyton Jones at 2022-08-29T04:18:57-04:00
Fix a bug in anyInRnEnvR

This bug was a subtle error in anyInRnEnvR, introduced by

    commit d4d3fe6e02c0eb2117dbbc9df72ae394edf50f06
    Author: Andreas Klebinger <klebinger.andreas at gmx.at>
    Date:   Sat Jul 9 01:19:52 2022 +0200

    Rule matching: Don't compute the FVs if we don't look at them.

The net result was #22028, where a rewrite rule would wrongly
match on a lambda.

The fix to that function is easy.

- - - - -
299b0653 by Matthew Pickering at 2022-08-29T11:22:16+01:00
hadrian: Use a stamp file to record when a package is built in a certain way

Before this patch which library ways we had built wasn't recorded
directly. So you would run into issues if you build the .conf file with
some library ways before switching the library ways which you wanted to
build.

Now there is one stamp file for each way, so in order to build a
specific way you can need that specific stamp file rather than going
indirectly via the .conf file.

- - - - -
f523b6ff by Matthew Pickering at 2022-08-29T11:22:16+01:00
hadrian: Inplace/Final package databases

There are now two different package databases per stage. An inplace
package database contains .conf files which point directly into the
build directories. The final package database contains .conf files which
point into the installed locations. The inplace .conf files are created
before any building happens and have fake ABI hash values. The final
.conf files are created after a package finished building and contains
the proper ABI has.

The motivation for this is to make the dependency structure more
fine-grained when building modules. Now a module depends just depends
directly on M.o from package p rather than the .conf file depend on the
.conf file for package p. So when all of a modules direct dependencies
have finished building we can start building it rather than waiting for
the whole package to finish.

The secondary motivation is that the multi-repl doesn't need to build
everything before starting the multi-repl session. We can just configure
the inplace package-db and use that in order to start the repl.

- - - - -
b4cacdde by Matthew Pickering at 2022-08-29T11:22:16+01:00
hadrian: Add some more packages to multi-cradle

The main improvement here is to pass `-this-unit-id` for executables so
that they can be added to the multi-cradle if desired as well as normal
library packages.

- - - - -
2609d718 by Matthew Pickering at 2022-08-29T11:22:16+01:00
hadrian: Need builders needed by Cabal Configure in parallel

Because of the use of withStaged (which needs the necessary builder)
when configuring a package, the builds of stage1:exe:ghc-bin and
stage1:exe:ghc-pkg where being linearised when building a specific
target like `binary-dist-dir`.

Thankfully the fix is quite local, to supply all the `withStaged`
arguments together so the needs can be batched together and hence
performed in parallel.

Fixes #22093

- - - - -
1e42c17d by Matthew Pickering at 2022-08-29T11:22:16+01:00
Remove stage1:exe:ghc-bin pre-build from CI script

CI builds stage1:exe:ghc-bin before the binary-dist target which
introduces some quite bad linearisation (see #22093) because we don't
build stage1 compiler in parallel with anything. Then when the
binary-dist target is started we have to build stage1:exe:ghc-pkg before
doing anything.

Fixes #22094

- - - - -


30 changed files:

- .gitlab-ci.yml
- .gitlab/ci.sh
- .gitlab/upload_ghc_libs.py
- − MAKEHELP.md
- − Makefile
- − bindisttest/ghc.mk
- boot
- compiler/GHC/Cmm/CLabel.hs
- compiler/GHC/Cmm/CommonBlockElim.hs
- compiler/GHC/Cmm/DebugBlock.hs
- compiler/GHC/Cmm/Lint.hs
- compiler/GHC/Cmm/Node.hs
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/CmmToAsm.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/CmmToAsm/Dwarf.hs
- compiler/GHC/CmmToAsm/Dwarf/Types.hs
- compiler/GHC/CmmToAsm/PIC.hs
- compiler/GHC/CmmToAsm/PPC/Ppr.hs
- compiler/GHC/CmmToAsm/Ppr.hs
- compiler/GHC/CmmToAsm/X86/Ppr.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Core.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/DmdAnal.hs
- compiler/GHC/Core/Opt/FloatOut.hs
- compiler/GHC/Core/Opt/SetLevels.hs
- compiler/GHC/Core/Opt/Simplify.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/094ed61a860e5f5821cac164fae8a29fc74819d3...1e42c17d642b56d34b5b1221dea914a9a9f4fa14

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/094ed61a860e5f5821cac164fae8a29fc74819d3...1e42c17d642b56d34b5b1221dea914a9a9f4fa14
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20220829/840610f0/attachment-0001.html>


More information about the ghc-commits mailing list