[Git][ghc/ghc][wip/T24634-oneshot-bytecode] 42 commits: configure: Set LD_STAGE0 appropiately when 9.10.1 is used as a boot compiler

Thu Jul 18 01:28:58 UTC 2024

Cheng Shao pushed to branch wip/T24634-oneshot-bytecode at Glasgow Haskell Compiler / GHC

Commits:
564981bd by Matthew Pickering at 2024-07-05T07:35:29-04:00
configure: Set LD_STAGE0 appropiately when 9.10.1 is used as a boot compiler

In 9.10.1 the "ld command" has been removed, so we fall back to using
the more precise "merge objects command" when it's available as
LD_STAGE0 is only used to set the object merging command in hadrian.

Fixes #24949

- - - - -
a949c792 by Matthew Pickering at 2024-07-05T07:35:29-04:00
hadrian: Don't build ghci object files for ./hadrian/ghci target

There is some convoluted logic which determines whether we build ghci
object files are not. In any case, if you set `ghcDynPrograms = pure
False` then it forces them to be built.

Given we aren't ever building executables with this flavour it's fine
to leave `ghcDynPrograms` as the default and it should be a bit faster
to build less.

Also fixes #24949

- - - - -
48bd8f8e by Matthew Pickering at 2024-07-05T07:36:06-04:00
hadrian: Remove STG dump from ticky_ghc flavour transformer

This adds 10-15 minutes to build time, it is a better strategy to
precisely enable dumps for the modules which show up prominently in a
ticky profile.

Given I am one of the only people regularly building ticky compilers I
think it's worthwhile to remove these.

Fixes #23635

- - - - -
5b1aefb7 by Matthew Pickering at 2024-07-05T07:36:06-04:00
hadrian: Add dump_stg flavour transformer

This allows you to write `--flavour=default+ticky_ghc+dump_stg` if you
really want STG for all modules.

- - - - -
ab2b60b6 by Sven Tennie at 2024-07-08T15:03:41-04:00
AArch64: Simplify stmtToInstrs type

There's no need to hand `Nothing`s around... (there was no case with a
`BlockId`.)

- - - - -
71a7fa8c by Sven Tennie at 2024-07-08T15:03:41-04:00
AArch64: Simplify stmtsToInstrs type

The `BlockId` parameter (`bid`) is never used, only handed around.
Deleting it simplifies the surrounding code.

- - - - -
8bf6fd68 by Simon Peyton Jones at 2024-07-08T15:04:17-04:00
Fix eta-expansion in Prep

As #25033 showed, we were eta-expanding in a way that broke a join point,
which messed up Note [CorePrep invariants].

The fix is rather easy.  See Wrinkle (EA1) of
Note [Eta expansion of arguments in CorePrep]

- - - - -
96acf823 by Sjoerd Visscher at 2024-07-09T06:16:14-04:00
One-shot Haddock

- - - - -
74ec4c06 by Sjoerd Visscher at 2024-07-09T06:16:14-04:00
Remove haddock-stdout test option

Superseded by output handling of Hadrian

- - - - -
ed8a8f0b by Rodrigo Mesquita at 2024-07-09T06:16:51-04:00
ghc-boot: Relax Cabal bound

Fixes #25013

- - - - -
3f9548fe by Matthew Pickering at 2024-07-09T06:17:36-04:00
ci: Unset ALEX/HAPPY variables when testing bootstrap jobs

Ticket #24826 reports a regression in 9.10.1 when building from a source
distribution. This patch is an attempt to reproduce the issue on CI by
more aggressively removing `alex` and `happy` from the environment.

- - - - -
aba2c9d4 by Andrea Bedini at 2024-07-09T06:17:36-04:00
hadrian: Ignore build-tool-depends fields in cabal files

hadrian does not utilise the build-tool-depends fields in cabal files
and their presence can cause issues when building source distribution
(see #24826)

Ideally Cabal would support building "full" source distributions which
would remove the need for workarounds in hadrian but for now we can
patch the build-tool-depends out of the cabal files.

Fixes #24826

- - - - -
12bb9e7b by Matthew Pickering at 2024-07-09T06:18:12-04:00
testsuite: Don't attempt to link when checking whether a way is supported

It is sufficient to check that the simple test file compiles as it will
fail if there are not the relevant library files for the requested way.

If you break a way so badly that even a simple executable fails to link
(as I did for profiled dynamic way), it will just mean the tests for
that way are skipped on CI rather than displayed.

- - - - -
46ec0a8e by Torsten Schmits at 2024-07-09T13:37:02+02:00
Improve docs for NondecreasingIndentation

The text stated that this affects indentation of layouts nested in do
expressions, while it actually affects that of do layouts nested in any
other.

- - - - -
dddc9dff by Zubin Duggal at 2024-07-12T11:41:24-04:00
compiler: Fingerprint -fwrite-if-simplified-core

We need to recompile if this flag is changed because later modules might depend on the
simplified core for this module if -fprefer-bytecode is enabled.

Fixes #24656

- - - - -
145a6477 by Matthew Pickering at 2024-07-12T11:42:00-04:00
Add support for building profiled dynamic way

The main payload of this change is to hadrian.

* Default settings will produced dynamic profiled objects
* `-fexternal-interpreter` is turned on in some situations when there is
  an incompatibility between host GHC and the way attempting to be
  built.
* Very few changes actually needed to GHC

There are also necessary changes to the bootstrap plans to work with the
vendored Cabal dependency. These changes should ideally be reverted by
the next GHC release.

In hadrian support is added for building profiled dynamic libraries
(nothing too exciting to see there)

Updates hadrian to use a vendored Cabal submodule, it is important that
we replace this usage with a released version of Cabal library before
the 9.12 release.

Fixes #21594

-------------------------
Metric Increase:
    libdir
-------------------------

- - - - -
414a6950 by Matthew Pickering at 2024-07-12T11:42:00-04:00
testsuite: Make find_so regex more precise

The hash contains lowercase [a-z0-9] and crucially not _p which meant we
sometimes matched on `libHS.._p` profiled shared libraries rather than
the normal shared library.

- - - - -
dee035bf by Alex Mason at 2024-07-12T11:42:41-04:00
ncg(aarch64): Add fsqrt instruction, byteSwap primitives [#24956]

Implements the FSQRT machop using native assembly rather than a C call.

Implements MO_BSwap by producing assembly to do the byte swapping
instead of producing a foreign call a C function.

In `tar`, the hot loop for `deserialise` got almost 4x faster by
avoiding the foreign call which caused spilling live variables to the
stack -- this means the loop did 4x more memory read/writing than
necessary in that particular case!

- - - - -
5104ee61 by Sylvain Henry at 2024-07-12T11:43:23-04:00
Linker: use m32 allocator for sections when NEED_PLT (#24432)

Use M32 allocator to avoid fragmentation when allocating ELF sections.
We already did this when NEED_PLT was undefined. Failing to do this led
to relocations impossible to fulfil (#24432).

- - - - -
52d66984 by Sylvain Henry at 2024-07-12T11:43:23-04:00
RTS: allow M32 allocation outside of 4GB range when assuming -fPIC

- - - - -
c34fef56 by Sylvain Henry at 2024-07-12T11:43:23-04:00
Linker: fix stub offset

Remove unjustified +8 offset that leads to memory corruption (cf
discussion in #24432).

- - - - -
280e4bf5 by Simon Peyton Jones at 2024-07-12T11:43:59-04:00
Make type-equality on synonyms a bit faster

This MR make equality fast for (S tys1 `eqType` S tys2),
where S is a non-forgetful type synonym.

It doesn't affect compile-time allocation much, but then comparison doesn't
allocate anyway.  But it seems like a Good Thing anyway.

See Note [Comparing type synonyms] in GHC.Core.TyCo.Compare
and Note [Forgetful type synonyms] in GHC.Core.TyCon

Addresses #25009.

- - - - -
cb83c347 by Alan Zimmerman at 2024-07-12T11:44:35-04:00
EPA: Bring back SrcSpan in EpaDelta

When processing files in ghc-exactprint, the usual workflow is to
first normalise it with makeDeltaAst, and then operate on it.

But we need the original locations to operate on it, in terms of
finding things.

So restore the original SrcSpan for reference in EpaDelta

- - - - -
7bcda869 by Matthew Pickering at 2024-07-12T11:45:11-04:00
Update alpine release job to 3.20

alpine 3.20 was recently released and uses a new python and sphinx
toolchain which could be useful to test.

- - - - -
43aa99b8 by Matthew Pickering at 2024-07-12T11:45:11-04:00
testsuite: workaround bug in python-3.12

There is some unexplained change to binding behaviour in python-3.12
which requires moving this import from the top-level into the scope of
the function.

I didn't feel any particular desire to do a deep investigation as to why
this changed as the code works when modified like this. No one in the
python IRC channel seemed to know what the problem was.

- - - - -
e3914028 by Adam Sandberg Ericsson at 2024-07-12T11:45:47-04:00
initialise mmap_32bit_base during RTS startup #24847
- - - - -
86b8ecee by Hécate Kleidukos at 2024-07-12T11:46:27-04:00
haddock: Only fetch supported languages and extensions once per Interface list

This reduces the number of operations done on each Interface, because
supported languages and extensions are determined from architecture and
operating system of the build host. This information remains stable
across Interfaces, and as such doesn not need to be recovered for each
Interface.

- - - - -
4f85366f by sheaf at 2024-07-13T05:58:14-04:00
Testsuite: use py-cpuinfo to compute CPU features

This replaces the rather hacky logic we had in place for checking
CPU features. In particular, this means that feature availability now
works properly on Windows.

- - - - -
41f1354d by Matthew Pickering at 2024-07-13T05:58:51-04:00
testsuite: Replace $CC with $TEST_CC

The TEST_CC variable should be set based on the test compiler, which may
be different to the compiler which is set to CC on your system (for
example when cross compiling).

Fixes #24946

- - - - -
572fbc44 by sheaf at 2024-07-15T08:30:32-04:00
isIrrefutableHsPat: consider COMPLETE pragmas

This patch ensures we taken into account COMPLETE pragmas when we
compute whether a pattern is irrefutable. In particular, if a pattern
synonym is the sole member of a COMPLETE pragma (without a result TyCon),
then we consider a pattern match on that pattern synonym to be irrefutable.

This affects the desugaring of do blocks, as it ensures we don't use
a "fail" operation.

Fixes #15681 #16618 #22004

- - - - -
84dadea9 by Zubin Duggal at 2024-07-15T08:31:09-04:00
haddock: Handle non-hs files, so that haddock can generate documentation for modules with
foreign imports and template haskell.

Fixes #24964

- - - - -
0b4ff9fa by Zubin Duggal at 2024-07-15T12:12:30-04:00
haddock: Keep track of warnings/deprecations from dependent packages in `InstalledInterface`
and use this to propagate these on items re-exported from dependent packages.

Fixes #25037

- - - - -
b8b4b212 by Zubin Duggal at 2024-07-15T12:12:30-04:00
haddock: Keep track of instance source locations in `InstalledInterface` and use this to add
source locations on out of package instances

Fixes #24929

- - - - -
559a7a7c by Matthew Pickering at 2024-07-15T12:13:05-04:00
ci: Refactor job_groups definition, split up by platform

The groups are now split up so it's easier to see which jobs are
generated for each platform

No change in behaviour, just refactoring.

- - - - -
20383006 by Matthew Pickering at 2024-07-16T11:48:25+01:00
ci: Replace debian 10 with debian 12 on validation jobs

Since debian 10 is now EOL we migrate onwards to debian 12 as the basis
for most platform independent validation jobs.

- - - - -
12d3b66c by Matthew Pickering at 2024-07-17T13:22:37-04:00
ghcup-metadata: Fix use of arch argument

The arch argument was ignored when making the jobname, which lead to
failures when generating metadata for the alpine_3_18-aarch64 bindist.

Fixes #25089

- - - - -
3db39b77 by Cheng Shao at 2024-07-18T01:26:02+00:00
compiler: always generate dynamic stub objects when generating bytecode

When foreign stub objects are compiled during bytecode generation
(e.g. when using whole core bindings), ensure they are dynamic so they
can be properly loaded.

- - - - -
074cc029 by Cheng Shao at 2024-07-18T01:26:02+00:00
compiler: generate stub objects before writing iface

This commit promotes foreign stub compilation a bit earlier before we
actually write iface, to make it possible to actually serialize them
into the final iface when compiling with
`-fwrite-if-simplified-core`. Also ensures dynamic stub objects are
generated if we're compiling with `-fwrite-if-simplified-core`.

- - - - -
9219540a by Cheng Shao at 2024-07-18T01:26:02+00:00
compiler: add mi_stub_objs to ModIface

This commit adds the mi_stub_objs field to ModIface, representing
serialized foreign stub dynamic objects that can be reloaded later
when compiling a downstream module with `-fprefer-byte-code` that
loads whole core bindings from an upstream module that requires
foreign stubs, see #24634 for such an example.

- - - - -
83d4537a by Cheng Shao at 2024-07-18T01:26:02+00:00
compiler: serialize foreign stub dynamic objects in interface file when compiling with -fwrite-if-simplified-core

This commit makes the compiler serialize foreign stub dynamic objects
in mi_stub_objs when compiling with -fwrite-if-simplified-core. This
is needed when we compile a downstream module with -fprefer-byte-code,
in which case we want to make use of whole core bindings in upstream
modules instead of having to load their dynamic objects, but the
foreign stub information has been lost across different ghc
invocations previously. Hence the fix in this commit. See #24634 for
such an example.

- - - - -
4d5dfcc5 by Cheng Shao at 2024-07-18T01:26:02+00:00
testsuite: fix T24634

This commit fixes T24634 and marks it as unbroken.

- - - - -
a5321003 by Cheng Shao at 2024-07-18T01:26:02+00:00
WIP: oneshot bytecode support

- - - - -

27 changed files:

- .gitlab-ci.yml
- .gitlab/ci.sh
- .gitlab/generate-ci/gen_ci.hs
- .gitlab/jobs.yaml
- .gitlab/rel_eng/fetch-gitlab-artifacts/fetch_gitlab.py
- .gitlab/rel_eng/mk-ghcup-metadata/mk_ghcup_metadata.py
- .gitmodules
- compiler/GHC.hs
- compiler/GHC/CmmToAsm/AArch64/CodeGen.hs
- compiler/GHC/CmmToAsm/AArch64/Instr.hs
- compiler/GHC/CmmToAsm/AArch64/Ppr.hs
- compiler/GHC/Core/ConLike.hs
- compiler/GHC/Core/Map/Type.hs
- compiler/GHC/Core/TyCo/Compare.hs
- compiler/GHC/Core/TyCon.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/Core/Unify.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Main.hs
- + compiler/GHC/Driver/Main.hs-boot
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Dump.hs
- compiler/GHC/Hs/Pat.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/29df022f7ea42494da0c771611be181aa4310d4c...a532100318e9902f3e2405f6e525e9ee3e834be0

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/29df022f7ea42494da0c771611be181aa4310d4c...a532100318e9902f3e2405f6e525e9ee3e834be0
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240717/f69aeb54/attachment-0001.html>