[Git][ghc/ghc][wip/hadrian-windows-bindist] 37 commits: users-guide: Note that GHC2021 doesn't include ExplicitNamespaces

Thu Aug 17 10:47:50 UTC 2023

Matthew Pickering pushed to branch wip/hadrian-windows-bindist at Glasgow Haskell Compiler / GHC

Commits:
e194ed2b by Ben Gamari at 2023-08-15T00:58:09-04:00
users-guide: Note that GHC2021 doesn't include ExplicitNamespaces

As noted in #23801.

- - - - -
d814bda9 by Ben Gamari at 2023-08-15T05:43:53-04:00
users-guide: Support both distutils and packaging

As noted in #23818, some old distributions (e.g. Debian 9) only include
`distutils` while newer distributions only include `packaging`.

Fixes #23818.

- - - - -
1726db3f by Ben Gamari at 2023-08-15T05:43:53-04:00
users-guide: Ensure extlinks is compatible with Sphinx <4

The semantics of the `extlinks` attribute annoyingly changed in Sphinx
4. Reflect this in our configuration. See #22690.

Fixes #23807.

- - - - -
173338cf by Matthew Pickering at 2023-08-15T22:00:24-04:00
ci: Run full-ci on master and release branches

Fixes #23737

- - - - -
bdab6898 by Bodigrim at 2023-08-15T22:01:03-04:00
Add @since pragmas for Data.Ord.clamp and GHC.Float.clamp

- - - - -
662d351b by Matthew Pickering at 2023-08-16T09:35:04-04:00
ghc-toolchain: Match CPP args with configure script

At the moment we need ghc-toolchain to precisely match the output as
provided by the normal configure script. The normal configure script
(FP_HSCPP_CMD_WITH_ARGS) branches on whether we are using clang or gcc
so we match that logic exactly in ghc-toolchain.

The old implementation (which checks if certain flags are supported) is
better but for now we have to match to catch any potential errors in the
configuration.

Ticket: #23720

- - - - -
09c6759e by Matthew Pickering at 2023-08-16T09:35:04-04:00
configure: Fix `-Wl,--no-as-needed` check

The check was failing because the args supplied by $$1 were quoted which
failed because then the C compiler thought they were an input file.

Fixes #23720

- - - - -
2129678b by Matthew Pickering at 2023-08-16T09:35:04-04:00
configure: Add flag which turns ghc-toolchain check into error

We want to catch these errors in CI, but first we need to a flag which
turns this check into an error.

- - - - -
6e2aa8e0 by Matthew Pickering at 2023-08-16T09:35:04-04:00
ci: Enable --enable-strict-ghc-toolchain-check for all CI jobs

This will cause any CI job to fail if we have a mismatch between what
ghc-toolchain reports and what ./configure natively reports. Fixing
these kinds of issues is highest priority for 9.10 release.

- - - - -
12d39e24 by Rodrigo Mesquita at 2023-08-16T09:35:04-04:00
Pass user-specified options to ghc-toolchain

The current user interface to configuring target toolchains is `./configure`.

In !9263 we added a new tool to configure target toolchains called
`ghc-toolchain`, but the blessed way of creating these toolchains is
still through configure.

However, we were not passing the user-specified options given with the
`./configure` invocation to the ghc-toolchain tool.

This commit remedies that by storing the user options and environment
variables in USER_* variables, which then get passed to GHC-toolchain.

The exception to the rule is the windows bundled toolchain, which
overrides the USER_* variables with whatever flags the windows bundled
toolchain requires to work.
We consider the bundled toolchain to be effectively the user specifying
options, since the actual user delegated that configuration work.

Closes #23678

- - - - -
f7b3c3a0 by Rodrigo Mesquita at 2023-08-16T09:35:04-04:00
ghc-toolchain: Parse javascript and ghcjs as a Arch and OS

- - - - -
8a0ae4ee by Rodrigo Mesquita at 2023-08-16T09:35:04-04:00
ghc-toolchain: Fix ranlib option

- - - - -
31e9ec96 by Rodrigo Mesquita at 2023-08-16T09:35:04-04:00
Check Link Works with -Werror

- - - - -
bc1998b3 by Matthew Pickering at 2023-08-16T09:35:04-04:00
Only check for no_compact_unwind support on darwin

While writing ghc-toolchain we noticed that the
FP_PROG_LD_NO_COMPACT_UNWIND check is subtly wrong. Specifically, we
pass -Wl,-no_compact_unwind to cc. However, ld.gold interprets this as
-n o_compact_unwind, which is a valid argument.

Fixes #23676

- - - - -
0283f36e by Matthew Pickering at 2023-08-16T09:35:04-04:00
Add some javascript special cases to ghc-toolchain

On javascript there isn't a choice of toolchain but some of the
configure checks were not accurately providing the correct answer.

1. The linker was reported as gnu LD because the --version output
   mentioned gnu LD.
2. The --target flag makes no sense on javascript but it was just
   ignored by the linker, so we add a special case to stop ghc-toolchain
   thinking that emcc supports --target when used as a linker.

- - - - -
a48ec5f8 by Matthew Pickering at 2023-08-16T09:35:04-04:00
check for emcc in gnu_LD check

- - - - -
50df2e69 by Matthew Pickering at 2023-08-16T09:35:04-04:00
Add ldOverrideWhitelist to only default to ldOverride on windows/linux

On some platforms - ie darwin, javascript etc we really do not want to
allow the user to use any linker other than the default one as this
leads to all kinds of bugs. Therefore it is a bit more prudant to add a
whitelist which specifies on which platforms it might be possible to use
a different linker.

- - - - -
a669a39c by Matthew Pickering at 2023-08-16T09:35:04-04:00
Fix plaform glob in FPTOOLS_SET_C_LD_FLAGS

A normal triple may look like

x86_64-unknown-linux

but when cross-compiling you get $target set to a quad such as..

aarch64-unknown-linux-gnu

Which should also match this check.

- - - - -
c52b6769 by Matthew Pickering at 2023-08-16T09:35:04-04:00
ghc-toolchain: Pass ld-override onto ghc-toolchain

- - - - -
039b484f by Matthew Pickering at 2023-08-16T09:35:04-04:00
ld override: Make whitelist override user given option

- - - - -
d2b63cbc by Matthew Pickering at 2023-08-16T09:35:05-04:00
ghc-toolchain: Add format mode to normalise differences before diffing.

The "format" mode takes an "--input" and "--ouput" target file and
formats it.

This is intended to be useful on windows where the
configure/ghc-toolchain target files can't be diffed very easily because
the path separators are different.

- - - - -
f2b39e4a by Matthew Pickering at 2023-08-16T09:35:05-04:00
ghc-toolchain: Bump ci-images commit to get new ghc-wasm-meta

We needed to remove -Wno-unused-command-line-argument from the arguments
passed in order for the configure check to report correctly.

See https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10976#note_516335

- - - - -
92103830 by Matthew Pickering at 2023-08-16T09:35:05-04:00
configure: MergeObjsCmd - distinguish between empty string and unset variable

If `MergeObjsCmd` is explicitly set to the empty string then we should
assume that MergeObjs is just not supported.

This is especially important for windows where we set MergeObjsCmd to ""
in m4/fp_setup_windows_toolchain.m4.

- - - - -
3500bb2c by Matthew Pickering at 2023-08-16T09:35:05-04:00
configure: Add proper check to see if object merging works

- - - - -
08c9a014 by Matthew Pickering at 2023-08-16T09:35:05-04:00
ghc-toolchain: If MergeObjsCmd is not set, replace setting with Nothing

If the user explicitly chooses to not set a MergeObjsCmd then it is
correct to use Nothing for tgtMergeObjs field in the Target file.

- - - - -
c9071d94 by Matthew Pickering at 2023-08-16T09:35:05-04:00
HsCppArgs: Augment the HsCppOptions

This is important when we pass -I when setting up the windows toolchain.

- - - - -
294a6d80 by Matthew Pickering at 2023-08-16T09:35:05-04:00
Set USER_CPP_ARGS when setting up windows toolchain

- - - - -
bde4b5d4 by Rodrigo Mesquita at 2023-08-16T09:35:05-04:00
Improve handling of Cc as a fallback

- - - - -
f4c1c3a3 by Rodrigo Mesquita at 2023-08-16T09:35:05-04:00
ghc-toolchain: Configure Cpp and HsCpp correctly when user specifies flags

In ghc-toolchain, we were only /not/ configuring required flags when the
user specified any flags at all for the  of the HsCpp and Cpp tools.

Otherwise, the linker takes into consideration the user specified flags
to determine whether to search for a better linker implementation, but
already configured the remaining GHC and platform-specific flags
regardless of the user options.

Other Tools consider the user options as a baseline for further
configuration (see `findProgram`), so #23689 is not applicable.

Closes #23689

- - - - -
bfe4ffac by Matthew Pickering at 2023-08-16T09:35:05-04:00
CPP_ARGS: Put new options after user specified options

This matches up with the behaviour of ghc-toolchain, so that the output
of both matches.

- - - - -
a6828173 by Gergő Érdi at 2023-08-16T09:35:41-04:00
If a defaulting plugin made progress, re-zonk wanteds before built-in defaulting

Fixes #23821.

- - - - -
0fc4424f by Matthew Pickering at 2023-08-17T11:46:05+01:00
hadrian: Uniformly pass buildOptions to all builders in runBuilder

In Builder.hs, runBuilderWith mostly ignores the buildOptions in BuildInfo.

This leads to hard to diagnose bugs as any build options you pass with
runBuilderWithCmdOptions are ignored for many builders.

Solution: Uniformly pass buildOptions to the invocation of cmd.

Fixes #23845

- - - - -
3273cdbc by Matthew Pickering at 2023-08-17T11:46:05+01:00
Abstract windows toolchain setup

This commit splits up the windows toolchain setup logic into two
functions.

* FP_INSTALL_WINDOWS_TOOLCHAIN - deals with downloading the toolchain if
  it isn't already downloaded
* FP_SETUP_WINDOWS_TOOLCHAIN - sets the environment variables to point
  to the correct place

FP_SETUP_WINDOWS_TOOLCHAIN is abstracted from the location of the mingw
toolchain and also the eventual location where we will install the
toolchain in the installed bindist.

This is the first step towards #23608

- - - - -
40ae0441 by Matthew Pickering at 2023-08-17T11:46:05+01:00
Generate build.mk for bindists

The config.mk.in script was relying on some variables which were
supposed to be set by build.mk but therefore never were when used to
install a bindist.

Specifically

* BUILD_PROF_LIBS to determine whether we had profiled libraries or not
* DYNAMIC_GHC_PROGRAMS to determine whether we had shared libraries or
  not

Not only were these never set but also not really accurate because you
could have shared libaries but still statically linked ghc executable.

In addition variables like GhcLibWays were just never used, so those
have been deleted from the script.

Now instead we generate a build.mk file which just directly specifies
which RtsWays we have supplied in the bindist and whether we have
DYNAMIC_GHC_PROGRAMS.

- - - - -
79c14f97 by Matthew Pickering at 2023-08-17T11:46:05+01:00
hadrian: Add reloc-binary-dist-* targets

This adds a command line option to build a "relocatable" bindist.

The bindist is created by first creating a normal bindist and then
installing it using the `RelocatableBuild=YES` option. This creates a
bindist without any wrapper scripts pointing to the libdir.

The motivation for this feature is that we want to ship relocatable
bindists on windows and this method is more uniform than the ad-hoc
method which lead to bugs such as #23608 and #23476

The relocatable bindist can be built with the "reloc-binary-dist" target
and supports the same suffixes as the normal "binary-dist" command to
specify the compression style.

- - - - -
364128f3 by Matthew Pickering at 2023-08-17T11:46:07+01:00
packaging: Fix installation scripts on windows/RelocatableBuild case

This includes quite a lot of small fixes which fix the installation
makefile to work on windows properly. This also required fixing the
RelocatableBuild variable which seemed to have been broken for a long
while.

Sam helped me a lot writing this patch by providing a windows machine to
test the changes. Without him it would have taken ages to tweak
everything.

Co-authored-by: sheaf <sam.derbyshire at gmail.com>

- - - - -
c37bb3f2 by Matthew Pickering at 2023-08-17T11:47:22+01:00
ci: Build relocatable bindist on windows

We now build the relocatable bindist target on windows, which means we
test and distribute the new method of creating a relocatable bindist.

- - - - -

30 changed files:

- .gitlab-ci.yml
- .gitlab/ci.sh
- .gitlab/generate-ci/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC/Tc/Solver.hs
- configure.ac
- distrib/configure.ac.in
- docs/users_guide/exts/control.rst
- docs/users_guide/flags.py
- docs/users_guide/ghc_config.py.in
- docs/users_guide/utils.py
- hadrian/bindist/Makefile
- hadrian/bindist/config.mk.in
- hadrian/cfg/default.target.in
- hadrian/src/Builder.hs
- hadrian/src/Hadrian/Builder/Ar.hs
- hadrian/src/Rules/BinaryDist.hs
- libraries/base/Data/Ord.hs
- libraries/base/GHC/Float.hs
- m4/find_merge_objects.m4
- m4/fp_cpp_cmd_with_args.m4
- m4/fp_hs_cpp_cmd_with_args.m4
- m4/fp_link_supports_no_as_needed.m4
- m4/fp_prog_ld_is_gnu.m4
- m4/fp_prog_ld_no_compact_unwind.m4
- m4/fp_settings.m4
- m4/fp_setup_windows_toolchain.m4
- m4/fptools_set_c_ld_flags.m4
- m4/ghc_toolchain.m4
- m4/prep_target_file.m4

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/13995dd998fa5a6f2fbfc1cdc9231bbf6e986a5d...c37bb3f2b4faeaa6d14a135c40348b0fb4c62010

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/13995dd998fa5a6f2fbfc1cdc9231bbf6e986a5d...c37bb3f2b4faeaa6d14a135c40348b0fb4c62010
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230817/34e3cb43/attachment-0001.html>