[Git][ghc/ghc][wip/global-package-db] 75 commits: rel_eng: Update hackage docs upload scripts

Sun Mar 17 11:19:45 UTC 2024

Matthew Pickering pushed to branch wip/global-package-db at Glasgow Haskell Compiler / GHC

Commits:
18ad1077 by Matthew Pickering at 2024-03-05T14:22:31-05:00
rel_eng: Update hackage docs upload scripts

This adds the upload of ghc-internal and ghc-experimental to our scripts
which upload packages to hackage.

- - - - -
bf47c9ba by Matthew Pickering at 2024-03-05T14:22:31-05:00
docs: Remove stray module comment from GHC.Profiling.Eras

- - - - -
37d9b340 by Matthew Pickering at 2024-03-05T14:22:31-05:00
Fix ghc-internal cabal file

The file mentioned some artifacts relating to the base library. I have
renamed these to the new ghc-internal variants.

- - - - -
23f2a478 by Matthew Pickering at 2024-03-05T14:22:31-05:00
Fix haddock source links and hyperlinked source

There were a few issues with the hackage links:

1. We were using the package id rather than the package name for the
   package links. This is fixed by now allowing the template to mention
   %pkg% or %pkgid% and substituing both appropiatly.
2. The `--haddock-base-url` flag is renamed to `--haddock-for-hackage`
   as the new base link works on a local or remote hackage server.
3. The "src" path including too much stuff, so cross-package source
   links were broken as the template was getting double expanded.

Fixes #24086

- - - - -
2fa336a9 by Ben Gamari at 2024-03-05T14:23:07-05:00
filepath: Bump submodule to 1.5.2.0

- - - - -
31217944 by Ben Gamari at 2024-03-05T14:23:07-05:00
os-string: Bump submodule to 2.0.2

- - - - -
4074a3f2 by Matthew Pickering at 2024-03-05T21:44:35-05:00
base: Reflect new era profiling RTS flags in GHC.RTS.Flags

* -he profiling mode
* -he profiling selector
* --automatic-era-increment

CLC proposal #254 - https://github.com/haskell/core-libraries-committee/issues/254

- - - - -
a8c0e31b by Sylvain Henry at 2024-03-05T21:45:14-05:00
JS: faster implementation for some numeric primitives (#23597)

Use faster implementations for the following primitives in the JS
backend by not using JavaScript's BigInt:
- plusInt64
- minusInt64
- minusWord64
- timesWord64
- timesInt64

Co-authored-by: Josh Meredith <joshmeredith2008 at gmail.com>

- - - - -
21e3f325 by Cheng Shao at 2024-03-05T21:45:52-05:00
rts: add -xr option to control two step allocator reserved space size

This patch adds a -xr RTS option to control the size of virtual memory
address space reserved by the two step allocator on a 64-bit platform,
see added documentation for explanation. Closes #24498.

- - - - -
dedcf102 by Cheng Shao at 2024-03-06T13:39:04-05:00
rts: expose HeapAlloc.h as public header

This commit exposes HeapAlloc.h as a public header. The intention is
to expose HEAP_ALLOCED/HEAP_ALLOCED_GC, so they can be used in
assertions in other public headers, and they may also be useful for
user code.

- - - - -
d19441d7 by Cheng Shao at 2024-03-06T13:39:04-05:00
rts: assert pointer is indeed heap allocated in Bdescr()

This commit adds an assertion to Bdescr() to assert the pointer is
indeed heap allocated. This is useful to rule out RTS bugs that
attempt to access non-existent block descriptor of a static closure, #24492
being one such example.

- - - - -
9a656a04 by Ben Gamari at 2024-03-06T13:39:39-05:00
ghc-experimental: Add dummy dependencies to work around #23942

This is a temporary measure to improve CI reliability until a proper
solution is developed.

Works around #23942.

- - - - -
1e84b924 by Simon Peyton Jones at 2024-03-06T13:39:39-05:00
Three compile perf improvements with deep nesting

These were changes are all triggered by #24471.

1. Make GHC.Core.Opt.SetLevels.lvlMFE behave better when there are
   many free variables.  See Note [Large free-variable sets].

2. Make GHC.Core.Opt.Arity.floatIn a bit lazier in its Cost argument.
   This benefits the common case where the ArityType turns out to
   be nullary. See Note [Care with nested expressions]

3. Make GHC.CoreToStg.Prep.cpeArg behave for deeply-nested
   expressions.  See Note [Eta expansion of arguments in CorePrep]
   wrinkle (EA2).

Compile times go down by up to 4.5%, and much more in artificial
cases. (Geo mean of compiler/perf changes is -0.4%.)

Metric Decrease:
    CoOpt_Read
    T10421
    T12425

- - - - -
c4b13113 by Hécate Moonlight at 2024-03-06T13:40:17-05:00
Use "module" instead of "library" when applicable in base haddocks

- - - - -
9cd9efb4 by Vladislav Zavialov at 2024-03-07T13:01:54+03:00
Rephrase error message to say "visible arguments" (#24318)

* Main change: make the error message generated by mkFunTysMsg more
  accurate by changing "value arguments" to "visible arguments".

* Refactor: define a new type synonym VisArity and use it instead of
  Arity in a few places.

It might be the case that there other places in the compiler that should
talk about visible arguments rather than value arguments, but I haven't
tried to find them all, focusing only on the error message reported in
the ticket.

- - - - -
d523a6a7 by Ben Gamari at 2024-03-07T19:40:45-05:00
Bump array submodule

- - - - -
7e55003c by Ben Gamari at 2024-03-07T19:40:45-05:00
Bump stm submodule

- - - - -
32d337ef by Ben Gamari at 2024-03-07T19:40:45-05:00
Introduce exception context

Here we introduce the `ExceptionContext` type and `ExceptionAnnotation`
class, allowing dynamically-typed user-defined annotations to be
attached to exceptions.

CLC Proposal: https://github.com/haskell/core-libraries-committee/issues/199
GHC Proposal: https://github.com/ghc-proposals/ghc-proposals/pull/330

- - - - -
39f3d922 by Ben Gamari at 2024-03-07T19:40:46-05:00
testsuite/interface-stability: Update documentation

- - - - -
fdea7ada by Ben Gamari at 2024-03-07T19:40:46-05:00
ghc-internal: comment formatting

- - - - -
4fba42ef by Ben Gamari at 2024-03-07T19:40:46-05:00
compiler: Default and warn ExceptionContext constraints

- - - - -
3886a205 by Ben Gamari at 2024-03-07T19:40:46-05:00
base: Introduce exception backtraces

Here we introduce the `Backtraces` type and associated machinery for
attaching these via `ExceptionContext`. These has a few compile-time
regressions (`T15703` and `T9872d`) due to the additional dependencies
in the exception machinery.

As well, there is a surprisingly large regression in the
`size_hello_artifact` test. This appears to be due to various `Integer` and
`Read` bits now being reachable at link-time. I believe it should be
possible to avoid this but I have accepted the change for now to get the
feature merged.

CLC Proposal: https://github.com/haskell/core-libraries-committee/issues/199
GHC Proposal: https://github.com/ghc-proposals/ghc-proposals/pull/330

Metric Increase:
    T15703
    T9872d
    size_hello_artifact

- - - - -
18c5409f by Ben Gamari at 2024-03-07T19:40:46-05:00
users guide: Release notes for exception backtrace work

- - - - -
f849c5fc by Ben Gamari at 2024-03-07T19:40:46-05:00
compiler: Don't show ExceptionContext of GhcExceptions

Most GhcExceptions are user-facing errors and therefore the
ExceptionContext has little value. Ideally we would enable
it in the DEBUG compiler but I am leaving this for future work.

- - - - -
dc646e6f by Sylvain Henry at 2024-03-07T19:40:46-05:00
Disable T9930fail for the JS target (cf #19174)

- - - - -
bfc09760 by Alan Zimmerman at 2024-03-07T19:41:22-05:00
Update showAstData to honour blanking of AnnParen

Also tweak rendering of SrcSpan to remove extra blank line.

- - - - -
50454a29 by Ben Gamari at 2024-03-08T03:32:42-05:00
ghc-internal: Eliminate GHC.Internal.Data.Kind

This was simply reexporting things from `ghc-prim`. Instead reexport
these directly from `Data.Kind`. Also add build ordering dependency to
work around #23942.

- - - - -
38a4b6ab by Ben Gamari at 2024-03-08T03:33:18-05:00
rts: Fix SET_HDR initialization of retainer set

This fixes a regression in retainer set profiling introduced by
b0293f78cb6acf2540389e22bdda420d0ab874da. Prior to that commit
the heap traversal word would be initialized by `SET_HDR` using
`LDV_RECORD_CREATE`. However, the commit added a `doingLDVProfiling`
check in `LDV_RECORD_CREATE`, meaning that this initialization no longer
happened.

Given that this initialization was awkwardly indirectly anyways, I have
fixed this by explicitly initializating the heap traversal word to
`NULL` in `SET_PROF_HDR`. This is equivalent to the previous behavior,
but much more direct.

Fixes #24513.

- - - - -
2859a637 by Ben Gamari at 2024-03-08T18:26:47-05:00
base: Use strerror_r instead of strerror

As noted by #24344, `strerror` is not necessarily thread-safe.
Thankfully, POSIX.1-2001 has long offered `strerror_r`, which is
safe to use.

Fixes #24344.

CLC discussion: https://github.com/haskell/core-libraries-committee/issues/249

- - - - -
edb9bf77 by Jade at 2024-03-09T03:39:38-05:00
Error messages: Improve Error messages for Data constructors in type signatures.

This patch improves the error messages from invalid type signatures by
trying to guess what the user did and suggesting an appropriate fix.

Partially fixes: #17879

- - - - -
cfb197e3 by Patrick at 2024-03-09T03:40:15-05:00
HieAst: add module name #24493

The main purpose of this is to tuck the module name `xxx` in `module xxx where` into the hieAst.
It should fix #24493.

The following have been done:
1. Renamed and update the `tcg_doc_hdr :: Maybe (LHsDoc GhcRn)` to `tcg_hdr_info :: (Maybe (LHsDoc GhcRn), Maybe (XRec GhcRn ModuleName))`
   To store the located module name information.
2. update the `RenamedSource` and `RenamedStuff` with extra `Maybe (XRec GhcRn ModuleName)` located module name information.
3. add test `testsuite/tests/hiefile/should_compile/T24493.hs` to ensure the module name is added and update several relevent tests.
4. accompanied submodule haddoc test update MR in https://gitlab.haskell.org/ghc/haddock/-/merge_requests/53

- - - - -
2341d81e by Vaibhav Sagar at 2024-03-09T03:40:54-05:00
GHC.Utils.Binary: fix a couple of typos

- - - - -
5580e1bd by Ben Gamari at 2024-03-09T03:41:30-05:00
rts: Drop .wasm suffix from .prof file names

This replicates the behavior on Windows, where `Hi.exe` will produce
profiling output named `Hi.prof` instead of `Hi.exe.prof`.

While in the area I also fixed the extension-stripping logic, which
incorrectly rewrote `Hi.exefoo` to `Hi.foo`.

Closes #24515.

- - - - -
259495ee by Cheng Shao at 2024-03-09T03:41:30-05:00
testsuite: drop exe extension from .hp & .prof filenames

See #24515 for details.

- - - - -
c477a8d2 by Ben Gamari at 2024-03-09T03:42:05-05:00
rts/linker: Enable GOT support on all platforms

There is nothing platform-dependent about our GOT implementation and
GOT support is needed by `T24171` on i386.

- - - - -
2e592857 by Vladislav Zavialov at 2024-03-09T03:42:41-05:00
Drop outdated comment on TcRnIllformedTypePattern

This should have been done in 0f0c53a501b but I missed it.

- - - - -
c554b4da by Ben Gamari at 2024-03-09T09:39:20-05:00
rts/CloneStack: Bounds check array write

- - - - -
15c590a5 by Ben Gamari at 2024-03-09T09:39:20-05:00
rts/CloneStack: Don't expose helper functions in header

- - - - -
e831ce31 by Ben Gamari at 2024-03-09T09:39:20-05:00
base: Move internals of GHC.InfoProv into GHC.InfoProv.Types

Such that we can add new helpers into GHC.InfoProv.Types without
breakage.

- - - - -
6948e24d by Ben Gamari at 2024-03-09T09:39:20-05:00
rts: Lazily decode IPE tables

Previously we would eagerly allocate `InfoTableEnt`s for each
info table registered in the info table provenance map. However, this
costs considerable memory and initialization time. Instead we now
lazily decode these tables. This allows us to use one-third the memory
*and* opens the door to taking advantage of sharing opportunities within
a module.

This required considerable reworking since lookupIPE now must be passed
its result buffer.

- - - - -
9204a04e by Ben Gamari at 2024-03-09T09:39:20-05:00
rts/IPE: Don't expose helper in header

- - - - -
308926ff by Ben Gamari at 2024-03-09T09:39:20-05:00
rts/IPE: Share module_name within a Node

This allows us to shave a 64-bit word off of the packed IPE entry size.

- - - - -
bebdea05 by Ben Gamari at 2024-03-09T09:39:20-05:00
IPE: Expose unit ID in InfoTableProv

Here we add the unit ID to the info table provenance structure.

- - - - -
6519c9ad by Ben Gamari at 2024-03-09T09:39:35-05:00
rts: Refactor GHC.Stack.CloneStack.decode

Don't allocate a Ptr constructor per frame.

- - - - -
ed0b69dc by Ben Gamari at 2024-03-09T09:39:35-05:00
base: Do not expose whereFrom# from GHC.Exts

- - - - -
2b1faea9 by Vladislav Zavialov at 2024-03-09T17:38:21-05:00
docs: Update info on TypeAbstractions

* Mention TypeAbstractions in 9.10.1-notes.rst
* Set the status to "Experimental".
* Add a "Since: GHC 9.x" comment to each section.

- - - - -
f8b88918 by Ben Gamari at 2024-03-09T21:21:46-05:00
ci-images: Bump Alpine image to bootstrap with 9.8.2

- - - - -
705e6927 by Ben Gamari at 2024-03-09T21:21:46-05:00
testsuite: Mark T24171 as fragile due to #24512

I will fix this but not in time for 9.10.1-alpha1

- - - - -
c74196e1 by Ben Gamari at 2024-03-09T21:21:46-05:00
testsuite: Mark linker_unload_native as fragile

In particular this fails on platforms without `dlinfo`. I plan to
address this but not before 9.10.1-alpha1.

- - - - -
f4d87f7a by Ben Gamari at 2024-03-09T21:21:46-05:00
configure: Bump version to 9.10

- - - - -
88df9a5f by Ben Gamari at 2024-03-09T21:21:46-05:00
Bump transformers submodule to 0.6.1.1

- - - - -
8176d5e8 by Ben Gamari at 2024-03-09T21:21:46-05:00
testsuite: Increase ulimit for T18623

1 MByte was just too tight and failed intermittently on some platforms
(e.g. CentOS 7). Bumping the limit to 8 MByte should provide sufficient
headroom.

Fixes #23139.

- - - - -
c74b38a3 by Ben Gamari at 2024-03-09T21:21:46-05:00
base: Bump version to 4.20.0.0

- - - - -
b2937fc3 by Ben Gamari at 2024-03-09T21:21:46-05:00
ghc-internal: Set initial version at 9.1001.0

This provides PVP compliance while maintaining a clear correspondence
between GHC releases and `ghc-internal` versions.

- - - - -
4ae7d868 by Ben Gamari at 2024-03-09T21:21:46-05:00
ghc-prim: Bump version to 0.11.0

- - - - -
50798dc6 by Ben Gamari at 2024-03-09T21:21:46-05:00
template-haskell: Bump version to 2.22.0.0

- - - - -
8564f976 by Ben Gamari at 2024-03-09T21:21:46-05:00
base-exports: Accommodate spurious whitespace changes in 32-bit output

It appears that this was

- - - - -
9d4f0e98 by Ben Gamari at 2024-03-09T21:21:46-05:00
users-guide: Move exception backtrace relnotes to 9.10

This was previously mistakenly added to the GHC 9.8 release notes.

- - - - -
145eae60 by Ben Gamari at 2024-03-09T21:21:46-05:00
gitlab/rel_eng: Fix name of Rocky8 artifact

- - - - -
39c2a630 by Ben Gamari at 2024-03-09T21:21:46-05:00
gitlab/rel_eng: Fix path of generate_jobs_metadata

- - - - -
aed034de by Ben Gamari at 2024-03-09T21:21:46-05:00
gitlab/upload: Rework recompression

The old `combine` approach was quite fragile due to use of filename
globbing. Moreover, it didn't parallelize well. This refactoring
makes the goal more obvious, parallelizes better, and is more robust.

- - - - -
dc207d06 by Ben Gamari at 2024-03-10T08:56:08-04:00
configure: Bump GHC version to 9.11

Bumps haddock submodule.

- - - - -
8b2513e8 by Ben Gamari at 2024-03-11T01:20:03-04:00
rts/linker: Don't unload code when profiling is enabled

The heap census may contain references (e.g. `Counter.identity`) to
static data which must be available when the census is reported at the
end of execution.

Fixes #24512.

- - - - -
7810b4c3 by Ben Gamari at 2024-03-11T01:20:03-04:00
rts/linker: Don't unload native objects when dlinfo isn't available

To do so is unsafe as we have no way of identifying references to
symbols provided by the object.

Fixes #24513. Fixes #23993.

- - - - -
0590764c by Ben Gamari at 2024-03-11T01:20:39-04:00
rel_eng/upload: Purge both $rel_name/ and $ver/

This is necessary for prereleases, where GHCup accesses the release via
`$ver/`

- - - - -
b85a4631 by Brandon Chinn at 2024-03-12T19:25:56-04:00
Remove duplicate code normalising slashes

- - - - -
c91946f9 by Brandon Chinn at 2024-03-12T19:25:56-04:00
Simplify regexes with raw strings

- - - - -
1a5f53c6 by Brandon Chinn at 2024-03-12T19:25:57-04:00
Don't normalize backslashes in characters

- - - - -
7ea971d3 by Andrei Borzenkov at 2024-03-12T19:26:32-04:00
Fix compiler crash caused by implicit RHS quantification in type synonyms (#24470)

- - - - -
39f3ac3e by Cheng Shao at 2024-03-12T19:27:11-04:00
Revert "compiler: make genSym use C-based atomic increment on non-JS 32-bit platforms"

This reverts commit 615eb855416ce536e02ed935ecc5a6f25519ae16. It was
originally intended to fix #24449, but it was merely sweeping the bug
under the rug. 3836a110577b5c9343915fd96c1b2c64217e0082 has properly
fixed the fragile test, and we no longer need the C version of genSym.
Furthermore, the C implementation causes trouble when compiling with
clang that targets i386 due to alignment warning and libatomic linking
issue, so it makes sense to revert it.

- - - - -
e6bfb85c by Cheng Shao at 2024-03-12T19:27:11-04:00
compiler: fix out-of-bound memory access of genSym on 32-bit

This commit fixes an unnoticed out-of-bound memory access of genSym on
32-bit. ghc_unique_inc is 32-bit sized/aligned on 32-bit platforms,
but we mistakenly treat it as a Word64 pointer in genSym, and
therefore will accidentally load 2 garbage higher bytes, or with a
small but non-zero chance, overwrite something else in the data
section depends on how the linker places the data segments. This
regression was introduced in !11802 and fixed here.

- - - - -
77171cd1 by Ben Orchard at 2024-03-14T09:00:40-04:00
Note mutability of array and address access primops

Without an understanding of immutable vs. mutable memory, the index
primop family have a potentially non-intuitive type signature:

    indexOffAddr :: Addr# -> Int# -> a
    readOffAddr  :: Addr# -> Int# -> State# d -> (# State# d, a #)

indexOffAddr# might seem like a free generality improvement, which it
certainly is not!

This change adds a brief note on mutability expectations for most
index/read/write access primops.

- - - - -
7da7f8f6 by Alan Zimmerman at 2024-03-14T09:01:15-04:00
EPA: Fix regression discarding comments in contexts

Closes #24533

- - - - -
382ac566 by Matthew Pickering at 2024-03-17T11:19:39+00:00
Read global package database from settings file

Before this patch, the global package database was always assumed to be
in libdir </> package.conf.d.

This causes issues in GHC's build system because there are sometimes
situations where the package database you need to use is not located in
the same place as the settings file.

* The stage1 compiler needs to use stage1 libraries, so we should set
  "Global Package DB" for the stage1 compiler to the stage1 package
  database.
* Stage 2 cross compilers need to use stage2 libraries, so likewise, we
  should set the package database path to `_build/stage2/lib/`

* The normal situation is where the stage2 compiler uses stage1
  libraries. Then everything lines up.

* When installing we have rearranged everything so that the settings
  file and package database line up properly, so then everything should
  continue to work as before. In this case we set the relative package
  db path to `package.conf.d`, so it resolves the same as before.

* ghc-pkg needs to be modified as well to look in the settings file fo
  the package database rather than assuming the global package database
  location relative to the lib folder.

* Cabal/cabal-install will work correctly because they query the global
  package database using `--print-global-package-db`.

A reasonable question is why not generate the "right" settings files in
the right places in GHC's build system. In order to do this you would
need to engineer wrappers for all executables to point to a specific
libdir. There are also situations where the same package db is used by
two different compilers with two different settings files (think stage2
cross compiler and stage3 compiler).

In short, this 10 line patch allows for some reasonable simplifications
in Hadrian at very little cost to anything else.

Fixes #24502

- - - - -
bc8d1842 by Matthew Pickering at 2024-03-17T11:19:39+00:00
hadrian: Remove stage1 testsuite wrappers logic

Now instead of producing wrappers which pass the global package database
argument to ghc and ghc-pkg, we write the location of the correct
package database into the settings file so you can just use the intree
compiler directly.

- - - - -

30 changed files:

- .gitlab-ci.yml
- .gitlab/generate-ci/gen_ci.hs
- .gitlab/jobs.yaml
- .gitlab/rel_eng/default.nix
- .gitlab/rel_eng/fetch-gitlab-artifacts/fetch_gitlab.py
- .gitlab/rel_eng/mk-ghcup-metadata/README.mkd
- + .gitlab/rel_eng/recompress-all
- .gitlab/rel_eng/upload.sh
- .gitlab/rel_eng/upload_ghc_libs.py
- compiler/GHC.hs
- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/SetLevels.hs
- compiler/GHC/Core/Predicate.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Dump.hs
- compiler/GHC/HsToCore/Docs.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Parser/Annotation.hs
- compiler/GHC/Parser/Errors/Ppr.hs
- compiler/GHC/Parser/Errors/Types.hs
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Settings/IO.hs
- compiler/GHC/StgToCmm/InfoTableProv.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/HsType.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8e3cb48ce42cfd6ea83528ae70df486e011bb017...bc8d184209e1e9767a7229d7f5ad048f5a4f22ab

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8e3cb48ce42cfd6ea83528ae70df486e011bb017...bc8d184209e1e9767a7229d7f5ad048f5a4f22ab
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240317/efc34145/attachment-0001.html>