[Git][ghc/ghc][wip/expand-do] 19 commits: Restructure IPE buffer layout

Thu Jun 1 16:39:38 UTC 2023

Apoorv Ingle pushed to branch wip/expand-do at Glasgow Haskell Compiler / GHC

Commits:
f3556d6c by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Restructure IPE buffer layout

Reference ticket #21766

This commit restructures IPE buffer list entries to not contain
references to their corresponding info tables. IPE buffer list nodes now
point to two lists of equal length, one holding the list of info table
pointers and one holding the corresponding entries for each info table.
This will allow the entry data to be compressed without losing the
references to the info tables.

- - - - -
5d1f2411 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Add IPE compression to configure

Reference ticket #21766

Adds an `--enable-ipe-data-compreesion` flag to the configure script
which will check for libzstd and set the appropriate flags to allow for
IPE data compression in the compiler

- - - - -
b7a640ac by Finley McIlwaine at 2023-06-01T04:53:12-04:00
IPE data compression

Reference ticket #21766

When IPE data compression is enabled, compress the emitted IPE buffer
entries and decompress them in the RTS.

- - - - -
5aef5658 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Fix libzstd detection in configure and RTS

Ensure that `HAVE_LIBZSTD` gets defined to either 0 or 1 in all cases
and properly check that before IPE data decompression in the RTS. See
ticket #21766.

- - - - -
69563c97 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Add note describing IPE data compression

See ticket #21766

- - - - -
7872e2b6 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Fix byte order of IPE data, fix IPE tests

Make sure byte order of written IPE buffer entries matches target.

Make sure the IPE-related tests properly access the fields of IPE buffer
entry nodes with the new IPE layout.

This commit also introduces checks to avoid importing modules if IPE
compression is not enabled.

See ticket #21766.

- - - - -
0e85099b by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Fix IPE data decompression buffer allocation

Capacity of buffers allocated for decompressed IPE data was
incorrect due to a misuse of the `ZSTD_findFrameCompressedSize`
function. Fix by always storing decompressed size of IPE data in IPE
buffer list nodes and using `ZSTD_findFrameCompressedSize` to determine
the size of the compressed data.

See ticket #21766

- - - - -
a0048866 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Add optional dependencies to ./configure output

Changes the configure script to indicate whether libnuma, libzstd, or
libdw are being used as dependencies due to their optional features
being enabled.

- - - - -
09d93bd0 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Add IPE-enabled builds to CI

- Adds an IPE job to the CI pipeline which is triggered by the ~IPE label
- Introduces CI logic to enable IPE data compression
- Enables uncompressed IPE data on debug CI job
- Regenerates jobs.yaml

MR https://gitlab.haskell.org/ghc/ci-images/-/merge_requests/112 on the
images repository is meant to ensure that the proper images have
libzstd-dev installed.

- - - - -
3ded9a1c by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Update user's guide and release notes, small fixes

Add mention of IPE data compression to user's guide and the release
notes for 9.8.1. Also note the impact compression has on binary size in
both places.

Change IpeBufferListNode compression check so only the value `1`
indicates compression.

See ticket #21766

- - - - -
41b41577 by Finley McIlwaine at 2023-06-01T04:53:12-04:00
Remove IPE enabled builds from CI

We don't need to explicitly specify the +ipe transformer to test IPE data
since there are tests which manually enable IPE information. This commit does
leave zstd IPE data compression enabled on the debian CI jobs.

- - - - -
982bef3a by Krzysztof Gogolewski at 2023-06-01T04:53:49-04:00
Fix build with 9.2

GHC.Tc.Zonk.Type uses an equality constraint.
ghc.nix currently provides 9.2.

- - - - -
1c96bc3d by Krzysztof Gogolewski at 2023-06-01T10:56:11-04:00
Output Lint errors to stderr instead of stdout

This is a continuation of 7b095b99, which fixed warnings but not errors.

Refs #13342

- - - - -
8e81f140 by sheaf at 2023-06-01T10:56:51-04:00
Refactor lookupExactOrOrig & friends

This refactors the panoply of renamer lookup functions relating to
lookupExactOrOrig to more graciously handle Exact and Orig names.

In particular, we avoid the situation in which we would add Exact/Orig
GREs to the tcg_used_gres field, which could cause a panic in bestImport
like in #23240.

Fixes #23428

- - - - -
5d415bfd by Krzysztof Gogolewski at 2023-06-01T10:57:31-04:00
Use the one-shot trick for UM and RewriteM functors

As described in Note [The one-shot state monad trick],
we shouldn't use derived Functor instances for monads using
one-shot. This was done for most of them, but UM and RewriteM
were missed.

- - - - -
2c38551e by Krzysztof Gogolewski at 2023-06-01T10:58:08-04:00
Fix testsuite skipping Lint

setTestOpts() is used to modify the test options for an entire .T file,
rather than a single test.
If there was a test using collect_compiler_stats, all of the tests
in the same file had lint disabled.

Fixes #21247

- - - - -
00a1e50b by Krzysztof Gogolewski at 2023-06-01T10:58:44-04:00
Add testcases for already fixed #16432

They were fixed by 40c7daed0.

Fixes #16432

- - - - -
7d14d9be by Apoorv Ingle at 2023-06-01T11:39:05-05:00
Fixes #18324 #23147
Expands do notation before typechecking using `HsExpansion`
- Adds testcases T18324, T18324b, DoubleMatch
Expands
- Do statements
- Monadic do statements
- monadic fix blocks
- make sure fail is used for pattern match failures in bind statments
- Makes sure unused binds generate warnings
- runs the pattern match check in generated lambda exprs to avoid getting suprious pattern match failures. c.f. pmcheck/should_compile/DoubleMatch.hs
- PopSrcSpan in HsExpr to pop error context
- Discards default monad fail alternatives that are spuriously generated
- Make sure we check for generated loc span for checking if the (>>) is user written or expanded for /do/ purposes

- - - - -
4a53ccf6 by Apoorv Ingle at 2023-06-01T11:39:06-05:00
- Add PopSrcSpan (XXExprGhcRn) in appropriate places while expanding statements
- correct source spans displayed for warnDiscardedDoBindings
- use `mkExpandStmt` to store original stmts along with expanded expr for using the right context for error message printing
- improves error messages for applicative do
- remove special case from isMatchContextPmChecked (long distance info is now properly propogated)
- set correct src spans to statement expansions
- Match ctxt while type checking HsLam is different if the lambda match is due to an expression generated from a do block
- call tcExpr and not tcApp in PopSrcSpan so make sure impredicativity works fine
- look into XExprs in tcInferAppHead_maybe for infering the type to make T18324 typecheck and run

- - - - -

30 changed files:

- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Unify.hs
- compiler/GHC/Driver/CodeOutput.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/Hs/Instances.hs
- compiler/GHC/Hs/Pat.hs
- compiler/GHC/Hs/Utils.hs
- compiler/GHC/HsToCore/Arrows.hs
- compiler/GHC/HsToCore/Binds.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/HsToCore/Match.hs
- compiler/GHC/HsToCore/Match/Constructor.hs
- compiler/GHC/HsToCore/Monad.hs
- compiler/GHC/HsToCore/Pmc.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Rename/Env.hs
- compiler/GHC/Rename/Expr.hs
- compiler/GHC/Rename/Module.hs
- compiler/GHC/Rename/Names.hs
- compiler/GHC/Rename/Unbound.hs
- compiler/GHC/Rename/Utils.hs
- compiler/GHC/Stg/Lint.hs
- compiler/GHC/StgToCmm/InfoTableProv.hs
- compiler/GHC/Tc/Deriv/Generate.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/App.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/be6ecf94873df437026961022a31ec7068fabfc5...4a53ccf61d2ba5856ee1b4ca8ef108baef977c3a

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/be6ecf94873df437026961022a31ec7068fabfc5...4a53ccf61d2ba5856ee1b4ca8ef108baef977c3a
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230601/fe38596c/attachment-0001.html>