[Git][ghc/ghc][wip/sand-witch/check- at -binders] 31 commits: JS: handle stored null StablePtr

Wed Feb 14 09:03:10 UTC 2024

Andrei Borzenkov pushed to branch wip/sand-witch/check- at -binders at Glasgow Haskell Compiler / GHC

Commits:
df9fd9f7 by Sylvain Henry at 2024-02-12T12:18:42-05:00
JS: handle stored null StablePtr

Some Haskell codes unsafely cast StablePtr into ptr to compare against
NULL. E.g. in direct-sqlite:

  if castStablePtrToPtr aggStPtr /= nullPtr then

where `aggStPtr` is read (`peek`) from zeroed memory initially.

We fix this by giving these StablePtr the same representation as other
null pointers. It's safe because StablePtr at offset 0 is unused (for
this exact reason).

- - - - -
55346ede by Sylvain Henry at 2024-02-12T12:18:42-05:00
JS: disable MergeObjsMode test

This isn't implemented for JS backend objects.

- - - - -
aef587f6 by Sylvain Henry at 2024-02-12T12:18:42-05:00
JS: add support for linking C sources

Support linking C sources with JS output of the JavaScript backend.
See the added documentation in the users guide.

The implementation simply extends the JS linker to use the objects (.o)
that were already produced by the emcc compiler and which were filtered
out previously. I've also added some options to control the link with C
functions (see the documentation about pragmas).

With this change I've successfully compiled the direct-sqlite package
which embeds the sqlite.c database code. Some wrappers are still
required (see the documentation about wrappers) but everything generic
enough to be reused for other libraries have been integrated into
rts/js/mem.js.

- - - - -
b71b392f by Sylvain Henry at 2024-02-12T12:18:42-05:00
JS: avoid EMCC logging spurious failure

emcc would sometime output messages like:

  cache:INFO: generating system asset: symbol_lists/424b44514e43d789148e69e4e7d1c7fdc0350b79.json... (this will be cached in "/emsdk/upstream/emscripten/cache/symbol_lists/424b44514e43d789148e69e4e7d1c7fdc0350b79.json" for subsequent builds)
  cache:INFO:  - ok

Cf https://github.com/emscripten-core/emscripten/issues/18607

This breaks our tests matching the stderr output. We avoid this by setting EMCC_LOGGING=0

- - - - -
ff2c0cc9 by Simon Peyton Jones at 2024-02-12T12:19:17-05:00
Remove a dead comment

Just remove an out of date block of commented-out code, and tidy up
the relevant Notes.  See #8317.

- - - - -
bedb4f0d by Teo Camarasu at 2024-02-12T18:50:33-05:00
nonmoving: Add support for heap profiling

Add support for heap profiling while using the nonmoving collector.

We greatly simply the implementation by disabling concurrent collection for
GCs when heap profiling is enabled. This entails that the marked objects on
the nonmoving heap are exactly the live objects.

Note that we match the behaviour for live bytes accounting by taking the size
of objects on the nonmoving heap to be that of the segment's block
rather than the object itself.

Resolves #22221

- - - - -
d0d5acb5 by Teo Camarasu at 2024-02-12T18:51:09-05:00
doc: Add requires prof annotation to options that require it

Resolves #24421

- - - - -
57bb8c92 by Cheng Shao at 2024-02-13T14:07:49-05:00
deriveConstants: add needed constants for wasm backend

This commit adds needed constants to deriveConstants. They are used by
RTS code in the wasm backend to support the JSFFI logic.

- - - - -
615eb855 by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: make genSym use C-based atomic increment on non-JS 32-bit platforms

The pure Haskell implementation causes i386 regression in unrelated
work that can be fixed by using C-based atomic increment, see added
comment for details.

- - - - -
a9918891 by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: allow JSFFI for wasm32

This commit allows the javascript calling convention to be used when
the target platform is wasm32.

- - - - -
8771a53b by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: allow boxed JSVal as a foreign type

This commit allows the boxed JSVal type to be used as a foreign
argument/result type.

- - - - -
053c92b3 by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: ensure ctors have the right priority on wasm32

This commit fixes the priorities of ctors generated by GHC codegen on
wasm32, see the referred note for details.

- - - - -
b7942e0a by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: add JSFFI desugar logic for wasm32

This commit adds JSFFI desugar logic for the wasm backend.

- - - - -
2c1dca76 by Cheng Shao at 2024-02-13T14:07:49-05:00
compiler: add JavaScriptFFI to supported extension list on wasm32

This commit adds JavaScriptFFI as a supported extension when the
target platform is wasm32.

- - - - -
9ad0e2b4 by Cheng Shao at 2024-02-13T14:07:49-05:00
rts/ghc-internal: add JSFFI support logic for wasm32

This commit adds rts/ghc-internal logic to support the wasm backend's
JSFFI functionality.

- - - - -
e9ebea66 by Cheng Shao at 2024-02-13T14:07:49-05:00
ghc-internal: fix threadDelay for wasm in browsers

This commit fixes broken threadDelay for wasm when it runs in
browsers, see added note for detailed explanation.

- - - - -
f85f3fdb by Cheng Shao at 2024-02-13T14:07:49-05:00
utils: add JSFFI utility code

This commit adds JavaScript util code to utils to support the wasm
backend's JSFFI functionality:

- jsffi/post-link.mjs, a post-linker to process the linked wasm module
  and emit a small complement JavaScript ESM module to be used with it
  at runtime
- jsffi/prelude.js, a tiny bit of prelude code as the JavaScript side
  of runtime logic
- jsffi/test-runner.mjs, run the jsffi test cases

Co-authored-by: amesgen <amesgen at amesgen.de>

- - - - -
77e91500 by Cheng Shao at 2024-02-13T14:07:49-05:00
hadrian: distribute jsbits needed for wasm backend's JSFFI support

The post-linker.mjs/prelude.js files are now distributed in the
bindist libdir, so when using the wasm backend's JSFFI feature, the
user wouldn't need to fetch them from a ghc checkout manually.

- - - - -
c47ba1c3 by Cheng Shao at 2024-02-13T14:07:49-05:00
testsuite: add opts.target_wrapper

This commit adds opts.target_wrapper which allows overriding the
target wrapper on a per test case basis when testing a cross target.
This is used when testing the wasm backend's JSFFI functionality; the
rest of the cases are tested using wasmtime, though the jsffi cases
are tested using the node.js based test runner.

- - - - -
8e048675 by Cheng Shao at 2024-02-13T14:07:49-05:00
testsuite: T22774 should work for wasm JSFFI

T22774 works since the wasm backend now supports the JSFFI feature.

- - - - -
1d07f9a6 by Cheng Shao at 2024-02-13T14:07:49-05:00
testsuite: add JSFFI test cases for wasm backend

This commit adds a few test cases for the wasm backend's JSFFI
functionality, as well as a simple README to instruct future
contributors to add new test cases.

- - - - -
b8997080 by Cheng Shao at 2024-02-13T14:07:49-05:00
docs: add documentation for wasm backend JSFFI

This commit adds changelog and user facing documentation for the wasm
backend's JSFFI feature.

- - - - -
ffeb000d by David Binder at 2024-02-13T14:08:30-05:00
Add tests from libraries/process/tests and libraries/Win32/tests to GHC

These tests were previously part of the libraries, which themselves are
submodules of the GHC repository. This commit moves the tests directly
to the GHC repository.

- - - - -
5a932cf2 by David Binder at 2024-02-13T14:08:30-05:00
Do not execute win32 tests on non-windows runners

- - - - -
500d8cb8 by Jade at 2024-02-13T14:09:07-05:00
prevent GHCi (and runghc) from suggesting other symbols when not finding main

Fixes: #23996

- - - - -
b19ec331 by Cheng Shao at 2024-02-13T14:09:46-05:00
rts: update xxHash to v0.8.2

- - - - -
4a97bdb8 by Cheng Shao at 2024-02-13T14:09:46-05:00
rts: use XXH3_64bits hash on all 64-bit platforms

This commit enables XXH3_64bits hash to be used on all 64-bit
platforms. Previously it was only enabled on x86_64, so platforms like
aarch64 silently falls back to using XXH32 which degrades the hashing
function quality.

- - - - -
ee01de7d by Cheng Shao at 2024-02-13T14:09:46-05:00
rts: define XXH_INLINE_ALL

This commit cleans up how we include the xxhash.h header and only
define XXH_INLINE_ALL, which is sufficient to inline the xxHash
functions without symbol collision.

- - - - -
0e01e1db by Alan Zimmerman at 2024-02-14T02:13:22-05:00
EPA: Move EpAnn out of extension points

Leaving a few that are too tricky, maybe some other time.

Also
 - remove some unneeded helpers from Parser.y
 - reduce allocations with strictness annotations

Updates haddock submodule

Metric Decrease:
    parsing001

- - - - -
de589554 by Andreas Klebinger at 2024-02-14T02:13:59-05:00
Fix ffi callbacks with >6 args and non-64bit args.

Check for ptr/int arguments rather than 64-bit width arguments when counting
integer register arguments.
The old approach broke when we stopped using exclusively W64-sized types to represent
sub-word sized integers.

Fixes #24314

- - - - -
2df79dfe by Andrei Borzenkov at 2024-02-14T13:01:12+04:00
Parser, renamer, type checker for @a-binders (17594)

GHC Proposal 448 introduces binders for invisible type arguments
(@a-binders) in various contexts. This patch implements @-binders
in lambda patterns and function equations:

  {-# LANGUAGE TypeAbstractions #-}

  id1 :: a -> a
  id1 @t x = x :: t      -- @t-binder on the LHS of a function equation

  higherRank :: (forall a. (Num a, Bounded a) => a -> a) -> (Int8, Int16)
  higherRank f = (f 42, f 42)

  ex :: (Int8, Int16)
  ex = higherRank (\ @a x -> maxBound @a - x )
                         -- @a-binder in a lambda pattern in an argument
                         -- to a higher-order function

Syntax
------

To represent those @-binders in the AST, the list of patterns in Match
now uses ArgPat instead of Pat:

  data Match p body
     = Match {
         ...
-        m_pats  :: [LPat p],
+        m_pats  :: [LArgPat p],
         ...
   }

+ data ArgPat pass
+   = VisPat (XVisPat pass) (LPat pass)
+   | InvisPat (XInvisPat pass) (HsTyPat (NoGhcTc pass))
+   | XArgPat !(XXArgPat pass)

The VisPat constructor represents patterns for visible arguments,
which include ordinary value-level arguments and required type arguments
(neither is prefixed with a @), while InvisPat represents invisible type
arguments (prefixed with a @).

Parser
------

In the grammar (Parser.y), the lambda and lambda-cases productions of
aexp non-terminal were updated to accept argpats instead of apats:

  aexp : ...
-        | '\\' apats '->' exp
+        | '\\' argpats '->' exp
         ...
-        | '\\' 'lcases' altslist(apats)
+        | '\\' 'lcases' altslist(argpats)
         ...

+ argpat : apat
+        | PREFIX_AT atype

Function left-hand sides did not require any changes to the grammar, as
they were already parsed with productions capable of parsing @-binders.
Those binders were being rejected in post-processing (isFunLhs), and now
we accept them.

In Parser.PostProcess, patterns are constructed with the help of
PatBuilder, which is used as an intermediate data structure when
disambiguating between FunBind and PatBind. In this patch we define
ArgPatBuilder to accompany PatBuilder. ArgPatBuilder is a short-lived
data structure produced in isFunLhs and consumed in checkFunBind.

Renamer
-------

Renaming of @-binders builds upon prior work on type patterns,
implemented in 2afbddb0f24, which guarantees proper scoping and
shadowing behavior of bound type variables.

This patch merely defines rnLArgPatsAndThen to process a mix of visible
and invisible patterns:

+ rnLArgPatsAndThen :: NameMaker -> [LArgPat GhcPs] -> CpsRn [LArgPat GhcRn]
+ rnLArgPatsAndThen mk = mapM (wrapSrcSpanCps rnArgPatAndThen) where
+   rnArgPatAndThen (VisPat x p)    = ... rnLPatAndThen ...
+   rnArgPatAndThen (InvisPat _ tp) = ... rnHsTyPat ...

Common logic between rnArgPats and rnPats is factored out into the
rn_pats_general helper.

Type checker
------------

Type-checking of @-binders builds upon prior work on lazy skolemisation,
implemented in f5d3e03c56f.

This patch extends tcMatchPats to handle @-binders. Now it takes and
returns a list of LArgPat rather than LPat:

  tcMatchPats ::
              ...
-             -> [LPat GhcRn]
+             -> [LArgPat GhcRn]
              ...
-             -> TcM ([LPat GhcTc], a)
+             -> TcM ([LArgPat GhcTc], a)

Invisible binders in the Match are matched up with invisible foralls in
the type. This is done with a new clause in the `loop` worker of
tcMatchPats:

  loop :: [LArgPat GhcRn] -> [ExpPatType] -> TcM ([LArgPat GhcTc], a)
  loop (L l apat : pats) (ExpForAllPatTy (Bndr tv vis) : pat_tys)
    ...
    -- NEW CLAUSE:
    | InvisPat _ tp <- apat, isInvisibleForAllTyFlag vis
    = ...

In addition to that, tcMatchPats no longer discards type patterns. This
is done by filterOutErasedPats in the desugarer instead.

- - - - -

30 changed files:

- CODEOWNERS
- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Builtin/Names/TH.hs
- compiler/GHC/CmmToAsm/Wasm/Asm.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/Core/Opt/ConstantFold.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Driver/Config/StgToJS.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Binds.hs
- compiler/GHC/Hs/Decls.hs
- compiler/GHC/Hs/Dump.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/Hs/ImpExp.hs
- compiler/GHC/Hs/Instances.hs
- compiler/GHC/Hs/Pat.hs
- compiler/GHC/Hs/Type.hs
- compiler/GHC/Hs/Utils.hs
- compiler/GHC/HsToCore/Arrows.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/HsToCore/Foreign/C.hs
- compiler/GHC/HsToCore/Foreign/Decl.hs
- + compiler/GHC/HsToCore/Foreign/Wasm.hs
- compiler/GHC/HsToCore/Match.hs
- compiler/GHC/HsToCore/Pmc/Desugar.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/HsToCore/Ticks.hs
- compiler/GHC/HsToCore/Utils.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/72b9c8ec3b874cd2505e3b016917f6261a5d6f68...2df79dfe652f481521a3594ac0676e99ccb6dc2c

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/72b9c8ec3b874cd2505e3b016917f6261a5d6f68...2df79dfe652f481521a3594ac0676e99ccb6dc2c
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240214/b1278d7a/attachment-0001.html>