[Git][ghc/ghc][wip/foundation-ghci] 8 commits: compiler: use fromAscList when applicable
Matthew Pickering (@mpickering)
gitlab at gitlab.haskell.org
Mon Feb 24 10:58:44 UTC 2025
Matthew Pickering pushed to branch wip/foundation-ghci at Glasgow Haskell Compiler / GHC
Commits:
1c8e608a by Cheng Shao at 2025-02-21T20:18:41-05:00
compiler: use fromAscList when applicable
This patch uses fromAscList (with O(n) complexity) instead of fromList
(with O(nlogn) complexity) in certain Binary instances. It's safe to
do so since the corresponding serialization logic is based on toList
(same as toAscList).
- - - - -
549e0aff by Ben Gamari at 2025-02-21T20:19:18-05:00
rts/linker/MachO: Mark internal symbols as static
There is no reason why these should have external linkage.
- - - - -
fbf3d020 by Cheng Shao at 2025-02-22T07:41:01-05:00
wasm: bump dyld v8 heap size limit
This patch sets `--max-old-space-size=65536` as wasm dyld shebang
arguments to lessen v8 heap pressure in certain workloads that load
the full ghc package. It doesn't really commit 64G memory but it does
help reduce v8 gc overhead.
- - - - -
cb60da24 by Cheng Shao at 2025-02-22T07:41:01-05:00
wasm: fix dyld for shared libraries created by llvm 20.x
This patch fixes wasm dyld script for shared libraries created by llvm
20.x. The __wasm_apply_data_relocs function is now optional and may be
omitted for shared libraries without any runtime relocatable data
segments, so only call __wasm_apply_data_relocs when it's present.
- - - - -
7eea38c8 by Cheng Shao at 2025-02-22T07:41:37-05:00
driver: fix wasm backend sysroot lookup logic when -flto is passed
For the wasm target, the driver calls `wasm32-wasi-clang
--print-search-dirs` and finds the sysroot directory that contains
libc.so etc, then passes the directory path to dyld. However, when GHC
is configured with -flto as a part of C/C++ compiler flags, the clang
driver would insert a llvm-lto specific directory in the
--print-search-dirs output and the driver didn't take that into
account. This patch fixes it and always selects the non-lto sysroot
directory to be passed to dyld. This is one small step towards
supporting building all cbits with lto for wasm.
- - - - -
f3bfe31e by Cheng Shao at 2025-02-23T14:06:25-05:00
wasm: add Note [Variable passing in JSFFI] as !13583 follow up
This patch adds a note to explain how the magic variables like
`__ghc_wasm_jsffi_dyld` are brought into scope of JSFFI code snippets,
as follow up work of !13583.
- - - - -
c318be56 by Cheng Shao at 2025-02-23T14:07:02-05:00
libffi: update to 3.4.7
Bumps libffi submodule.
- - - - -
34c4f826 by Matthew Pickering at 2025-02-24T10:58:01+00:00
testsuite: Run foundation test in ghci and ghci-opt ways
Running this test using optimised core is a good test for implementing
numeric primops in the interpreter.
This test currently fails in `ghci-opt` way when breakpoints are
enabled so they are disabled for now.
- - - - -
8 changed files:
- compiler/GHC.hs
- compiler/GHC/Utils/Binary.hs
- libffi-tarballs
- libraries/ghci/GHCi/ObjLink.hs
- rts/linker/MachO.c
- testsuite/tests/numeric/should_run/all.T
- utils/jsffi/dyld.mjs
- utils/jsffi/post-link.mjs
Changes:
=====================================
compiler/GHC.hs
=====================================
@@ -717,11 +717,7 @@ setTopSessionDynFlags dflags = do
#if defined(wasm32_HOST_ARCH)
let libdir = sorry "cannot spawn child process on wasm"
#else
- libdir <- liftIO $ do
- libdirs <- Loader.getGccSearchDirectory logger dflags "libraries"
- case libdirs of
- [_, libdir] -> pure libdir
- _ -> panic "corrupted wasi-sdk installation"
+ libdir <- liftIO $ last <$> Loader.getGccSearchDirectory logger dflags "libraries"
#endif
let profiled = ways dflags `hasWay` WayProf
way_tag = if profiled then "_p" else ""
=====================================
compiler/GHC/Utils/Binary.hs
=====================================
@@ -940,8 +940,8 @@ instance Binary a => Binary [a] where
-- | This instance doesn't rely on the determinism of the keys' 'Ord' instance,
-- so it works e.g. for 'Name's too.
instance (Binary a, Ord a) => Binary (Set a) where
- put_ bh s = put_ bh (Set.toList s)
- get bh = Set.fromList <$> get bh
+ put_ bh s = put_ bh (Set.toAscList s)
+ get bh = Set.fromAscList <$> get bh
instance Binary a => Binary (NonEmpty a) where
put_ bh = put_ bh . NonEmpty.toList
@@ -2086,5 +2086,5 @@ source location as part of a larger structure.
--------------------------------------------------------------------------------
instance (Binary v) => Binary (IntMap v) where
- put_ bh m = put_ bh (IntMap.toList m)
- get bh = IntMap.fromList <$> get bh
+ put_ bh m = put_ bh (IntMap.toAscList m)
+ get bh = IntMap.fromAscList <$> get bh
=====================================
libffi-tarballs
=====================================
@@ -1 +1 @@
-Subproject commit 89a9b01c5647c8f0d3899435b99df690f582e9f1
+Subproject commit cb280851187d7b509d341be7b50c9a239810feb0
=====================================
libraries/ghci/GHCi/ObjLink.hs
=====================================
@@ -76,6 +76,9 @@ loadDLL f =
evaluate =<< js_loadDLL (toJSString f)
pure $ Right nullPtr
+-- See Note [Variable passing in JSFFI] for where
+-- __ghc_wasm_jsffi_dyld comes from
+
foreign import javascript safe "__ghc_wasm_jsffi_dyld.loadDLL($1)"
js_loadDLL :: JSString -> IO ()
=====================================
rts/linker/MachO.c
=====================================
@@ -51,7 +51,7 @@
/* often times we need to extend some value of certain number of bits
* int an int64_t for e.g. relative offsets.
*/
-int64_t signExtend(uint64_t val, uint8_t bits);
+static int64_t signExtend(uint64_t val, uint8_t bits);
/* Helper functions to check some instruction properties */
static bool isVectorOp(uint32_t *p);
static bool isLoadStore(uint32_t *p);
@@ -60,17 +60,17 @@ static bool isLoadStore(uint32_t *p);
* where we want to write the address offset to. Thus decoding as well
* as encoding is needed.
*/
-bool fitsBits(size_t bits, int64_t value);
-int64_t decodeAddend(ObjectCode * oc, Section * section,
+static bool fitsBits(size_t bits, int64_t value);
+static int64_t decodeAddend(ObjectCode * oc, Section * section,
MachORelocationInfo * ri);
-void encodeAddend(ObjectCode * oc, Section * section,
+static void encodeAddend(ObjectCode * oc, Section * section,
MachORelocationInfo * ri, int64_t addend);
/* Global Offset Table logic */
-bool isGotLoad(MachORelocationInfo * ri);
-bool needGotSlot(MachONList * symbol);
-bool makeGot(ObjectCode * oc);
-void freeGot(ObjectCode * oc);
+static bool isGotLoad(MachORelocationInfo * ri);
+static bool needGotSlot(MachONList * symbol);
+static bool makeGot(ObjectCode * oc);
+static void freeGot(ObjectCode * oc);
#endif /* aarch64_HOST_ARCH */
/*
@@ -265,7 +265,7 @@ resolveImports(
#if defined(aarch64_HOST_ARCH)
/* aarch64 linker by moritz angermann <moritz at lichtzwerge.de> */
-int64_t
+static int64_t
signExtend(uint64_t val, uint8_t bits) {
return (int64_t)(val << (64-bits)) >> (64-bits);
}
@@ -280,7 +280,7 @@ isLoadStore(uint32_t *p) {
return (*p & 0x3B000000) == 0x39000000;
}
-int64_t
+static int64_t
decodeAddend(ObjectCode * oc, Section * section, MachORelocationInfo * ri) {
/* the instruction. It is 32bit wide */
@@ -350,7 +350,7 @@ decodeAddend(ObjectCode * oc, Section * section, MachORelocationInfo * ri) {
barf("unsupported relocation type: %d\n", ri->r_type);
}
-inline bool
+inline static bool
fitsBits(size_t bits, int64_t value) {
if(bits == 64) return true;
if(bits > 64) barf("fits_bits with %zu bits and an 64bit integer!", bits);
@@ -358,7 +358,7 @@ fitsBits(size_t bits, int64_t value) {
|| -1 == (value >> bits); // All bits on: -1
}
-void
+static void
encodeAddend(ObjectCode * oc, Section * section,
MachORelocationInfo * ri, int64_t addend) {
uint32_t * p = (uint32_t*)((uint8_t*)section->start + ri->r_address);
@@ -440,7 +440,7 @@ encodeAddend(ObjectCode * oc, Section * section,
barf("unsupported relocation type: %d\n", ri->r_type);
}
-bool
+static bool
isGotLoad(struct relocation_info * ri) {
return ri->r_type == ARM64_RELOC_GOT_LOAD_PAGE21
|| ri->r_type == ARM64_RELOC_GOT_LOAD_PAGEOFF12;
@@ -450,7 +450,7 @@ isGotLoad(struct relocation_info * ri) {
* Check if we need a global offset table slot for a
* given symbol
*/
-bool
+static bool
needGotSlot(MachONList * symbol) {
return (symbol->n_type & N_EXT) /* is an external symbol */
&& (N_UNDF == (symbol->n_type & N_TYPE) /* and is undefined */
@@ -458,7 +458,7 @@ needGotSlot(MachONList * symbol) {
* different section */
}
-bool
+static bool
makeGot(ObjectCode * oc) {
size_t got_slots = 0;
@@ -484,7 +484,7 @@ makeGot(ObjectCode * oc) {
return EXIT_SUCCESS;
}
-void
+static void
freeGot(ObjectCode * oc) {
/* sanity check */
if(NULL != oc->info->got_start && oc->info->got_size > 0) {
=====================================
testsuite/tests/numeric/should_run/all.T
=====================================
@@ -82,7 +82,7 @@ test('IntegerToFloat', normal, compile_and_run, [''])
test('T20291', normal, compile_and_run, [''])
test('T22282', normal, compile_and_run, [''])
test('T22671', js_fragile(24259), compile_and_run, [''])
-test('foundation', [when(js_arch(), run_timeout_multiplier(2)), js_fragile(24259)], compile_and_run, ['-O -package transformers'])
+test('foundation', [when(js_arch(), run_timeout_multiplier(2)), js_fragile(24259), extra_ways(['optasm','ghci','ghci-opt'])], compile_and_run, ['-package transformers -fno-break-points'])
test('T24066', normal, compile_and_run, [''])
test('div01', normal, compile_and_run, [''])
test('T24245', normal, compile_and_run, [''])
=====================================
utils/jsffi/dyld.mjs
=====================================
@@ -1,4 +1,4 @@
-#!/usr/bin/env -S node --disable-warning=ExperimentalWarning --experimental-wasm-type-reflection --max-old-space-size=8192 --no-turbo-fast-api-calls --wasm-lazy-validation
+#!/usr/bin/env -S node --disable-warning=ExperimentalWarning --experimental-wasm-type-reflection --max-old-space-size=65536 --no-turbo-fast-api-calls --wasm-lazy-validation
// Note [The Wasm Dynamic Linker]
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -629,7 +629,8 @@ class DyLD {
// Fulfill the ghc_wasm_jsffi imports. Use new Function()
// instead of eval() to prevent bindings in this local scope to
- // be accessed by JSFFI code snippets.
+ // be accessed by JSFFI code snippets. See Note [Variable passing in JSFFI]
+ // for what's going on here.
Object.assign(
import_obj.ghc_wasm_jsffi,
new Function(
@@ -796,12 +797,17 @@ class DyLD {
const init = () => {
// See
- // https://github.com/llvm/llvm-project/blob/llvmorg-19.1.1/lld/wasm/Writer.cpp#L1430,
- // there's also __wasm_init_memory (not relevant yet, we don't
+ // https://gitlab.haskell.org/haskell-wasm/llvm-project/-/blob/release/20.x/lld/wasm/Writer.cpp#L1450,
+ // __wasm_apply_data_relocs is now optional so only call it if
+ // it exists (we know for sure it exists for libc.so though).
+ // There's also __wasm_init_memory (not relevant yet, we don't
// use passive segments) & __wasm_apply_global_relocs but
// those are included in the start function and should have
- // been called upon instantiation.
- instance.exports.__wasm_apply_data_relocs();
+ // been called upon instantiation, see
+ // Writer::createStartFunction().
+ if (instance.exports.__wasm_apply_data_relocs) {
+ instance.exports.__wasm_apply_data_relocs();
+ }
instance.exports._initialize();
};
=====================================
utils/jsffi/post-link.mjs
=====================================
@@ -52,6 +52,47 @@ export function parseSections(mod) {
return recs;
}
+// Note [Variable passing in JSFFI]
+// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+//
+// The JSFFI code snippets can access variables in globalThis,
+// arguments like $1, $2, etc, plus a few magic variables: __exports,
+// __ghc_wasm_jsffi_dyld and __ghc_wasm_jsffi_finalization_registry.
+// How are these variables passed to JSFFI code? Remember, we strive
+// to keep the globalThis namespace hygiene and maintain the ability
+// to have multiple Haskell-wasm apps coexisting in the same JS
+// context, so we must not pass magic variables as global variables
+// even though they may seem globally unique.
+//
+// The solution is simple: put them in the JS lambda binder position.
+// Though there are different layers of lambdas here:
+//
+// 1. User writes "$1($2, await $3)" in a JSFFI code snippet. No
+// explicit binder here, the snippet is either an expression or
+// some statements.
+// 2. GHC doesn't know JS syntax but it knows JS function arity from
+// HS type signature, as well as if the JS function is async/sync
+// from safe/unsafe annotation. So it infers the JS binder (like
+// "async ($1, $2, $3)") and emits a (name,binder,body) tuple into
+// the ghc_wasm_jsffi custom section.
+// 3. After link-time we collect these tuples to make a JS object
+// mapping names to binder=>body, and this JS object will be used
+// to fulfill the ghc_wasm_jsffi wasm imports. This JS object is
+// returned by an outer layer of lambda which is in charge of
+// passing magic variables.
+//
+// In case of post-linker for statically linked wasm modules,
+// __ghc_wasm_jsffi_dyld won't work so is omitted, and
+// __ghc_wasm_jsffi_finalization_registry can be created inside the
+// outer JS lambda. Only __exports is exposed as user-visible API
+// since it's up to the user to perform knot-tying by assigning the
+// instance exports back to the (initially empty) __exports object
+// passed to this lambda.
+//
+// In case of dyld, all magic variables are dyld-session-global
+// variables; dyld uses new Function() to make the outer lambda, then
+// immediately invokes it by passing the right magic variables.
+
export async function postLink(mod) {
let src = (
await fs.readFile(path.join(import.meta.dirname, "prelude.mjs"), {
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/d0b2005b6428d4b194b7dda445dbfe96460519b5...34c4f826faafc5c94206fa99cb98e335654fef06
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/d0b2005b6428d4b194b7dda445dbfe96460519b5...34c4f826faafc5c94206fa99cb98e335654fef06
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250224/f5a820cb/attachment-0001.html>
More information about the ghc-commits
mailing list