[Git][ghc/ghc][wip/az/ghc-cpp] 66 commits: wasm: bump dyld v8 heap size limit
Alan Zimmerman (@alanz)
gitlab at gitlab.haskell.org
Sun Feb 23 19:44:38 UTC 2025
Alan Zimmerman pushed to branch wip/az/ghc-cpp at Glasgow Haskell Compiler / GHC
Commits:
fbf3d020 by Cheng Shao at 2025-02-22T07:41:01-05:00
wasm: bump dyld v8 heap size limit
This patch sets `--max-old-space-size=65536` as wasm dyld shebang
arguments to lessen v8 heap pressure in certain workloads that load
the full ghc package. It doesn't really commit 64G memory but it does
help reduce v8 gc overhead.
- - - - -
cb60da24 by Cheng Shao at 2025-02-22T07:41:01-05:00
wasm: fix dyld for shared libraries created by llvm 20.x
This patch fixes wasm dyld script for shared libraries created by llvm
20.x. The __wasm_apply_data_relocs function is now optional and may be
omitted for shared libraries without any runtime relocatable data
segments, so only call __wasm_apply_data_relocs when it's present.
- - - - -
7eea38c8 by Cheng Shao at 2025-02-22T07:41:37-05:00
driver: fix wasm backend sysroot lookup logic when -flto is passed
For the wasm target, the driver calls `wasm32-wasi-clang
--print-search-dirs` and finds the sysroot directory that contains
libc.so etc, then passes the directory path to dyld. However, when GHC
is configured with -flto as a part of C/C++ compiler flags, the clang
driver would insert a llvm-lto specific directory in the
--print-search-dirs output and the driver didn't take that into
account. This patch fixes it and always selects the non-lto sysroot
directory to be passed to dyld. This is one small step towards
supporting building all cbits with lto for wasm.
- - - - -
ff928764 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
GHC-CPP: first rough proof of concept
Processes
#define FOO
#ifdef FOO
x = 1
#endif
Into
[ITcppIgnored [L loc ITcppDefine]
,ITcppIgnored [L loc ITcppIfdef]
,ITvarid "x"
,ITequal
,ITinteger (IL {il_text = SourceText "1", il_neg = False, il_value = 1})
,ITcppIgnored [L loc ITcppEndif]
,ITeof]
In time, ITcppIgnored will be pushed into a comment
- - - - -
da6615ed by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Tidy up before re-visiting the continuation mechanic
- - - - -
aeac4a5d by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Switch preprocessor to continuation passing style
Proof of concept, needs tidying up
- - - - -
6df63535 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Small cleanup
- - - - -
a08503b0 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Get rid of some cruft
- - - - -
f086dcf7 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Starting to integrate.
Need to get the pragma recognised and set
- - - - -
14ce600d by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Make cppTokens extend to end of line, and process CPP comments
- - - - -
92e6812b by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Remove unused ITcppDefined
- - - - -
ba90c0ae by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Allow spaces between # and keyword for preprocessor directive
- - - - -
0055fa54 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Process CPP continuation lines
They are emited as separate ITcppContinue tokens.
Perhaps the processing should be more like a comment, and keep on
going to the end.
BUT, the last line needs to be slurped as a whole.
- - - - -
46d91d5f by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Accumulate CPP continuations, process when ready
Can be simplified further, we only need one CPP token
- - - - -
cccb3346 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Simplify Lexer interface. Only ITcpp
We transfer directive lines through it, then parse them from scratch
in the preprocessor.
- - - - -
68dd079a by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Deal with directive on last line, with no trailing \n
- - - - -
9d583667 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Start parsing and processing the directives
- - - - -
59f79d31 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Prepare for processing include files
- - - - -
4c0f12b3 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Move PpState into PreProcess
And initParserState, initPragState too
- - - - -
cb497566 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Process nested include files
Also move PpState out of Lexer.x, so it is easy to evolve it in a ghci
session, loading utils/check-cpp/Main.hs
- - - - -
7dc66b46 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Split into separate files
- - - - -
bbb2f472 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Starting on expression parser.
But it hangs. Time for Text.Parsec.Expr
- - - - -
f63dc0fe by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Start integrating the ghc-cpp work
>From https://github.com/alanz/ghc-cpp
- - - - -
f208872a by Alan Zimmerman at 2025-02-23T15:18:20+00:00
WIP
- - - - -
78514d73 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Fixup after rebase
- - - - -
917468b8 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
WIP
- - - - -
e42cfc2e by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Fixup after rebase, including all tests pass
- - - - -
9ac7fd76 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Change pragma usage to GHC_CPP from GhcCPP
- - - - -
1426bcc2 by Alan Zimmerman at 2025-02-23T15:18:20+00:00
Some comments
- - - - -
a396a5bb by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Reformat
- - - - -
c6ea4685 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Delete unused file
- - - - -
85b6fd6f by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Rename module Parse to ParsePP
- - - - -
4b3946d1 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Clarify naming in the parser
- - - - -
73c2900a by Alan Zimmerman at 2025-02-23T15:18:21+00:00
WIP. Switching to alex/happy to be able to work in-tree
Since Parsec is not available
- - - - -
51c4a5d8 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Layering is now correct
- GHC lexer, emits CPP tokens
- accumulated in Preprocessor state
- Lexed by CPP lexer, CPP command extracted, tokens concated with
spaces (to get rid of token pasting via comments)
- if directive lexed and parsed by CPP lexer/parser, and evaluated
- - - - -
b55c8f2f by Alan Zimmerman at 2025-02-23T15:18:21+00:00
First example working
Loading Example1.hs into ghci, getting the right results
```
{-# LANGUAGE GHC_CPP #-}
module Example1 where
y = 3
x =
"hello"
"bye now"
foo = putStrLn x
```
- - - - -
cb8f0a6f by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Rebase, and all tests pass except whitespace for generated parser
- - - - -
f929de12 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
More plumbing. Ready for testing tomorrow.
- - - - -
ee8584e5 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Proress. Renamed module State from Types
And at first blush it seems to handle preprocessor scopes properly.
- - - - -
4f177010 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Insert basic GHC version macros into parser
__GLASGOW_HASKELL__
__GLASGOW_HASKELL_FULL_VERSION__
__GLASGOW_HASKELL_PATCHLEVEL1__
__GLASGOW_HASKELL_PATCHLEVEL2__
- - - - -
7981fdd8 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Re-sync check-cpp for easy ghci work
- - - - -
2076926b by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Get rid of warnings
- - - - -
3f0a5a8d by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Rework macro processing, in check-cpp
Macros kept at the top level, looked up via name, multiple arity
versions per name can be stored
- - - - -
db73b4e6 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
WIP. Can crack arguments for #define
Next step it to crack out args in an expansion
- - - - -
0a2538bc by Alan Zimmerman at 2025-02-23T15:18:21+00:00
WIP on arg parsing.
- - - - -
594bb1f1 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Progress. Still screwing up nested parens.
- - - - -
ebe89299 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Seems to work, but has redundant code
- - - - -
372a22b8 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Remove redundant code
- - - - -
606681f4 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Reformat
- - - - -
72e98ce1 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Expand args, single pass
Still need to repeat until fixpoint
- - - - -
18c7fd9a by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Fixed point expansion
- - - - -
01920521 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Sync the playground to compiler
- - - - -
b971c127 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Working on dumping the GHC_CPP result
But We need to keep the BufSpan in a comment
- - - - -
484a397d by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Keep BufSpan in queued comments in GHC.Parser.Lexer
- - - - -
40c8a163 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Getting close to being able to print the combined tokens
showing what is in and what is out
- - - - -
3b41540c by Alan Zimmerman at 2025-02-23T15:18:21+00:00
First implementation of dumpGhcCpp.
Example output
First dumps all macros in the state, then the source, showing which
lines are in and which are out
------------------------------
- |#define FOO(A,B) A + B
- |#define FOO(A,B,C) A + B + C
- |#if FOO(1,FOO(3,4)) == 8
- |-- a comment
|x = 1
- |#else
- |x = 5
- |#endif
- - - - -
96230e57 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Clean up a bit
- - - - -
fae2f839 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Add -ddump-ghc-cpp option and a test based on it
- - - - -
129083c9 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Restore Lexer.x rules, we need them for continuation lines
- - - - -
3ffdf8e0 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Lexer.x: trying to sort out the span for continuations
- We need to match on \n at the end of the line
- We cannot simply back up for it
- - - - -
412ea7cc by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Inserts predefined macros. But does not dump properly
Because the cpp tokens have a trailing newline
- - - - -
00f1086e by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Remove unnecessary LExer rules
We *need* the ones that explicitly match to the end of the line.
- - - - -
425fb060 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Generate correct span for ITcpp
Dump now works, except we do not render trailing `\` for continuation
lines. This is good enough for use in test output.
- - - - -
fd786418 by Alan Zimmerman at 2025-02-23T15:18:21+00:00
Reduce duplication in lexer
- - - - -
c201cfda by Alan Zimmerman at 2025-02-23T17:03:16+00:00
Tweaks
- - - - -
ed06a20a by Alan Zimmerman at 2025-02-23T19:42:31+00:00
Insert min_version predefined macros into state
The mechanism now works. Still need to flesh out the full set.
- - - - -
67 changed files:
- compiler/GHC.hs
- compiler/GHC/Cmm/Lexer.x
- compiler/GHC/Cmm/Parser.y
- compiler/GHC/Cmm/Parser/Monad.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/Config/Parser.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Parser.hs-boot
- compiler/GHC/Parser.y
- compiler/GHC/Parser/Annotation.hs
- compiler/GHC/Parser/HaddockLex.x
- compiler/GHC/Parser/Header.hs
- compiler/GHC/Parser/Lexer.x
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Parser/PostProcess/Haddock.hs
- + compiler/GHC/Parser/PreProcess.hs
- + compiler/GHC/Parser/PreProcess/Eval.hs
- + compiler/GHC/Parser/PreProcess/Lexer.x
- + compiler/GHC/Parser/PreProcess/Macro.hs
- + compiler/GHC/Parser/PreProcess/ParsePP.hs
- + compiler/GHC/Parser/PreProcess/Parser.y
- + compiler/GHC/Parser/PreProcess/ParserM.hs
- + compiler/GHC/Parser/PreProcess/State.hs
- compiler/GHC/Parser/Utils.hs
- compiler/GHC/SysTools/Cpp.hs
- compiler/ghc.cabal.in
- docs/users_guide/debugging.rst
- ghc/GHCi/UI.hs
- hadrian/src/Rules/SourceDist.hs
- hadrian/stack.yaml.lock
- libraries/ghc-internal/src/GHC/Internal/LanguageExtensions.hs
- testsuite/tests/count-deps/CountDepsParser.stdout
- testsuite/tests/driver/T4437.hs
- testsuite/tests/ghc-api/T11579.hs
- + testsuite/tests/ghc-cpp/GhcCpp01.hs
- + testsuite/tests/ghc-cpp/GhcCpp01.stderr
- + testsuite/tests/ghc-cpp/all.T
- testsuite/tests/interface-stability/template-haskell-exports.stdout
- testsuite/tests/linters/notes.stdout
- + utils/check-cpp/.ghci
- + utils/check-cpp/.gitignore
- + utils/check-cpp/Eval.hs
- + utils/check-cpp/Example1.hs
- + utils/check-cpp/Example2.hs
- + utils/check-cpp/Lexer.x
- + utils/check-cpp/Macro.hs
- + utils/check-cpp/Main.hs
- + utils/check-cpp/ParsePP.hs
- + utils/check-cpp/ParseSimulate.hs
- + utils/check-cpp/Parser.y
- + utils/check-cpp/ParserM.hs
- + utils/check-cpp/PreProcess.hs
- + utils/check-cpp/README.md
- + utils/check-cpp/State.hs
- + utils/check-cpp/run.sh
- utils/check-exact/Main.hs
- utils/check-exact/Parsers.hs
- utils/check-exact/Preprocess.hs
- utils/check-exact/Utils.hs
- utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Parser.hs
- utils/haddock/haddock-api/src/Haddock/Parser.hs
- utils/haddock/haddock-api/src/Haddock/Types.hs
- utils/jsffi/dyld.mjs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/fc86f7bec83fd2da6794b2d0938f4b37697b6527...ed06a20ad8c09e92f270943efe19c0f7efa05625
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/fc86f7bec83fd2da6794b2d0938f4b37697b6527...ed06a20ad8c09e92f270943efe19c0f7efa05625
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250223/bc3ec81e/attachment-0001.html>
More information about the ghc-commits
mailing list