[Git][ghc/ghc][wip/t21766] 12 commits: JS: avoid head/tail and unpackFS
Finley McIlwaine (@FinleyMcIlwaine)
gitlab at gitlab.haskell.org
Wed Feb 8 16:00:35 UTC 2023
Finley McIlwaine pushed to branch wip/t21766 at Glasgow Haskell Compiler / GHC
Commits:
c1670c6b by Sylvain Henry at 2023-02-07T21:25:18-05:00
JS: avoid head/tail and unpackFS
- - - - -
a9912de7 by Krzysztof Gogolewski at 2023-02-07T21:25:53-05:00
testsuite: Fix Python warnings (#22856)
- - - - -
cc81f32e by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Restructure IPE buffer layout
Reference ticket #21766
This commit restructures IPE buffer list entries to not contain
references to their corresponding info tables. IPE buffer list nodes now
point to two lists of equal length, one holding the list of info table
pointers and one holding the corresponding entries for each info table.
This will allow the entry data to be compressed without losing the
references to the info tables.
- - - - -
d28eab3d by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Add IPE compression to configure
Reference ticket #21766
Adds an `--enable-ipe-data-compreesion` flag to the configure script
which will check for libzstd and set the appropriate flags to allow for
IPE data compression in the compiler
- - - - -
008024e8 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
IPE data compression
Reference ticket #21766
When IPE data compression is enabled, compress the emitted IPE buffer
entries and decompress them in the RTS.
- - - - -
2158fbe5 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Fix libzstd detection in configure and RTS
Ensure that `HAVE_LIBZSTD` gets defined to either 0 or 1 in all cases
and properly check that before IPE data decompression in the RTS. See
ticket #21766.
- - - - -
bcc45839 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Add note describing IPE data compression
See ticket #21766
- - - - -
cca0f881 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Fix byte order of IPE data, fix IPE tests
Make sure byte order of written IPE buffer entries matches target.
Make sure the IPE-related tests properly access the fields of IPE buffer
entry nodes with the new IPE layout.
This commit also introduces checks to avoid importing modules if IPE
compression is not enabled.
See ticket #21766.
- - - - -
d8deebcf by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Fix IPE data decompression buffer allocation
Capacity of buffers allocated for decompressed IPE data was
incorrect due to a misuse of the `ZSTD_findFrameCompressedSize`
function. Fix by always storing decompressed size of IPE data in IPE
buffer list nodes and using `ZSTD_findFrameCompressedSize` to determine
the size of the compressed data.
See ticket #21766
- - - - -
38a6dca8 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Add optional dependencies to ./configure output
Changes the configure script to indicate whether libnuma, libzstd, or
libdw are being used as dependencies due to their optional features
being enabled.
- - - - -
49b95513 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Add IPE-enabled builds to CI
- Adds an IPE job to the CI pipeline which is triggered by the ~IPE label
- Introduces CI logic to enable IPE data compression
- Enables uncompressed IPE data on debug CI job
- Regenerates jobs.yaml
MR https://gitlab.haskell.org/ghc/ci-images/-/merge_requests/112 on the
images repository is meant to ensure that the proper images have
libzstd-dev installed.
- - - - -
2142c448 by Finley McIlwaine at 2023-02-08T09:00:04-07:00
Update user's guide and release notes
Add mention of IPE data compression to user's guide and the release
notes for 9.8.1. Also note the impact compression has on binary size in
both places.
See ticket #21766
- - - - -
25 changed files:
- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC/StgToCmm/InfoTableProv.hs
- compiler/GHC/StgToJS/Printer.hs
- compiler/ghc.cabal.in
- configure.ac
- docs/users_guide/9.8.1-notes.rst
- docs/users_guide/debug-info.rst
- hadrian/cfg/system.config.in
- hadrian/src/Oracles/Flag.hs
- hadrian/src/Oracles/Setting.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- m4/fp_find_libnuma.m4
- + m4/fp_find_libzstd.m4
- rts/IPE.c
- rts/IPE.h
- rts/include/rts/IPE.h
- rts/rts.cabal.in
- testsuite/driver/runtests.py
- testsuite/driver/testlib.py
- testsuite/tests/rts/ipe/ipeEventLog_fromMap.c
- testsuite/tests/rts/ipe/ipeMap.c
- testsuite/tests/rts/ipe/ipe_lib.c
- testsuite/tests/rts/ipe/ipe_lib.h
Changes:
=====================================
.gitlab/gen_ci.hs
=====================================
@@ -140,6 +140,8 @@ data BuildConfig
, llvmBootstrap :: Bool
, withAssertions :: Bool
, withNuma :: Bool
+ , withZstd :: Bool
+ , withIpe :: Bool
, crossTarget :: Maybe String
, crossEmulator :: CrossEmulator
, configureWrapper :: Maybe String
@@ -152,10 +154,11 @@ data BuildConfig
-- Extra arguments to pass to ./configure due to the BuildConfig
configureArgsStr :: BuildConfig -> String
configureArgsStr bc = unwords $
- ["--enable-unregisterised"| unregisterised bc ]
+ ["--enable-unregisterised"| unregisterised bc ]
++ ["--disable-tables-next-to-code" | not (tablesNextToCode bc) ]
++ ["--with-intree-gmp" | Just _ <- pure (crossTarget bc) ]
++ ["--with-system-libffi" | crossTarget bc == Just "wasm32-wasi" ]
+ ++ ["--enable-ipe-data-compression" | withZstd bc ]
-- Compute the hadrian flavour from the BuildConfig
mkJobFlavour :: BuildConfig -> Flavour
@@ -165,11 +168,18 @@ mkJobFlavour BuildConfig{..} = Flavour buildFlavour opts
[Dwarf | withDwarf] ++
[FullyStatic | fullyStatic] ++
[ThreadSanitiser | threadSanitiser] ++
- [NoSplitSections | noSplitSections, buildFlavour == Release ]
+ [NoSplitSections | noSplitSections, buildFlavour == Release ] ++
+ [Ipe | withIpe]
data Flavour = Flavour BaseFlavour [FlavourTrans]
-data FlavourTrans = Llvm | Dwarf | FullyStatic | ThreadSanitiser | NoSplitSections
+data FlavourTrans =
+ Llvm
+ | Dwarf
+ | FullyStatic
+ | ThreadSanitiser
+ | NoSplitSections
+ | Ipe
data BaseFlavour = Release | Validate | SlowValidate deriving Eq
@@ -187,6 +197,8 @@ vanilla = BuildConfig
, llvmBootstrap = False
, withAssertions = False
, withNuma = False
+ , withZstd = False
+ , withIpe = False
, crossTarget = Nothing
, crossEmulator = NoEmulator
, configureWrapper = Nothing
@@ -218,6 +230,11 @@ debug = vanilla { buildFlavour = SlowValidate
, withNuma = True
}
+ipe :: BuildConfig
+ipe = vanilla { withIpe = True
+ , withZstd = True
+ }
+
static :: BuildConfig
static = vanilla { fullyStatic = True }
@@ -306,17 +323,18 @@ testEnv arch opsys bc = intercalate "-" $
-- | The hadrian flavour string we are going to use for this build
flavourString :: Flavour -> String
-flavourString (Flavour base trans) = baseString base ++ concatMap (("+" ++) . flavourString) trans
+flavourString (Flavour base trans) = base_string base ++ concatMap (("+" ++) . flavour_string) trans
where
- baseString Release = "release"
- baseString Validate = "validate"
- baseString SlowValidate = "slow-validate"
+ base_string Release = "release"
+ base_string Validate = "validate"
+ base_string SlowValidate = "slow-validate"
- flavourString Llvm = "llvm"
- flavourString Dwarf = "debug_info"
- flavourString FullyStatic = "fully_static"
- flavourString ThreadSanitiser = "thread_sanitizer"
- flavourString NoSplitSections = "no_split_sections"
+ flavour_string Llvm = "llvm"
+ flavour_string Dwarf = "debug_info"
+ flavour_string FullyStatic = "fully_static"
+ flavour_string ThreadSanitiser = "thread_sanitizer"
+ flavour_string NoSplitSections = "no_split_sections"
+ flavour_string Ipe = "ipe"
-- The path to the docker image (just for linux builders)
dockerImage :: Arch -> Opsys -> Maybe String
@@ -509,7 +527,7 @@ manualRule rules = rules { when = Manual }
-- For example, even if you don't explicitly disable a rule it will end up in the
-- rule list with the OFF state.
enumRules :: OnOffRules -> [OnOffRule]
-enumRules o = map lkup rules
+enumRules o = map lkup rulesList
where
enabled_rules = rule_set o
lkup r = OnOffRule (if S.member r enabled_rules then On else Off) r
@@ -544,6 +562,7 @@ data Rule = FastCI -- ^ Run this job when the fast-ci label is set
| Nightly -- ^ Only run this job in the nightly pipeline
| LLVMBackend -- ^ Only run this job when the "LLVM backend" label is present
| FreeBSDLabel -- ^ Only run this job when the "FreeBSD" label is set.
+ | IpeData -- ^ Only run this job when the "IPE" label is set
| Disable -- ^ Don't run this job.
deriving (Bounded, Enum, Ord, Eq)
@@ -568,12 +587,14 @@ ruleString On ReleaseOnly = "$RELEASE_JOB == \"yes\""
ruleString Off ReleaseOnly = "$RELEASE_JOB != \"yes\""
ruleString On Nightly = "$NIGHTLY"
ruleString Off Nightly = "$NIGHTLY == null"
+ruleString On IpeData = "$CI_MERGE_REQUEST_LABELS =~ /.*IPE.*/"
+ruleString Off IpeData = true
ruleString On Disable = false
ruleString Off Disable = true
-- Enumeration of all the rules
-rules :: [Rule]
-rules = [minBound .. maxBound]
+rulesList :: [Rule]
+rulesList = [minBound .. maxBound]
-- | A 'Job' is the description of a single job in a gitlab pipeline. The
-- job contains all the information about how to do the build but can be further
@@ -856,7 +877,7 @@ job_groups =
, modifyValidateJobs manual tsan_jobs
, modifyValidateJobs manual (validateBuilds Amd64 (Linux Debian10) noTntc)
, addValidateRule LLVMBackend (validateBuilds Amd64 (Linux Debian10) llvm)
-
+ , addValidateRule IpeData (validateBuilds Amd64 (Linux Debian10) ipe)
, disableValidate (standardBuilds Amd64 (Linux Debian11))
-- We still build Deb9 bindists for now due to Ubuntu 18 and Linux Mint 19
-- not being at EOL until April 2023 and they still need tinfo5.
=====================================
.gitlab/jobs.yaml
=====================================
@@ -35,7 +35,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -97,7 +97,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -155,7 +155,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -213,7 +213,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -276,7 +276,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -335,7 +335,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -394,7 +394,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -453,7 +453,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -518,7 +518,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -579,7 +579,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -640,7 +640,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -701,7 +701,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -764,7 +764,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -825,7 +825,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -888,7 +888,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -948,7 +948,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1007,7 +1007,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1066,7 +1066,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1126,7 +1126,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1185,7 +1185,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1244,7 +1244,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1268,6 +1268,65 @@
"XZ_OPT": "-9"
}
},
+ "nightly-x86_64-linux-deb10-validate+ipe": {
+ "after_script": [
+ ".gitlab/ci.sh save_cache",
+ ".gitlab/ci.sh clean",
+ "cat ci_timings"
+ ],
+ "allow_failure": false,
+ "artifacts": {
+ "expire_in": "8 weeks",
+ "paths": [
+ "ghc-x86_64-linux-deb10-validate+ipe.tar.xz",
+ "junit.xml"
+ ],
+ "reports": {
+ "junit": "junit.xml"
+ },
+ "when": "always"
+ },
+ "cache": {
+ "key": "x86_64-linux-deb10-$CACHE_REV",
+ "paths": [
+ "cabal-cache",
+ "toolchain"
+ ]
+ },
+ "dependencies": [],
+ "image": "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb10:$DOCKER_REV",
+ "needs": [
+ {
+ "artifacts": false,
+ "job": "hadrian-ghc-in-ghci"
+ }
+ ],
+ "rules": [
+ {
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "when": "on_success"
+ }
+ ],
+ "script": [
+ "sudo chown ghc:ghc -R .",
+ ".gitlab/ci.sh setup",
+ ".gitlab/ci.sh configure",
+ ".gitlab/ci.sh build_hadrian",
+ ".gitlab/ci.sh test_hadrian"
+ ],
+ "stage": "full-build",
+ "tags": [
+ "x86_64-linux"
+ ],
+ "variables": {
+ "BIGNUM_BACKEND": "gmp",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-validate+ipe",
+ "BUILD_FLAVOUR": "validate+ipe",
+ "CONFIGURE_ARGS": "--enable-ipe-data-compression",
+ "TEST_ENV": "x86_64-linux-deb10-validate+ipe",
+ "XZ_OPT": "-9"
+ }
+ },
"nightly-x86_64-linux-deb10-validate+llvm": {
"after_script": [
".gitlab/ci.sh save_cache",
@@ -1303,7 +1362,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1362,7 +1421,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1423,7 +1482,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1484,7 +1543,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1546,7 +1605,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1605,7 +1664,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1664,7 +1723,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1725,7 +1784,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1787,7 +1846,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1848,7 +1907,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1908,7 +1967,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1967,7 +2026,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2022,7 +2081,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2081,7 +2140,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2144,7 +2203,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2208,7 +2267,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2268,7 +2327,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2328,7 +2387,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2394,7 +2453,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2458,7 +2517,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2522,7 +2581,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2583,7 +2642,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2643,7 +2702,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2703,7 +2762,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2763,7 +2822,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2823,7 +2882,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2885,7 +2944,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2947,7 +3006,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3010,7 +3069,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3071,7 +3130,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3131,7 +3190,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3187,7 +3246,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3247,7 +3306,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3311,7 +3370,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3375,7 +3434,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*FreeBSD.*/) && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*FreeBSD.*/) && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3435,7 +3494,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3495,7 +3554,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3557,7 +3616,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3616,7 +3675,7 @@
"rules": [
{
"allow_failure": true,
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "manual"
}
],
@@ -3674,7 +3733,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3733,7 +3792,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3791,7 +3850,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3814,6 +3873,64 @@
"TEST_ENV": "x86_64-linux-deb10-validate+debug_info"
}
},
+ "x86_64-linux-deb10-validate+ipe": {
+ "after_script": [
+ ".gitlab/ci.sh save_cache",
+ ".gitlab/ci.sh clean",
+ "cat ci_timings"
+ ],
+ "allow_failure": false,
+ "artifacts": {
+ "expire_in": "2 weeks",
+ "paths": [
+ "ghc-x86_64-linux-deb10-validate+ipe.tar.xz",
+ "junit.xml"
+ ],
+ "reports": {
+ "junit": "junit.xml"
+ },
+ "when": "always"
+ },
+ "cache": {
+ "key": "x86_64-linux-deb10-$CACHE_REV",
+ "paths": [
+ "cabal-cache",
+ "toolchain"
+ ]
+ },
+ "dependencies": [],
+ "image": "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb10:$DOCKER_REV",
+ "needs": [
+ {
+ "artifacts": false,
+ "job": "hadrian-ghc-in-ghci"
+ }
+ ],
+ "rules": [
+ {
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*IPE.*/) && (\"true\" == \"true\")",
+ "when": "on_success"
+ }
+ ],
+ "script": [
+ "sudo chown ghc:ghc -R .",
+ ".gitlab/ci.sh setup",
+ ".gitlab/ci.sh configure",
+ ".gitlab/ci.sh build_hadrian",
+ ".gitlab/ci.sh test_hadrian"
+ ],
+ "stage": "full-build",
+ "tags": [
+ "x86_64-linux"
+ ],
+ "variables": {
+ "BIGNUM_BACKEND": "gmp",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-validate+ipe",
+ "BUILD_FLAVOUR": "validate+ipe",
+ "CONFIGURE_ARGS": "--enable-ipe-data-compression",
+ "TEST_ENV": "x86_64-linux-deb10-validate+ipe"
+ }
+ },
"x86_64-linux-deb10-validate+llvm": {
"after_script": [
".gitlab/ci.sh save_cache",
@@ -3849,7 +3966,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && ($CI_MERGE_REQUEST_LABELS =~ /.*LLVM backend.*/) && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && ($CI_MERGE_REQUEST_LABELS =~ /.*LLVM backend.*/) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3908,7 +4025,7 @@
"rules": [
{
"allow_failure": true,
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "manual"
}
],
@@ -3968,7 +4085,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4028,7 +4145,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4089,7 +4206,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4145,7 +4262,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
=====================================
compiler/GHC/StgToCmm/InfoTableProv.hs
=====================================
@@ -1,66 +1,187 @@
+{-# LANGUAGE CPP #-}
+
module GHC.StgToCmm.InfoTableProv (emitIpeBufferListNode) where
+import Foreign
+
+#if defined(HAVE_LIBZSTD)
+import Foreign.C.Types
+import qualified Data.ByteString.Internal as BSI
+import GHC.IO (unsafePerformIO)
+#endif
+
+import GHC.Data.FastString (fastStringToShortText)
import GHC.Prelude
import GHC.Platform
+import GHC.Types.SrcLoc (pprUserRealSpan, srcSpanFile)
import GHC.Unit.Module
import GHC.Utils.Outputable
-import GHC.Types.SrcLoc (pprUserRealSpan, srcSpanFile)
-import GHC.Data.FastString (fastStringToShortText)
+import GHC.Cmm
import GHC.Cmm.CLabel
-import GHC.Cmm.Expr
import GHC.Cmm.Utils
import GHC.StgToCmm.Config
-import GHC.StgToCmm.Lit (newByteStringCLit)
import GHC.StgToCmm.Monad
-import GHC.StgToCmm.Utils
import GHC.Data.ShortText (ShortText)
import qualified GHC.Data.ShortText as ST
-import qualified Data.Map.Strict as M
import Control.Monad.Trans.State.Strict
+
import qualified Data.ByteString as BS
import qualified Data.ByteString.Builder as BSB
import qualified Data.ByteString.Lazy as BSL
+import qualified Data.Map.Strict as M
+
+{-
+Note [Compression and Decompression of IPE data]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Compiling with `-finfo-table-map` causes build results to include a map from
+info tables to source positions called the info table provenance entry (IPE)
+map. See Note [Mapping Info Tables to Source Positions]. The IPE information
+can grow the size of build results significantly. At the time of writing, a
+default build of GHC results in a total of 109M of libHSghc-*.so build results.
+A default+ipe build of GHC (see ./hadrian/doc/flavours.md) results in 262M of
+libHSghc-*.so build results without compression.
-emitIpeBufferListNode :: Module
- -> [InfoProvEnt]
- -> FCode ()
+We reduce the impact of IPE data on the size of build results by compressing
+the data before it is emitted using the zstd compression library. See
+Note [The Info Table Provenance Entry (IPE) Map] for information on the layout
+of IPE data on disk and in the RTS. We cannot simply compress all data held in
+the IPE entry buffer, as the pointers to info tables must be converted to
+memory addresses during linking. Therefore, we can only compress the strings
+table and the IPE entries themselves (which essentially only consist of indices
+into the strings table).
+
+With compression, a default+ipe build of GHC results in a total of 205M of
+libHSghc-*.so build results. This is over a 20% reduction from the uncompressed
+case.
+
+Decompression happens lazily, as it only occurs when the IPE map is
+constructed (which is also done lazily on first lookup or traversal). During
+construction, the 'compressed' field of each IPE buffer list node is examined.
+If the field indicates that the data has been compressed, the entry data and
+strings table are decompressed before continuing with the normal IPE map
+construction.
+-}
+
+emitIpeBufferListNode ::
+ Module
+ -> [InfoProvEnt]
+ -> FCode ()
emitIpeBufferListNode _ [] = return ()
emitIpeBufferListNode this_mod ents = do
cfg <- getStgToCmmConfig
- let ctx = stgToCmmContext cfg
+
+ tables_lbl <- mkStringLitLabel <$> newUnique
+ strings_lbl <- mkStringLitLabel <$> newUnique
+ entries_lbl <- mkStringLitLabel <$> newUnique
+
+ let ctx = stgToCmmContext cfg
platform = stgToCmmPlatform cfg
+ int n = mkIntCLit platform n
+
+ (cg_ipes, strtab) = flip runState emptyStringTable $ do
+ module_name <- lookupStringTable $ ST.pack $ renderWithContext ctx (ppr this_mod)
+ mapM (toCgIPE platform ctx module_name) ents
+
+ tables :: [CmmStatic]
+ tables = map (CmmStaticLit . CmmLabel . ipeInfoTablePtr) cg_ipes
+
+ uncompressed_strings :: BS.ByteString
+ uncompressed_strings = getStringTableStrings strtab
+
+ strings_bytes :: BS.ByteString
+ strings_bytes = compress defaultCompressionLevel uncompressed_strings
+
+ strings :: [CmmStatic]
+ strings = [CmmString strings_bytes]
+
+ uncompressed_entries :: BS.ByteString
+ uncompressed_entries = toIpeBufferEntries (platformByteOrder platform) cg_ipes
- let (cg_ipes, strtab) = flip runState emptyStringTable $ do
- module_name <- lookupStringTable $ ST.pack $ renderWithContext ctx (ppr this_mod)
- mapM (toCgIPE platform ctx module_name) ents
-
- let -- Emit the fields of an IpeBufferEntry struct.
- toIpeBufferEntry :: CgInfoProvEnt -> [CmmLit]
- toIpeBufferEntry cg_ipe =
- [ CmmLabel (ipeInfoTablePtr cg_ipe)
- , strtab_offset (ipeTableName cg_ipe)
- , strtab_offset (ipeClosureDesc cg_ipe)
- , strtab_offset (ipeTypeDesc cg_ipe)
- , strtab_offset (ipeLabel cg_ipe)
- , strtab_offset (ipeModuleName cg_ipe)
- , strtab_offset (ipeSrcFile cg_ipe)
- , strtab_offset (ipeSrcSpan cg_ipe)
- , int32 0
- ]
-
- int n = mkIntCLit platform n
- int32 n = CmmInt n W32
- strtab_offset (StrTabOffset n) = int32 (fromIntegral n)
-
- strings <- newByteStringCLit (getStringTableStrings strtab)
- let lits = [ zeroCLit platform -- 'next' field
- , strings -- 'strings' field
- , int $ length cg_ipes -- 'count' field
- ] ++ concatMap toIpeBufferEntry cg_ipes
- emitDataLits (mkIPELabel this_mod) lits
+ entries_bytes :: BS.ByteString
+ entries_bytes = compress defaultCompressionLevel uncompressed_entries
+
+ entries :: [CmmStatic]
+ entries = [CmmString entries_bytes]
+
+ ipe_buffer_lbl :: CLabel
+ ipe_buffer_lbl = mkIPELabel this_mod
+
+ ipe_buffer_node :: [CmmStatic]
+ ipe_buffer_node = map CmmStaticLit
+ [ -- 'next' field
+ zeroCLit platform
+
+ -- 'compressed' field
+ , int do_compress
+
+ -- 'count' field
+ , int $ length cg_ipes
+
+ -- 'tables' field
+ , CmmLabel tables_lbl
+
+ -- 'entries' field
+ , CmmLabel entries_lbl
+
+ -- 'entries_size' field (decompressed size)
+ , int $ BS.length uncompressed_entries
+
+ -- 'string_table' field
+ , CmmLabel strings_lbl
+
+ -- 'string_table_size' field (decompressed size)
+ , int $ BS.length uncompressed_strings
+ ]
+
+ -- Emit the list of info table pointers
+ emitDecl $ CmmData
+ (Section Data tables_lbl)
+ (CmmStaticsRaw tables_lbl tables)
+
+ -- Emit the strings table
+ emitDecl $ CmmData
+ (Section Data strings_lbl)
+ (CmmStaticsRaw strings_lbl strings)
+
+ -- Emit the list of IPE buffer entries
+ emitDecl $ CmmData
+ (Section Data entries_lbl)
+ (CmmStaticsRaw entries_lbl entries)
+
+ -- Emit the IPE buffer list node
+ emitDecl $ CmmData
+ (Section Data ipe_buffer_lbl)
+ (CmmStaticsRaw ipe_buffer_lbl ipe_buffer_node)
+
+-- | Emit the fields of an IpeBufferEntry struct for each entry in a given list.
+toIpeBufferEntries ::
+ ByteOrder -- ^ Byte order to write the data in
+ -> [CgInfoProvEnt] -- ^ List of IPE buffer entries
+ -> BS.ByteString
+toIpeBufferEntries byte_order cg_ipes =
+ BSL.toStrict . BSB.toLazyByteString . mconcat
+ $ map (mconcat . map word32Builder . to_ipe_buf_ent) cg_ipes
+ where
+ to_ipe_buf_ent :: CgInfoProvEnt -> [Word32]
+ to_ipe_buf_ent cg_ipe =
+ [ ipeTableName cg_ipe
+ , ipeClosureDesc cg_ipe
+ , ipeTypeDesc cg_ipe
+ , ipeLabel cg_ipe
+ , ipeModuleName cg_ipe
+ , ipeSrcFile cg_ipe
+ , ipeSrcSpan cg_ipe
+ , 0 -- padding
+ ]
+
+ word32Builder :: Word32 -> BSB.Builder
+ word32Builder = case byte_order of
+ BigEndian -> BSB.word32BE
+ LittleEndian -> BSB.word32LE
toCgIPE :: Platform -> SDocContext -> StrTabOffset -> InfoProvEnt -> State StringTable CgInfoProvEnt
toCgIPE platform ctx module_name ipe = do
@@ -76,7 +197,7 @@ toCgIPE platform ctx module_name ipe = do
coords = renderWithContext ctx (pprUserRealSpan False span)
in (file, coords)
label <- lookupStringTable $ ST.pack label_str
- src_file <- lookupStringTable $ src_loc_file
+ src_file <- lookupStringTable src_loc_file
src_span <- lookupStringTable $ ST.pack src_loc_span
return $ CgInfoProvEnt { ipeInfoTablePtr = infoTablePtr ipe
, ipeTableName = table_name
@@ -104,7 +225,7 @@ data StringTable = StringTable { stStrings :: DList ShortText
, stLookup :: !(M.Map ShortText StrTabOffset)
}
-newtype StrTabOffset = StrTabOffset Int
+type StrTabOffset = Word32
emptyStringTable :: StringTable
emptyStringTable =
@@ -129,9 +250,50 @@ lookupStringTable str = state $ \st ->
, stLength = stLength st + ST.byteLength str + 1
, stLookup = M.insert str res (stLookup st)
}
- res = StrTabOffset (stLength st)
+ res = fromIntegral (stLength st)
in (res, st')
+do_compress :: Int
+compress :: Int -> BS.ByteString -> BS.ByteString
+#if !defined(HAVE_LIBZSTD)
+do_compress = 0
+compress _ bs = bs
+#else
+do_compress = 1
+
+compress clvl (BSI.PS srcForeignPtr off len) = unsafePerformIO $
+ withForeignPtr srcForeignPtr $ \srcPtr -> do
+ maxCompressedSize <- zstd_compress_bound $ fromIntegral len
+ dstForeignPtr <- BSI.mallocByteString (fromIntegral maxCompressedSize)
+ withForeignPtr dstForeignPtr $ \dstPtr -> do
+ compressedSize <- fromIntegral <$>
+ zstd_compress
+ dstPtr
+ maxCompressedSize
+ (srcPtr `plusPtr` off)
+ (fromIntegral len)
+ (fromIntegral clvl)
+ BSI.create compressedSize $ \p -> BSI.memcpy p dstPtr compressedSize
+
+foreign import ccall unsafe "ZSTD_compress"
+ zstd_compress ::
+ Ptr dst -- ^ Destination buffer
+ -> CSize -- ^ Capacity of destination buffer
+ -> Ptr src -- ^ Source buffer
+ -> CSize -- ^ Size of source buffer
+ -> CInt -- ^ Compression level
+ -> IO CSize
+
+-- | Compute the maximum compressed size for a given source buffer size
+foreign import ccall unsafe "ZSTD_compressBound"
+ zstd_compress_bound ::
+ CSize -- ^ Size of source buffer
+ -> IO CSize
+#endif
+
+defaultCompressionLevel :: Int
+defaultCompressionLevel = 3
+
newtype DList a = DList ([a] -> [a])
emptyDList :: DList a
=====================================
compiler/GHC/StgToJS/Printer.hs
=====================================
@@ -108,19 +108,17 @@ ghcjsRenderJsV r (JHash m)
where
quoteIfRequired :: FastString -> Doc
quoteIfRequired x
- | isUnquotedKey x' = text x'
- | otherwise = PP.squotes (text x')
- where x' = unpackFS x
-
- isUnquotedKey :: String -> Bool
- isUnquotedKey x | null x = False
- | all isDigit x = True
- | otherwise = validFirstIdent (head x)
- && all validOtherIdent (tail x)
+ | isUnquotedKey x = ftext x
+ | otherwise = PP.squotes (ftext x)
+ isUnquotedKey :: FastString -> Bool
+ isUnquotedKey fs = case unpackFS fs of
+ [] -> False
+ s@(c:cs) -> all isDigit s || (validFirstIdent c && all validOtherIdent cs)
validFirstIdent c = c == '_' || c == '$' || isAlpha c
validOtherIdent c = isAlpha c || isDigit c
+
ghcjsRenderJsV r v = renderJsV defaultRenderJs r v
prettyBlock :: RenderJs -> [JStat] -> Doc
=====================================
compiler/ghc.cabal.in
=====================================
@@ -57,6 +57,10 @@ Flag build-tool-depends
Description: Use build-tool-depends
Default: True
+Flag with-libzstd
+ Default: False
+ Manual: True
+
Library
Default-Language: Haskell2010
Exposed: False
@@ -71,6 +75,10 @@ Library
if flag(build-tool-depends)
build-tool-depends: alex:alex >= 3.2.6, happy:happy >= 1.20.0, genprimopcode:genprimopcode, deriveConstants:deriveConstants
+ if flag(with-libzstd)
+ extra-libraries: zstd
+ CPP-Options: -DHAVE_LIBZSTD
+
Build-Depends: base >= 4.11 && < 4.19,
deepseq >= 1.4 && < 1.5,
directory >= 1 && < 1.4,
=====================================
configure.ac
=====================================
@@ -1122,6 +1122,10 @@ AC_DEFINE_UNQUOTED([RTS_LINKER_USE_MMAP], [$RtsLinkerUseMmap],
GHC_ADJUSTORS_METHOD([Target])
AC_SUBST([UseLibffiForAdjustors])
+dnl ** IPE data compression
+dnl --------------------------------------------------------------
+FP_FIND_LIBZSTD
+
dnl ** Other RTS features
dnl --------------------------------------------------------------
FP_FIND_LIBDW
@@ -1267,6 +1271,17 @@ echo "\
makeinfo : $MAKEINFO
git : $GIT
cabal-install : $CABAL
+"
+
+USING_LIBNUMA=$(if [ "$HaveLibNuma" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+USING_LIBZSTD=$(if [ "$HaveLibZstd" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+USING_LIBDW=$(if [ "$USE_LIBDW" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+
+echo "\
+ Using optional dependencies:
+ libnuma : $USING_LIBNUMA
+ libzstd : $USING_LIBZSTD
+ libdw : $USING_LIBDW
Using LLVM tools
clang : $ClangCmd
=====================================
docs/users_guide/9.8.1-notes.rst
=====================================
@@ -21,6 +21,17 @@ Compiler
foo (\x -> x*2 + x)
+- The compiler may now be configured to compress the debugging information
+ included in :ghc-flag:`-finfo-table-map` enabled binaries. To do so, one must
+ build GHC from source (see
+ `here<https://gitlab.haskell.org/ghc/ghc/-/wikis/building>` for directions)
+ and supply the ``--enable-ipe-data-compression`` flag to the ``configure``
+ script. **Note**: This feature requires that the machine building GHC has
+ `libzstd <https://github.com/facebook/zstd/>`_ installed.
+
+ In a test compiling GHC itself, the size of the :ghc-flag:`-finfo-table-map`
+ enabled build results was reduced by over 20% when compression was enabled.
+
GHCi
~~~~
=====================================
docs/users_guide/debug-info.rst
=====================================
@@ -370,9 +370,23 @@ to a source location. This lookup table is generated by using the ``-finfo-table
also want more precise information about constructor info tables then you
should also use :ghc-flag:`-fdistinct-constructor-tables`.
- This flag will increase the binary size by quite a lot, depending on how
- big your project is. For compiling a project the size of GHC the overhead was
- about 200 megabytes.
+ The :ghc-flag:`-finfo-table-map` flag will increase the binary size by quite
+ a lot, depending on how big your project is. For compiling a project the
+ size of GHC the overhead was about 200 megabytes.
+
+ :since: 9.8
+
+ If you wish to reduce the size of :ghc-flag:`-finfo-table-map` enabled
+ binaries, consider building GHC from source and supplying the
+ ``--enable-ipe-data-compression`` flag to the ``configure`` script. This
+ will cause GHC to compress the :ghc-flag:`-finfo-table-map` related
+ debugging information included in binaries using the `libzstd
+ <https://github.com/facebook/zstd/>`_ compression library. **Note**: This
+ feature requires that the machine building GHC has
+ `libzstd <https://github.com/facebook/zstd/>`_ installed.
+
+ In a test compiling GHC itself, the size of the :ghc-flag:`-finfo-table-map`
+ enabled build results was reduced by over 20% when compression was enabled.
.. ghc-flag:: -fdistinct-constructor-tables
:shortdesc: Generate a fresh info table for each usage
=====================================
hadrian/cfg/system.config.in
=====================================
@@ -200,10 +200,14 @@ libdw-lib-dir = @LibdwLibDir@
libnuma-include-dir = @LibNumaIncludeDir@
libnuma-lib-dir = @LibNumaLibDir@
+libzstd-include-dir = @LibZstdIncludeDir@
+libzstd-lib-dir = @LibZstdLibDir@
+
# Optional Dependencies:
#=======================
use-lib-dw = @UseLibdw@
+use-lib-zstd = @UseLibZstd@
use-lib-numa = @UseLibNuma@
use-lib-m = @UseLibm@
use-lib-rt = @UseLibrt@
=====================================
hadrian/src/Oracles/Flag.hs
=====================================
@@ -35,6 +35,7 @@ data Flag = ArSupportsAtFile
| UseLibffiForAdjustors
| UseLibdw
| UseLibnuma
+ | UseLibzstd
| UseLibm
| UseLibrt
| UseLibdl
@@ -65,6 +66,7 @@ flag f = do
UseLibffiForAdjustors -> "use-libffi-for-adjustors"
UseLibdw -> "use-lib-dw"
UseLibnuma -> "use-lib-numa"
+ UseLibzstd -> "use-lib-zstd"
UseLibm -> "use-lib-m"
UseLibrt -> "use-lib-rt"
UseLibdl -> "use-lib-dl"
=====================================
hadrian/src/Oracles/Setting.hs
=====================================
@@ -60,6 +60,8 @@ data Setting = BuildArch
| LibdwLibDir
| LibnumaIncludeDir
| LibnumaLibDir
+ | LibZstdIncludeDir
+ | LibZstdLibDir
| LlvmTarget
| ProjectGitCommitId
| ProjectName
@@ -162,6 +164,8 @@ setting key = lookupSystemConfig $ case key of
LibdwLibDir -> "libdw-lib-dir"
LibnumaIncludeDir -> "libnuma-include-dir"
LibnumaLibDir -> "libnuma-lib-dir"
+ LibZstdIncludeDir -> "libzstd-include-dir"
+ LibZstdLibDir -> "libzstd-lib-dir"
LlvmTarget -> "llvm-target"
ProjectGitCommitId -> "project-git-commit-id"
ProjectName -> "project-name"
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -291,6 +291,7 @@ rtsCabalFlags = mconcat
, flag "CabalNeedLibpthread" UseLibpthread
, flag "CabalHaveLibbfd" UseLibbfd
, flag "CabalHaveLibNuma" UseLibnuma
+ , flag "CabalHaveLibZstd" UseLibzstd
, flag "CabalNeedLibatomic" NeedLibatomic
, flag "CabalUseSystemLibFFI" UseSystemFfi
, flag "CabalLibffiAdjustors" UseLibffiForAdjustors
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -77,6 +77,7 @@ packageArgs = do
[ andM [expr ghcWithInterpreter, notStage0] `cabalFlag` "internal-interpreter"
, notM cross `cabalFlag` "terminfo"
, arg "-build-tool-depends"
+ , flag UseLibzstd `cabalFlag` "with-libzstd"
]
, builder (Haddock BuildPackage) ? arg ("--optghc=-I" ++ path) ]
@@ -286,6 +287,8 @@ rtsPackageArgs = package rts ? do
libdwLibraryDir <- getSetting LibdwLibDir
libnumaIncludeDir <- getSetting LibnumaIncludeDir
libnumaLibraryDir <- getSetting LibnumaLibDir
+ libzstdIncludeDir <- getSetting LibZstdIncludeDir
+ libzstdLibraryDir <- getSetting LibZstdLibDir
-- Arguments passed to GHC when compiling C and .cmm sources.
let ghcArgs = mconcat
@@ -392,6 +395,7 @@ rtsPackageArgs = package rts ? do
, builder (Cabal Setup) ? mconcat
[ cabalExtraDirs libdwIncludeDir libdwLibraryDir
, cabalExtraDirs libnumaIncludeDir libnumaLibraryDir
+ , cabalExtraDirs libzstdIncludeDir libzstdLibraryDir
, useSystemFfi ? cabalExtraDirs ffiIncludeDir ffiLibraryDir
]
, builder (Cc (FindCDependencies CDep)) ? cArgs
=====================================
m4/fp_find_libnuma.m4
=====================================
@@ -30,7 +30,7 @@ AC_DEFUN([FP_FIND_LIBNUMA],
[Enable NUMA memory policy and thread affinity support in the
runtime system via numactl's libnuma [default=auto]])])
- if test "$enable_numa" != "no" ; then
+ if test "$enable_numa" = "yes" ; then
CFLAGS2="$CFLAGS"
CFLAGS="$LIBNUMA_CFLAGS $CFLAGS"
LDFLAGS2="$LDFLAGS"
@@ -41,7 +41,7 @@ AC_DEFUN([FP_FIND_LIBNUMA],
if test "$ac_cv_header_numa_h$ac_cv_header_numaif_h" = "yesyes" ; then
AC_CHECK_LIB(numa, numa_available,HaveLibNuma=1)
fi
- if test "$enable_numa:$HaveLibNuma" = "yes:0" ; then
+ if test "$HaveLibNuma" = "0" ; then
AC_MSG_ERROR([Cannot find system libnuma (required by --enable-numa)])
fi
=====================================
m4/fp_find_libzstd.m4
=====================================
@@ -0,0 +1,79 @@
+AC_DEFUN([FP_FIND_LIBZSTD],
+[
+ dnl ** Is IPE data compression enabled?
+ dnl --------------------------------------------------------------
+ AC_ARG_ENABLE(
+ ipe-data-compression,
+ [AS_HELP_STRING(
+ [--enable-ipe-data-compression],
+ [Enable compression of info table provenance entries using the
+ zstd compression library [default=no]]
+ )],
+ [FP_CAPITALIZE_YES_NO(["$enableval"], [EnableIpeDataCompression])],
+ [EnableIpeDataCompression=NO]
+ )
+
+ HaveLibZstd=0
+ if test "$EnableIpeDataCompression" = "YES"; then
+ dnl ** Have zstd?
+ dnl --------------------------------------------------------------
+ AC_ARG_WITH(
+ libzstd-libraries,
+ [AS_HELP_STRING(
+ [--with-libzstd-libraries=ARG],
+ [Find libraries for libzstd in ARG [default=system default]]
+ )],
+ [
+ LibZstdLibDir="$withval"
+ LIBZSTD_LDFLAGS="-L$withval"
+ ]
+ )
+
+ AC_SUBST(LibZstdLibDir)
+
+ AC_ARG_WITH(
+ libzstd-includes,
+ [AS_HELP_STRING(
+ [--with-libzstd-includes=ARG],
+ [Find includes for libzstd in ARG [default=system default]]
+ )],
+ [
+ LibZstdIncludeDir="$withval"
+ LIBZSTD_CFLAGS="-I$withval"
+ ]
+ )
+
+ AC_SUBST(LibZstdIncludeDir)
+
+ CFLAGS2="$CFLAGS"
+ CFLAGS="$LIBZSTD_CFLAGS $CFLAGS"
+ LDFLAGS2="$LDFLAGS"
+ LDFLAGS="$LIBZSTD_LDFLAGS $LDFLAGS"
+
+ AC_CHECK_HEADERS([zstd.h])
+
+ if test "$ac_cv_header_zstd_h" = "yes" ; then
+ AC_CHECK_LIB(zstd,ZSTD_versionString,HaveLibZstd=1)
+ fi
+ if test "$HaveLibZstd" = "0" ; then
+ AC_MSG_ERROR(
+ [Cannot find system libzstd (required by
+ --enable-ipe-data-compression)]
+ )
+ fi
+
+ CFLAGS="$CFLAGS2"
+ LDFLAGS="$LDFLAGS2"
+ fi
+
+ AC_DEFINE_UNQUOTED([HAVE_LIBZSTD], [$HaveLibZstd], [Define to 1 if you
+ wish to compress IPE data in compiler results (requires libzstd)])
+
+ if test $HaveLibZstd = "1" ; then
+ AC_SUBST([UseLibZstd],[YES])
+ AC_SUBST([CabalHaveLibZstd],[True])
+ else
+ AC_SUBST([UseLibZstd],[NO])
+ AC_SUBST([CabalHaveLibZstd],[False])
+ fi
+])
=====================================
rts/IPE.c
=====================================
@@ -20,6 +20,10 @@
#include <fs_rts.h>
#include <string.h>
+#if HAVE_LIBZSTD == 1
+#include <zstd.h>
+#endif
+
#if defined(TRACING)
#include "Trace.h"
#endif
@@ -36,8 +40,9 @@ collecting IPE lists on registration.
It's a singly linked list of IPE list buffers (IpeBufferListNode). These are
emitted by the code generator, with generally one produced per module. Each
-contains an array of IPE entries and a link field (which is used to link
-buffers onto the pending list.
+contains a pointer to a list of IPE entries, a pointer to a list of info
+table pointers, and a link field (which is used to link buffers onto the
+pending list.
For reasons of space efficiency, IPE entries are represented slightly
differently in the object file than the InfoProvEnt which we ultimately expose
@@ -77,23 +82,23 @@ void exitIpe(void) { }
#endif // THREADED_RTS
-static InfoProvEnt ipeBufferEntryToIpe(const IpeBufferListNode *node, const IpeBufferEntry *ent)
+static InfoProvEnt ipeBufferEntryToIpe(const char *strings, const StgInfoTable *tbl, const IpeBufferEntry ent)
{
- const char *strings = node->string_table;
return (InfoProvEnt) {
- .info = ent->info,
+ .info = tbl,
.prov = {
- .table_name = &strings[ent->table_name],
- .closure_desc = &strings[ent->closure_desc],
- .ty_desc = &strings[ent->ty_desc],
- .label = &strings[ent->label],
- .module = &strings[ent->module_name],
- .src_file = &strings[ent->src_file],
- .src_span = &strings[ent->src_span]
+ .table_name = &strings[ent.table_name],
+ .closure_desc = &strings[ent.closure_desc],
+ .ty_desc = &strings[ent.ty_desc],
+ .label = &strings[ent.label],
+ .module = &strings[ent.module_name],
+ .src_file = &strings[ent.src_file],
+ .src_span = &strings[ent.src_span]
}
};
}
+
#if defined(TRACING)
static void traceIPEFromHashTable(void *data STG_UNUSED, StgWord key STG_UNUSED,
const void *value) {
@@ -105,8 +110,18 @@ void dumpIPEToEventLog(void) {
// Dump pending entries
IpeBufferListNode *cursor = RELAXED_LOAD(&ipeBufferList);
while (cursor != NULL) {
+ IpeBufferEntry *entries;
+ char *strings;
+
+ // Decompress if compressed
+ decompressIPEBufferListNodeIfCompressed(cursor, &entries, &strings);
+
for (uint32_t i = 0; i < cursor->count; i++) {
- const InfoProvEnt ent = ipeBufferEntryToIpe(cursor, &cursor->entries[i]);
+ const InfoProvEnt ent = ipeBufferEntryToIpe(
+ strings,
+ cursor->tables[i],
+ entries[i]
+ );
traceIPE(&ent);
}
cursor = cursor->next;
@@ -120,6 +135,7 @@ void dumpIPEToEventLog(void) {
RELEASE_LOCK(&ipeMapLock);
}
+
#else
void dumpIPEToEventLog(void) { }
@@ -169,16 +185,84 @@ void updateIpeMap() {
}
while (pending != NULL) {
- IpeBufferListNode *currentNode = pending;
- InfoProvEnt *ip_ents = stgMallocBytes(sizeof(InfoProvEnt) * currentNode->count, "updateIpeMap");
- for (uint32_t i = 0; i < currentNode->count; i++) {
- const IpeBufferEntry *ent = ¤tNode->entries[i];
- ip_ents[i] = ipeBufferEntryToIpe(currentNode, ent);
- insertHashTable(ipeMap, (StgWord) ent->info, &ip_ents[i]);
+ IpeBufferListNode *current_node = pending;
+ const IpeBufferEntry *entries;
+ const char *strings;
+
+ // Decompress if compressed
+ decompressIPEBufferListNodeIfCompressed(current_node, &entries, &strings);
+
+ // Convert the on-disk IPE buffer entry representation (IpeBufferEntry)
+ // into the runtime representation (InfoProvEnt)
+ InfoProvEnt *ip_ents = stgMallocBytes(
+ sizeof(InfoProvEnt) * current_node->count,
+ "updateIpeMap: ip_ents"
+ );
+ for (uint32_t i = 0; i < current_node->count; i++) {
+ const IpeBufferEntry ent = entries[i];
+ const StgInfoTable *tbl = current_node->tables[i];
+ ip_ents[i] = ipeBufferEntryToIpe(strings, tbl, ent);
+ insertHashTable(ipeMap, (StgWord) tbl, &ip_ents[i]);
}
- pending = currentNode->next;
+ pending = current_node->next;
}
RELEASE_LOCK(&ipeMapLock);
}
+
+/* Decompress the IPE data and strings table referenced by an IPE buffer list
+node if it is compressed. No matter whether the data is compressed, the pointers
+referenced by the 'entries_dst' and 'string_table_dst' parameters will point at
+the decompressed IPE data and string table for the given node, respectively,
+upon return from this function.
+*/
+void decompressIPEBufferListNodeIfCompressed(IpeBufferListNode *node, IpeBufferEntry **entries_dst, char **string_table_dst) {
+ if (node->compressed) {
+ // The IPE list buffer node indicates that the strings table and
+ // entries list has been compressed. If zstd is not available, fail.
+ // If zstd is available, decompress.
+#if HAVE_LIBZSTD == 0
+ barf("An IPE buffer list node has been compressed, but the \
+ decompression library (zstd) is not available.");
+#else
+ size_t compressed_sz = ZSTD_findFrameCompressedSize(
+ node->string_table,
+ node->string_table_size
+ );
+ char *decompressed_strings = stgMallocBytes(
+ node->string_table_size,
+ "updateIpeMap: decompressed_strings"
+ );
+ ZSTD_decompress(
+ decompressed_strings,
+ node->string_table_size,
+ node->string_table,
+ compressed_sz
+ );
+ *string_table_dst = decompressed_strings;
+
+ // Decompress the IPE data
+ compressed_sz = ZSTD_findFrameCompressedSize(
+ node->entries,
+ node->entries_size
+ );
+ void *decompressed_entries = stgMallocBytes(
+ node->entries_size,
+ "updateIpeMap: decompressed_entries"
+ );
+ ZSTD_decompress(
+ decompressed_entries,
+ node->entries_size,
+ node->entries,
+ compressed_sz
+ );
+ *entries_dst = decompressed_entries;
+#endif // HAVE_LIBZSTD == 0
+
+ } else {
+ // Not compressed, no need to decompress
+ *entries_dst = node->entries;
+ *string_table_dst = node->string_table;
+ }
+}
=====================================
rts/IPE.h
=====================================
@@ -17,5 +17,6 @@ void dumpIPEToEventLog(void);
void updateIpeMap(void);
void initIpe(void);
void exitIpe(void);
+void decompressIPEBufferListNodeIfCompressed(IpeBufferListNode*, IpeBufferEntry**, char**);
#include "EndPrivate.h"
=====================================
rts/include/rts/IPE.h
=====================================
@@ -52,9 +52,6 @@ typedef uint32_t StringIdx;
// The size of this must be a multiple of the word size
// to ensure correct packing.
typedef struct {
- // When TNTC is enabled this will point to the entry code
- // not the info table itself.
- const StgInfoTable *info;
StringIdx table_name;
StringIdx closure_desc;
StringIdx ty_desc;
@@ -69,10 +66,23 @@ GHC_STATIC_ASSERT(sizeof(IpeBufferEntry) % (WORD_SIZE_IN_BITS / 8) == 0, "sizeof
typedef struct IpeBufferListNode_ {
struct IpeBufferListNode_ *next;
+
// Everything below is read-only and generated by the codegen
- const char *string_table;
+
+ // This flag should be treated as a boolean
+ StgWord compressed;
+
StgWord count;
- IpeBufferEntry entries[];
+
+ // When TNTC is enabled, these will point to the entry code
+ // not the info table itself.
+ StgInfoTable **tables;
+
+ IpeBufferEntry *entries;
+ StgWord entries_size; // decompressed size
+
+ char *string_table;
+ StgWord string_table_size; // decompressed size
} IpeBufferListNode;
void registerInfoProvList(IpeBufferListNode *node);
=====================================
rts/rts.cabal.in
=====================================
@@ -45,6 +45,8 @@ flag libdw
default: @CabalHaveLibdw@
flag libnuma
default: @CabalHaveLibNuma@
+flag libzstd
+ default: @CabalHaveLibZstd@
flag 64bit
default: @Cabal64bit@
flag leading-underscore
@@ -212,6 +214,8 @@ library
extra-libraries: elf dw
if flag(libnuma)
extra-libraries: numa
+ if flag(libzstd)
+ extra-libraries: zstd
if !flag(smp)
cpp-options: -DNOSMP
=====================================
testsuite/driver/runtests.py
=====================================
@@ -601,6 +601,7 @@ else:
if args.junit:
junit(t).write(args.junit)
+ args.junit.close()
if config.only_report_hadrian_deps:
print("WARNING - skipping all tests and only reporting required hadrian dependencies:", config.hadrian_deps)
=====================================
testsuite/driver/testlib.py
=====================================
@@ -1347,7 +1347,7 @@ def do_test(name: TestName,
# if found and instead have the testsuite decide on what to do
# with the output.
def override_options(pre_cmd):
- if config.verbose >= 5 and bool(re.match('\$make', pre_cmd, re.I)):
+ if config.verbose >= 5 and bool(re.match(r'\$make', pre_cmd, re.I)):
return pre_cmd.replace(' -s' , '') \
.replace('--silent', '') \
.replace('--quiet' , '')
@@ -1989,7 +1989,7 @@ def split_file(in_fn: Path, delimiter: str, out1_fn: Path, out2_fn: Path):
with out1_fn.open('w', encoding='utf8', newline='') as out1:
with out2_fn.open('w', encoding='utf8', newline='') as out2:
line = infile.readline()
- while re.sub('^\s*','',line) != delimiter and line != '':
+ while re.sub(r'^\s*','',line) != delimiter and line != '':
out1.write(line)
line = infile.readline()
@@ -2399,20 +2399,20 @@ def normalise_errmsg(s: str) -> str:
# warning message to get clean output.
if config.msys:
s = re.sub('Failed to remove file (.*); error= (.*)$', '', s)
- s = re.sub('DeleteFile "(.+)": permission denied \(Access is denied\.\)(.*)$', '', s)
+ s = re.sub(r'DeleteFile "(.+)": permission denied \(Access is denied\.\)(.*)$', '', s)
# filter out unsupported GNU_PROPERTY_TYPE (5), which is emitted by LLVM10
# and not understood by older binutils (ar, ranlib, ...)
- s = modify_lines(s, lambda l: re.sub('^(.+)warning: (.+): unsupported GNU_PROPERTY_TYPE \(5\) type: 0xc000000(.*)$', '', l))
+ s = modify_lines(s, lambda l: re.sub(r'^(.+)warning: (.+): unsupported GNU_PROPERTY_TYPE \(5\) type: 0xc000000(.*)$', '', l))
- s = re.sub('ld: warning: passed .* min versions \(.*\) for platform macOS. Using [\.0-9]+.','',s)
+ s = re.sub(r'ld: warning: passed .* min versions \(.*\) for platform macOS. Using [\.0-9]+.','',s)
s = re.sub('ld: warning: -sdk_version and -platform_version are not compatible, ignoring -sdk_version','',s)
# ignore superfluous dylibs passed to the linker.
s = re.sub('ld: warning: .*, ignoring unexpected dylib file\n','',s)
# ignore LLVM Version mismatch garbage; this will just break tests.
s = re.sub('You are using an unsupported version of LLVM!.*\n','',s)
- s = re.sub('Currently only [\.0-9]+ is supported. System LLVM version: [\.0-9]+.*\n','',s)
- s = re.sub('We will try though\.\.\..*\n','',s)
+ s = re.sub('Currently only [\\.0-9]+ is supported. System LLVM version: [\\.0-9]+.*\n','',s)
+ s = re.sub('We will try though\\.\\.\\..*\n','',s)
# ignore warning about strip invalidating signatures
s = re.sub('.*strip: changes being made to the file will invalidate the code signature in.*\n','',s)
# clang may warn about unused argument when used as assembler
@@ -2475,8 +2475,8 @@ def normalise_slashes_( s: str ) -> str:
return s
def normalise_exe_( s: str ) -> str:
- s = re.sub('\.exe', '', s)
- s = re.sub('\.jsexe', '', s)
+ s = re.sub(r'\.exe', '', s)
+ s = re.sub(r'\.jsexe', '', s)
return s
def normalise_output( s: str ) -> str:
@@ -2494,14 +2494,14 @@ def normalise_output( s: str ) -> str:
# ghci outputs are pretty unstable with -fexternal-dynamic-refs, which is
# requires for -fPIC
s = re.sub(' -fexternal-dynamic-refs\n','',s)
- s = re.sub('ld: warning: passed .* min versions \(.*\) for platform macOS. Using [\.0-9]+.','',s)
+ s = re.sub(r'ld: warning: passed .* min versions \(.*\) for platform macOS. Using [\.0-9]+.','',s)
s = re.sub('ld: warning: -sdk_version and -platform_version are not compatible, ignoring -sdk_version','',s)
# ignore superfluous dylibs passed to the linker.
s = re.sub('ld: warning: .*, ignoring unexpected dylib file\n','',s)
# ignore LLVM Version mismatch garbage; this will just break tests.
s = re.sub('You are using an unsupported version of LLVM!.*\n','',s)
- s = re.sub('Currently only [\.0-9]+ is supported. System LLVM version: [\.0-9]+.*\n','',s)
- s = re.sub('We will try though\.\.\..*\n','',s)
+ s = re.sub('Currently only [\\.0-9]+ is supported. System LLVM version: [\\.0-9]+.*\n','',s)
+ s = re.sub('We will try though\\.\\.\\..*\n','',s)
# ignore warning about strip invalidating signatures
s = re.sub('.*strip: changes being made to the file will invalidate the code signature in.*\n','',s)
# clang may warn about unused argument when used as assembler
=====================================
testsuite/tests/rts/ipe/ipeEventLog_fromMap.c
=====================================
@@ -19,7 +19,7 @@ int main(int argc, char *argv[]) {
registerInfoProvList(list2);
// Query an IPE to initialize the underlying hash map.
- lookupIPE(list1->entries[0].info);
+ lookupIPE(list1->tables[0]);
// Trace all IPE events.
dumpIPEToEventLog();
=====================================
testsuite/tests/rts/ipe/ipeMap.c
=====================================
@@ -40,15 +40,23 @@ void shouldFindNothingInAnEmptyIPEMap(Capability *cap) {
}
HaskellObj shouldFindOneIfItHasBeenRegistered(Capability *cap) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *));
+ node->entries = malloc(sizeof(IpeBufferEntry));
+
StringTable st;
init_string_table(&st);
HaskellObj fortyTwo = UNTAG_CLOSURE(rts_mkInt(cap, 42));
- node->entries[0] = makeAnyProvEntry(cap, &st, fortyTwo, 42);
- node->count = 1;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 1;
+ node->tables[0] = get_itbl(fortyTwo);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 42);
+ node->entries_size = sizeof(IpeBufferEntry);
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
@@ -72,15 +80,23 @@ HaskellObj shouldFindOneIfItHasBeenRegistered(Capability *cap) {
void shouldFindTwoIfTwoHaveBeenRegistered(Capability *cap,
HaskellObj fortyTwo) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *));
+ node->entries = malloc(sizeof(IpeBufferEntry));
+
StringTable st;
init_string_table(&st);
HaskellObj twentyThree = UNTAG_CLOSURE(rts_mkInt8(cap, 23));
- node->entries[0] = makeAnyProvEntry(cap, &st, twentyThree, 23);
- node->count = 1;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 1;
+ node->tables[0] = get_itbl(twentyThree);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 23);
+ node->entries_size = sizeof(IpeBufferEntry);
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
@@ -103,17 +119,26 @@ void shouldFindTwoIfTwoHaveBeenRegistered(Capability *cap,
}
void shouldFindTwoFromTheSameList(Capability *cap) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + 2 * sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *) * 2);
+ node->entries = malloc(sizeof(IpeBufferEntry) * 2);
+
StringTable st;
init_string_table(&st);
HaskellObj one = UNTAG_CLOSURE(rts_mkInt16(cap, 1));
HaskellObj two = UNTAG_CLOSURE(rts_mkInt32(cap, 2));
- node->entries[0] = makeAnyProvEntry(cap, &st, one, 1);
- node->entries[1] = makeAnyProvEntry(cap, &st, two, 2);
- node->count = 2;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 2;
+ node->tables[0] = get_itbl(one);
+ node->tables[1] = get_itbl(two);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 1);
+ node->entries[1] = makeAnyProvEntry(cap, &st, 2);
+ node->entries_size = sizeof(IpeBufferEntry) * 2;
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
=====================================
testsuite/tests/rts/ipe/ipe_lib.c
=====================================
@@ -25,9 +25,8 @@ uint32_t add_string(StringTable *st, const char *s) {
return n;
}
-IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj closure, int i) {
+IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, int i) {
IpeBufferEntry provEnt;
- provEnt.info = get_itbl(closure);
unsigned int tableNameLength = strlen("table_name_") + 3 /* digits */ + 1 /* null character */;
char *tableName = malloc(sizeof(char) * tableNameLength);
@@ -69,15 +68,27 @@ IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj clo
IpeBufferListNode *makeAnyProvEntries(Capability *cap, int start, int end) {
const int n = end - start;
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + n * sizeof(IpeBufferEntry));
+
+ // Allocate buffers for IpeBufferListNode
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *) * n);
+ node->entries = malloc(sizeof(IpeBufferEntry) * n);
+
StringTable st;
init_string_table(&st);
+
+ // Make the entries and fill the buffers
for (int i=start; i < end; i++) {
HaskellObj closure = rts_mkInt(cap, 42);
- node->entries[i] = makeAnyProvEntry(cap, &st, closure, i);
+ node->tables[i] = get_itbl(closure);
+ node->entries[i] = makeAnyProvEntry(cap, &st, i);
}
+
+ // Set the rest of the fields
node->next = NULL;
+ node->compressed = 0;
node->count = n;
node->string_table = st.buffer;
+
return node;
}
=====================================
testsuite/tests/rts/ipe/ipe_lib.h
=====================================
@@ -12,6 +12,6 @@ void init_string_table(StringTable *st);
uint32_t add_string(StringTable *st, const char *s);
IpeBufferListNode *makeAnyProvEntries(Capability *cap, int start, int end);
-IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj closure, int i);
+IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, int i);
void dumpIPEToEventLog(void);
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/76f179c87bec4465b3316c0c29ecdc80a204b826...2142c4481197b5544d42a9937763812168be9e4b
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/76f179c87bec4465b3316c0c29ecdc80a204b826...2142c4481197b5544d42a9937763812168be9e4b
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230208/a757035a/attachment-0001.html>
More information about the ghc-commits
mailing list