[Git][ghc/ghc][wip/t21766] 13 commits: Don't allow . in overloaded labels
Finley McIlwaine (@FinleyMcIlwaine)
gitlab at gitlab.haskell.org
Wed Feb 8 01:49:09 UTC 2023
Finley McIlwaine pushed to branch wip/t21766 at Glasgow Haskell Compiler / GHC
Commits:
b17fb3d9 by sheaf at 2023-02-07T10:51:33-05:00
Don't allow . in overloaded labels
This patch removes . from the list of allowed characters in a non-quoted
overloaded label, as it was realised this steals syntax, e.g. (#.).
Users who want this functionality will have to add quotes around the
label, e.g. `#"17.28"`.
Fixes #22821
- - - - -
5dce04ee by romes at 2023-02-07T10:52:10-05:00
Update kinds in comments in GHC.Core.TyCon
Use `Type` instead of star kind (*)
Fix comment with incorrect kind * to have kind `Constraint`
- - - - -
92916194 by Ben Gamari at 2023-02-07T10:52:48-05:00
Revert "Use fix-sized equality primops for fixed size boxed types"
This reverts commit 024020c38126f3ce326ff56906d53525bc71690c.
This was never applied to master/9.6 originally.
See #20405 for why using these primops is a bad idea.
(cherry picked from commit b1d109ad542e4c37ae5af6ace71baf2cb509d865)
- - - - -
caf6c5fa by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Restructure IPE buffer layout
Reference ticket #21766
This commit restructures IPE buffer list entries to not contain
references to their corresponding info tables. IPE buffer list nodes now
point to two lists of equal length, one holding the list of info table
pointers and one holding the corresponding entries for each info table.
This will allow the entry data to be compressed without losing the
references to the info tables.
- - - - -
cbedbb40 by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Add IPE compression to configure
Reference ticket #21766
Adds an `--enable-ipe-data-compreesion` flag to the configure script
which will check for libzstd and set the appropriate flags to allow for
IPE data compression in the compiler
- - - - -
f02355f7 by Finley McIlwaine at 2023-02-07T12:20:09-07:00
IPE data compression
Reference ticket #21766
When IPE data compression is enabled, compress the emitted IPE buffer
entries and decompress them in the RTS.
- - - - -
9984fb5c by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Fix libzstd detection in configure and RTS
Ensure that `HAVE_LIBZSTD` gets defined to either 0 or 1 in all cases
and properly check that before IPE data decompression in the RTS. See
ticket #21766.
- - - - -
db651fe1 by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Add note describing IPE data compression
See ticket #21766
- - - - -
f501788b by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Fix byte order of IPE data, fix IPE tests
Make sure byte order of written IPE buffer entries matches target.
Make sure the IPE-related tests properly access the fields of IPE buffer
entry nodes with the new IPE layout.
This commit also introduces checks to avoid importing modules if IPE
compression is not enabled.
See ticket #21766.
- - - - -
e4f47dff by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Fix IPE data decompression buffer allocation
Capacity of buffers allocated for decompressed IPE data was
incorrect due to a misuse of the `ZSTD_findFrameCompressedSize`
function. Fix by always storing decompressed size of IPE data in IPE
buffer list nodes and using `ZSTD_findFrameCompressedSize` to determine
the size of the compressed data.
See ticket #21766
- - - - -
90c04ab0 by Finley McIlwaine at 2023-02-07T12:20:09-07:00
Add optional dependencies to ./configure output
Changes the configure script to indicate whether libnuma, libzstd, or
libdw are being used as dependencies due to their optional features
being enabled.
- - - - -
2fc71ba8 by Finley McIlwaine at 2023-02-07T12:26:10-07:00
Add IPE-enabled builds to CI
- Adds an IPE job to the CI pipeline which is triggered by the ~IPE label
- Introduces CI logic to enable IPE data compression
- Enables uncompressed IPE data on debug CI job
- Regenerates jobs.yaml
MR https://gitlab.haskell.org/ghc/ci-images/-/merge_requests/112 on the
images repository is meant to ensure that the proper images have
libzstd-dev installed.
- - - - -
a240bf49 by Finley McIlwaine at 2023-02-07T18:45:45-07:00
Update user's guide and release notes
Add mention of IPE data compression to user's guide and the release
notes for 9.8.1. Also note the impact compression has on binary size in
both places.
See ticket #21766
- - - - -
29 changed files:
- .gitlab/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC/Core/TyCon.hs
- compiler/GHC/Parser/Lexer.x
- compiler/GHC/StgToCmm/InfoTableProv.hs
- compiler/ghc.cabal.in
- configure.ac
- docs/users_guide/9.6.1-notes.rst
- docs/users_guide/9.8.1-notes.rst
- docs/users_guide/debug-info.rst
- hadrian/cfg/system.config.in
- hadrian/src/Oracles/Flag.hs
- hadrian/src/Oracles/Setting.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- libraries/base/GHC/Int.hs
- libraries/base/GHC/Word.hs
- m4/fp_find_libnuma.m4
- + m4/fp_find_libzstd.m4
- rts/IPE.c
- rts/IPE.h
- rts/include/rts/IPE.h
- rts/rts.cabal.in
- testsuite/tests/overloadedrecflds/should_run/T11671_run.hs
- testsuite/tests/printer/Test22771.hs
- testsuite/tests/rts/ipe/ipeEventLog_fromMap.c
- testsuite/tests/rts/ipe/ipeMap.c
- testsuite/tests/rts/ipe/ipe_lib.c
- testsuite/tests/rts/ipe/ipe_lib.h
Changes:
=====================================
.gitlab/gen_ci.hs
=====================================
@@ -140,6 +140,8 @@ data BuildConfig
, llvmBootstrap :: Bool
, withAssertions :: Bool
, withNuma :: Bool
+ , withZstd :: Bool
+ , withIpe :: Bool
, crossTarget :: Maybe String
, crossEmulator :: CrossEmulator
, configureWrapper :: Maybe String
@@ -152,10 +154,11 @@ data BuildConfig
-- Extra arguments to pass to ./configure due to the BuildConfig
configureArgsStr :: BuildConfig -> String
configureArgsStr bc = unwords $
- ["--enable-unregisterised"| unregisterised bc ]
+ ["--enable-unregisterised"| unregisterised bc ]
++ ["--disable-tables-next-to-code" | not (tablesNextToCode bc) ]
++ ["--with-intree-gmp" | Just _ <- pure (crossTarget bc) ]
++ ["--with-system-libffi" | crossTarget bc == Just "wasm32-wasi" ]
+ ++ ["--enable-ipe-data-compression" | withZstd bc ]
-- Compute the hadrian flavour from the BuildConfig
mkJobFlavour :: BuildConfig -> Flavour
@@ -165,11 +168,18 @@ mkJobFlavour BuildConfig{..} = Flavour buildFlavour opts
[Dwarf | withDwarf] ++
[FullyStatic | fullyStatic] ++
[ThreadSanitiser | threadSanitiser] ++
- [NoSplitSections | noSplitSections, buildFlavour == Release ]
+ [NoSplitSections | noSplitSections, buildFlavour == Release ] ++
+ [Ipe | withIpe]
data Flavour = Flavour BaseFlavour [FlavourTrans]
-data FlavourTrans = Llvm | Dwarf | FullyStatic | ThreadSanitiser | NoSplitSections
+data FlavourTrans =
+ Llvm
+ | Dwarf
+ | FullyStatic
+ | ThreadSanitiser
+ | NoSplitSections
+ | Ipe
data BaseFlavour = Release | Validate | SlowValidate deriving Eq
@@ -187,6 +197,8 @@ vanilla = BuildConfig
, llvmBootstrap = False
, withAssertions = False
, withNuma = False
+ , withZstd = False
+ , withIpe = False
, crossTarget = Nothing
, crossEmulator = NoEmulator
, configureWrapper = Nothing
@@ -216,8 +228,17 @@ debug = vanilla { buildFlavour = SlowValidate
, withAssertions = True
-- WithNuma so at least one job tests Numa
, withNuma = True
+
+ -- Build with IPE in debug so at least one job tests
+ -- uncompressed IPE data
+ , withIpe = True
}
+ipe :: BuildConfig
+ipe = vanilla { withIpe = True
+ , withZstd = True
+ }
+
static :: BuildConfig
static = vanilla { fullyStatic = True }
@@ -306,17 +327,18 @@ testEnv arch opsys bc = intercalate "-" $
-- | The hadrian flavour string we are going to use for this build
flavourString :: Flavour -> String
-flavourString (Flavour base trans) = baseString base ++ concatMap (("+" ++) . flavourString) trans
+flavourString (Flavour base trans) = base_string base ++ concatMap (("+" ++) . flavour_string) trans
where
- baseString Release = "release"
- baseString Validate = "validate"
- baseString SlowValidate = "slow-validate"
+ base_string Release = "release"
+ base_string Validate = "validate"
+ base_string SlowValidate = "slow-validate"
- flavourString Llvm = "llvm"
- flavourString Dwarf = "debug_info"
- flavourString FullyStatic = "fully_static"
- flavourString ThreadSanitiser = "thread_sanitizer"
- flavourString NoSplitSections = "no_split_sections"
+ flavour_string Llvm = "llvm"
+ flavour_string Dwarf = "debug_info"
+ flavour_string FullyStatic = "fully_static"
+ flavour_string ThreadSanitiser = "thread_sanitizer"
+ flavour_string NoSplitSections = "no_split_sections"
+ flavour_string Ipe = "ipe"
-- The path to the docker image (just for linux builders)
dockerImage :: Arch -> Opsys -> Maybe String
@@ -509,7 +531,7 @@ manualRule rules = rules { when = Manual }
-- For example, even if you don't explicitly disable a rule it will end up in the
-- rule list with the OFF state.
enumRules :: OnOffRules -> [OnOffRule]
-enumRules o = map lkup rules
+enumRules o = map lkup rulesList
where
enabled_rules = rule_set o
lkup r = OnOffRule (if S.member r enabled_rules then On else Off) r
@@ -544,6 +566,7 @@ data Rule = FastCI -- ^ Run this job when the fast-ci label is set
| Nightly -- ^ Only run this job in the nightly pipeline
| LLVMBackend -- ^ Only run this job when the "LLVM backend" label is present
| FreeBSDLabel -- ^ Only run this job when the "FreeBSD" label is set.
+ | IpeData -- ^ Only run this job when the "IPE" label is set
| Disable -- ^ Don't run this job.
deriving (Bounded, Enum, Ord, Eq)
@@ -568,12 +591,14 @@ ruleString On ReleaseOnly = "$RELEASE_JOB == \"yes\""
ruleString Off ReleaseOnly = "$RELEASE_JOB != \"yes\""
ruleString On Nightly = "$NIGHTLY"
ruleString Off Nightly = "$NIGHTLY == null"
+ruleString On IpeData = "$CI_MERGE_REQUEST_LABELS =~ /.*IPE.*/"
+ruleString Off IpeData = true
ruleString On Disable = false
ruleString Off Disable = true
-- Enumeration of all the rules
-rules :: [Rule]
-rules = [minBound .. maxBound]
+rulesList :: [Rule]
+rulesList = [minBound .. maxBound]
-- | A 'Job' is the description of a single job in a gitlab pipeline. The
-- job contains all the information about how to do the build but can be further
@@ -853,10 +878,11 @@ job_groups =
, validateBuilds Amd64 (Linux Debian10) nativeInt
, fastCI (validateBuilds Amd64 (Linux Debian10) unreg)
, fastCI (validateBuilds Amd64 (Linux Debian10) debug)
+ , disableValidate (validateBuilds Amd64 (Linux Debian10) ipe)
, modifyValidateJobs manual tsan_jobs
, modifyValidateJobs manual (validateBuilds Amd64 (Linux Debian10) noTntc)
, addValidateRule LLVMBackend (validateBuilds Amd64 (Linux Debian10) llvm)
-
+ , addValidateRule IpeData (validateBuilds Amd64 (Linux Debian10) ipe)
, disableValidate (standardBuilds Amd64 (Linux Debian11))
-- We still build Deb9 bindists for now due to Ubuntu 18 and Linux Mint 19
-- not being at EOL until April 2023 and they still need tinfo5.
=====================================
.gitlab/jobs.yaml
=====================================
@@ -35,7 +35,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -97,7 +97,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -155,7 +155,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -213,7 +213,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -276,7 +276,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -335,7 +335,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -394,7 +394,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -453,7 +453,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -518,7 +518,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -579,7 +579,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -640,7 +640,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -701,7 +701,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -764,7 +764,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -825,7 +825,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -888,7 +888,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -948,7 +948,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1007,7 +1007,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1031,7 +1031,7 @@
"XZ_OPT": "-9"
}
},
- "nightly-x86_64-linux-deb10-numa-slow-validate": {
+ "nightly-x86_64-linux-deb10-numa-slow-validate+ipe": {
"after_script": [
".gitlab/ci.sh save_cache",
".gitlab/ci.sh clean",
@@ -1041,7 +1041,7 @@
"artifacts": {
"expire_in": "8 weeks",
"paths": [
- "ghc-x86_64-linux-deb10-numa-slow-validate.tar.xz",
+ "ghc-x86_64-linux-deb10-numa-slow-validate+ipe.tar.xz",
"junit.xml"
],
"reports": {
@@ -1066,7 +1066,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1083,11 +1083,11 @@
],
"variables": {
"BIGNUM_BACKEND": "gmp",
- "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-numa-slow-validate",
- "BUILD_FLAVOUR": "slow-validate",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-numa-slow-validate+ipe",
+ "BUILD_FLAVOUR": "slow-validate+ipe",
"CONFIGURE_ARGS": "",
"ENABLE_NUMA": "1",
- "TEST_ENV": "x86_64-linux-deb10-numa-slow-validate",
+ "TEST_ENV": "x86_64-linux-deb10-numa-slow-validate+ipe",
"XZ_OPT": "-9"
}
},
@@ -1126,7 +1126,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1185,7 +1185,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1244,7 +1244,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1268,6 +1268,65 @@
"XZ_OPT": "-9"
}
},
+ "nightly-x86_64-linux-deb10-validate+ipe": {
+ "after_script": [
+ ".gitlab/ci.sh save_cache",
+ ".gitlab/ci.sh clean",
+ "cat ci_timings"
+ ],
+ "allow_failure": false,
+ "artifacts": {
+ "expire_in": "8 weeks",
+ "paths": [
+ "ghc-x86_64-linux-deb10-validate+ipe.tar.xz",
+ "junit.xml"
+ ],
+ "reports": {
+ "junit": "junit.xml"
+ },
+ "when": "always"
+ },
+ "cache": {
+ "key": "x86_64-linux-deb10-$CACHE_REV",
+ "paths": [
+ "cabal-cache",
+ "toolchain"
+ ]
+ },
+ "dependencies": [],
+ "image": "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb10:$DOCKER_REV",
+ "needs": [
+ {
+ "artifacts": false,
+ "job": "hadrian-ghc-in-ghci"
+ }
+ ],
+ "rules": [
+ {
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "when": "on_success"
+ }
+ ],
+ "script": [
+ "sudo chown ghc:ghc -R .",
+ ".gitlab/ci.sh setup",
+ ".gitlab/ci.sh configure",
+ ".gitlab/ci.sh build_hadrian",
+ ".gitlab/ci.sh test_hadrian"
+ ],
+ "stage": "full-build",
+ "tags": [
+ "x86_64-linux"
+ ],
+ "variables": {
+ "BIGNUM_BACKEND": "gmp",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-validate+ipe",
+ "BUILD_FLAVOUR": "validate+ipe",
+ "CONFIGURE_ARGS": "--enable-ipe-data-compression",
+ "TEST_ENV": "x86_64-linux-deb10-validate+ipe",
+ "XZ_OPT": "-9"
+ }
+ },
"nightly-x86_64-linux-deb10-validate+llvm": {
"after_script": [
".gitlab/ci.sh save_cache",
@@ -1303,7 +1362,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1362,7 +1421,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1423,7 +1482,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1484,7 +1543,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1546,7 +1605,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1605,7 +1664,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1664,7 +1723,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1725,7 +1784,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1787,7 +1846,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1848,7 +1907,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1908,7 +1967,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -1967,7 +2026,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2022,7 +2081,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2081,7 +2140,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2144,7 +2203,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2208,7 +2267,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2268,7 +2327,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2328,7 +2387,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2394,7 +2453,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2458,7 +2517,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2522,7 +2581,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2583,7 +2642,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2643,7 +2702,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2703,7 +2762,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2763,7 +2822,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2823,7 +2882,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2885,7 +2944,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -2947,7 +3006,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3010,7 +3069,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3071,7 +3130,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3131,7 +3190,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3187,7 +3246,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3247,7 +3306,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB == \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3311,7 +3370,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3375,7 +3434,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*FreeBSD.*/) && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*FreeBSD.*/) && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3435,7 +3494,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3495,7 +3554,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3557,7 +3616,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3616,7 +3675,7 @@
"rules": [
{
"allow_failure": true,
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "manual"
}
],
@@ -3639,7 +3698,7 @@
"TEST_ENV": "x86_64-linux-deb10-no_tntc-validate"
}
},
- "x86_64-linux-deb10-numa-slow-validate": {
+ "x86_64-linux-deb10-numa-slow-validate+ipe": {
"after_script": [
".gitlab/ci.sh save_cache",
".gitlab/ci.sh clean",
@@ -3649,7 +3708,7 @@
"artifacts": {
"expire_in": "2 weeks",
"paths": [
- "ghc-x86_64-linux-deb10-numa-slow-validate.tar.xz",
+ "ghc-x86_64-linux-deb10-numa-slow-validate+ipe.tar.xz",
"junit.xml"
],
"reports": {
@@ -3674,7 +3733,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3691,11 +3750,11 @@
],
"variables": {
"BIGNUM_BACKEND": "gmp",
- "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-numa-slow-validate",
- "BUILD_FLAVOUR": "slow-validate",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-numa-slow-validate+ipe",
+ "BUILD_FLAVOUR": "slow-validate+ipe",
"CONFIGURE_ARGS": "",
"ENABLE_NUMA": "1",
- "TEST_ENV": "x86_64-linux-deb10-numa-slow-validate"
+ "TEST_ENV": "x86_64-linux-deb10-numa-slow-validate+ipe"
}
},
"x86_64-linux-deb10-unreg-validate": {
@@ -3733,7 +3792,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3791,7 +3850,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3814,6 +3873,64 @@
"TEST_ENV": "x86_64-linux-deb10-validate+debug_info"
}
},
+ "x86_64-linux-deb10-validate+ipe": {
+ "after_script": [
+ ".gitlab/ci.sh save_cache",
+ ".gitlab/ci.sh clean",
+ "cat ci_timings"
+ ],
+ "allow_failure": false,
+ "artifacts": {
+ "expire_in": "2 weeks",
+ "paths": [
+ "ghc-x86_64-linux-deb10-validate+ipe.tar.xz",
+ "junit.xml"
+ ],
+ "reports": {
+ "junit": "junit.xml"
+ },
+ "when": "always"
+ },
+ "cache": {
+ "key": "x86_64-linux-deb10-$CACHE_REV",
+ "paths": [
+ "cabal-cache",
+ "toolchain"
+ ]
+ },
+ "dependencies": [],
+ "image": "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb10:$DOCKER_REV",
+ "needs": [
+ {
+ "artifacts": false,
+ "job": "hadrian-ghc-in-ghci"
+ }
+ ],
+ "rules": [
+ {
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && ($CI_MERGE_REQUEST_LABELS =~ /.*IPE.*/) && (\"true\" == \"true\")",
+ "when": "on_success"
+ }
+ ],
+ "script": [
+ "sudo chown ghc:ghc -R .",
+ ".gitlab/ci.sh setup",
+ ".gitlab/ci.sh configure",
+ ".gitlab/ci.sh build_hadrian",
+ ".gitlab/ci.sh test_hadrian"
+ ],
+ "stage": "full-build",
+ "tags": [
+ "x86_64-linux"
+ ],
+ "variables": {
+ "BIGNUM_BACKEND": "gmp",
+ "BIN_DIST_NAME": "ghc-x86_64-linux-deb10-validate+ipe",
+ "BUILD_FLAVOUR": "validate+ipe",
+ "CONFIGURE_ARGS": "--enable-ipe-data-compression",
+ "TEST_ENV": "x86_64-linux-deb10-validate+ipe"
+ }
+ },
"x86_64-linux-deb10-validate+llvm": {
"after_script": [
".gitlab/ci.sh save_cache",
@@ -3849,7 +3966,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && ($CI_MERGE_REQUEST_LABELS =~ /.*LLVM backend.*/) && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && ($CI_MERGE_REQUEST_LABELS =~ /.*LLVM backend.*/) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -3908,7 +4025,7 @@
"rules": [
{
"allow_failure": true,
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "manual"
}
],
@@ -3968,7 +4085,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4028,7 +4145,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4089,7 +4206,7 @@
],
"rules": [
{
- "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "($CI_MERGE_REQUEST_LABELS !~ /.*fast-ci.*/) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
@@ -4145,7 +4262,7 @@
],
"rules": [
{
- "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
+ "if": "(\"true\" == \"true\") && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null) && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\") && (\"true\" == \"true\")",
"when": "on_success"
}
],
=====================================
compiler/GHC/Core/TyCon.hs
=====================================
@@ -194,7 +194,7 @@ Note [Type synonym families]
* Type synonym families, also known as "type functions", map directly
onto the type functions in FC:
- type family F a :: *
+ type family F a :: Type
type instance F Int = Bool
..etc...
@@ -210,11 +210,11 @@ Note [Type synonym families]
type instance F (F Int) = ... -- BAD!
* Translation of type family decl:
- type family F a :: *
+ type family F a :: Type
translates to
a FamilyTyCon 'F', whose FamTyConFlav is OpenSynFamilyTyCon
- type family G a :: * where
+ type family G a :: Type where
G Int = Bool
G Bool = Char
G a = ()
@@ -229,7 +229,7 @@ Note [Data type families]
See also Note [Wrappers for data instance tycons] in GHC.Types.Id.Make
* Data type families are declared thus
- data family T a :: *
+ data family T a :: Type
data instance T Int = T1 | T2 Bool
Here T is the "family TyCon".
@@ -321,7 +321,7 @@ See also Note [Wrappers for data instance tycons] in GHC.Types.Id.Make
should not think of a data family T as a *type function* at all, not
even an injective one! We can't allow even injective type functions
on the LHS of a type function:
- type family injective G a :: *
+ type family injective G a :: Type
type instance F (G Int) = Bool
is no good, even if G is injective, because consider
type instance G Int = Bool
@@ -572,21 +572,21 @@ Since they are user-callable we must get their type-argument visibility
information right; and that info is in the TyConBinders.
Here is an example:
- data App a b = MkApp (a b) -- App :: forall {k}. (k->*) -> k -> *
+ data App a b = MkApp (a b) -- App :: forall {k}. (k->Type) -> k -> Type
The TyCon has
- tyConTyBinders = [ Named (Bndr (k :: *) Inferred), Anon (k->*), Anon k ]
+ tyConTyBinders = [ Named (Bndr (k :: Type) Inferred), Anon (k->Type), Anon k ]
The TyConBinders for App line up with App's kind, given above.
But the DataCon MkApp has the type
- MkApp :: forall {k} (a:k->*) (b:k). a b -> App k a b
+ MkApp :: forall {k} (a:k->Type) (b:k). a b -> App k a b
That is, its ForAllTyBinders should be
- dataConUnivTyVarBinders = [ Bndr (k:*) Inferred
- , Bndr (a:k->*) Specified
+ dataConUnivTyVarBinders = [ Bndr (k:Type) Inferred
+ , Bndr (a:k->Type) Specified
, Bndr (b:k) Specified ]
So tyConTyVarBinders converts TyCon's TyConBinders into TyVarBinders:
@@ -620,8 +620,8 @@ They fit together like so:
type App a (b :: k) = a b
- tyConBinders = [ Bndr (k::*) (NamedTCB Inferred)
- , Bndr (a:k->*) AnonTCB
+ tyConBinders = [ Bndr (k::Type) (NamedTCB Inferred)
+ , Bndr (a:k->Type) AnonTCB
, Bndr (b:k) AnonTCB ]
Note that there are three binders here, including the
@@ -636,13 +636,13 @@ They fit together like so:
that TyVar may scope over some other part of the TyCon's definition. Eg
type T a = a -> a
we have
- tyConBinders = [ Bndr (a:*) AnonTCB ]
+ tyConBinders = [ Bndr (a:Type) AnonTCB ]
synTcRhs = a -> a
So the 'a' scopes over the synTcRhs
* From the tyConBinders and tyConResKind we can get the tyConKind
E.g for our App example:
- App :: forall k. (k->*) -> k -> *
+ App :: forall k. (k->Type) -> k -> Type
We get a 'forall' in the kind for each NamedTCB, and an arrow
for each AnonTCB
@@ -725,15 +725,15 @@ instance Binary TyConBndrVis where
-- things such as:
--
-- 1) Data declarations: @data Foo = ...@ creates the @Foo@ type constructor of
--- kind @*@
+-- kind @Type@
--
-- 2) Type synonyms: @type Foo = ...@ creates the @Foo@ type constructor
--
-- 3) Newtypes: @newtype Foo a = MkFoo ...@ creates the @Foo@ type constructor
--- of kind @* -> *@
+-- of kind @Type -> Type@
--
-- 4) Class declarations: @class Foo where@ creates the @Foo@ type constructor
--- of kind @*@
+-- of kind @Constraint@
--
-- This data type also encodes a number of primitive, built in type constructors
-- such as those for function and tuple types.
@@ -1252,16 +1252,16 @@ data FamTyConFlav
--
-- These are introduced by either a top level declaration:
--
- -- > data family T a :: *
+ -- > data family T a :: Type
--
-- Or an associated data type declaration, within a class declaration:
--
-- > class C a b where
- -- > data T b :: *
+ -- > data T b :: Type
DataFamilyTyCon
TyConRepName
- -- | An open type synonym family e.g. @type family F x y :: * -> *@
+ -- | An open type synonym family e.g. @type family F x y :: Type -> Type@
| OpenSynFamilyTyCon
-- | A closed type synonym family e.g.
=====================================
compiler/GHC/Parser/Lexer.x
=====================================
@@ -163,7 +163,6 @@ $small = [$ascsmall $unismall \_]
$uniidchar = \x07 -- Trick Alex into handling Unicode. See Note [Unicode in Alex].
$idchar = [$small $large $digit $uniidchar \']
-$labelchar = [$small $large $digit $uniidchar \' \.]
$unigraphic = \x06 -- Trick Alex into handling Unicode. See Note [Unicode in Alex].
$graphic = [$small $large $symbol $digit $idchar $special $unigraphic \"\']
@@ -455,7 +454,7 @@ $unigraphic / { isSmartQuote } { smart_quote_error }
}
<0> {
- "#" $labelchar+ / { ifExtension OverloadedLabelsBit } { skip_one_varid_src ITlabelvarid }
+ "#" $idchar+ / { ifExtension OverloadedLabelsBit } { skip_one_varid_src ITlabelvarid }
"#" \" / { ifExtension OverloadedLabelsBit } { lex_quoted_label }
}
=====================================
compiler/GHC/StgToCmm/InfoTableProv.hs
=====================================
@@ -1,66 +1,187 @@
+{-# LANGUAGE CPP #-}
+
module GHC.StgToCmm.InfoTableProv (emitIpeBufferListNode) where
+import Foreign
+
+#if defined(HAVE_LIBZSTD)
+import Foreign.C.Types
+import qualified Data.ByteString.Internal as BSI
+import GHC.IO (unsafePerformIO)
+#endif
+
+import GHC.Data.FastString (fastStringToShortText)
import GHC.Prelude
import GHC.Platform
+import GHC.Types.SrcLoc (pprUserRealSpan, srcSpanFile)
import GHC.Unit.Module
import GHC.Utils.Outputable
-import GHC.Types.SrcLoc (pprUserRealSpan, srcSpanFile)
-import GHC.Data.FastString (fastStringToShortText)
+import GHC.Cmm
import GHC.Cmm.CLabel
-import GHC.Cmm.Expr
import GHC.Cmm.Utils
import GHC.StgToCmm.Config
-import GHC.StgToCmm.Lit (newByteStringCLit)
import GHC.StgToCmm.Monad
-import GHC.StgToCmm.Utils
import GHC.Data.ShortText (ShortText)
import qualified GHC.Data.ShortText as ST
-import qualified Data.Map.Strict as M
import Control.Monad.Trans.State.Strict
+
import qualified Data.ByteString as BS
import qualified Data.ByteString.Builder as BSB
import qualified Data.ByteString.Lazy as BSL
+import qualified Data.Map.Strict as M
+
+{-
+Note [Compression and Decompression of IPE data]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Compiling with `-finfo-table-map` causes build results to include a map from
+info tables to source positions called the info table provenance entry (IPE)
+map. See Note [Mapping Info Tables to Source Positions]. The IPE information
+can grow the size of build results significantly. At the time of writing, a
+default build of GHC results in a total of 109M of libHSghc-*.so build results.
+A default+ipe build of GHC (see ./hadrian/doc/flavours.md) results in 262M of
+libHSghc-*.so build results without compression.
-emitIpeBufferListNode :: Module
- -> [InfoProvEnt]
- -> FCode ()
+We reduce the impact of IPE data on the size of build results by compressing
+the data before it is emitted using the zstd compression library. See
+Note [The Info Table Provenance Entry (IPE) Map] for information on the layout
+of IPE data on disk and in the RTS. We cannot simply compress all data held in
+the IPE entry buffer, as the pointers to info tables must be converted to
+memory addresses during linking. Therefore, we can only compress the strings
+table and the IPE entries themselves (which essentially only consist of indices
+into the strings table).
+
+With compression, a default+ipe build of GHC results in a total of 205M of
+libHSghc-*.so build results. This is over a 20% reduction from the uncompressed
+case.
+
+Decompression happens lazily, as it only occurs when the IPE map is
+constructed (which is also done lazily on first lookup or traversal). During
+construction, the 'compressed' field of each IPE buffer list node is examined.
+If the field indicates that the data has been compressed, the entry data and
+strings table are decompressed before continuing with the normal IPE map
+construction.
+-}
+
+emitIpeBufferListNode ::
+ Module
+ -> [InfoProvEnt]
+ -> FCode ()
emitIpeBufferListNode _ [] = return ()
emitIpeBufferListNode this_mod ents = do
cfg <- getStgToCmmConfig
- let ctx = stgToCmmContext cfg
+
+ tables_lbl <- mkStringLitLabel <$> newUnique
+ strings_lbl <- mkStringLitLabel <$> newUnique
+ entries_lbl <- mkStringLitLabel <$> newUnique
+
+ let ctx = stgToCmmContext cfg
platform = stgToCmmPlatform cfg
+ int n = mkIntCLit platform n
+
+ (cg_ipes, strtab) = flip runState emptyStringTable $ do
+ module_name <- lookupStringTable $ ST.pack $ renderWithContext ctx (ppr this_mod)
+ mapM (toCgIPE platform ctx module_name) ents
+
+ tables :: [CmmStatic]
+ tables = map (CmmStaticLit . CmmLabel . ipeInfoTablePtr) cg_ipes
+
+ uncompressed_strings :: BS.ByteString
+ uncompressed_strings = getStringTableStrings strtab
+
+ strings_bytes :: BS.ByteString
+ strings_bytes = compress defaultCompressionLevel uncompressed_strings
+
+ strings :: [CmmStatic]
+ strings = [CmmString strings_bytes]
+
+ uncompressed_entries :: BS.ByteString
+ uncompressed_entries = toIpeBufferEntries (platformByteOrder platform) cg_ipes
- let (cg_ipes, strtab) = flip runState emptyStringTable $ do
- module_name <- lookupStringTable $ ST.pack $ renderWithContext ctx (ppr this_mod)
- mapM (toCgIPE platform ctx module_name) ents
-
- let -- Emit the fields of an IpeBufferEntry struct.
- toIpeBufferEntry :: CgInfoProvEnt -> [CmmLit]
- toIpeBufferEntry cg_ipe =
- [ CmmLabel (ipeInfoTablePtr cg_ipe)
- , strtab_offset (ipeTableName cg_ipe)
- , strtab_offset (ipeClosureDesc cg_ipe)
- , strtab_offset (ipeTypeDesc cg_ipe)
- , strtab_offset (ipeLabel cg_ipe)
- , strtab_offset (ipeModuleName cg_ipe)
- , strtab_offset (ipeSrcFile cg_ipe)
- , strtab_offset (ipeSrcSpan cg_ipe)
- , int32 0
- ]
-
- int n = mkIntCLit platform n
- int32 n = CmmInt n W32
- strtab_offset (StrTabOffset n) = int32 (fromIntegral n)
-
- strings <- newByteStringCLit (getStringTableStrings strtab)
- let lits = [ zeroCLit platform -- 'next' field
- , strings -- 'strings' field
- , int $ length cg_ipes -- 'count' field
- ] ++ concatMap toIpeBufferEntry cg_ipes
- emitDataLits (mkIPELabel this_mod) lits
+ entries_bytes :: BS.ByteString
+ entries_bytes = compress defaultCompressionLevel uncompressed_entries
+
+ entries :: [CmmStatic]
+ entries = [CmmString entries_bytes]
+
+ ipe_buffer_lbl :: CLabel
+ ipe_buffer_lbl = mkIPELabel this_mod
+
+ ipe_buffer_node :: [CmmStatic]
+ ipe_buffer_node = map CmmStaticLit
+ [ -- 'next' field
+ zeroCLit platform
+
+ -- 'compressed' field
+ , int do_compress
+
+ -- 'count' field
+ , int $ length cg_ipes
+
+ -- 'tables' field
+ , CmmLabel tables_lbl
+
+ -- 'entries' field
+ , CmmLabel entries_lbl
+
+ -- 'entries_size' field (decompressed size)
+ , int $ BS.length uncompressed_entries
+
+ -- 'string_table' field
+ , CmmLabel strings_lbl
+
+ -- 'string_table_size' field (decompressed size)
+ , int $ BS.length uncompressed_strings
+ ]
+
+ -- Emit the list of info table pointers
+ emitDecl $ CmmData
+ (Section Data tables_lbl)
+ (CmmStaticsRaw tables_lbl tables)
+
+ -- Emit the strings table
+ emitDecl $ CmmData
+ (Section Data strings_lbl)
+ (CmmStaticsRaw strings_lbl strings)
+
+ -- Emit the list of IPE buffer entries
+ emitDecl $ CmmData
+ (Section Data entries_lbl)
+ (CmmStaticsRaw entries_lbl entries)
+
+ -- Emit the IPE buffer list node
+ emitDecl $ CmmData
+ (Section Data ipe_buffer_lbl)
+ (CmmStaticsRaw ipe_buffer_lbl ipe_buffer_node)
+
+-- | Emit the fields of an IpeBufferEntry struct for each entry in a given list.
+toIpeBufferEntries ::
+ ByteOrder -- ^ Byte order to write the data in
+ -> [CgInfoProvEnt] -- ^ List of IPE buffer entries
+ -> BS.ByteString
+toIpeBufferEntries byte_order cg_ipes =
+ BSL.toStrict . BSB.toLazyByteString . mconcat
+ $ map (mconcat . map word32Builder . to_ipe_buf_ent) cg_ipes
+ where
+ to_ipe_buf_ent :: CgInfoProvEnt -> [Word32]
+ to_ipe_buf_ent cg_ipe =
+ [ ipeTableName cg_ipe
+ , ipeClosureDesc cg_ipe
+ , ipeTypeDesc cg_ipe
+ , ipeLabel cg_ipe
+ , ipeModuleName cg_ipe
+ , ipeSrcFile cg_ipe
+ , ipeSrcSpan cg_ipe
+ , 0 -- padding
+ ]
+
+ word32Builder :: Word32 -> BSB.Builder
+ word32Builder = case byte_order of
+ BigEndian -> BSB.word32BE
+ LittleEndian -> BSB.word32LE
toCgIPE :: Platform -> SDocContext -> StrTabOffset -> InfoProvEnt -> State StringTable CgInfoProvEnt
toCgIPE platform ctx module_name ipe = do
@@ -76,7 +197,7 @@ toCgIPE platform ctx module_name ipe = do
coords = renderWithContext ctx (pprUserRealSpan False span)
in (file, coords)
label <- lookupStringTable $ ST.pack label_str
- src_file <- lookupStringTable $ src_loc_file
+ src_file <- lookupStringTable src_loc_file
src_span <- lookupStringTable $ ST.pack src_loc_span
return $ CgInfoProvEnt { ipeInfoTablePtr = infoTablePtr ipe
, ipeTableName = table_name
@@ -104,7 +225,7 @@ data StringTable = StringTable { stStrings :: DList ShortText
, stLookup :: !(M.Map ShortText StrTabOffset)
}
-newtype StrTabOffset = StrTabOffset Int
+type StrTabOffset = Word32
emptyStringTable :: StringTable
emptyStringTable =
@@ -129,9 +250,50 @@ lookupStringTable str = state $ \st ->
, stLength = stLength st + ST.byteLength str + 1
, stLookup = M.insert str res (stLookup st)
}
- res = StrTabOffset (stLength st)
+ res = fromIntegral (stLength st)
in (res, st')
+do_compress :: Int
+compress :: Int -> BS.ByteString -> BS.ByteString
+#if !defined(HAVE_LIBZSTD)
+do_compress = 0
+compress _ bs = bs
+#else
+do_compress = 1
+
+compress clvl (BSI.PS srcForeignPtr off len) = unsafePerformIO $
+ withForeignPtr srcForeignPtr $ \srcPtr -> do
+ maxCompressedSize <- zstd_compress_bound $ fromIntegral len
+ dstForeignPtr <- BSI.mallocByteString (fromIntegral maxCompressedSize)
+ withForeignPtr dstForeignPtr $ \dstPtr -> do
+ compressedSize <- fromIntegral <$>
+ zstd_compress
+ dstPtr
+ maxCompressedSize
+ (srcPtr `plusPtr` off)
+ (fromIntegral len)
+ (fromIntegral clvl)
+ BSI.create compressedSize $ \p -> BSI.memcpy p dstPtr compressedSize
+
+foreign import ccall unsafe "ZSTD_compress"
+ zstd_compress ::
+ Ptr dst -- ^ Destination buffer
+ -> CSize -- ^ Capacity of destination buffer
+ -> Ptr src -- ^ Source buffer
+ -> CSize -- ^ Size of source buffer
+ -> CInt -- ^ Compression level
+ -> IO CSize
+
+-- | Compute the maximum compressed size for a given source buffer size
+foreign import ccall unsafe "ZSTD_compressBound"
+ zstd_compress_bound ::
+ CSize -- ^ Size of source buffer
+ -> IO CSize
+#endif
+
+defaultCompressionLevel :: Int
+defaultCompressionLevel = 3
+
newtype DList a = DList ([a] -> [a])
emptyDList :: DList a
=====================================
compiler/ghc.cabal.in
=====================================
@@ -57,6 +57,10 @@ Flag build-tool-depends
Description: Use build-tool-depends
Default: True
+Flag with-libzstd
+ Default: False
+ Manual: True
+
Library
Default-Language: Haskell2010
Exposed: False
@@ -71,6 +75,10 @@ Library
if flag(build-tool-depends)
build-tool-depends: alex:alex >= 3.2.6, happy:happy >= 1.20.0, genprimopcode:genprimopcode, deriveConstants:deriveConstants
+ if flag(with-libzstd)
+ extra-libraries: zstd
+ CPP-Options: -DHAVE_LIBZSTD
+
Build-Depends: base >= 4.11 && < 4.19,
deepseq >= 1.4 && < 1.5,
directory >= 1 && < 1.4,
=====================================
configure.ac
=====================================
@@ -1122,6 +1122,10 @@ AC_DEFINE_UNQUOTED([RTS_LINKER_USE_MMAP], [$RtsLinkerUseMmap],
GHC_ADJUSTORS_METHOD([Target])
AC_SUBST([UseLibffiForAdjustors])
+dnl ** IPE data compression
+dnl --------------------------------------------------------------
+FP_FIND_LIBZSTD
+
dnl ** Other RTS features
dnl --------------------------------------------------------------
FP_FIND_LIBDW
@@ -1267,6 +1271,17 @@ echo "\
makeinfo : $MAKEINFO
git : $GIT
cabal-install : $CABAL
+"
+
+USING_LIBNUMA=$(if [ "$HaveLibNuma" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+USING_LIBZSTD=$(if [ "$HaveLibZstd" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+USING_LIBDW=$(if [ "$USE_LIBDW" = "1" ]; then echo "YES"; else echo "NO"; fi;)
+
+echo "\
+ Using optional dependencies:
+ libnuma : $USING_LIBNUMA
+ libzstd : $USING_LIBZSTD
+ libdw : $USING_LIBDW
Using LLVM tools
clang : $ClangCmd
=====================================
docs/users_guide/9.6.1-notes.rst
=====================================
@@ -84,7 +84,7 @@ Language
This extends the variety syntax for constructing labels under :extension:`OverloadedLabels`.
Examples of newly allowed syntax:
- Leading capital letters: `#Foo` equivalant to `getLabel @"Foo"`
- - Numeric characters: `#3.14` equivalent to `getLabel @"3.14"`
+ - Numeric characters: `#1728` equivalent to `getLabel @"1728"`
- Arbitrary strings: `#"Hello, World!"` equivalent to `getLabel @"Hello, World!"`
Compiler
=====================================
docs/users_guide/9.8.1-notes.rst
=====================================
@@ -21,6 +21,17 @@ Compiler
foo (\x -> x*2 + x)
+- The compiler may now be configured to compress the debugging information
+ included in :ghc-flag:`-finfo-table-map` enabled binaries. To do so, one must
+ build GHC from source (see
+ `here<https://gitlab.haskell.org/ghc/ghc/-/wikis/building>` for directions)
+ and supply the ``--enable-ipe-data-compression`` flag to the ``configure``
+ script. **Note**: This feature requires that the machine building GHC has
+ `libzstd <https://github.com/facebook/zstd/>`_ installed.
+
+ In a test compiling GHC itself, the size of the :ghc-flag:`-finfo-table-map`
+ enabled build results was reduced by over 20% when compression was enabled.
+
GHCi
~~~~
=====================================
docs/users_guide/debug-info.rst
=====================================
@@ -370,9 +370,23 @@ to a source location. This lookup table is generated by using the ``-finfo-table
also want more precise information about constructor info tables then you
should also use :ghc-flag:`-fdistinct-constructor-tables`.
- This flag will increase the binary size by quite a lot, depending on how
- big your project is. For compiling a project the size of GHC the overhead was
- about 200 megabytes.
+ The :ghc-flag:`-finfo-table-map` flag will increase the binary size by quite
+ a lot, depending on how big your project is. For compiling a project the
+ size of GHC the overhead was about 200 megabytes.
+
+ :since: 9.8
+
+ If you wish to reduce the size of :ghc-flag:`-finfo-table-map` enabled
+ binaries, consider building GHC from source and supplying the
+ ``--enable-ipe-data-compression`` flag to the ``configure`` script. This
+ will cause GHC to compress the :ghc-flag:`-finfo-table-map` related
+ debugging information included in binaries using the `libzstd
+ <https://github.com/facebook/zstd/>`_ compression library. **Note**: This
+ feature requires that the machine building GHC has
+ `libzstd <https://github.com/facebook/zstd/>`_ installed.
+
+ In a test compiling GHC itself, the size of the :ghc-flag:`-finfo-table-map`
+ enabled build results was reduced by over 20% when compression was enabled.
.. ghc-flag:: -fdistinct-constructor-tables
:shortdesc: Generate a fresh info table for each usage
=====================================
hadrian/cfg/system.config.in
=====================================
@@ -200,10 +200,14 @@ libdw-lib-dir = @LibdwLibDir@
libnuma-include-dir = @LibNumaIncludeDir@
libnuma-lib-dir = @LibNumaLibDir@
+libzstd-include-dir = @LibZstdIncludeDir@
+libzstd-lib-dir = @LibZstdLibDir@
+
# Optional Dependencies:
#=======================
use-lib-dw = @UseLibdw@
+use-lib-zstd = @UseLibZstd@
use-lib-numa = @UseLibNuma@
use-lib-m = @UseLibm@
use-lib-rt = @UseLibrt@
=====================================
hadrian/src/Oracles/Flag.hs
=====================================
@@ -35,6 +35,7 @@ data Flag = ArSupportsAtFile
| UseLibffiForAdjustors
| UseLibdw
| UseLibnuma
+ | UseLibzstd
| UseLibm
| UseLibrt
| UseLibdl
@@ -65,6 +66,7 @@ flag f = do
UseLibffiForAdjustors -> "use-libffi-for-adjustors"
UseLibdw -> "use-lib-dw"
UseLibnuma -> "use-lib-numa"
+ UseLibzstd -> "use-lib-zstd"
UseLibm -> "use-lib-m"
UseLibrt -> "use-lib-rt"
UseLibdl -> "use-lib-dl"
=====================================
hadrian/src/Oracles/Setting.hs
=====================================
@@ -60,6 +60,8 @@ data Setting = BuildArch
| LibdwLibDir
| LibnumaIncludeDir
| LibnumaLibDir
+ | LibZstdIncludeDir
+ | LibZstdLibDir
| LlvmTarget
| ProjectGitCommitId
| ProjectName
@@ -162,6 +164,8 @@ setting key = lookupSystemConfig $ case key of
LibdwLibDir -> "libdw-lib-dir"
LibnumaIncludeDir -> "libnuma-include-dir"
LibnumaLibDir -> "libnuma-lib-dir"
+ LibZstdIncludeDir -> "libzstd-include-dir"
+ LibZstdLibDir -> "libzstd-lib-dir"
LlvmTarget -> "llvm-target"
ProjectGitCommitId -> "project-git-commit-id"
ProjectName -> "project-name"
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -291,6 +291,7 @@ rtsCabalFlags = mconcat
, flag "CabalNeedLibpthread" UseLibpthread
, flag "CabalHaveLibbfd" UseLibbfd
, flag "CabalHaveLibNuma" UseLibnuma
+ , flag "CabalHaveLibZstd" UseLibzstd
, flag "CabalNeedLibatomic" NeedLibatomic
, flag "CabalUseSystemLibFFI" UseSystemFfi
, flag "CabalLibffiAdjustors" UseLibffiForAdjustors
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -77,6 +77,7 @@ packageArgs = do
[ andM [expr ghcWithInterpreter, notStage0] `cabalFlag` "internal-interpreter"
, notM cross `cabalFlag` "terminfo"
, arg "-build-tool-depends"
+ , flag UseLibzstd `cabalFlag` "with-libzstd"
]
, builder (Haddock BuildPackage) ? arg ("--optghc=-I" ++ path) ]
@@ -286,6 +287,8 @@ rtsPackageArgs = package rts ? do
libdwLibraryDir <- getSetting LibdwLibDir
libnumaIncludeDir <- getSetting LibnumaIncludeDir
libnumaLibraryDir <- getSetting LibnumaLibDir
+ libzstdIncludeDir <- getSetting LibZstdIncludeDir
+ libzstdLibraryDir <- getSetting LibZstdLibDir
-- Arguments passed to GHC when compiling C and .cmm sources.
let ghcArgs = mconcat
@@ -392,6 +395,7 @@ rtsPackageArgs = package rts ? do
, builder (Cabal Setup) ? mconcat
[ cabalExtraDirs libdwIncludeDir libdwLibraryDir
, cabalExtraDirs libnumaIncludeDir libnumaLibraryDir
+ , cabalExtraDirs libzstdIncludeDir libzstdLibraryDir
, useSystemFfi ? cabalExtraDirs ffiIncludeDir ffiLibraryDir
]
, builder (Cc (FindCDependencies CDep)) ? cArgs
=====================================
libraries/base/GHC/Int.hs
=====================================
@@ -69,8 +69,8 @@ instance Eq Int8 where
(/=) = neInt8
eqInt8, neInt8 :: Int8 -> Int8 -> Bool
-eqInt8 (I8# x) (I8# y) = isTrue# (x `eqInt8#` y)
-neInt8 (I8# x) (I8# y) = isTrue# (x `neInt8#` y)
+eqInt8 (I8# x) (I8# y) = isTrue# ((int8ToInt# x) ==# (int8ToInt# y))
+neInt8 (I8# x) (I8# y) = isTrue# ((int8ToInt# x) /=# (int8ToInt# y))
{-# INLINE [1] eqInt8 #-}
{-# INLINE [1] neInt8 #-}
@@ -280,8 +280,8 @@ instance Eq Int16 where
(/=) = neInt16
eqInt16, neInt16 :: Int16 -> Int16 -> Bool
-eqInt16 (I16# x) (I16# y) = isTrue# (x `eqInt16#` y)
-neInt16 (I16# x) (I16# y) = isTrue# (x `neInt16#` y)
+eqInt16 (I16# x) (I16# y) = isTrue# ((int16ToInt# x) ==# (int16ToInt# y))
+neInt16 (I16# x) (I16# y) = isTrue# ((int16ToInt# x) /=# (int16ToInt# y))
{-# INLINE [1] eqInt16 #-}
{-# INLINE [1] neInt16 #-}
@@ -488,8 +488,8 @@ instance Eq Int32 where
(/=) = neInt32
eqInt32, neInt32 :: Int32 -> Int32 -> Bool
-eqInt32 (I32# x) (I32# y) = isTrue# (x `eqInt32#` y)
-neInt32 (I32# x) (I32# y) = isTrue# (x `neInt32#` y)
+eqInt32 (I32# x) (I32# y) = isTrue# ((int32ToInt# x) ==# (int32ToInt# y))
+neInt32 (I32# x) (I32# y) = isTrue# ((int32ToInt# x) /=# (int32ToInt# y))
{-# INLINE [1] eqInt32 #-}
{-# INLINE [1] neInt32 #-}
=====================================
libraries/base/GHC/Word.hs
=====================================
@@ -78,8 +78,8 @@ instance Eq Word8 where
(/=) = neWord8
eqWord8, neWord8 :: Word8 -> Word8 -> Bool
-eqWord8 (W8# x) (W8# y) = isTrue# (x `eqWord8#` y)
-neWord8 (W8# x) (W8# y) = isTrue# (x `neWord8#` y)
+eqWord8 (W8# x) (W8# y) = isTrue# ((word8ToWord# x) `eqWord#` (word8ToWord# y))
+neWord8 (W8# x) (W8# y) = isTrue# ((word8ToWord# x) `neWord#` (word8ToWord# y))
{-# INLINE [1] eqWord8 #-}
{-# INLINE [1] neWord8 #-}
@@ -268,8 +268,8 @@ instance Eq Word16 where
(/=) = neWord16
eqWord16, neWord16 :: Word16 -> Word16 -> Bool
-eqWord16 (W16# x) (W16# y) = isTrue# (x `eqWord16#` y)
-neWord16 (W16# x) (W16# y) = isTrue# (x `neWord16#` y)
+eqWord16 (W16# x) (W16# y) = isTrue# ((word16ToWord# x) `eqWord#` (word16ToWord# y))
+neWord16 (W16# x) (W16# y) = isTrue# ((word16ToWord# x) `neWord#` (word16ToWord# y))
{-# INLINE [1] eqWord16 #-}
{-# INLINE [1] neWord16 #-}
@@ -500,8 +500,8 @@ instance Eq Word32 where
(/=) = neWord32
eqWord32, neWord32 :: Word32 -> Word32 -> Bool
-eqWord32 (W32# x) (W32# y) = isTrue# (x `eqWord32#` y)
-neWord32 (W32# x) (W32# y) = isTrue# (x `neWord32#` y)
+eqWord32 (W32# x) (W32# y) = isTrue# ((word32ToWord# x) `eqWord#` (word32ToWord# y))
+neWord32 (W32# x) (W32# y) = isTrue# ((word32ToWord# x) `neWord#` (word32ToWord# y))
{-# INLINE [1] eqWord32 #-}
{-# INLINE [1] neWord32 #-}
=====================================
m4/fp_find_libnuma.m4
=====================================
@@ -30,7 +30,7 @@ AC_DEFUN([FP_FIND_LIBNUMA],
[Enable NUMA memory policy and thread affinity support in the
runtime system via numactl's libnuma [default=auto]])])
- if test "$enable_numa" != "no" ; then
+ if test "$enable_numa" = "yes" ; then
CFLAGS2="$CFLAGS"
CFLAGS="$LIBNUMA_CFLAGS $CFLAGS"
LDFLAGS2="$LDFLAGS"
@@ -41,7 +41,7 @@ AC_DEFUN([FP_FIND_LIBNUMA],
if test "$ac_cv_header_numa_h$ac_cv_header_numaif_h" = "yesyes" ; then
AC_CHECK_LIB(numa, numa_available,HaveLibNuma=1)
fi
- if test "$enable_numa:$HaveLibNuma" = "yes:0" ; then
+ if test "$HaveLibNuma" = "0" ; then
AC_MSG_ERROR([Cannot find system libnuma (required by --enable-numa)])
fi
=====================================
m4/fp_find_libzstd.m4
=====================================
@@ -0,0 +1,79 @@
+AC_DEFUN([FP_FIND_LIBZSTD],
+[
+ dnl ** Is IPE data compression enabled?
+ dnl --------------------------------------------------------------
+ AC_ARG_ENABLE(
+ ipe-data-compression,
+ [AS_HELP_STRING(
+ [--enable-ipe-data-compression],
+ [Enable compression of info table provenance entries using the
+ zstd compression library [default=no]]
+ )],
+ [FP_CAPITALIZE_YES_NO(["$enableval"], [EnableIpeDataCompression])],
+ [EnableIpeDataCompression=NO]
+ )
+
+ HaveLibZstd=0
+ if test "$EnableIpeDataCompression" = "YES"; then
+ dnl ** Have zstd?
+ dnl --------------------------------------------------------------
+ AC_ARG_WITH(
+ libzstd-libraries,
+ [AS_HELP_STRING(
+ [--with-libzstd-libraries=ARG],
+ [Find libraries for libzstd in ARG [default=system default]]
+ )],
+ [
+ LibZstdLibDir="$withval"
+ LIBZSTD_LDFLAGS="-L$withval"
+ ]
+ )
+
+ AC_SUBST(LibZstdLibDir)
+
+ AC_ARG_WITH(
+ libzstd-includes,
+ [AS_HELP_STRING(
+ [--with-libzstd-includes=ARG],
+ [Find includes for libzstd in ARG [default=system default]]
+ )],
+ [
+ LibZstdIncludeDir="$withval"
+ LIBZSTD_CFLAGS="-I$withval"
+ ]
+ )
+
+ AC_SUBST(LibZstdIncludeDir)
+
+ CFLAGS2="$CFLAGS"
+ CFLAGS="$LIBZSTD_CFLAGS $CFLAGS"
+ LDFLAGS2="$LDFLAGS"
+ LDFLAGS="$LIBZSTD_LDFLAGS $LDFLAGS"
+
+ AC_CHECK_HEADERS([zstd.h])
+
+ if test "$ac_cv_header_zstd_h" = "yes" ; then
+ AC_CHECK_LIB(zstd,ZSTD_versionString,HaveLibZstd=1)
+ fi
+ if test "$HaveLibZstd" = "0" ; then
+ AC_MSG_ERROR(
+ [Cannot find system libzstd (required by
+ --enable-ipe-data-compression)]
+ )
+ fi
+
+ CFLAGS="$CFLAGS2"
+ LDFLAGS="$LDFLAGS2"
+ fi
+
+ AC_DEFINE_UNQUOTED([HAVE_LIBZSTD], [$HaveLibZstd], [Define to 1 if you
+ wish to compress IPE data in compiler results (requires libzstd)])
+
+ if test $HaveLibZstd = "1" ; then
+ AC_SUBST([UseLibZstd],[YES])
+ AC_SUBST([CabalHaveLibZstd],[True])
+ else
+ AC_SUBST([UseLibZstd],[NO])
+ AC_SUBST([CabalHaveLibZstd],[False])
+ fi
+])
=====================================
rts/IPE.c
=====================================
@@ -20,6 +20,10 @@
#include <fs_rts.h>
#include <string.h>
+#if HAVE_LIBZSTD == 1
+#include <zstd.h>
+#endif
+
#if defined(TRACING)
#include "Trace.h"
#endif
@@ -36,8 +40,9 @@ collecting IPE lists on registration.
It's a singly linked list of IPE list buffers (IpeBufferListNode). These are
emitted by the code generator, with generally one produced per module. Each
-contains an array of IPE entries and a link field (which is used to link
-buffers onto the pending list.
+contains a pointer to a list of IPE entries, a pointer to a list of info
+table pointers, and a link field (which is used to link buffers onto the
+pending list.
For reasons of space efficiency, IPE entries are represented slightly
differently in the object file than the InfoProvEnt which we ultimately expose
@@ -77,23 +82,23 @@ void exitIpe(void) { }
#endif // THREADED_RTS
-static InfoProvEnt ipeBufferEntryToIpe(const IpeBufferListNode *node, const IpeBufferEntry *ent)
+static InfoProvEnt ipeBufferEntryToIpe(const char *strings, const StgInfoTable *tbl, const IpeBufferEntry ent)
{
- const char *strings = node->string_table;
return (InfoProvEnt) {
- .info = ent->info,
+ .info = tbl,
.prov = {
- .table_name = &strings[ent->table_name],
- .closure_desc = &strings[ent->closure_desc],
- .ty_desc = &strings[ent->ty_desc],
- .label = &strings[ent->label],
- .module = &strings[ent->module_name],
- .src_file = &strings[ent->src_file],
- .src_span = &strings[ent->src_span]
+ .table_name = &strings[ent.table_name],
+ .closure_desc = &strings[ent.closure_desc],
+ .ty_desc = &strings[ent.ty_desc],
+ .label = &strings[ent.label],
+ .module = &strings[ent.module_name],
+ .src_file = &strings[ent.src_file],
+ .src_span = &strings[ent.src_span]
}
};
}
+
#if defined(TRACING)
static void traceIPEFromHashTable(void *data STG_UNUSED, StgWord key STG_UNUSED,
const void *value) {
@@ -105,8 +110,18 @@ void dumpIPEToEventLog(void) {
// Dump pending entries
IpeBufferListNode *cursor = RELAXED_LOAD(&ipeBufferList);
while (cursor != NULL) {
+ IpeBufferEntry *entries;
+ char *strings;
+
+ // Decompress if compressed
+ decompressIPEBufferListNodeIfCompressed(cursor, &entries, &strings);
+
for (uint32_t i = 0; i < cursor->count; i++) {
- const InfoProvEnt ent = ipeBufferEntryToIpe(cursor, &cursor->entries[i]);
+ const InfoProvEnt ent = ipeBufferEntryToIpe(
+ strings,
+ cursor->tables[i],
+ entries[i]
+ );
traceIPE(&ent);
}
cursor = cursor->next;
@@ -120,6 +135,7 @@ void dumpIPEToEventLog(void) {
RELEASE_LOCK(&ipeMapLock);
}
+
#else
void dumpIPEToEventLog(void) { }
@@ -169,16 +185,84 @@ void updateIpeMap() {
}
while (pending != NULL) {
- IpeBufferListNode *currentNode = pending;
- InfoProvEnt *ip_ents = stgMallocBytes(sizeof(InfoProvEnt) * currentNode->count, "updateIpeMap");
- for (uint32_t i = 0; i < currentNode->count; i++) {
- const IpeBufferEntry *ent = ¤tNode->entries[i];
- ip_ents[i] = ipeBufferEntryToIpe(currentNode, ent);
- insertHashTable(ipeMap, (StgWord) ent->info, &ip_ents[i]);
+ IpeBufferListNode *current_node = pending;
+ const IpeBufferEntry *entries;
+ const char *strings;
+
+ // Decompress if compressed
+ decompressIPEBufferListNodeIfCompressed(current_node, &entries, &strings);
+
+ // Convert the on-disk IPE buffer entry representation (IpeBufferEntry)
+ // into the runtime representation (InfoProvEnt)
+ InfoProvEnt *ip_ents = stgMallocBytes(
+ sizeof(InfoProvEnt) * current_node->count,
+ "updateIpeMap: ip_ents"
+ );
+ for (uint32_t i = 0; i < current_node->count; i++) {
+ const IpeBufferEntry ent = entries[i];
+ const StgInfoTable *tbl = current_node->tables[i];
+ ip_ents[i] = ipeBufferEntryToIpe(strings, tbl, ent);
+ insertHashTable(ipeMap, (StgWord) tbl, &ip_ents[i]);
}
- pending = currentNode->next;
+ pending = current_node->next;
}
RELEASE_LOCK(&ipeMapLock);
}
+
+/* Decompress the IPE data and strings table referenced by an IPE buffer list
+node if it is compressed. No matter whether the data is compressed, the pointers
+referenced by the 'entries_dst' and 'string_table_dst' parameters will point at
+the decompressed IPE data and string table for the given node, respectively,
+upon return from this function.
+*/
+void decompressIPEBufferListNodeIfCompressed(IpeBufferListNode *node, IpeBufferEntry **entries_dst, char **string_table_dst) {
+ if (node->compressed) {
+ // The IPE list buffer node indicates that the strings table and
+ // entries list has been compressed. If zstd is not available, fail.
+ // If zstd is available, decompress.
+#if HAVE_LIBZSTD == 0
+ barf("An IPE buffer list node has been compressed, but the \
+ decompression library (zstd) is not available.");
+#else
+ size_t compressed_sz = ZSTD_findFrameCompressedSize(
+ node->string_table,
+ node->string_table_size
+ );
+ char *decompressed_strings = stgMallocBytes(
+ node->string_table_size,
+ "updateIpeMap: decompressed_strings"
+ );
+ ZSTD_decompress(
+ decompressed_strings,
+ node->string_table_size,
+ node->string_table,
+ compressed_sz
+ );
+ *string_table_dst = decompressed_strings;
+
+ // Decompress the IPE data
+ compressed_sz = ZSTD_findFrameCompressedSize(
+ node->entries,
+ node->entries_size
+ );
+ void *decompressed_entries = stgMallocBytes(
+ node->entries_size,
+ "updateIpeMap: decompressed_entries"
+ );
+ ZSTD_decompress(
+ decompressed_entries,
+ node->entries_size,
+ node->entries,
+ compressed_sz
+ );
+ *entries_dst = decompressed_entries;
+#endif // HAVE_LIBZSTD == 0
+
+ } else {
+ // Not compressed, no need to decompress
+ *entries_dst = node->entries;
+ *string_table_dst = node->string_table;
+ }
+}
=====================================
rts/IPE.h
=====================================
@@ -17,5 +17,6 @@ void dumpIPEToEventLog(void);
void updateIpeMap(void);
void initIpe(void);
void exitIpe(void);
+void decompressIPEBufferListNodeIfCompressed(IpeBufferListNode*, IpeBufferEntry**, char**);
#include "EndPrivate.h"
=====================================
rts/include/rts/IPE.h
=====================================
@@ -52,9 +52,6 @@ typedef uint32_t StringIdx;
// The size of this must be a multiple of the word size
// to ensure correct packing.
typedef struct {
- // When TNTC is enabled this will point to the entry code
- // not the info table itself.
- const StgInfoTable *info;
StringIdx table_name;
StringIdx closure_desc;
StringIdx ty_desc;
@@ -69,10 +66,23 @@ GHC_STATIC_ASSERT(sizeof(IpeBufferEntry) % (WORD_SIZE_IN_BITS / 8) == 0, "sizeof
typedef struct IpeBufferListNode_ {
struct IpeBufferListNode_ *next;
+
// Everything below is read-only and generated by the codegen
- const char *string_table;
+
+ // This flag should be treated as a boolean
+ StgWord compressed;
+
StgWord count;
- IpeBufferEntry entries[];
+
+ // When TNTC is enabled, these will point to the entry code
+ // not the info table itself.
+ StgInfoTable **tables;
+
+ IpeBufferEntry *entries;
+ StgWord entries_size; // decompressed size
+
+ char *string_table;
+ StgWord string_table_size; // decompressed size
} IpeBufferListNode;
void registerInfoProvList(IpeBufferListNode *node);
=====================================
rts/rts.cabal.in
=====================================
@@ -45,6 +45,8 @@ flag libdw
default: @CabalHaveLibdw@
flag libnuma
default: @CabalHaveLibNuma@
+flag libzstd
+ default: @CabalHaveLibZstd@
flag 64bit
default: @Cabal64bit@
flag leading-underscore
@@ -212,6 +214,8 @@ library
extra-libraries: elf dw
if flag(libnuma)
extra-libraries: numa
+ if flag(libzstd)
+ extra-libraries: zstd
if !flag(smp)
cpp-options: -DNOSMP
=====================================
testsuite/tests/overloadedrecflds/should_run/T11671_run.hs
=====================================
@@ -12,8 +12,9 @@ import GHC.Prim (Addr#)
instance KnownSymbol symbol => IsLabel symbol String where
fromLabel = symbolVal (Proxy :: Proxy symbol)
-(#) :: String -> Int -> String
+(#), (#.) :: String -> Int -> String
(#) _ i = show i
+_ #. i = show i
f :: Addr# -> Int -> String
f _ i = show i
@@ -26,13 +27,13 @@ main = traverse_ putStrLn
, #type
, #Foo
, #3
- , #199.4
+ , #"199.4"
, #17a23b
, #f'a'
, #'a'
, #'
, #''notTHSplice
- , #...
+ , #"..."
, #привет
, #こんにちは
, #"3"
=====================================
testsuite/tests/printer/Test22771.hs
=====================================
@@ -14,8 +14,9 @@ import GHC.Prim (Addr#)
instance KnownSymbol symbol => IsLabel symbol String where
fromLabel = symbolVal (Proxy :: Proxy symbol)
-(#) :: String -> Int -> String
+(#), (#.) :: String -> Int -> String
(#) _ i = show i
+_ #. i = show i
f :: Addr# -> Int -> String
f _ i = show i
@@ -28,13 +29,13 @@ main = traverse_ putStrLn
, #type
, #Foo
, #3
- , #199.4
+ , #"199.4"
, #17a23b
, #f'a'
, #'a'
, #'
, #''notTHSplice
- , #...
+ , #"..."
, #привет
, #こんにちは
, #"3"
=====================================
testsuite/tests/rts/ipe/ipeEventLog_fromMap.c
=====================================
@@ -19,7 +19,7 @@ int main(int argc, char *argv[]) {
registerInfoProvList(list2);
// Query an IPE to initialize the underlying hash map.
- lookupIPE(list1->entries[0].info);
+ lookupIPE(list1->tables[0]);
// Trace all IPE events.
dumpIPEToEventLog();
=====================================
testsuite/tests/rts/ipe/ipeMap.c
=====================================
@@ -40,15 +40,23 @@ void shouldFindNothingInAnEmptyIPEMap(Capability *cap) {
}
HaskellObj shouldFindOneIfItHasBeenRegistered(Capability *cap) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *));
+ node->entries = malloc(sizeof(IpeBufferEntry));
+
StringTable st;
init_string_table(&st);
HaskellObj fortyTwo = UNTAG_CLOSURE(rts_mkInt(cap, 42));
- node->entries[0] = makeAnyProvEntry(cap, &st, fortyTwo, 42);
- node->count = 1;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 1;
+ node->tables[0] = get_itbl(fortyTwo);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 42);
+ node->entries_size = sizeof(IpeBufferEntry);
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
@@ -72,15 +80,23 @@ HaskellObj shouldFindOneIfItHasBeenRegistered(Capability *cap) {
void shouldFindTwoIfTwoHaveBeenRegistered(Capability *cap,
HaskellObj fortyTwo) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *));
+ node->entries = malloc(sizeof(IpeBufferEntry));
+
StringTable st;
init_string_table(&st);
HaskellObj twentyThree = UNTAG_CLOSURE(rts_mkInt8(cap, 23));
- node->entries[0] = makeAnyProvEntry(cap, &st, twentyThree, 23);
- node->count = 1;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 1;
+ node->tables[0] = get_itbl(twentyThree);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 23);
+ node->entries_size = sizeof(IpeBufferEntry);
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
@@ -103,17 +119,26 @@ void shouldFindTwoIfTwoHaveBeenRegistered(Capability *cap,
}
void shouldFindTwoFromTheSameList(Capability *cap) {
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + 2 * sizeof(IpeBufferEntry));
+ // Allocate buffers for IPE buffer list node
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *) * 2);
+ node->entries = malloc(sizeof(IpeBufferEntry) * 2);
+
StringTable st;
init_string_table(&st);
HaskellObj one = UNTAG_CLOSURE(rts_mkInt16(cap, 1));
HaskellObj two = UNTAG_CLOSURE(rts_mkInt32(cap, 2));
- node->entries[0] = makeAnyProvEntry(cap, &st, one, 1);
- node->entries[1] = makeAnyProvEntry(cap, &st, two, 2);
- node->count = 2;
node->next = NULL;
+ node->compressed = 0;
+ node->count = 2;
+ node->tables[0] = get_itbl(one);
+ node->tables[1] = get_itbl(two);
+ node->entries[0] = makeAnyProvEntry(cap, &st, 1);
+ node->entries[1] = makeAnyProvEntry(cap, &st, 2);
+ node->entries_size = sizeof(IpeBufferEntry) * 2;
node->string_table = st.buffer;
+ node->string_table_size = st.size;
registerInfoProvList(node);
=====================================
testsuite/tests/rts/ipe/ipe_lib.c
=====================================
@@ -25,9 +25,8 @@ uint32_t add_string(StringTable *st, const char *s) {
return n;
}
-IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj closure, int i) {
+IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, int i) {
IpeBufferEntry provEnt;
- provEnt.info = get_itbl(closure);
unsigned int tableNameLength = strlen("table_name_") + 3 /* digits */ + 1 /* null character */;
char *tableName = malloc(sizeof(char) * tableNameLength);
@@ -69,15 +68,27 @@ IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj clo
IpeBufferListNode *makeAnyProvEntries(Capability *cap, int start, int end) {
const int n = end - start;
- IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode) + n * sizeof(IpeBufferEntry));
+
+ // Allocate buffers for IpeBufferListNode
+ IpeBufferListNode *node = malloc(sizeof(IpeBufferListNode));
+ node->tables = malloc(sizeof(StgInfoTable *) * n);
+ node->entries = malloc(sizeof(IpeBufferEntry) * n);
+
StringTable st;
init_string_table(&st);
+
+ // Make the entries and fill the buffers
for (int i=start; i < end; i++) {
HaskellObj closure = rts_mkInt(cap, 42);
- node->entries[i] = makeAnyProvEntry(cap, &st, closure, i);
+ node->tables[i] = get_itbl(closure);
+ node->entries[i] = makeAnyProvEntry(cap, &st, i);
}
+
+ // Set the rest of the fields
node->next = NULL;
+ node->compressed = 0;
node->count = n;
node->string_table = st.buffer;
+
return node;
}
=====================================
testsuite/tests/rts/ipe/ipe_lib.h
=====================================
@@ -12,6 +12,6 @@ void init_string_table(StringTable *st);
uint32_t add_string(StringTable *st, const char *s);
IpeBufferListNode *makeAnyProvEntries(Capability *cap, int start, int end);
-IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, HaskellObj closure, int i);
+IpeBufferEntry makeAnyProvEntry(Capability *cap, StringTable *st, int i);
void dumpIPEToEventLog(void);
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/921e9536350c9210fae2604a603d214b3c673ca6...a240bf498894b60c4b4b1c1935fbe0a15bbc74a2
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/921e9536350c9210fae2604a603d214b3c673ca6...a240bf498894b60c4b4b1c1935fbe0a15bbc74a2
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230207/691b798f/attachment-0001.html>
More information about the ghc-commits
mailing list