[Git][ghc/ghc][wip/22188] Add flag to control whether self-recompilation information is written to interface
Matthew Pickering (@mpickering)
gitlab at gitlab.haskell.org
Thu Feb 27 11:19:20 UTC 2025
Matthew Pickering pushed to branch wip/22188 at Glasgow Haskell Compiler / GHC
Commits:
2b89c263 by Matthew Pickering at 2025-02-27T11:17:40+00:00
Add flag to control whether self-recompilation information is written to interface
This patch adds the flag -fwrite-self-recomp-info which controls whether
interface files contain the information necessary to answer the
question:
Do I need to recompile myself or is this current interface file
suitable?
Why? Most packages are only built once either by a distribution or cabal
and then placed into an immutable store, after which we will never ask
this question. Therefore we can derive two benefits from omitting this
information.
* Primary motivation: It vastly reduces the surface area for creating
non-deterministic interface files. See issue #10424 which motivated a
proper fix to that issue. Distributions have long contained versions
of GHC which just have broken self-recompilation checking (in order to
get deterministic interface files).
* Secondary motivation: This reduces the size of interface files
slightly.. the `mi_usages` field can be quite big but probably this
isn't such a great benefit.
* Third motivation: Conceptually clarity about which parts of an
interface file are used in order to **communicate** with subsequent
packages about the **interface** for a module. And which parts are
used to self-communicate during recompilation checking.
In addition to this, the change alerted me to the incorrect
implemenation of the reifyModule function. See #8489 for more discussion
about how to fix this if anyone was so inclined. For now I just added a
warning `-Wreify-module-missing-info` which triggers if the module you
are trying to reify doesn't have a suitable interface. Interfaces which
are unsuitable include:
* The GHC.Prim interface, which is a fake interface
* Interfaces compiled with -fno-write-self-recomp-info
The main tracking issue is #22188 but fixes issues such as #10424 in a
proper way.
- - - - -
26 changed files:
- compiler/GHC.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/HsToCore.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Types/Error/Codes.hs
- compiler/GHC/Unit/Module/ModGuts.hs
- compiler/GHC/Unit/Module/ModIface.hs
- docs/users_guide/phases.rst
- docs/users_guide/using-warnings.rst
- + testsuite/tests/annotations/should_compile/th/annth_compunits.stderr
- + testsuite/tests/annotations/should_compile/th/annth_make.stderr
- + testsuite/tests/driver/self-recomp/Makefile
- + testsuite/tests/driver/self-recomp/SelfRecomp01.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp02.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp03.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.hs
- + testsuite/tests/driver/self-recomp/SelfRecomp04.stdout
- + testsuite/tests/driver/self-recomp/all.T
Changes:
=====================================
compiler/GHC.hs
=====================================
@@ -104,10 +104,8 @@ module GHC (
mi_module,
mi_sig_of,
mi_hsc_src,
- mi_src_hash,
mi_hi_bytes,
mi_deps,
- mi_usages,
mi_exports,
mi_used_th,
mi_fixities,
=====================================
compiler/GHC/Driver/DynFlags.hs
=====================================
@@ -1178,7 +1178,8 @@ defaultFlags settings
Opt_ShowErrorContext,
Opt_SuppressStgReps,
Opt_UnoptimizedCoreForInterpreter,
- Opt_SpecialiseIncoherents
+ Opt_SpecialiseIncoherents,
+ Opt_WriteSelfRecompInfo
]
++ [f | (ns,f) <- optLevelFlags, 0 `elem` ns]
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -692,6 +692,7 @@ data GeneralFlag
| Opt_ExposeOverloadedUnfoldings
| Opt_KeepAutoRules -- ^Keep auto-generated rules even if they seem to have become useless
| Opt_WriteInterface -- forces .hi files to be written even with -fno-code
+ | Opt_WriteSelfRecompInfo
| Opt_WriteHie -- generate .hie files
-- JavaScript opts
@@ -1076,6 +1077,7 @@ data WarningFlag =
| Opt_WarnDataKindsTC -- Since 9.10
| Opt_WarnDefaultedExceptionContext -- Since 9.10
| Opt_WarnViewPatternSignatures -- Since 9.12
+ | Opt_WarnReifyModuleMissingInfo -- Since 9.6
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1192,6 +1194,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnDataKindsTC -> "data-kinds-tc" :| []
Opt_WarnDefaultedExceptionContext -> "defaulted-exception-context" :| []
Opt_WarnViewPatternSignatures -> "view-pattern-signatures" :| []
+ Opt_WarnReifyModuleMissingInfo -> "reify-module-missing-info" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1333,7 +1336,9 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnInconsistentFlags,
Opt_WarnDataKindsTC,
Opt_WarnTypeEqualityOutOfScope,
- Opt_WarnViewPatternSignatures
+ Opt_WarnViewPatternSignatures,
+ Opt_WarnTypeEqualityRequiresOperators,
+ Opt_WarnReifyModuleMissingInfo
]
-- | Things you get with -W
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2358,6 +2358,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnDataKindsTC -> warnSpec x
Opt_WarnDefaultedExceptionContext -> warnSpec x
Opt_WarnViewPatternSignatures -> warnSpec x
+ Opt_WarnReifyModuleMissingInfo -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
@@ -2528,6 +2529,7 @@ fFlagsDeps = [
flagSpec "use-rpaths" Opt_RPath,
flagSpec "write-interface" Opt_WriteInterface,
flagSpec "write-if-simplified-core" Opt_WriteIfSimplifiedCore,
+ flagSpec "write-self-recomp-info" Opt_WriteSelfRecompInfo,
flagSpec "write-ide-info" Opt_WriteHie,
flagSpec "unbox-small-strict-fields" Opt_UnboxSmallStrictFields,
flagSpec "unbox-strict-fields" Opt_UnboxStrictFields,
=====================================
compiler/GHC/HsToCore.hs
=====================================
@@ -22,14 +22,12 @@ import GHC.Driver.DynFlags
import GHC.Driver.Config
import GHC.Driver.Config.Core.Lint ( endPassHscEnvIO )
import GHC.Driver.Config.HsToCore.Ticks
-import GHC.Driver.Config.HsToCore.Usage
import GHC.Driver.Env
import GHC.Driver.Backend
import GHC.Driver.Plugins
import GHC.Hs
-import GHC.HsToCore.Usage
import GHC.HsToCore.Monad
import GHC.HsToCore.Errors.Types
import GHC.HsToCore.Expr
@@ -42,7 +40,7 @@ import GHC.HsToCore.Docs
import GHC.Tc.Types
import GHC.Tc.Types.Origin ( Position(..) )
-import GHC.Tc.Utils.Monad ( finalSafeMode, fixSafeInstances, initIfaceLoad )
+import GHC.Tc.Utils.Monad ( finalSafeMode, fixSafeInstances )
import GHC.Tc.Module ( runTcInteractive )
import GHC.Core.Type
@@ -100,6 +98,7 @@ import GHC.Unit.Module.Deps
import Data.List (partition)
import Data.IORef
import Data.Traversable (for)
+import GHC.Iface.Make (mkRecompUsageInfo)
{-
************************************************************************
@@ -127,12 +126,10 @@ deSugar hsc_env
tcg_fix_env = fix_env,
tcg_inst_env = inst_env,
tcg_fam_inst_env = fam_inst_env,
- tcg_merged = merged,
tcg_warns = warns,
tcg_anns = anns,
tcg_binds = binds,
tcg_imp_specs = imp_specs,
- tcg_dependent_files = dependent_files,
tcg_ev_binds = ev_binds,
tcg_th_foreign_files = th_foreign_files_var,
tcg_fords = fords,
@@ -228,8 +225,7 @@ deSugar hsc_env
; endPassHscEnvIO hsc_env name_ppr_ctx CoreDesugarOpt ds_binds ds_rules_for_imps
- ; let used_names = mkUsedNames tcg_env
- pluginModules = map lpModule (loadedPlugins (hsc_plugins hsc_env))
+ ; let pluginModules = map lpModule (loadedPlugins (hsc_plugins hsc_env))
home_unit = hsc_home_unit hsc_env
; let deps = mkDependencies home_unit
(tcg_mod tcg_env)
@@ -237,17 +233,10 @@ deSugar hsc_env
(map mi_module pluginModules)
; used_th <- readIORef tc_splice_used
- ; dep_files <- readIORef dependent_files
; safe_mode <- finalSafeMode dflags tcg_env
- ; (needed_mods, needed_pkgs) <- readIORef (tcg_th_needed_deps tcg_env)
-
- ; let uc = initUsageConfig hsc_env
- ; let plugins = hsc_plugins hsc_env
- ; let fc = hsc_FC hsc_env
- ; let unit_env = hsc_unit_env hsc_env
- ; usages <- initIfaceLoad hsc_env $
- mkUsageInfo uc plugins fc unit_env mod (imp_mods imports) used_names
- dep_files merged needed_mods needed_pkgs
+
+ ; usages <- mkRecompUsageInfo hsc_env tcg_env
+
-- id_mod /= mod when we are processing an hsig, but hsigs
-- never desugared and compiled (there's no code!)
-- Consequently, this should hold for any ModGuts that make
=====================================
compiler/GHC/Iface/Binary.hs
=====================================
@@ -50,7 +50,7 @@ import GHC.Types.Name.Cache
import GHC.Types.SrcLoc
import GHC.Platform
import GHC.Settings.Constants
-import GHC.Utils.Fingerprint
+--import GHC.Utils.Fingerprint
import GHC.Iface.Type (IfaceType(..), getIfaceType, putIfaceType, ifaceTypeSharedByte)
import Control.Monad
@@ -115,7 +115,7 @@ readBinIfaceHeader
-> CheckHiWay
-> TraceBinIFace
-> FilePath
- -> IO (Fingerprint, ReadBinHandle)
+ -> IO ReadBinHandle
readBinIfaceHeader profile _name_cache checkHiWay traceBinIFace hi_path = do
let platform = profilePlatform profile
@@ -157,8 +157,7 @@ readBinIfaceHeader profile _name_cache checkHiWay traceBinIFace hi_path = do
when (checkHiWay == CheckHiWay) $
errorOnMismatch "mismatched interface file profile tag" tag check_tag
- src_hash <- get bh
- pure (src_hash, bh)
+ pure bh
-- | Read an interface file.
--
@@ -171,12 +170,12 @@ readBinIface
-> FilePath
-> IO ModIface
readBinIface profile name_cache checkHiWay traceBinIface hi_path = do
- (src_hash, bh) <- readBinIfaceHeader profile name_cache checkHiWay traceBinIface hi_path
+ bh <- readBinIfaceHeader profile name_cache checkHiWay traceBinIface hi_path
mod_iface <- getIfaceWithExtFields name_cache bh
return $ mod_iface
- & addSourceFingerprint src_hash
+-- & addSourceFingerprint src_hash
getIfaceWithExtFields :: NameCache -> ReadBinHandle -> IO ModIface
@@ -260,7 +259,6 @@ writeBinIface profile traceBinIface compressionLevel hi_path mod_iface = do
put_ bh (show hiVersion)
let tag = profileBuildTag profile
put_ bh tag
- put_ bh (mi_src_hash mod_iface)
putIfaceWithExtFields traceBinIface compressionLevel bh mod_iface
=====================================
compiler/GHC/Iface/Load.hs
=====================================
@@ -1259,30 +1259,33 @@ pprModIfaceSimple unit_state iface =
-- The UnitState is used to pretty-print units
pprModIface :: UnitState -> ModIface -> SDoc
pprModIface unit_state iface
- = vcat [ text "interface"
+ = vcat $ [ text "interface"
<+> ppr (mi_module iface) <+> pp_hsc_src (mi_hsc_src iface)
+ <+> whenIsSelfRecomp (text "[self-recomp]")
<+> (if mi_orphan exts then text "[orphan module]" else Outputable.empty)
<+> (if mi_finsts exts then text "[family instance module]" else Outputable.empty)
<+> (if mi_hpc iface then text "[hpc]" else Outputable.empty)
<+> integer hiVersion
- , nest 2 (text "interface hash:" <+> ppr (mi_iface_hash exts))
, nest 2 (text "ABI hash:" <+> ppr (mi_mod_hash exts))
, nest 2 (text "export-list hash:" <+> ppr (mi_exp_hash exts))
+ , whenIsSelfRecomp $ vcat
+ [ nest 2 (text "interface hash:" <+> ppr (mi_iface_hash exts))
+ , nest 2 (text "flag hash:" <+> ppr (mi_flag_hash exts))
+ , nest 2 (text "opt_hash:" <+> ppr (mi_opt_hash exts))
+ , nest 2 (text "hpc_hash:" <+> ppr (mi_hpc_hash exts))
+ , nest 2 (text "plugin_hash:" <+> ppr (mi_plugin_hash exts))
+ ]
, nest 2 (text "orphan hash:" <+> ppr (mi_orphan_hash exts))
- , nest 2 (text "flag hash:" <+> ppr (mi_flag_hash exts))
- , nest 2 (text "opt_hash:" <+> ppr (mi_opt_hash exts))
- , nest 2 (text "hpc_hash:" <+> ppr (mi_hpc_hash exts))
- , nest 2 (text "plugin_hash:" <+> ppr (mi_plugin_hash exts))
- , nest 2 (text "src_hash:" <+> ppr (mi_src_hash iface))
, nest 2 (text "sig of:" <+> ppr (mi_sig_of iface))
, nest 2 (text "used TH splices:" <+> ppr (mi_used_th iface))
+ , whenIsSelfRecomp $ nest 2 (text "src_hash:" <+> ppr (mi_src_hash iface))
, nest 2 (text "where")
, text "exports:"
, nest 2 (vcat (map pprExport (mi_exports iface)))
, text "defaults:"
, nest 2 (vcat (map ppr (mi_defaults iface)))
, pprDeps unit_state (mi_deps iface)
- , vcat (map pprUsage (mi_usages iface))
+ , whenIsSelfRecomp $ vcat (map pprUsage (mi_usages iface))
, vcat (map pprIfaceAnnotation (mi_anns iface))
, pprFixities (mi_fixities iface)
, vcat [ppr ver $$ nest 2 (ppr decl) | (ver,decl) <- mi_decls iface]
@@ -1302,6 +1305,9 @@ pprModIface unit_state iface
]
where
exts = mi_final_exts iface
+ whenIsSelfRecomp action =
+ if (isSelfRecompilationInterface iface) then action else empty
+
pp_hsc_src HsBootFile = text "[boot]"
pp_hsc_src HsigFile = text "[hsig]"
pp_hsc_src HsSrcFile = Outputable.empty
=====================================
compiler/GHC/Iface/Make.hs
=====================================
@@ -13,6 +13,7 @@ module GHC.Iface.Make
( mkPartialIface
, mkFullIface
, mkIfaceTc
+ , mkRecompUsageInfo
, mkIfaceExports
)
where
@@ -237,14 +238,11 @@ mkIfaceTc hsc_env safe_mode mod_details mod_summary mb_program
tcg_import_decls = import_decls,
tcg_rdr_env = rdr_env,
tcg_fix_env = fix_env,
- tcg_merged = merged,
tcg_warns = warns,
tcg_hpc = other_hpc_info,
- tcg_th_splice_used = tc_splice_used,
- tcg_dependent_files = dependent_files
+ tcg_th_splice_used = tc_splice_used
}
= do
- let used_names = mkUsedNames tc_result
let pluginModules = map lpModule (loadedPlugins (hsc_plugins hsc_env))
let home_unit = hsc_home_unit hsc_env
let deps = mkDependencies home_unit
@@ -253,47 +251,58 @@ mkIfaceTc hsc_env safe_mode mod_details mod_summary mb_program
(map mi_module pluginModules)
let hpc_info = emptyHpcInfo other_hpc_info
used_th <- readIORef tc_splice_used
- dep_files <- (readIORef dependent_files)
- (needed_links, needed_pkgs) <- readIORef (tcg_th_needed_deps tc_result)
- let uc = initUsageConfig hsc_env
- plugins = hsc_plugins hsc_env
- fc = hsc_FC hsc_env
- unit_env = hsc_unit_env hsc_env
- -- Do NOT use semantic module here; this_mod in mkUsageInfo
- -- is used solely to decide if we should record a dependency
- -- or not. When we instantiate a signature, the semantic
- -- module is something we want to record dependencies for,
- -- but if you pass that in here, we'll decide it's the local
- -- module and does not need to be recorded as a dependency.
- -- See Note [Identity versus semantic module]
- usages <- initIfaceLoad hsc_env $ mkUsageInfo uc plugins fc unit_env this_mod (imp_mods imports) used_names
- dep_files merged needed_links needed_pkgs
+ usage <- mkRecompUsageInfo hsc_env tc_result
docs <- extractDocs (ms_hspp_opts mod_summary) tc_result
let partial_iface = mkIface_ hsc_env
this_mod (fromMaybe [] mb_program) hsc_src
used_th deps rdr_env import_decls
fix_env warns hpc_info
- (imp_trust_own_pkg imports) safe_mode usages
+ (imp_trust_own_pkg imports) safe_mode usage
docs mod_summary
mod_details
mkFullIface hsc_env partial_iface Nothing Nothing NoStubs []
+mkRecompUsageInfo :: HscEnv -> TcGblEnv -> IO (Maybe [Usage])
+mkRecompUsageInfo hsc_env tc_result = do
+ let dflags = hsc_dflags hsc_env
+ if not (gopt Opt_WriteSelfRecompInfo dflags)
+ then return Nothing
+ else do
+ let used_names = mkUsedNames tc_result
+ dep_files <- (readIORef (tcg_dependent_files tc_result))
+ (needed_links, needed_pkgs) <- readIORef (tcg_th_needed_deps tc_result)
+ let uc = initUsageConfig hsc_env
+ plugins = hsc_plugins hsc_env
+ fc = hsc_FC hsc_env
+ unit_env = hsc_unit_env hsc_env
+
+ -- Do NOT use semantic module here; this_mod in mkUsageInfo
+ -- is used solely to decide if we should record a dependency
+ -- or not. When we instantiate a signature, the semantic
+ -- module is something we want to record dependencies for,
+ -- but if you pass that in here, we'll decide it's the local
+ -- module and does not need to be recorded as a dependency.
+ -- See Note [Identity versus semantic module]
+ usages <- initIfaceLoad hsc_env $ mkUsageInfo uc plugins fc unit_env (tcg_mod tc_result) (imp_mods (tcg_imports tc_result)) used_names
+ dep_files (tcg_merged tc_result) needed_links needed_pkgs
+ return (Just usages)
+
mkIface_ :: HscEnv -> Module -> CoreProgram -> HscSource
-> Bool -> Dependencies -> GlobalRdrEnv -> [ImportUserSpec]
-> NameEnv FixItem -> Warnings GhcRn -> HpcInfo
-> Bool
-> SafeHaskellMode
- -> [Usage]
+ -> Maybe [Usage]
-> Maybe Docs
-> ModSummary
-> ModDetails
-> PartialModIface
mkIface_ hsc_env
this_mod core_prog hsc_src used_th deps rdr_env import_decls fix_env src_warns
- hpc_info pkg_trust_req safe_mode usages
+ hpc_info pkg_trust_req safe_mode musages
docs mod_summary
ModDetails{ md_defaults = defaults,
md_insts = insts,
@@ -343,6 +352,9 @@ mkIface_ hsc_env
annotations = map mkIfaceAnnotation anns
icomplete_matches = map mkIfaceCompleteMatch complete_matches
!rdrs = maybeGlobalRdrEnv rdr_env
+ self_recomp = case musages of
+ Nothing -> NoSelfRecomp
+ Just usages -> ModIfaceSelfRecomp (ms_hs_hash mod_summary) usages
emptyPartialModIface this_mod
-- Need to record this because it depends on the -instantiated-with flag
@@ -351,8 +363,8 @@ mkIface_ hsc_env
then Nothing
else Just semantic_mod)
& set_mi_hsc_src hsc_src
+ & set_mi_self_recomp self_recomp
& set_mi_deps deps
- & set_mi_usages usages
& set_mi_exports (mkIfaceExports exports)
& set_mi_defaults (defaultsToIfaceDefaults defaults)
@@ -377,7 +389,6 @@ mkIface_ hsc_env
& set_mi_docs docs
& set_mi_final_exts ()
& set_mi_ext_fields emptyExtensibleFields
- & set_mi_src_hash (ms_hs_hash mod_summary)
& set_mi_hi_bytes PartialIfaceBinHandle
where
=====================================
compiler/GHC/Iface/Recomp.hs
=====================================
@@ -171,6 +171,7 @@ data RecompReason
= UnitDepRemoved UnitId
| ModulePackageChanged FastString
| SourceFileChanged
+ | NoSelfRecompInfo
| ThisUnitIdChanged
| ImpurePlugin
| PluginsChanged
@@ -204,6 +205,7 @@ instance Outputable RecompReason where
UnitDepRemoved uid -> ppr uid <+> text "removed"
ModulePackageChanged s -> ftext s <+> text "package changed"
SourceFileChanged -> text "Source file changed"
+ NoSelfRecompInfo -> text "Old interface lacks recompilation info"
ThisUnitIdChanged -> text "-this-unit-id changed"
ImpurePlugin -> text "Impure plugin forced recompilation"
PluginsChanged -> text "Plugins changed"
@@ -380,6 +382,8 @@ checkVersions hsc_env mod_summary iface
-- but we ALSO must make sure the instantiation matches up. See
-- test case bkpcabal04!
; hsc_env <- getTopEnv
+ ; if not (isSelfRecompilationInterface iface)
+ then return $ outOfDateItemBecause NoSelfRecompInfo Nothing else do {
; if mi_src_hash iface /= ms_hs_hash mod_summary
then return $ outOfDateItemBecause SourceFileChanged Nothing else do {
; if not (isHomeModule home_unit (mi_module iface))
@@ -414,7 +418,7 @@ checkVersions hsc_env mod_summary iface
| u <- mi_usages iface]
; case recomp of (NeedsRecompile reason) -> return $ OutOfDateItem reason (Just iface) ; _ -> do {
; return $ UpToDateItem iface
- }}}}}}}
+ }}}}}}}}
where
logger = hsc_logger hsc_env
dflags = hsc_dflags hsc_env
@@ -1213,18 +1217,6 @@ addFingerprints hsc_env iface0
sorted_extra_decls :: Maybe [IfaceBindingX IfaceMaybeRhs IfaceTopBndrInfo]
sorted_extra_decls = sortOn binding_key <$> mi_extra_decls iface0
- -- the flag hash depends on:
- -- - (some of) dflags
- -- it returns two hashes, one that shouldn't change
- -- the abi hash and one that should
- flag_hash <- fingerprintDynFlags hsc_env this_mod putNameLiterally
-
- opt_hash <- fingerprintOptFlags dflags putNameLiterally
-
- hpc_hash <- fingerprintHpcFlags dflags putNameLiterally
-
- plugin_hash <- fingerprintPlugins (hsc_plugins hsc_env)
-
-- the ABI hash depends on:
-- - decls
-- - export list
@@ -1238,29 +1230,17 @@ addFingerprints hsc_env iface0
mi_warns iface0,
mi_foreign iface0)
- -- The interface hash depends on:
- -- - the ABI hash, plus
- -- - the source file hash,
- -- - the module level annotations,
- -- - usages
- -- - deps (home and external packages, dependent files)
- -- - hpc
- iface_hash <- computeFingerprint putNameLiterally
- (mod_hash,
- mi_src_hash iface0,
- ann_fn (mkVarOccFS (fsLit "module")), -- See mkIfaceAnnCache
- mi_usages iface0,
- sorted_deps,
- mi_hpc iface0)
+ -- the flag hash depends on:
+ -- - (some of) dflags
+ -- it returns two hashes, one that shouldn't change
+ -- the abi hash and one that should
+ self_recomp <- if gopt Opt_WriteSelfRecompInfo dflags
+ then mkSelfRecomp mod_hash sorted_deps
+ else return NoSelfRecompBackend
let
final_iface_exts = ModIfaceBackend
- { mi_iface_hash = iface_hash
- , mi_mod_hash = mod_hash
- , mi_flag_hash = flag_hash
- , mi_opt_hash = opt_hash
- , mi_hpc_hash = hpc_hash
- , mi_plugin_hash = plugin_hash
+ { mi_mod_hash = mod_hash
, mi_orphan = not ( all ifRuleAuto orph_rules
-- See Note [Orphans and auto-generated rules]
&& null orph_insts
@@ -1272,6 +1252,7 @@ addFingerprints hsc_env iface0
, mi_export_warn_fn = export_warn_fn
, mi_fix_fn = fix_fn
, mi_hash_fn = lookupOccEnv local_env
+ , mi_self_recomp_backend_info = self_recomp
}
final_iface = completePartialModIface iface0
sorted_decls sorted_extra_decls final_iface_exts
@@ -1287,6 +1268,38 @@ addFingerprints hsc_env iface0
(non_orph_fis, orph_fis) = mkOrphMap ifFamInstOrph (mi_fam_insts iface0)
ann_fn = mkIfaceAnnCache (mi_anns iface0)
+ mkSelfRecomp mod_hash sorted_deps = do
+
+ flag_hash <- fingerprintDynFlags hsc_env this_mod putNameLiterally
+
+ opt_hash <- fingerprintOptFlags dflags putNameLiterally
+
+ hpc_hash <- fingerprintHpcFlags dflags putNameLiterally
+
+ plugin_hash <- fingerprintPlugins (hsc_plugins hsc_env)
+
+ -- The interface hash depends on:
+ -- - the ABI hash, plus
+ -- - the source file hash,
+ -- - the module level annotations,
+ -- - usages
+ -- - deps (home and external packages, dependent files)
+ -- - hpc
+ iface_hash <- computeFingerprint putNameLiterally
+ (mod_hash,
+ mi_src_hash iface0,
+ ann_fn (mkVarOccFS (fsLit "module")), -- See mkIfaceAnnCache
+ mi_usages iface0,
+ sorted_deps,
+ mi_hpc iface0)
+
+ return (ModIfaceSelfRecompBackend
+ { mi_sr_flag_hash = flag_hash
+ , mi_sr_hpc_hash = hpc_hash
+ , mi_sr_opt_hash = opt_hash
+ , mi_sr_iface_hash = iface_hash
+ , mi_sr_plugin_hash = plugin_hash })
+
-- | Retrieve the orphan hashes 'mi_orphan_hash' for a list of modules
-- (in particular, the orphan modules which are transitively imported by the
-- current module).
=====================================
compiler/GHC/Tc/Errors/Ppr.hs
=====================================
@@ -1994,6 +1994,11 @@ instance Diagnostic TcRnMessage where
TcRnUnexpectedTypeSyntaxInTerms syntax -> mkSimpleDecorated $
text "Unexpected" <+> pprTypeSyntaxName syntax
+ TcRnReifyModuleMissingInfo m -> mkSimpleDecorated $
+ vcat [ (ppr m) <+> text "can't be reified due to missing information in its interface file."
+ , text "Possible cause:" <+> ppr m <+> text "was compiled with -fno-write-self-recomp-info" ]
+
+
diagnosticReason :: TcRnMessage -> DiagnosticReason
diagnosticReason = \case
TcRnUnknownMessage m
@@ -2638,6 +2643,9 @@ instance Diagnostic TcRnMessage where
-> ErrorWithoutFlag
TcRnUnexpectedTypeSyntaxInTerms{}
-> ErrorWithoutFlag
+ TcRnReifyModuleMissingInfo {} ->
+ WarningWithFlag Opt_WarnReifyModuleMissingInfo
+
diagnosticHints = \case
TcRnUnknownMessage m
@@ -3320,6 +3328,8 @@ instance Diagnostic TcRnMessage where
-> noHints
TcRnUnexpectedTypeSyntaxInTerms syntax
-> [suggestExtension (typeSyntaxExtension syntax)]
+ TcRnReifyModuleMissingInfo {} -> noHints
+
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Tc/Errors/Types.hs
=====================================
@@ -4395,6 +4395,12 @@ data TcRnMessage where
Test cases: T24159_type_syntax_rn_fail
-}
TcRnUnexpectedTypeSyntaxInTerms :: TypeSyntax -> TcRnMessage
+ {-| TcRnReifyModuleMissingInfo is a warning triggered by attempting to
+ call reifyModule on a module whose interface file lacks the necessary information
+ to satisfy the query. This normally occurs when the module is compiled with `-fno-write-self-recomp-info`.
+ -}
+ TcRnReifyModuleMissingInfo:: Module -> TcRnMessage
+
deriving Generic
----
=====================================
compiler/GHC/Tc/Gen/Splice.hs
=====================================
@@ -2875,6 +2875,10 @@ modToTHMod :: Module -> TH.Module
modToTHMod m = TH.Module (TH.PkgName $ unitString $ moduleUnit m)
(TH.ModName $ moduleNameString $ moduleName m)
+-- | Note that reifyModule will not work if the module is compiled with `-fno-write-self-recomp-info`
+-- because the implementation works by consulting the `mi_usages` field which is intended to be only
+-- used for recompilation checking. See #8489 for a ticket which tracks improvement
+-- of this function.
reifyModule :: TH.Module -> TcM TH.ModuleInfo
reifyModule (TH.Module (TH.PkgName pkgString) (TH.ModName mString)) = do
this_mod <- getModule
@@ -2887,9 +2891,20 @@ reifyModule (TH.Module (TH.PkgName pkgString) (TH.ModName mString)) = do
reifyFromIface reifMod = do
iface <- loadInterfaceForModule (text "reifying module from TH for" <+> ppr reifMod) reifMod
- let usages = [modToTHMod m | usage <- mi_usages iface,
- Just m <- [usageToModule (moduleUnit reifMod) usage] ]
- return $ TH.ModuleInfo usages
+ case mi_self_recomp_info iface of
+ NoSelfRecomp -> do
+ -- Arguably this should fail here but GHC.Prim always has NoSelfRecomp, so
+ -- any existing traversals would just stop working. Now they will start warning
+ -- and a user is expected to add a special case to avoid GHC.Prim in their traversal.
+
+ -- An alternative would be to add that special case for GHC.Prim here and make it a hard
+ -- error if reifyModule was attempted to be used with these partial interface files.
+ addDiagnosticTc (TcRnReifyModuleMissingInfo reifMod)
+ return (TH.ModuleInfo [])
+ ModIfaceSelfRecomp{ mi_sr_usages } -> do
+ let usages = [modToTHMod m | usage <- mi_sr_usages
+ , Just m <- [usageToModule (moduleUnit reifMod) usage] ]
+ return $ TH.ModuleInfo usages
usageToModule :: Unit -> Usage -> Maybe Module
usageToModule _ (UsageFile {}) = Nothing
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -730,6 +730,10 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "TcRnInvalidInvisTyVarBndr" = 57916
GhcDiagnosticCode "TcRnInvisBndrWithoutSig" = 92337
+ -- TODO
+ GhcDiagnosticCode "TcRnReifyModuleMissingInfo" = 89264
+
+
-- IllegalNewtypeReason
GhcDiagnosticCode "DoesNotHaveSingleField" = 23517
GhcDiagnosticCode "IsNonLinear" = 38291
=====================================
compiler/GHC/Unit/Module/ModGuts.hs
=====================================
@@ -53,7 +53,7 @@ data ModGuts
mg_exports :: ![AvailInfo], -- ^ What it exports
mg_deps :: !Dependencies, -- ^ What it depends on, directly or
-- otherwise
- mg_usages :: ![Usage], -- ^ What was used? Used for interfaces.
+ mg_usages :: !(Maybe [Usage]), -- ^ What was used? Used for interfaces.
mg_used_th :: !Bool, -- ^ Did we run a TH splice?
mg_rdr_env :: !GlobalRdrEnv, -- ^ Top-level lexical environment
=====================================
compiler/GHC/Unit/Module/ModIface.hs
=====================================
@@ -15,7 +15,6 @@ module GHC.Unit.Module.ModIface
, mi_sig_of
, mi_hsc_src
, mi_deps
- , mi_usages
, mi_exports
, mi_used_th
, mi_fixities
@@ -36,8 +35,8 @@ module GHC.Unit.Module.ModIface
, mi_docs
, mi_final_exts
, mi_ext_fields
- , mi_src_hash
, mi_hi_bytes
+ , mi_self_recomp_info
)
, pattern ModIface
, restoreFromOldModIface
@@ -45,10 +44,9 @@ module GHC.Unit.Module.ModIface
, set_mi_module
, set_mi_sig_of
, set_mi_hsc_src
- , set_mi_src_hash
+ , set_mi_self_recomp
, set_mi_hi_bytes
, set_mi_deps
- , set_mi_usages
, set_mi_exports
, set_mi_used_th
, set_mi_fixities
@@ -69,10 +67,15 @@ module GHC.Unit.Module.ModIface
, set_mi_docs
, set_mi_final_exts
, set_mi_ext_fields
+ , mi_usages
+ , mi_src_hash
, completePartialModIface
, IfaceBinHandle(..)
, PartialModIface
, ModIfaceBackend (..)
+ , ModIfaceSelfRecompBackend (..)
+ , ModIfaceSelfRecomp (..)
+ , isSelfRecompilationInterface
, IfaceDeclExts
, IfaceBackendExts
, IfaceExport
@@ -85,6 +88,11 @@ module GHC.Unit.Module.ModIface
, mi_semantic_module
, mi_free_holes
, mi_mnwib
+ , mi_flag_hash
+ , mi_iface_hash
+ , mi_opt_hash
+ , mi_hpc_hash
+ , mi_plugin_hash
, renameFreeHoles
, emptyPartialModIface
, emptyFullModIface
@@ -125,6 +133,9 @@ import GHC.Utils.Binary
import Control.DeepSeq
import Control.Exception
+import GHC.Utils.Panic
+import GHC.Utils.Outputable
+import GHC.Utils.Misc
{- Note [Interface file stages]
@@ -151,19 +162,11 @@ type ModIface = ModIface_ 'ModIfaceFinal
-- * Or computed just before writing the iface to disk. (Hashes)
-- In order to fully instantiate it.
data ModIfaceBackend = ModIfaceBackend
- { mi_iface_hash :: !Fingerprint
- -- ^ Hash of the whole interface
- , mi_mod_hash :: !Fingerprint
+ { mi_mod_hash :: !Fingerprint
-- ^ Hash of the ABI only
- , mi_flag_hash :: !Fingerprint
- -- ^ Hash of the important flags used when compiling the module, excluding
- -- optimisation flags
- , mi_opt_hash :: !Fingerprint
- -- ^ Hash of optimisation flags
- , mi_hpc_hash :: !Fingerprint
- -- ^ Hash of hpc flags
- , mi_plugin_hash :: !Fingerprint
- -- ^ Hash of plugins
+ , mi_self_recomp_backend_info :: !ModIfaceSelfRecompBackend
+ -- ^ Information needed for checking self-recompilation.
+ -- See Note [Self recompilation information in interface files]
, mi_orphan :: !WhetherHasOrphans
-- ^ Whether this module has orphans
, mi_finsts :: !WhetherHasFamInst
@@ -219,6 +222,83 @@ data IfaceBinHandle (phase :: ModIfacePhase) where
-- (e.g., set to 'Nothing').
FullIfaceBinHandle :: !(Strict.Maybe FullBinData) -> IfaceBinHandle 'ModIfaceFinal
+-- | The information for a module which is only used when deciding whether to recompile
+-- itself. In particular the external interface of a module is recorded by the ABI
+-- hash
+data ModIfaceSelfRecompBackend = NoSelfRecompBackend | ModIfaceSelfRecompBackend {
+ mi_sr_flag_hash :: !Fingerprint
+ -- ^ Hash of the important flags used when compiling the module, excluding
+ -- optimisation flags
+ , mi_sr_iface_hash :: !Fingerprint
+ -- ^ Hash of the whole interface
+ , mi_sr_opt_hash :: !Fingerprint
+ -- ^ Hash of optimisation flags
+ , mi_sr_hpc_hash :: !Fingerprint
+ -- ^ Hash of hpc flags
+ , mi_sr_plugin_hash :: !Fingerprint
+ -- ^ Hash of plugins
+}
+withSelfRecompBackend :: HasCallStack => (ModIfaceSelfRecompBackend-> t) -> ModIfaceBackend-> t
+
+withSelfRecompBackend f mi =
+ case mi_self_recomp_backend_info mi of
+ NoSelfRecompBackend -> panic "Trying to use self-recomp info"
+ x -> f x
+
+mi_flag_hash :: HasCallStack => ModIfaceBackend -> Fingerprint
+mi_flag_hash = withSelfRecompBackend mi_sr_flag_hash
+mi_iface_hash :: HasCallStack => ModIfaceBackend -> Fingerprint
+mi_iface_hash = withSelfRecompBackend mi_sr_iface_hash
+mi_opt_hash :: HasCallStack => ModIfaceBackend -> Fingerprint
+mi_opt_hash = withSelfRecompBackend mi_sr_opt_hash
+mi_hpc_hash :: HasCallStack => ModIfaceBackend -> Fingerprint
+mi_hpc_hash = withSelfRecompBackend mi_sr_hpc_hash
+mi_plugin_hash :: HasCallStack => ModIfaceBackend -> Fingerprint
+mi_plugin_hash = withSelfRecompBackend mi_sr_plugin_hash
+
+isSelfRecompilationInterface :: ModIface -> Bool
+isSelfRecompilationInterface iface =
+ case mi_self_recomp_info iface of
+ NoSelfRecomp -> False
+ ModIfaceSelfRecomp {} -> True
+
+{-
+Note [Self recompilation information in interface files]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The flag -fwrite-self-recomp-info controls whether
+interface files contain the information necessary to answer the
+question:
+
+ Do I need to recompile myself or is this current interface file
+ suitable?
+
+Why? Most packages are only built once either by a distribution or cabal
+and then placed into an immutable store, after which we will never ask
+this question. Therefore we can derive two benefits from omitting this
+information.
+
+* Primary motivation: It vastly reduces the surface area for creating
+ non-deterministic interface files. See issue #10424 which motivated a
+ proper fix to that issue. Distributions have long contained versions
+ of GHC which just have broken self-recompilation checking (in order to
+ get deterministic interface files).
+
+* Secondary motivation: This reduces the size of interface files
+ slightly.. the `mi_usages` field can be quite big but probably this
+ isn't such a great benefit.
+
+* Third motivation: Conceptually clarity about which parts of an
+ interface file are used in order to **communicate** with subsequent
+ packages about the **interface** for a module. And which parts are
+ used to self-communicate during recompilation checking.
+
+The main tracking issue is #22188 but fixes issues such as #10424 in a
+proper way.
+
+-}
+
+
-- | A 'ModIface' plus a 'ModDetails' summarises everything we know
-- about a compiled module. The 'ModIface' is the stuff *before* linking,
-- and can be written out to an interface file. The 'ModDetails is after
@@ -246,16 +326,6 @@ data ModIface_ (phase :: ModIfacePhase)
-- consulted for directly-imported modules, but not
-- for anything else (hence lazy)
- mi_usages_ :: [Usage],
- -- ^ Usages; kept sorted so that it's easy to decide
- -- whether to write a new iface file (changing usages
- -- doesn't affect the hash of this module)
- -- NOT STRICT! we read this field lazily from the interface file
- -- It is *only* consulted by the recompilation checker
- --
- -- The elements must be *deterministically* sorted to guarantee
- -- deterministic interface files
-
mi_exports_ :: ![IfaceExport],
-- ^ Exports
-- Kept sorted by (mod,occ), to make version comparisons easier
@@ -353,13 +423,15 @@ data ModIface_ (phase :: ModIfacePhase)
-- chosen over `ByteString`s.
--
- mi_src_hash_ :: !Fingerprint,
- -- ^ Hash of the .hs source, used for recompilation checking.
- mi_hi_bytes_ :: !(IfaceBinHandle phase)
+ mi_hi_bytes_ :: !(IfaceBinHandle phase),
-- ^ A serialised in-memory buffer of this 'ModIface'.
-- If this handle is given, we can avoid serialising the 'ModIface'
-- when writing this 'ModIface' to disk, and write this buffer to disk instead.
-- See Note [Sharing of ModIface].
+
+ mi_self_recomp_info_ :: !ModIfaceSelfRecomp
+ -- ^ Information needed for checking self-recompilation.
+ -- See Note [Self recompilation information in interface files]
}
-- Enough information to reconstruct the top level environment for a module
@@ -372,6 +444,37 @@ data IfaceTopEnv
instance NFData IfaceTopEnv where
rnf (IfaceTopEnv a b) = rnf a `seq` rnf b
+data ModIfaceSelfRecomp = NoSelfRecomp
+ | ModIfaceSelfRecomp { mi_sr_src_hash :: !Fingerprint
+ -- ^ Hash of the .hs source, used for recompilation checking.
+ , mi_sr_usages :: [Usage]
+ -- ^ Usages; kept sorted so that it's easy to decide
+ -- whether to write a new iface file (changing usages
+ -- doesn't affect the hash of this module)
+ -- NOT STRICT! we read this field lazily from the interface file
+ -- It is *only* consulted by the recompilation checker
+ }
+
+instance Outputable ModIfaceSelfRecomp where
+ ppr NoSelfRecomp = text "NoSelfRecomp"
+ ppr (ModIfaceSelfRecomp{mi_sr_src_hash, mi_sr_usages}) = vcat [text "Self-Recomp"
+ , nest 2 (vcat [text "src hash:" <+> ppr mi_sr_src_hash
+ , text "usages:" <+> ppr (length mi_sr_usages)])]
+
+withSelfRecomp :: HasCallStack => (ModIfaceSelfRecomp-> t) -> ModIface_ phase -> t
+withSelfRecomp f mi =
+ case mi_self_recomp_info mi of
+ NoSelfRecomp -> panic "Trying to use self-recomp info"
+ x -> f x
+
+mi_usages :: HasCallStack => ModIface_ phase -> [Usage]
+mi_usages = withSelfRecomp mi_sr_usages
+mi_src_hash :: HasCallStack => ModIface_ phase -> Fingerprint
+mi_src_hash = withSelfRecomp mi_sr_src_hash
+
+
+
+
{-
Note [Strictness in ModIface]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -446,15 +549,52 @@ renameFreeHoles fhs insts =
-- It wasn't actually a hole
| otherwise = emptyUniqDSet
+instance Binary ModIfaceSelfRecompBackend where
+ put_ bh NoSelfRecompBackend = put_ bh (0 :: Int)
+ put_ bh (ModIfaceSelfRecompBackend {mi_sr_flag_hash, mi_sr_iface_hash, mi_sr_plugin_hash, mi_sr_opt_hash, mi_sr_hpc_hash}) = do
+ put_ bh (1 :: Int)
+ put_ bh mi_sr_flag_hash
+ put_ bh mi_sr_iface_hash
+ put_ bh mi_sr_plugin_hash
+ put_ bh mi_sr_opt_hash
+ put_ bh mi_sr_hpc_hash
+
+ get bh = do
+ (tag :: Int) <- get bh
+ case tag of
+ 0 -> return NoSelfRecompBackend
+ 1 -> do
+ mi_sr_flag_hash <- get bh
+ mi_sr_iface_hash <- get bh
+ mi_sr_plugin_hash <- get bh
+ mi_sr_opt_hash <- get bh
+ mi_sr_hpc_hash <- get bh
+ return (ModIfaceSelfRecompBackend {mi_sr_flag_hash, mi_sr_iface_hash, mi_sr_plugin_hash, mi_sr_opt_hash, mi_sr_hpc_hash})
+ x -> pprPanic "get_ModIfaceSelfRecomp" (ppr x)
+
+instance Binary ModIfaceSelfRecomp where
+ put_ bh NoSelfRecomp = put_ bh (0 :: Int)
+ put_ bh (ModIfaceSelfRecomp{mi_sr_src_hash, mi_sr_usages}) = do
+ put_ bh (1 :: Int)
+ put_ bh mi_sr_src_hash
+ lazyPut bh mi_sr_usages
+
+ get bh = do
+ (tag :: Int) <- get bh
+ case tag of
+ 0 -> return NoSelfRecomp
+ 1 -> do
+ src_hash <- get bh
+ usages <- {-# SCC "bin_usages" #-} lazyGet bh
+ return $ ModIfaceSelfRecomp { mi_sr_src_hash = src_hash, mi_sr_usages = usages }
+ x -> pprPanic "get_ModIfaceSelfRecomp" (ppr x)
+
-- See Note [Strictness in ModIface] about where we use lazyPut vs put
instance Binary ModIface where
put_ bh (PrivateModIface {
mi_module_ = mod,
mi_sig_of_ = sig_of,
mi_hsc_src_ = hsc_src,
- mi_src_hash_ = _src_hash, -- Don't `put_` this in the instance
- -- because we are going to write it
- -- out separately in the actual file
mi_hi_bytes_ = _hi_bytes, -- We don't serialise the 'mi_hi_bytes_', as it itself
-- may contain an in-memory byte array buffer for this
-- 'ModIface'. If we used 'put_' on this 'ModIface', then
@@ -462,7 +602,6 @@ instance Binary ModIface where
-- the byte array.
-- See Note [Private fields in ModIface]
mi_deps_ = deps,
- mi_usages_ = usages,
mi_exports_ = exports,
mi_used_th_ = used_th,
mi_fixities_ = fixities,
@@ -483,13 +622,10 @@ instance Binary ModIface where
mi_ext_fields_ = _ext_fields, -- Don't `put_` this in the instance so we
-- can deal with it's pointer in the header
-- when we write the actual file
+ mi_self_recomp_info_ = self_recomp,
mi_final_exts_ = ModIfaceBackend {
- mi_iface_hash = iface_hash,
+ mi_self_recomp_backend_info = self_recomp_backend,
mi_mod_hash = mod_hash,
- mi_flag_hash = flag_hash,
- mi_opt_hash = opt_hash,
- mi_hpc_hash = hpc_hash,
- mi_plugin_hash = plugin_hash,
mi_orphan = orphan,
mi_finsts = hasFamInsts,
mi_exp_hash = exp_hash,
@@ -498,16 +634,12 @@ instance Binary ModIface where
put_ bh mod
put_ bh sig_of
put_ bh hsc_src
- put_ bh iface_hash
+ put_ bh self_recomp
+ put_ bh self_recomp_backend
put_ bh mod_hash
- put_ bh flag_hash
- put_ bh opt_hash
- put_ bh hpc_hash
- put_ bh plugin_hash
put_ bh orphan
put_ bh hasFamInsts
lazyPut bh deps
- lazyPut bh usages
put_ bh exports
put_ bh exp_hash
put_ bh used_th
@@ -532,16 +664,12 @@ instance Binary ModIface where
mod <- get bh
sig_of <- get bh
hsc_src <- get bh
- iface_hash <- get bh
+ self_recomp_info <- get bh
+ self_recomp_backend_info <- get bh
mod_hash <- get bh
- flag_hash <- get bh
- opt_hash <- get bh
- hpc_hash <- get bh
- plugin_hash <- get bh
orphan <- get bh
hasFamInsts <- get bh
deps <- lazyGet bh
- usages <- {-# SCC "bin_usages" #-} lazyGet bh
exports <- {-# SCC "bin_exports" #-} get bh
exp_hash <- get bh
used_th <- get bh
@@ -565,15 +693,12 @@ instance Binary ModIface where
mi_module_ = mod,
mi_sig_of_ = sig_of,
mi_hsc_src_ = hsc_src,
- mi_src_hash_ = fingerprint0, -- placeholder because this is dealt
- -- with specially when the file is read
mi_hi_bytes_ =
-- We can't populate this field here, as we are
-- missing the 'mi_ext_fields_' field, which is
-- handled in 'getIfaceWithExtFields'.
FullIfaceBinHandle Strict.Nothing,
mi_deps_ = deps,
- mi_usages_ = usages,
mi_exports_ = exports,
mi_used_th_ = used_th,
mi_anns_ = anns,
@@ -595,13 +720,10 @@ instance Binary ModIface where
mi_docs_ = docs,
mi_ext_fields_ = emptyExtensibleFields, -- placeholder because this is dealt
-- with specially when the file is read
+ mi_self_recomp_info_ = self_recomp_info,
mi_final_exts_ = ModIfaceBackend {
- mi_iface_hash = iface_hash,
+ mi_self_recomp_backend_info = self_recomp_backend_info,
mi_mod_hash = mod_hash,
- mi_flag_hash = flag_hash,
- mi_opt_hash = opt_hash,
- mi_hpc_hash = hpc_hash,
- mi_plugin_hash = plugin_hash,
mi_orphan = orphan,
mi_finsts = hasFamInsts,
mi_exp_hash = exp_hash,
@@ -622,10 +744,8 @@ emptyPartialModIface mod
{ mi_module_ = mod,
mi_sig_of_ = Nothing,
mi_hsc_src_ = HsSrcFile,
- mi_src_hash_ = fingerprint0,
mi_hi_bytes_ = PartialIfaceBinHandle,
mi_deps_ = noDependencies,
- mi_usages_ = [],
mi_exports_ = [],
mi_used_th_ = False,
mi_fixities_ = [],
@@ -645,7 +765,8 @@ emptyPartialModIface mod
mi_complete_matches_ = [],
mi_docs_ = Nothing,
mi_final_exts_ = (),
- mi_ext_fields_ = emptyExtensibleFields
+ mi_ext_fields_ = emptyExtensibleFields,
+ mi_self_recomp_info_ = NoSelfRecomp
}
emptyFullModIface :: Module -> ModIface
@@ -654,12 +775,8 @@ emptyFullModIface mod =
{ mi_decls_ = []
, mi_hi_bytes_ = FullIfaceBinHandle Strict.Nothing
, mi_final_exts_ = ModIfaceBackend
- { mi_iface_hash = fingerprint0,
- mi_mod_hash = fingerprint0,
- mi_flag_hash = fingerprint0,
- mi_opt_hash = fingerprint0,
- mi_hpc_hash = fingerprint0,
- mi_plugin_hash = fingerprint0,
+ { mi_mod_hash = fingerprint0,
+ mi_self_recomp_backend_info = NoSelfRecompBackend, -- TODO
mi_orphan = False,
mi_finsts = False,
mi_exp_hash = fingerprint0,
@@ -689,18 +806,17 @@ instance ( NFData (IfaceBackendExts (phase :: ModIfacePhase))
, NFData (IfaceDeclExts (phase :: ModIfacePhase))
) => NFData (ModIface_ phase) where
rnf (PrivateModIface
- { mi_module_, mi_sig_of_, mi_hsc_src_, mi_hi_bytes_, mi_deps_, mi_usages_
+ { mi_module_, mi_sig_of_, mi_hsc_src_, mi_hi_bytes_, mi_deps_
, mi_exports_, mi_used_th_, mi_fixities_, mi_warns_, mi_anns_
, mi_decls_, mi_defaults_, mi_extra_decls_, mi_foreign_, mi_top_env_, mi_insts_
, mi_fam_insts_, mi_rules_, mi_hpc_, mi_trust_, mi_trust_pkg_
, mi_complete_matches_, mi_docs_, mi_final_exts_
- , mi_ext_fields_, mi_src_hash_ })
+ , mi_ext_fields_ })
= rnf mi_module_
`seq` rnf mi_sig_of_
`seq` mi_hsc_src_
`seq` mi_hi_bytes_
`seq` mi_deps_
- `seq` mi_usages_
`seq` mi_exports_
`seq` rnf mi_used_th_
`seq` mi_fixities_
@@ -721,16 +837,14 @@ instance ( NFData (IfaceBackendExts (phase :: ModIfacePhase))
`seq` rnf mi_docs_
`seq` mi_final_exts_
`seq` mi_ext_fields_
- `seq` rnf mi_src_hash_
`seq` ()
instance NFData (ModIfaceBackend) where
- rnf (ModIfaceBackend{ mi_iface_hash, mi_mod_hash, mi_flag_hash, mi_opt_hash
- , mi_hpc_hash, mi_plugin_hash, mi_orphan, mi_finsts, mi_exp_hash
+ rnf (ModIfaceBackend{ mi_mod_hash
+ , mi_orphan, mi_finsts, mi_exp_hash
, mi_orphan_hash, mi_decl_warn_fn, mi_export_warn_fn, mi_fix_fn
, mi_hash_fn})
- = rnf mi_iface_hash
- `seq` rnf mi_mod_hash
+ = rnf mi_mod_hash
`seq` rnf mi_flag_hash
`seq` rnf mi_opt_hash
`seq` rnf mi_hpc_hash
@@ -744,6 +858,18 @@ instance NFData (ModIfaceBackend) where
`seq` rnf mi_fix_fn
`seq` rnf mi_hash_fn
+instance NFData ModIfaceSelfRecompBackend where
+ -- Sufficient as all fields are strict (and simple)
+ rnf NoSelfRecompBackend = ()
+ -- Written like this so if you add another field you have to think about it
+ rnf !(ModIfaceSelfRecompBackend _ _ _ _ _) = ()
+instance NFData ModIfaceSelfRecomp where
+ -- Sufficient as all fields are strict (and simple)
+ rnf NoSelfRecomp = ()
+ -- MP: Note does not deeply force Usages but the old ModIface logic didn't either, so
+ -- I left it as a shallow force.
+ rnf (ModIfaceSelfRecomp src_hash usages) = src_hash `seq` usages `seq` ()
+
forceModIface :: ModIface -> IO ()
forceModIface iface = () <$ (evaluate $ force iface)
@@ -806,7 +932,7 @@ completePartialModIface partial decls extra_decls final_exts = partial
--
-- The 'mi_src_hash' is computed outside of 'ModIface_' based on the 'ModSummary'.
addSourceFingerprint :: Fingerprint -> ModIface_ phase -> ModIface_ phase
-addSourceFingerprint val iface = iface { mi_src_hash_ = val }
+addSourceFingerprint _val _iface = error "todo" --iface { mi_src_hash_ = val }
-- | Copy fields that aren't serialised to disk to the new 'ModIface_'.
-- This includes especially hashes that are usually stored in the interface
@@ -819,7 +945,7 @@ restoreFromOldModIface :: ModIface_ phase -> ModIface_ phase -> ModIface_ phase
restoreFromOldModIface old new = new
{ mi_top_env_ = mi_top_env_ old
, mi_hsc_src_ = mi_hsc_src_ old
- , mi_src_hash_ = mi_src_hash_ old
+-- , mi_src_hash_ = error "todo" --mi_src_hash_ old
}
set_mi_module :: Module -> ModIface_ phase -> ModIface_ phase
@@ -831,8 +957,11 @@ set_mi_sig_of val iface = clear_mi_hi_bytes $ iface { mi_sig_of_ = val }
set_mi_hsc_src :: HscSource -> ModIface_ phase -> ModIface_ phase
set_mi_hsc_src val iface = clear_mi_hi_bytes $ iface { mi_hsc_src_ = val }
-set_mi_src_hash :: Fingerprint -> ModIface_ phase -> ModIface_ phase
-set_mi_src_hash val iface = clear_mi_hi_bytes $ iface { mi_src_hash_ = val }
+set_mi_self_recomp :: ModIfaceSelfRecomp -> ModIface_ phase -> ModIface_ phase
+set_mi_self_recomp val iface = clear_mi_hi_bytes $ iface { mi_self_recomp_info_ = val }
+
+--set_mi_src_hash :: Fingerprint -> ModIface_ phase -> ModIface_ phase
+--set_mi_src_hash val iface = clear_mi_hi_bytes $ iface { mi_src_hash_ = val }
set_mi_hi_bytes :: IfaceBinHandle phase -> ModIface_ phase -> ModIface_ phase
set_mi_hi_bytes val iface = iface { mi_hi_bytes_ = val }
@@ -840,8 +969,8 @@ set_mi_hi_bytes val iface = iface { mi_hi_bytes_ = val }
set_mi_deps :: Dependencies -> ModIface_ phase -> ModIface_ phase
set_mi_deps val iface = clear_mi_hi_bytes $ iface { mi_deps_ = val }
-set_mi_usages :: [Usage] -> ModIface_ phase -> ModIface_ phase
-set_mi_usages val iface = clear_mi_hi_bytes $ iface { mi_usages_ = val }
+--set_mi_usages :: [Usage] -> ModIface_ phase -> ModIface_ phase
+--set_mi_usages val iface = clear_mi_hi_bytes $ iface { mi_usages_ = val }
set_mi_exports :: [IfaceExport] -> ModIface_ phase -> ModIface_ phase
set_mi_exports val iface = clear_mi_hi_bytes $ iface { mi_exports_ = val }
@@ -992,20 +1121,19 @@ However, with the pragma, the correct core is generated:
{-# COMPLETE ModIface #-}
pattern ModIface ::
- Module -> Maybe Module -> HscSource -> Dependencies -> [Usage] ->
+ Module -> Maybe Module -> HscSource -> Dependencies ->
[IfaceExport] -> Bool -> [(OccName, Fixity)] -> IfaceWarnings ->
[IfaceAnnotation] -> [IfaceDeclExts phase] ->
Maybe [IfaceBindingX IfaceMaybeRhs IfaceTopBndrInfo] -> IfaceForeign ->
[IfaceDefault] -> Maybe IfaceTopEnv -> [IfaceClsInst] -> [IfaceFamInst] -> [IfaceRule] ->
AnyHpcUsage -> IfaceTrustInfo -> Bool -> [IfaceCompleteMatch] -> Maybe Docs ->
- IfaceBackendExts phase -> ExtensibleFields -> Fingerprint -> IfaceBinHandle phase ->
+ IfaceBackendExts phase -> ExtensibleFields -> IfaceBinHandle phase -> ModIfaceSelfRecomp ->
ModIface_ phase
pattern ModIface
{ mi_module
, mi_sig_of
, mi_hsc_src
, mi_deps
- , mi_usages
, mi_exports
, mi_used_th
, mi_fixities
@@ -1026,14 +1154,13 @@ pattern ModIface
, mi_docs
, mi_final_exts
, mi_ext_fields
- , mi_src_hash
, mi_hi_bytes
+ , mi_self_recomp_info
} <- PrivateModIface
{ mi_module_ = mi_module
, mi_sig_of_ = mi_sig_of
, mi_hsc_src_ = mi_hsc_src
, mi_deps_ = mi_deps
- , mi_usages_ = mi_usages
, mi_exports_ = mi_exports
, mi_used_th_ = mi_used_th
, mi_fixities_ = mi_fixities
@@ -1054,6 +1181,6 @@ pattern ModIface
, mi_docs_ = mi_docs
, mi_final_exts_ = mi_final_exts
, mi_ext_fields_ = mi_ext_fields
- , mi_src_hash_ = mi_src_hash
, mi_hi_bytes_ = mi_hi_bytes
+ , mi_self_recomp_info_ = mi_self_recomp_info
}
=====================================
docs/users_guide/phases.rst
=====================================
@@ -702,6 +702,24 @@ Options affecting code generation
depend on the optimisation level. Any definitions which are already included in
an interface file (via an unfolding for an exported identifier) are reused.
+.. ghc-flag:: -fwrite-self-recomp-info
+ :shortdesc: Write information for self-recompilation checking in an interface file
+ :type: dynamic
+ :category: codegen
+
+ :default: on
+
+ Include information in an interface file which can be used in future to determine
+ whether we need to recompile a module or can reuse the existing interface.
+
+ This is intended to be turned off in situations where you know you will never try
+ to recompile a module, such as when compiling a package for distribution.
+ The advantage is that by omitting unecessary information to do with dependencies
+ there is less chance of build paths leaking into the interface file and affecting
+ determinism.
+
+
+
.. ghc-flag:: -fobject-code
:shortdesc: Generate object code
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2398,6 +2398,7 @@ of ``-W(no-)*``.
triggered whenever this happens, and can be addressed by enabling the
extension.
+<<<<<<< HEAD
.. ghc-flag:: -Wloopy-superclass-solve
:shortdesc: *(deprecated)* warn when creating potentially-loopy superclass constraint evidence
:type: dynamic
@@ -2596,6 +2597,22 @@ of ``-W(no-)*``.
To make the code forwards-compatible and silence the warning, users are
advised to add parentheses manually.
+=======
+.. ghc-flag:: -Wreify-module-missing-info
+ :shortdesc: warn when `reifyModule`
+ :type: dynamic
+ :reverse: -Wno-reify-module-missing-info
+
+ :since: 9.6.1
+
+ The `reifyModule` function in the Template Haskell API can fail to find the necessary
+ information when an interface file is generated with `-fno-write-self-recomp-info`. This
+ is due to a shortcoming in `reifyModule` tracked by :ghc-ticket:`8489`.
+
+ This flag warns the user when they try to call `reifyModule` on a module where this
+ situation occurs so that they know the traversal has ended prematurely.
+
+>>>>>>> d73b6b36c4b (Add flag to control whether self-recompilation information is written to interface)
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
It turns on heavyweight intra-pass sanity-checking within GHC. (It checks GHC's
=====================================
testsuite/tests/annotations/should_compile/th/annth_compunits.stderr
=====================================
@@ -0,0 +1,11 @@
+
+annth.hs:12:4: warning: [GHC-89264] [-Wreify-module-missing-info (in -Wdefault)]
+ • GHC.Prim can't be reified due to missing information in its interface file.
+ Possible cause: GHC.Prim was compiled with -fno-write-self-recomp-info
+ • In the untyped splice:
+ $(do anns <- traverseModuleAnnotations
+ runIO $ print (anns :: [String])
+ anns <- reifyAnnotations (AnnLookupName 'testValue)
+ runIO $ print (anns :: [String])
+ anns <- reifyAnnotations (AnnLookupName 'testValueTH)
+ ....)
=====================================
testsuite/tests/annotations/should_compile/th/annth_make.stderr
=====================================
@@ -0,0 +1,11 @@
+
+annth.hs:12:4: warning: [GHC-89264] [-Wreify-module-missing-info (in -Wdefault)]
+ • GHC.Prim can't be reified due to missing information in its interface file.
+ Possible cause: GHC.Prim was compiled with -fno-write-self-recomp-info
+ • In the untyped splice:
+ $(do anns <- traverseModuleAnnotations
+ runIO $ print (anns :: [String])
+ anns <- reifyAnnotations (AnnLookupName 'testValue)
+ runIO $ print (anns :: [String])
+ anns <- reifyAnnotations (AnnLookupName 'testValueTH)
+ ....)
=====================================
testsuite/tests/driver/self-recomp/Makefile
=====================================
@@ -0,0 +1,38 @@
+TOP=../../..
+include $(TOP)/mk/boilerplate.mk
+include $(TOP)/mk/test.mk
+
+TEST_HC_OPTS_NO_RTSOPTS = $(filter-out -rtsopts,$(TEST_HC_OPTS))
+
+# -----------------------------------------------------------------------------
+# One-shot compilations, non-hierarchical modules
+
+# Check that modifying flags doesn't affect interface
+SelfRecomp01:
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp01.hs -fno-write-self-recomp-info -v0
+ "$(TEST_HC)" --show-iface SelfRecomp01.hi > iface1
+ rm SelfRecomp01.hi
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp01.hs -fno-write-self-recomp-info -Iidir -v0
+ "$(TEST_HC)" --show-iface SelfRecomp01.hi > iface2
+ diff iface1 iface2
+
+# Check that the result of addDependentFile doesn't end up in interface
+SelfRecomp02:
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp02.hs -fno-write-self-recomp-info -v0
+ "$(TEST_HC)" --show-iface SelfRecomp02.hi > iface
+ [ -z $(grep iface SelfRecomp02.hs) ]
+
+# Check that modifying source doesn't affect interface
+SelfRecomp03:
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp03.hs -fno-write-self-recomp-info -v0
+ "$(TEST_HC)" --show-iface SelfRecomp03.hi > iface1
+ rm SelfRecomp03.hi
+ echo "" >> SelfRecomp03.hs
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp03.hs -fno-write-self-recomp-info -v0
+ "$(TEST_HC)" --show-iface SelfRecomp03.hi > iface2
+ diff iface1 iface2
+
+# Check that if you don't have recompilation info then you always recompile.
+SelfRecomp04:
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp04.hs -fno-write-self-recomp-info -fhide-source-paths
+ "$(TEST_HC)" $(TEST_HC_OPTS) SelfRecomp04.hs -fno-write-self-recomp-info -fhide-source-paths
=====================================
testsuite/tests/driver/self-recomp/SelfRecomp01.hs
=====================================
@@ -0,0 +1,2 @@
+module SelfRecomp01 where
+
=====================================
testsuite/tests/driver/self-recomp/SelfRecomp02.hs
=====================================
@@ -0,0 +1,6 @@
+{-# LANGUAGE TemplateHaskell #-}
+module SelfRecomp02 where
+
+import Language.Haskell.TH.Syntax
+
+main = $(addDependentFile "SelfRecomp02.hs" >> [| print () |])
=====================================
testsuite/tests/driver/self-recomp/SelfRecomp03.hs
=====================================
@@ -0,0 +1,2 @@
+module SelfRecomp03 where
+
=====================================
testsuite/tests/driver/self-recomp/SelfRecomp04.hs
=====================================
@@ -0,0 +1 @@
+module SelfRecomp04 where
=====================================
testsuite/tests/driver/self-recomp/SelfRecomp04.stdout
=====================================
@@ -0,0 +1,2 @@
+[1 of 1] Compiling SelfRecomp04
+[1 of 1] Compiling SelfRecomp04 [Old interface lacks recompilation info]
=====================================
testsuite/tests/driver/self-recomp/all.T
=====================================
@@ -0,0 +1,4 @@
+test('SelfRecomp01', normal, makefile_test, ['SelfRecomp01'])
+test('SelfRecomp02', normal, makefile_test, ['SelfRecomp02'])
+test('SelfRecomp03', [copy_files], makefile_test, ['SelfRecomp03'])
+test('SelfRecomp04', normal, makefile_test, ['SelfRecomp04'])
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/2b89c2638a3fd8a01b0d01cb8455a24b6c9436c7
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/2b89c2638a3fd8a01b0d01cb8455a24b6c9436c7
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20250227/6878b626/attachment-0001.html>
More information about the ghc-commits
mailing list