The GHC(i)/RTS linker and Template Haskell

[Alexis King]

Hi all,

I’ve recently been trying to better understand how and where time is spent
at compile-time when running Template Haskell splices, and one of the areas
I’ve been struggling to figure out is the operation of the linker. From
reading the source code, here’s a summary of what I think I’ve figured out
so far:

   - TH splices are executed using the GHCi interpreter, though it may be
   internal or external (if -fexternal-interpreter is used).

   - Regardless of which mode is used, TH splices need their dependencies
   loaded into the interpreter context before they can be run. This is handled
   by the call to loadDecls in hscCompileCoreExpr', which in turn calls
   loadDependencies in GHC.Linker.Loader.

   - loadDependencies loads packages and modules in different ways. Package
   dependencies are just loaded via the appropriate built shared libraries,
   but modules from the current package have to be loaded a different way, via
   loadObjects (also in GHC.Linker.Loader).

Here, however, is where I get a bit lost. GHC has two strategies for
loading individual objects, which it chooses between depending on whether
the current value of interpreterDynamic is True. But I don’t actually
understand what interpreterDynamic means! The Haddock comment just says
that it determines whether or not the “interpreter uses the Dynamic way”,
but I don’t see why that matters. My understanding was that GHCi *always*
requires dynamic linking, since it is, after all, loading code dynamically.
Under what circumstances would interpreterDynamic ever be False?

Furthermore, I don’t actually understand precisely how and why this
influences the choice of loading strategy. In the case that
interpreterDynamic is True, GHC appears to convert the desired dyn_o object
into a shared library by calling the system linker, then loads that, which
can be very slow but otherwise works. However, when interpreterDynamic is
False, it loads the object directly. Both paths eventually call into “the
RTS linker”, implemented in rts/Linker.c, to actually load the resulting
object.

I have found precious little information on what the RTS linker does, in
which contexts it’s used, or how precisely it works. Note
[runtime-linker-phases] at the top of Linker.c has some information, but
it’s mostly a high-level description of what the code actually does rather
than an explanation of its role in the bigger picture. Does anyone know of
any resources they could possibly point me to that help to explain how all
the pieces fit together here? I’ve spent quite a bit of time reading the
code, but I’m afraid I still haven’t managed to see the forest for the
trees.

Thanks,
Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220531/8d59ca9f/attachment.html>