RFC: ghc's dynamic linker

[Duncan Coutts]

I'm afraid this is going to be another ghc feature request. Let me try
to explain why it would be worth it.

People want to write programs using Haskell as an extension language.
Think of a plugin system or an embeded language. This is currently
possible with ghc but requires some cunning hacks and fiddling with the
linker and it has some limiations.

The difficulty is obviously in accessing the linker. It would be great
if the linker was made avaliable directly in some ghc-specific library.
(I though this was the case but apparently not.) In particular so that
such programs don't need an instalation of ghc lying around to be able
to run. Ie the dyamic linker could be statically linked into a program.
I'd guess that this stuff is already in ghci so maybe it'd just be a
matter of exposing it for other progrms to use.

The second problem is that for an extension language, the dynamicaly
loaded modules need to be able to call functions in the 'host' program.
This means that the linker needs access to the host program's symbol
table (ie its own symbol table), or at least the portion of it that
provides the plugin API.

Simon PJ noted that this might be a nice feature for ghci as well as for
enabling plugin systems. It might allow ghci to not have to load a copy
of the standard librarys since it could link programs to the copy that
is already loaded inside ghci itself. This would save time and memory
when loading programs into ghci.

He also suggested a couple possible ways this might be used. The simpest
might be to add (yet!) another flag while compiling a module to indicate
that the symbol table for the current module should be made avaliable to
the dyamic linker. One would then compile the host program's API module
with this flag so extension modules can link against it.

A more general scheme might be to add pragmas to explicity indicate the
external symbols to associate with haskell functions. This sounds like
more work however and I don't know that the extra generality is necessary.

He also suggested I beg Simon Marlow to think about a sensible
implementation. :-)

The model for using a plugin system would be quite simple and easy to
use. One writes a module which imports the API module, the .hi file for
which is provided by the host program. The module is then compiled to a
.o file. That's it, the host program can then dynamically load the .o
file and link it against the actual API code in its own runtime image.

The particular application I have in mind is adding Haskell as an
extension language to hIDE the GTK Haskell IDE.
http://www.dtek.chalmers.se/~d99josve/hide/

I believe there are a number of people who would be interested in making
dynamic linking easier and might help out with testing and documentation.

So how difficult would this be? Is it worth it?

Duncan Coutts