[Haskell-cafe] status of dynamic loading

[Gwern Branwen]

On 2008.05.20 15:34:07 -0700, Evan Laforge <qdunkan at gmail.com> scribbled 2.2K characters:
> I have an application where I'd like to load and run code at runtime.
> As far as I know, the possibilities are:
>
> Use hs-plugins.  I'm not sure if this still works on modern ghc, or if
> it works on my platform (darwin intel), but I managed to get this
> compiled after "fixing" a use of STArray whose arity apparently
> changed, but I probably broke it because now I get "Error in array
> index" all the time.

It's probably going to break again soon; at the very least, newer Cabals break it here. So I don't find that surprising. hs-plugins has been problematic for a long time; I don't think anyone could recommend it (unless they're dons) in good conscience - I certainly couldn't.

> I gather Austin Seipp has been doing some work
> on this.  Is there a newer version that should work?  An easy fix for
> the error?

I personally haven't heard of anything; vaguely something with the GHC API comes to mind, but nothing about hs-plugins.

> Then there's the ghc api, which I gather is seen as the way forward
> from hs-plugins, saving it from hi-parsing brittleness and platform
> specific breakage.  I think Yi uses this, apparently in in the Shim
> package.  I poked around some, but wasn't too clear to me what was
> going on, if I was looking in the right place, and if this is a
> reasonable way forward.

GHC API will be the future, at some point, but it's definitely in flux. I know I had a rough time adapting some of the good docs on the 6.6 GHC API to work with 6.8.2; which I suppose is a good thing in the long run, but causes trouble for the here and now.

Yi itself I think uses the API now, yes. The previous reload stuff from 0.3 wasn't too hot, and so JYP went and added in all this Shim stuff I don't understand.

> lambdabot also appears to have its own dynamic loading mechanism, I
> haven't looked at the source enough yet to understand what it's doing.
>  Maybe there's some documentation out there.

So far as I know, lambdabot uses the 'old' architecture Yi was using: <http://www.cse.unsw.edu.au/~dons/papers/SC05.html>. That is, a small kernel which loads in fresh code and data. (On the other hand, the lambdabot.cabal claims to use plugins, so what do I know?)

> The low-tech solution, used by xmonad, would be to pass the "dynamic"
> code to main and just recompile the app for each instance, passing in
> an instance specific set of config arguments.  Of course, this isn't
> really dynamic at all, and compiling all the various instances could
> be clunky and awkward, but I know it works and could possibly be
> reasonable if I also write my own little language for the stuff that
> really has to be dynamic (anyone got one of these?  lisplike would be
> easy and ok, an haskell subset might be nicer...  I would like to
> avoid having *two* languages to write extensions in).

This is actually a very nice solution in many ways; I think it gets you a lot of power and extensibility at a very minimal cost in complexity and bugginess. At least, XMonad has been much better at providing dynamic changes and updates via xmonad.hs, for very few bugs or problems, than anything I see with Yi (which has many, many fewer users).

If you can use this solution, you probably should. I added in an XMonad-style of configuration to the autoproc package <http://hackage.haskell.org/cgi-bin/hackage-scripts/package/autoproc>, and it worked very nicely and was relatively easy to code up.

> The last solution I can think of is to export an RPC API instead of
> dynamically loading code.  The difficulty here is coming up with a
> vocabulary that can be flattened and shipped to another program
> without becoming an excessive maintenance issue or being too
> inflexible.
>
> All in all, the dynamic loading approach of hs-plugins seems like the
> easiest and most flexible, provided I can get its staged type
> checking.  Before I go trying to hack hs-plugins into working, or
> poking around inside yi or lambdabot, can anyone point me at some
> documentation, resources, advice, etc.?
>
> Thanks!

--
gwern
CACI NSO Al Chelsea UKUSA HTCIA insurgency BUDS Ft. Blacklisted
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20080520/d2d5e80c/attachment.bin