Announce: hs-plugins-0.9.6

[Donald Bruce Stewart]

claus.reinke:
> I think you're providing workarounds for some serious short-
> comings in Haskell! while a fully integrated design would be 
> nicer (as in the old orthogonally persistent systems - google
> for "type-safe linguistic reflection"; or more recently in Clean), 
> I look forward to having at least a pragmatically oriented 
> subset of the features available. some questions:
>
> is there any particular reason that your list of compatible
> platforms seems limited to unix-like systems? or would it 
> work on other GHC platforms as well, such as windows
> (after all, ghci works there..)?

Only as these are the systems that I've tested on, or that others have.
I don't have access to a Windows machine. hs-plugins should only be
limited to platforms with a working GHC dynamic loader, which is all the
systems I've tested on, plus Windows. Any unixy dependencies should be
considered bugs, and will be removed as I find them.

> what is the relation of this to -package ghc? at the moment, 
> Simon M. is very busy with all the other things people want, 
> but most of the functionality you provide I would hope to 
> find there when it comes (will hs-plugins be implemented on 
> top of -package ghc, or will its functionality be integrated, 

Currently hs-plugins is entirely separate to GHC. This could change as
-package ghc becomes more widely used. It would certainly make
maintainence across GHC releases easier.

The lowest layer of hs-plugins, the dynamic loader, reimplements several
utilities of GHC: the .hi file parser, the package.conf parser.  And, of
course, it contains a binding to the GHC dynamic linker. These
interfaces could all be made much simpler by -package ghc. Clean
interfaces to -package ghc would make, for example, the load() call just
a call into GHC. Also, the makeWith function parses and merges Haskell
source. Using GHC's parser for this removes the limitations of the
Language.Haskell parser.

The other aspect of hs-plugins is the runtime compilation stuff. I think
this part of hs-plugins (i.e. Eval.Haskell, Eval.Meta, Printf) should be
seen as prototype runtime metaprogramming system. 

Ideally it would use -package ghc for type checking and compilation. The
points where hs-plugins currently invokes GHC externally as a type
checker, or as a code generator, should be replaced with calls to the
-package ghc interfaces to these components. I'm already playing around
with this, but it is still in the alpha phase.

So, ultimately, -package ghc may replace some low levels of hs-plugins.
Some of the higher levels could turn into nice interfaces to -package
ghc components, or into a library along the lines of Template Haskell.

> or will there be separate implementations)? you mention it 
> once on page 26, in a way that suggests there should be 
> other references to it in this document, which I can't find..
> how bad/good are the startup/load times?

Some timings for the various flavour of load() are in the Haskell
Workshop paper. An unchecked plugin load is identical to loading a
module in GHCi, and has a similar cost. 

The interesting 'meta' functions require an external GHC process to
compile code, which has a non-trivial cost. For example, compiling a
printf string to a new function takes around 0.5s on my laptop. Once
that is done, it can be applied to arguments at the same speed as normal
native code.

I see -package ghc as the best way to reduce this cost, in a similar way
as Template Haskell uses GHCi internally for its compile-time eval
features.

> I could use some more runtime reflection support in Haskell 
> immediately - I'm currently starting ghci as an external process
> to load&run generated code, which allows me to make it look 
> as if the system just continued running, and to process some 
> error messages, but it doesn't really allow me to combine new 
> and old code, which is what hs-plugins seems to offer; 

Yes. This is exactly what it does. You should be able to do some
interesting things with the framework as it stands.

> however, I don't want to limit my code to unix systems, and I 
> would prefer some convergence on one standardised API.

This would be good. As I see it, the runtime compilation aspects (which
is what you are most interested in?) is really staged computation, so
I'm currently working on an interface along the lines of Shields, Sheard
and SPJ's "Dynamic Typing as Staged Type Inference" paper. That is, a
runtime version of TH. A quick sketch of run(), defer() and splice() are
in Eval.Meta. This is currently my idea of the final interface, but the
entire system is a few months from being really comfortable.

There is still more work to be done here, particularly in relation to
dynamic typing. Data.Dynamic is too restrictive for a comfortable
runtime metaprogramming system. I'm currently looking at a better
dynamics, like that used in hs-plugins' pdynload(), and using -package
ghc for type checking. Still very preliminary though. 

-- Don