[Haskell-cafe] Re: ANNOUNCE: Harpy -- run-time code generation library

[Dirk Kleeblatt]

apfelmus wrote:
> Dirk Kleeblatt wrote:
>> apfelmus wrote:
[...]
>> So, yes, the code position is only used after the definition of the
>> label. But the "look up in a map"-part makes the jmp operation strict in
>> the label parameter.
> 
> Ah, I slowly get what you mean. In short, the point is that you're
> reinventing lazy evaluation because you're writing directly to a memory
> buffer and writing a raw _|_ is impossible. So, you invent a scheme to
> delay the evaluation of the _|_ until they are defined.

Exactly.

> I thought that you'd simply store the instructions in a list/monoid like
> the Builder-Monoid from Data.Binary

Currently, we don't.

> Here's an illustration with a fictional robodog toy assembly language
> 
>   import Control.Monad.RWS
> 
>   type Address = Int
>   data Dogcode = Bark | Bite | Jump Address

Nice! :-)

> If you really want to directly write to a memory buffer,

It makes interleaving code generation, execution of the buffer, 
generating more code, executing a little bit and so on (see below) 
easier, but...

> However, because of omnipresent lazy evaluation, it is unclear whether
> "directly" writing to a buffer instead of first building a monoid does
> give a speed/space advantage. So, your current approach may well be
> slower than a naïve "first (difference) list, then raw memory" approach.

... after seeing some benchmarks in the last hours, I guess this is 
right: Doing the optimizations that the binary package does might give 
laziness and speed at the same time.

>> We could omit the map, and just remember where to patch the code, but
>> then we'd need to call explicitly some function after code generation
>> that does the patching. We had implemented this, but found the current
>> solution easier to use, since backpatching is completely automatic and
>> hidden from the user.
> 
> Huh, I thought that runCodeGen would do an additional backpatch pass as
> transparently?

Currently, it doesn't. And in the current implementation, it's not 
necessary.

>>> I also think that having liftIO in the CodeGen-monad is plain wrong. I
>>> mean, CodeGen is a monad that generates code without any execution
>> note that runCodeGen runs the code _generation_, executing the
>> generated code is done _within_ the CodeGen monad via the functions
>> generated by callDecl (or the predefined functions in the Harpy.Call
>> module). This is even more intertwined, but intentional.
> 
> Huh? That means that code gets executed during it's own generation? But
> why do you mix separate concerns? I don't see what use this is besides
> being an opportunity to mess up.

One of our projects is a just-in-time compiler for a functional 
language. Here, compilation is done lazily: A starting stub is compiled 
and executed, when the execution reaches some point for which no code 
has been generated yet the next bit of code is compiled, and the program 
is resumed, and so on. So execution and code generation are interleaved.

Another project is related to dependent type checking as described by 
Benjamin Grégoire and Xavier Leroy in [1]. Here, functions can occur in 
types, and the cited paper describes how these functions can be executed 
by compiled code. During type checking. So type checking, code 
generation, and execution are interleaved.

>> Of course, again a different design is possible, making runCodeGen
>> return a binary code object, that can be called from the IO monad. But
>> then, the user has to care about releasing code buffers, and not to have
>> unevaluated closures having code pointers to already released run-time
>> generated code.
> 
> Huh, why does the user have to care? Shouldn't wrapping the raw memory
> block into a Foreign.ForeignPtr do the job?

Well, both projects manage a separate memory block which is used like a 
heap by generated code. This heap contains closures that have code 
pointers into generated code. And these are subject to our (not yet 
implemented... ;-) ) garbage collectors, not the Haskell collector.

Kind regards,
   Dirk

[1] http://pauillac.inria.fr/~xleroy/publi/strong-reduction.pdf

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3212 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20070515/7d1ed476/smime.bin