Static values language extension proposal

[Jost Berthold]

On 01/28/2014 06:03 PM, Carter Schonwald wrote:
> Theres actually a missing piece of information in this thread: what
> are the example computations that are being sent? My understanding is
> that erlang has not way to send file handles, shared variables,
> Tvars, Mvars, memory mapped binary files, GPU code / memory pointers
> , and other fun unportable things between nodes, and I don't really
> expect / see how we can hope to sanely do that in haskell!
>
> [...]"exactly the same binary, running on a cluster of homogeneous
> machines with the exact same hardware, with a modern linux distro "
> [...]
>
> Nathan Howell was actually doing some experimentation with one
> strategy for this special case here
> https://github.com/alphaHeavy/vacuum-tube  as a deeply rts twiddling
> bit of hackery so you could in fact "serialize arbitrary closures"
> between homogeneous machines running the exact same code (and with
> address randomization disabled too i think)

When mentioning Nathan's approach (based on foreign primops), let me
point to a more complete, RTS-backed implementation; work done by myself
and itself based on a long-standing runtime support for a parallel 
Haskell on distributed memory systems.
The latest instance of this rts-based serialisation was reported in the 
Haskell-implementors' workshop 2013 ( 
www.haskell.org/wikiupload/2/28/HIW2013PackingAPI.pdf ); code is on 
github (https://github.com/jberthold/rts-serialisation)

Some technical remarks:
-Nathan's prim.op approach is awesome, but it is not easy to get its 
interplay with garbage collection right. It is on my list to take a look 
at this code again and see how far we can push the envelope.

-About address randomisation: The RTS-based serialisation uses relative 
locations from a known offset to handle it. A more concerning detail is 
that CAFs must be reverted rather than discarded during GC (currently 
they are just retained, not satisfactory for long-running code).

-About sending arbitrary closures: indeed it does not make any sense to 
transfer MVars and IORefs (file handles, StablePtrs, etc). My approach 
is to solve this dynamically by exception handling. I can imagine that 
there is a sensible combination of RTS support with a suitable type 
class framework (Static, for one), but lazy evaluation, especially lazy 
I/O, complicates matters.

/ Jost Berthold