Remote GHCi
Andrew Gibiansky
andrew.gibiansky at gmail.com
Thu Nov 19 17:37:43 UTC 2015
Simon,
That's good to hear. That addresses one of the complex use cases for
dynCompileExpr that we have.
I'd like to present two more and see how we can address them. They both
follow the same pattern so I will just describe the first one.
In order to let interpreted Haskell code display images and other complex
media to the frontend, the interpreted code uses unsafePerformIO to create
a TChan. This TChan holds images (roughly speaking). The interpreted
Haskell has access to a function which writes to this TChan.
The compiled Haskell runs the interpreted Haskell in a separate thread.
That thread then writes to this TChan (which lives in the interpreted
context). In order to access values in the TChan, IHaskell uses
dynCompileExpr on a value of type `IO [Image]`. This value is an IO action
that reads from the TChan that lives in the interpreted context. This value
is then run in the compiled code, giving IHaskell access to the results
generated by the interpreter.
I do not see how this can be easily replicated with a remote GHCi. The
value is `IO [Image]`, so it is not serializable. `[Image]` is
serializable; however, in order to get the Image values from the TChan, we
would have to run interpreted code, and these images are generated during
the runtime of interpreted code already, so we would need to be running two
separate interpreted threads at once. This has to be completely invisible
to the user and seems to me like it might be a logistical nightmare... And,
if I understand correctly, GHC/GHCi are not thread safe, so I can't just
use the new equivalent of dynCompileExpr from two threads. (Can I?)
Thanks,
-- Andrew
On Thu, Nov 19, 2015 at 1:44 AM, Simon Marlow <marlowsd at gmail.com> wrote:
> Hi Andrew -
>
> Since the interpreted code is running in a separate process that we spawn
> using createProcess, we can set stdin/stdout/stderr to be whatever we like,
> including new pipes. GHC itself needs two pipes to communicate with the
> sub-process, but those use separate file descriptors from the std Handles.
>
> So I think the answer is yes, we can support that more easily with remote
> GHCi. I'll think about what API we can provide for it.
>
> Cheers,
> Simon
>
>
> On 18/11/2015 16:26, Andrew Gibiansky wrote:
>
>> Simon,
>>
>> I'd like to hear how we can support what IHaskell does with remote GHCi.
>>
>> One core functionality that we use dynCompileExpr for (not quite
>> dynCompileExpr, but similar) is getting the standard output of code that
>> is being run. Any time code is run, we
>>
>> 1. Create a unix pipe.
>> 2. Set stdout to point to that pipe using dupTo.
>> 3. Use hscStmt with unsafeCoerce to get the other end of the pipe in the
>> compiled context.
>> 4. Run the statement in the interpreted context in a separate thread;
>> meanwhile, read from the pipe to get the stdout of the code running in
>> the interpreted context.
>> 5. When it is done running, move stdout back to point to the read stdout
>> and close the unix pipe file handle.
>> 6. Send the stdout (both intermediate values and the final value) to the
>> frontend to display to the user.
>>
>> The key here is that we can access directly the file handle created by
>> the interpreted code. If the interpreted code is remote, we clearly
>> cannot read from a pipe it creates. In your remote GHCi, how could we
>> solve this problem?
>>
>> In general, how would stdin and stdout work? Would there be a clean way
>> to feed the remote process its stdin and receive its stdout and stderr?
>> That would effectively mean stdin/stdout/stderr are configurable which
>> would be a godsend for IHaskell.
>>
>> -- Andrew
>>
>> On Wed, Nov 18, 2015 at 1:45 AM, Simon Marlow <marlowsd at gmail.com
>> <mailto:marlowsd at gmail.com>> wrote:
>>
>> On 18/11/2015 01:41, Manuel M T Chakravarty wrote:
>>
>> Hi Simon,
>>
>> While this is an interesting proposal, Haskell for Mac strongly
>> relies on running interpreted code in the same process. I’m using
>> ’dynCompileExpr’ as well as ’hscStmtWithLocation’ and some other
>> stuff.
>>
>>
>> Let me say first of all that I'm not going to remove anything, so
>> there's no need to worry. But I'd like to explore exactly what you
>> need, so that we can see whether there's a way to accommodate it
>> with a separate-process implementation.
>>
>> hscStmtWithLocation is part of the core GHCi functionality, it is
>> definitely supported. It has a slightly different signature:
>>
>> hscStmtWithLocation :: HscEnv
>> -> String -- ^ The statement
>> -> String -- ^ The source
>> -> Int -- ^ Starting line
>> -> IO ( Maybe ([Id]
>> , RemoteHValue {- IO [HValue] -}
>> , FixityEnv))
>>
>> RemoteHValue is a reference to a value in the interpreter's context.
>> These have to be evaluated via an explicit API, rather than just
>> unsafeCoercing HValue as we do now. (this is not strictly speaking
>> part of the GHC API, so a separate but interesting question is: why
>> did you need to use this directly, and what should we add to the GHC
>> API?)
>>
>> I believe that many uses of dynCompileExpr can be changed so that
>> the code using the resulting value is moved into the interpreter's
>> context, and then there's no problem.
>>
>> This is quite crucial for some of the interactive
>> functionality. Imagine a game where the game engine is in Swift
>> linked into the main application and the game logic is in
>> *interpreted* Haskell code. The engine calls into the Haskell code
>> multiple times per frame of the animation and for all
>> keyboard/mouse/etc input (using StablePtr and ForeignPtr to
>> construct
>> the scene graph across the Swift and Haskell heap).
>>
>>
>> So my question is, why wouldn't you run the whole game engine in the
>> interpreter's context? That's what would happen if you were to load
>> the program into GHCi and run it. Directly calling back and forth
>> between the client of the GHC API and the program being interpreted
>> is arguably a strange thing to do, and it's kind of accidental that
>> we allow it.
>>
>> I actually also might have a use for the architecture that you are
>> proposing. However, I really would like to keep the ability to, at
>> least, optionally run interpreted code in the same process
>> (without
>> profiling etc). Do you think we could have both?
>>
>>
>> We can certainly have both, it's straightforward to implement, but I
>> don't get to throw away some of the hacks we have to support
>> same-process execution, which would be a shame. We just add more
>> code rather than
>>
>>
>> Cheers,
>> Manuel
>>
>> Simon Marlow <marlowsd at gmail.com <mailto:marlowsd at gmail.com
>> >>:
>>
>> Hi folks - I've been thinking about changing the way we run
>> interpreted code so that it would be run in a separate
>> process. It turns out this has quite a few benefits, and
>> would let us kill some of the really awkward hacks we have
>> in GHC to work around problems that arise because we're
>> running interpreted code and the compiler on the same runtime.
>>
>> I summarised the idea here:
>> https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi
>>
>> I'd be interested to hear if anyone has any thoughts around
>> this, particularly if doing this would make your life
>> difficult in some way. Are people relying on dynCompileExpr
>> for anything?
>>
>> Cheers,
>> Simon
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20151119/f4ed1602/attachment.html>
More information about the ghc-devs
mailing list