[Yhc] Re: some initial questions

[Simon Marlow]

Thomas Shackell wrote:

> The first thing to note is that concurrency is implemented in Yhc by the 
> interpretter running some instructions for one process, then some more 
> instructions for the next process and so on.
> 
> This is rather than using one OS thread per Haskell process because all 
> processes can access the global heap, so if using OS threads it would be 
> necessary to either lock every heap access (incredibly slow) or use some 
> of the truely nasty tricks that concurrent GHC uses.

I resent that :-)   There are other choices too: one OS thread per 
Haskell thread works fine if there's a Giant Lock around the whole 
runtime (this is the approach that OCaml uses, for instance).  Your 
approach is clearly the right one in the context of YHC, though.

I think it's fantastic that you guys have got concurrency working, BTW.

Can you really context switch between two arbitrary instructions?  Is 
the stack in a GC'able state at all times?

> The first problem, as you note, is that originally we only had one stack 
> but now every process needs its own stack. After considering possible 
> options I decided the easiest way was:
>    - process stacks become normal nodes in the heap, and are garbage
>      collected like other heap nodes. This preserves the property that
>      we can't run out of heap but not stack (and vice versa).
> 
>    - a process is initially allocated a small stack space in the heap
>      and if that runs out we simply allocate a new stack space (twice as
>      large as before) and let the old stack be garbage collected.

This is exactly what GHC does.  I found we had to be very careful with 
pointers to the old version of the stack, which occur from time to time. 
  eg. ThreadIds are just pointers to the TSO (== stack), they have to be 
indirected to the new stack when accessed.

> Now going back, the approach of running some instructions from one 
> thread then some more from another is fine providing every instruction 
> runs quickly. They all do, with one exception, PRIMITIVE. This is 
> because PRIMITIVE is used for IO actions/FFI calls.
> 
> The solution? Well the basic idea is "just spawn an OS thread to do the 
> IO action while the main thread continues running Haskell".
> 
> In detail, when PRIMITIVE is called for an IO action/FFI call:
> 
>    - an OS thread is taken from a pool of threads
> 
>    - the main thread tells the OS thread to run the IO action/FFI call
> 
>    - the main thread then reschedules so another haskell process can run
> 
>    - eventually the OS thread complete it's action and is put back in
>      the pool. It adds the process to the list of processes that have
>      completed their IO actions and are waiting to be rescheduled.
> 
>    - when the main thread next reschedules it checks the list of
>      completed IO actions and makes the corresponding blocked processes
>      ready to run again.

What happens if the FFI call invokes a callback?  (a foreign export, or 
foreign import wrapper)?  Can callbacks be invoked by multiple OS 
threads?  Are you planning to implement bound threads?

Cheers,
	Simmon