[Haskell-cafe] How to make asynchronous I/O composable and safe?

[David Barbour]

I favor a wait-free concurrency model based on the `vat` from E language.
Vats can be modeled very easily in Haskell, and in many other languages. I
currently use such a vat model for my Haskell projects. I describe aspects
of it at a few places:

* http://lambda-the-ultimate.org/node/4289#comment-65886 (vats and methods)
* http://lambda-the-ultimate.org/node/4325#comment-66645 (pipeline method
calls)
* http://awelonblue.wordpress.com/2011/10/06/vat-model-for-rdp/

Unfortunately, my code isn't generic. The vats I've implemented are
specialized (i.e. via extra stages and queues) primarily for efficient
processing of concurrent reactive dataflows.

Advantages of Vats:
* wait-free asynchronous IO, hence deadlock and starvation free
* first-class methods that help in many ways:
** distribute the dispatch burden (no need for a `main` switch statement)
** extensible (easy to add new methods without adjusting central code)
** transparent parallelization (calls may transparently invoke local or
remote methods)
** securable (control distribution and parameter-types of methods)
** easily model code distribution (e.g. create a method with monad action
parameters)
* simple state - variables local to each vat may be shared between methods
* coarse-grained `islands of consistency` are easy to reason about
* implicit batching between vats improves consistency AND efficiency
* easy to express incremental computation (as sequence of method calls)
* clean coarse-grained interaction with data-parallelism (e.g. spark
parameters)

Technically, the vat consistency model is not `composable`. Instead, it
works well based on coarse granularity and the natural limits of local
reasoning - i.e. actual use-cases only interact with one or two other vats
before they extend far enough that developers simply design assuming
arbitrary ordering and potential interference. Developers can build ad-hoc
consistency models atop vats easily enough. I use a temporal consistency
model atop vats, which is composable, but is feasible for my reactive
dataflow model primarily due to its updates being commutative.

That said, one should be careful about asserting transactions are
composable. Transactions don't scale well as they compose, having greater
opportunity for conflict, rework, starvation, priority inversion.
Transactions also don't interact well with the real-world IO, such as
continuous sensor streams or actuators. Despite the non-composability of
vat semantics, they at least scale better than transactions. cf.
http://awelonblue.wordpress.com/2011/07/05/transaction-tribulation/

The recent work on Cloud Haskell takes a similar inspiration from E's vats.
However, the focus of cloud haskell is different and consequently there are
a lot of subtle but important differences between cloud haskell processes
and my use of vats, e.g. regarding process identifiers, serializability
requirements, etc.
* http://research.microsoft.com/~simonpj/papers/parallel/remote.pdf (cloud
haskell)

Regards,

Dave


On Fri, Jan 13, 2012 at 9:24 PM, Joey Adams <joeyadams3.14159 at gmail.com>wrote:

> In Haskell, sound logic and a great type system lead to elegant,
> composable code in a variety of domains, such as:
>
>  * Expression evaluation
>  * Parsing
>  * Concurrent programming (thanks to STM)
>
> Asynchronous I/O is tricky.  However, Haskell currently does little to
> alleviate the complexity (at least for me).
>
> How can we structure network protocol APIs so that they stack well
> (e.g. only lock once, rather than locking each layer's connection
> state)?  How can we deal with I/O errors without having to think about
> them at every turn?
>
> For now, how can I structure my application's communication API so
> it's less messy to use?
>
> Thanks,
> - Joey
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120116/ab4cea6b/attachment.htm>