[Haskell-cafe] [ANN] dejafu-0.2.0.0: Overloadable primitives for testable, potentially non-deterministic, concurrency.

[Michael Walker]

Hi cafe,

I am happy to announce a new release of Déjà Fu, my library for testing
concurrent Haskell programs! Things have advanced significantly since
the paper I presented at the Haskell Symposium this year, both in terms
of performance and functionality.

Git:     https://github.com/barrucadu/dejafu
Hackage: https://hackage.haskell.org/package/dejafu-0.2.0.0

## What is it?

Déjà Fu is a library for developing and testing concurrent Haskell
programs, it provides a typeclass-abstraction over GHC's regular
concurrency API, allowing the concrete implementation to be swapped out.

Why do we need this? Well, concurrency is really hard to get right.
Empirical studies have found that many real-world concurrency bugs can
be exhibited with small test cases using as few as two threads: so it’s
not just big concurrent programs that are hard, small ones are too. We
as programmers just don’t seem to have a very good intuition for
traditional threads-and-shared-memory-style concurrency. The typical
approach to testing concurrent programs is to just run them lots of
times, but that doesn’t provide any hard coverage guarantees, and then
we need to wonder: how many runs do we need?

Fortunately, there has been a lot of research into testing concurrency
in the past several years. Systematic concurrency testing is an
approach where the source of nondeterminism, the actual scheduler, is
swapped out for one under the control of the testing framework. This
allows possible schedules to be systematically explored, giving real
coverage guarantees for our tests.

This is a library implementing systematic concurrency testing. It
provides two typeclasses, MonadConc to abstract over much of
Control.Concurrent and related modules, and MonadSTM, to similarly
abstract over Control.Monad.STM.

For examples, see the test suite and the async-dejafu source:
- https://github.com/barrucadu/dejafu/tree/master/dejafu-tests
- https://github.com/barrucadu/dejafu/tree/master/async-dejafu

If you want to read more:

- Recent technical report:
  http://misc.barrucadu.co.uk/pub/dejafu-techreport.pdf
- Haskell Symposium paper about version 0.1.0.0:
  http://www.barrucadu.co.uk/publications/dejafu-hs15.pdf
- A slew of things on my blog:
  http://www.barrucadu.co.uk/posts.html

## Changelog

New stuff:

- Relaxed memory: CRefs now behave just like IORefs, and testing can
  use one of three different memory models: sequential consistency,
  total store order (the default), and partial store order.

- Support for 'yield'.

- Integration with HUnit and tasty:
  https://hackage.haskell.org/package/hunit-dejafu
  https://hackage.haskell.org/package/tasty-dejafu

- Version of the async library reimplemented to use MonadConc:
  https://hackage.haskell.org/package/async-dejafu

- Fair bounding: which allows yield-based constructs like spinlocks
  to be tested without causing infinitely-long executions.

- Length bounding: which allows for potentially non-terminating
  computations to be tested.

Improvements:

- Now uses partial-order reduction to cut down on the number of
  schedules to try, which is a *huge* improvement over just pre-emption
  bounding, which is what 0.1.0.0 did.

- The 'autocheck' function has prettier output.

## How to use it:

If you’re not making use of any IO in your code other than for
concurrency, the transition to using MonadConc and MonadSTM will
probably just be a textual substitution:

- IO is replaced with MonadConc m => m
- STM with MonadSTM m => m
- *IORef with *CRef
- *MVar with *CVar
- *TVar with *CTVar
- Most other things have the same name, and so can be replaced by just
  swapping imports around.

If you are using other IO, you will need a gentle sprinkling of MonadIO
and liftIO in your code as well.

## Is this really just a drop-in replacement for IO/STM?

That’s the idea, yes.

More specifically, the IO instance of MonadConc and the STM instance of
MonadSTM just use the regular IO and STM functions, and so should have
no noticeable change in behaviour.

There are some differences which can lead to incorrect results when
testing, but which should not affect code when used in an IO or STM
context. Specifically: Control.Monad.Conc.Class.getNumCapabilities can
lie to encourage more concurrency when testing; and
Control.Exception.catch can catch exceptions from pure code, but
Control.Monad.Conc.Class.catch can’t (except for the IO instance).

-- 
Michael Walker (http://www.barrucadu.co.uk)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20151201/5ad607e5/attachment.sig>