[Haskell-cafe] IOSim on Hackage

[coot at coot.me]

Hi Robin,

Some time ago we received a question about Dejafu on GitHub. Duncan provided a longer answer so let me share the link: 

https://github.com/input-output-hk/io-sim/issues/53
(tldr: `io-sim` simulates time, while dejafu might provide nicer partial order reduction).

I haven't thoroughly reviewed `unliftio`, so my answer can only be partial.  My impression is that a subset of `unliftio` can be supported by `io-sim`, but not things that require `OS` support or cross language barrier, e.g. disk access, spawning processes, foreign calls.  With `io-sim` one has to implement such things as `IO` is not available in `IOSim` monad :smile: - but this isn't that hard as it sounds and gives some interesting opportunities. One of teams in IOG simulates file system API with `fs-sim` (https://github.com/input-output-hk/fs-sim) - they're planning to publish it on Hackage.  We implemented the Berkeley Socket interface (actually something a bit more general as we needed to support some Windows stuff too), which we also plan to publish on Hackage at some point.  The advantage of owning an implementation is that one can do failure injection, something that otherwise requires a middleman between the process and OS (or a special kernel driver) - which isn't that easy as writing something in Haskell.

To be honest I have too little experience with working with Dejafu to be able to give a very good comparison.  I spent a fair bit of time debugging deadlocks and race conditions with `io-sim`, and that's how `io-sim` support for debugging such bugs got quite decent. In particular all threads, `TVar`'s and `MVar`s can be named, one can emit traces when `TVar`s are committed - still debugging race conditions or deadlocks is fun :wink:.
 This is also useful for writing quickcheck property style tests, which have access to internal state changes, which becomes independent of thread scheduling (unlike `putStrLn` or something similar).

Two outstanding issues are:
* add support for `IORef`s: for `io-sim` this is streightforward, for `io-sim-por` (the partial reduction algorightm we have) probably complex, just because POR is complex in itself;
* add support for `MVar` inspection (as we do with `TVar`s; the simulated `MVar`s are implemented with `TVar`s).
There are probably some places were we missing some APIs from one of the packages which we want to support (`base`, `async`, `stm` & `time`). We want to keep up with their evolution.  The list of issues is quite short now (we churned quite a number of issues in the past!):
https://github.com/orgs/input-output-hk/projects/19/views/24  



Best regards,
Marcin

------- Original Message -------
On Monday, August 21st, 2023 at 15:30, Robin Palotai <palotai.robin at gmail.com> wrote:


> Nice! Always good to see more production tooling to emerge.
> A question, not that it is your responsibility to consider, but in case you have thoughts, would be keen to hear:
> 

> Are there any long term integrative plan on bringing the various Io/exception/concurrency abstractions on a compatible basis? Currently there's for example unliftio/Rio, which is already not compatible with dejafu. Dejafu is class based, but might not play well with these io-classes. Also, the emulated semantics might differ. Etc.
> 

> To a player who wants "just" some robust working ecosystem, all these choices and turbulence is quite hard to navigate or make choices about. And input about observed pros/cons and long term plans appreciated.
> 

> Thank you,
> Robin
> 

> On Sun, Aug 20, 2023, 23:02 <coot at coot.me> wrote:
> 

> > Hello dear Haskell-Cafe,
> > 

> > I realised that I haven't shared here that we (the networking team of IOG) released io-sim on Hackage. We found and fixed countless bugs thanks to it, including some concurrent ones. Here's a blog post which I wrote some time ago:
> > https://engineering.iog.io/2023-04-14-io-sim-annoucement
> > 

> > https://hackage.haskell.org/package/io-sim
> > https://hackage.haskell.org/package/io-classes
> > 

> > Cheers,
> > Marcin Szamotulski
> > _______________________________________________
> > Haskell-Cafe mailing list
> > To (un)subscribe, modify options or view archives go to:
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> > Only members subscribed via the mailman list are allowed to post.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 509 bytes
Desc: OpenPGP digital signature
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20230825/36870a70/attachment.sig>