[Haskell-cafe] Combining ST with STM

[Thomas Koster]

On 9 February 2016 at 14:43, Thomas Koster <tkoster at gmail.com> wrote:
> I have an STM transaction that needs some private, temporary state.
> The most obvious way is to simply pass pure state as arguments, but
> for efficiency, I would like this state to be some kind of mutable
> array, like STArray.
>
> I know, STM has TVars and TArray, but since this state is private to
> the transaction, I am wondering if using TVars/TArrays for private
> state might be overkill that will unnecessarily slow down the STM
> commit process. The private state is, by definition, not shared, so
> including it in the STM log and commit process is, as far as I can
> tell, pointless.
>
> ST and STArray still appear to be the most appropriate tools for the
> private state, because STRefs and STArrays really, really are private.
>
> So this basically means I want to interleave ST and STM in a "safe"
> way. That is, if the STM transaction retries, I want the ST state to
> be vaporised as well.
>
> Ideally, I would love to be able to say something like this:
>
> -- | Copy the value from the shared TVar into the private STRef.
> load :: TVar a -> STRef a -> STSTM s ()
> load shared private = do
>   value <- liftSTM (readTVar shared)
>   liftST (writeSTRef private value)
>
> Naturally, that STRef must originate from a call to newSTRef earlier
> in the same transaction and is private to it, just like the real ST
> monad. As far as I can tell, I am not trying to weaken either ST or
> STM in any way here.
>
> I found the STMonadTrans package on Hackage [1] that claims to
> implement ST as a monad transformer, STT, which sounds close to what I
> want. While its documentation does not mention STM, it does say that
> some monads are unsafe to use as a base monad for STT.
>
> Is STMonadTrans safe to use with STM?
>
> [1] https://hackage.haskell.org/package/STMonadTrans

On 9 February 2016 at 15:16, Ryan Yates <fryguybob at gmail.com> wrote:
> I also have found need for what I think you are describing but only in the
> context of transactional arrays where there are multiple fields to
> initialize while I know that the array is private to the creating thread.
> For now I'm adding the primitives I need as I go, but I would like to have
> better safer story.  You might be interested in how the stm-containers
> package uses ST to build internal nodes in transactions [1], [2].
>
> [1]:
> https://github.com/nikita-volkov/stm-containers/blob/master/library/STMContainers/HAMT/Nodes.hs#L36
> [2]:
> https://github.com/nikita-volkov/stm-containers/blob/master/library/STMContainers/WordArray.hs#L118

Thank you Ryan.

Indeed, it is by experimenting with stm-containers that the need for
mixing ST and STM arose. Where the previous iteration of my program
used plain ST transactions serialized with an MVar, I am experimenting
with stm-containers with the hope that I will see improved throughput
for transactions that do not overlap, which, I believe, could complete
in parallel, at least some of the time.

It seems stm-containers itself uses unsafeFreezeArray from the
"primitive" package. One difference though is that while my private
array would be thawed, modified and refrozen regularly, the
stm-containers WordArray stays immutable (not thawed) once frozen, as
far as I can tell.

Since I am using only a single array for the entire private state,
sprinkling some runST calls with unsafeThawArray/unsafeFreezeArray in
my STM transaction may be enough for my needs, as long as I am
exceptionally careful not to leak one of these arrays into or out of
any STM transaction demarcated by the "atomically" block. If anybody
knows of any reason why I should abort this idea, please speak up.

I noticed also that Data.Array.Unsafe in base also has unsafe freezing
and thawing. Is there a reason to use one over the other?

--
Thomas Koster