Avoiding BlockedIndefinitelyOnSTM exceptions

[Gabriel Gonzalez]

I don't quite understand your question, but I'll try to give a fuller 
explanation of the problem I was trying to solve to see if it perhaps 
answers your question.

The motivation behind `pipes-concurrency` was to make it easy for 
readers and writers to coordinate implicitly (using the garbage 
collector) instead of explicitly.  Here's an example of code that people 
would use `pipes-concurrency` to write:

     main = do
         (output, input) <- spawn Single
         a1 <- async $ runEffect $ someProducer >-> toOutput output
         a2 <- async $ runEffect $ fromInput input >-> someConsumer
         mapM_ wait [a1, a2]

For people unfamiliar with `pipes` or `pipes-concurrency`, what the 
above code does is it `spawn`s a `TMVar` that you can read or write.  
Then it forks two threads, one of which feeds some values to the `TMVar` 
and the other of which reads some values from the `TMVar`.

What was neat about the way `pipes-concurrency` works is that:

* If `someProducer` produces fewer values than `someConsumer` requests 
from `TMVar`, then `someConsumer` would detect that and gracefully 
terminate instead of trying to read a value from the 
now-permanently-empty `TMVar`.
* If `someConsumer` consumes fewer values than `someProducer` feeds to 
the `TMVar` then `someProducer` would detect that and gracefully 
terminate instead of trying to write a new value to the 
now-permanently-full `TMVar`.

Note that throwing an exception to either of these threads is not a 
suitable substitute for graceful termination in more sophisticated 
examples.  The rationale behind graceful termination is to allow running 
more logic in these threads after they are done reading or writing from 
the STM variable.

My feeling is that the garbage collector already knows that these reads 
or writes are doomed, so why not make use of that knowledge to 
gracefully recover from doomed transactions?

On 07/14/2014 12:19 AM, Neil Davies wrote:
> Gabriel
>
> Is the underlying issue one of “scope” - STM variables have global scope, would a batter approach to be to create scope of such things and then some overall recovery mechanism could handle such an exception within that scope?
>
> Neil
>
> On 14 Jul 2014, at 03:30, Gabriel Gonzalez <gabriel439 at gmail.com> wrote:
>
>> I have what may sound like an unusual request: I would like to automatically avoid `BlockedIndefinitelyOnSTM` exceptions with a primitive that looks something like this:
>>
>>     safe :: STM a -> STM (Maybe a)
>>
>> This hypothetical `safe` primitive would attempt a transaction, and if `ghc` detects that this transaction would fail because of an `BlockedIndefinitelyOnSTM` exception it will return `Nothing` instead of throwing an uncatchable exception.
>>
>> I originally simulated a limited form of this behavior using `pipes-concurrency`.  I instrumented the garbage collector (using weak references) to detect when an STM variable was garbage collected and to safely cancel any transactions that depended on those variables.  You can see the implementation here:
>>
>> https://github.com/Gabriel439/Haskell-Pipes-Concurrency-Library/blob/23e7e2dab472b7e4cde7bea31227a917ce5d5375/src/Pipes/Concurrent.hs#L170
>>
>> The original purpose behind this was to easily read and write to a channel without having to count references to the channel.  I reasoned that the garbage collector *already knew* how many open references there were to channel, so I thought "why not use the garbage collector to gracefully cancel transactions that would block just before they would trigger the exception?"
>>
>> This worked really well up until ghc-7.8 changed something and the above trick no longer works.  To be honest, I'm surprised that it ever worked at all, which is why I'm not requesting restoring the original behavior.  Instead, I think providing something like the above `safe` primitive would be nicer, if possible.
>>
>> Would it be possible to implement something like `safe`?
>>
>> Alternatively, is it possible to make the `BlockedIndefinitelyOnSTM` exception catchable?
>>
>> P.S.  I'm also interested in learning more about what may have caused the change in behavior in the transition from ghc-7.6 to ghc-7.8.  What changes were made to the interaction between STM and weak references that may have triggered this?
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> Glasgow-haskell-users at haskell.org
>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users