[Haskell-cafe] Speed of Error handling with Continuations vs. Eithers

[wren ng thornton]

Andrea Vezzosi wrote:
> On Thu, May 13, 2010 at 10:51 AM, wren ng thornton <wren at freegeek.org> wrote:
>> Andrea Vezzosi wrote:
>>> wren ng thornton  wrote:
>>>> With this change [1] I can't notice any difference for your benchmark[2].
>>>> Then again, all the runTest calls take 0 msec and I've had no luck making
>>>> the computation take much time; perhaps your computer can detect a
>>>> difference.
>>> On my machine, with ghc-6.12.1, yours and the original ErrCPS give
>>> quite similar results, both ~2x slower than Either.
>>> However it's important to note that these results are highly dependent
>>> on the monadic expressions being evaluated, with a different benchmark
>>> you can get an huge speedup with the CPS versions.
>>
>> That's very curious. After installing Criterion, my machine (OSX 10.5.8
>> 2.8GHz Intel Core2Duo, GHC 6.12.1 with -O2) shows only 1% difference between
>> my ErrCPS and Either on this benchmark. Alas, I can't print kernel density
>> graphs since Crieterion charts are broken on 6.12. It seems buggy that your
>> platform would behave so much differently...
> 
> I got the measurements from the original code, could you share the
> code that uses criterion instead?

The 1% number was buggy because I hadn't factored the generation of 
random lists out of the benchmark. But, having fixed that, I still can't 
replicate your numbers: I get 12us for Either, vs 17us for EitherCPS.

http://community.haskell.org/~wren/wren-extras/test/Control/Monad/ErrCPS/CriterionBenchmark.hs



Yet another version of the same benchmark, this time using Microbench:

http://community.haskell.org/~wren/wren-extras/test/Control/Monad/ErrCPS/MicrobenchBenchmark.hs

Microbench seems to replicate your numbers better: 2551.930ns vs 
4466.832ns (or 391.86 vs 223.87 calls per second)--- though this is 
getting into the range where there might be Int overflow issues 
corrupting the results (a similar problem showed up when benchmarking 
Data.Trie vs Data.Map), so it may warrant further investigation.


-- 
Live well,
~wren