[Haskell-cafe] How to correctly benchmark code with Criterion?

Antoine Latter aslatter at gmail.com
Thu Oct 18 15:42:47 CEST 2012


On Thu, Oct 18, 2012 at 4:23 AM, Janek S. <fremenzone at poczta.onet.pl> wrote:
> Dear list,
>
> during past few days I spent a lot of time trying to figure out how to write Criterion benchmarks,
> so that results don't get skewed by lazy evaluation. I want to benchmark different versions of an
> algorithm doing numerical computations on a vector. For that I need to create an input vector
> containing a few thousand elements. I decided to create random data, but that really doesn't
> matter - I could have as well use infinite lists instead of random ones.
>
> My problem is that I am not certain if I am creating my benchmark correctly. I wrote a function
> that creates data like this:
>
> dataBuild :: RandomGen g => g -> ([Double], [Double])
> dataBuild gen = (take 6 $ randoms gen, take 2048 $ randoms gen)
>
> And I create benchmark like this:
>
> bench "Lists" $ nf L.benchThisFunction (L.dataBuild gen)
>

The argument value will be evaluated by the first run of the
bench-mark, and then laziness will keep the value around for the next
few hundred runs that the "bench" function performs.

So the evaluation will be included in the benchmark, but if "bench" is
doing enough trials it will be statistical noise.

Antoine



More information about the Haskell-Cafe mailing list