[Haskell-cafe] Benchmarking two versions of the same package?

[Alp Mestanogullari]

Oh, Julian Arni just reminded me of Joachim Breitner's Gipeda:
https://github.com/nomeata/gipeda

This might do the trick, but any suggestion is still welcome!

On Thu, Jun 11, 2015 at 1:41 PM, Alp Mestanogullari <alpmestan at gmail.com>
wrote:

> Hi -cafe,
>
> While we can easily benchmark different functions or libraries easily with
> criterion, I can't think of a reasonably easy (and accurate!) way of
> benchmarking two versions of the same package. And not necessarily version
> as in cabal version -- one of the use cases I have in mind would be running
> a benchmark suite whenever a PR gets merged to the main branch of a
> library, so the benchmark would need to compare
> the performance of the library's-code-before-merging and after.
>
> This definitely can't be accomplished with something like criterion
> because we can't have two different instances of a package in scope for a
> module, even with -XPackageImports.
>
> If we separately build the same program against two instances of the same
> library and run the benchmarks separately, this might happen far apart
> enough that the machine running this might be under a different load. This
> does however seem to be the only actual solution? Run separately at two
> different commits, diff the numbers, report.
>
> I have felt the need for a solution to this for quite some time, but have
> never needed it bad enough that it became a priority. Thinking about it
> again today, I thought I should drop an email to the list and see if fellow
> haskellers have an easy-to-use solution for this? I may very well be
> overlooking something.
>
> --
> Alp Mestanogullari
>



-- 
Alp Mestanogullari
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150611/9259dd0b/attachment.html>