[Haskell-cafe] Benchmarking two versions of the same package?
roma at ro-che.info
Thu Jun 11 11:48:39 UTC 2015
In your CI script, you can rename one of your versions (say, mypkg to
mypkg-new) and then use -XPackageImports.
On 11/06/15 14:41, Alp Mestanogullari wrote:
> Hi -cafe,
> While we can easily benchmark different functions or libraries easily
> with criterion, I can't think of a reasonably easy (and accurate!) way
> of benchmarking two versions of the same package. And not necessarily
> version as in cabal version -- one of the use cases I have in mind would
> be running a benchmark suite whenever a PR gets merged to the main
> branch of a library, so the benchmark would need to compare
> the performance of the library's-code-before-merging and after.
> This definitely can't be accomplished with something like criterion
> because we can't have two different instances of a package in scope for
> a module, even with -XPackageImports.
> If we separately build the same program against two instances of the
> same library and run the benchmarks separately, this might happen far
> apart enough that the machine running this might be under a different
> load. This does however seem to be the only actual solution? Run
> separately at two different commits, diff the numbers, report.
> I have felt the need for a solution to this for quite some time, but
> have never needed it bad enough that it became a priority. Thinking
> about it again today, I thought I should drop an email to the list and
> see if fellow haskellers have an easy-to-use solution for this? I may
> very well be overlooking something.
> Alp Mestanogullari
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 819 bytes
Desc: OpenPGP digital signature
More information about the Haskell-Cafe