[Haskell-cafe] Benchmarking two versions of the same package?

Alp Mestanogullari alpmestan at gmail.com
Thu Jun 11 12:24:51 UTC 2015


Yeah, although I think this doesn't really solve the issue of using two
versions of the same package.

On Thu, Jun 11, 2015 at 1:52 PM, lucas di cioccio <lucas.dicioccio at gmail.com
> wrote:

> You should be able to use Laborantin to organize this kind of experiments.
> You'll have to write some boilerplate on how to "build" each package set.
> https://hackage.haskell.org/package/laborantin-hs
>
> --Lucas
>
> 2015-06-11 12:45 GMT+01:00 Alp Mestanogullari <alpmestan at gmail.com>:
>
>> Oh, Julian Arni just reminded me of Joachim Breitner's Gipeda:
>> https://github.com/nomeata/gipeda
>>
>> This might do the trick, but any suggestion is still welcome!
>>
>> On Thu, Jun 11, 2015 at 1:41 PM, Alp Mestanogullari <alpmestan at gmail.com>
>> wrote:
>>
>>> Hi -cafe,
>>>
>>> While we can easily benchmark different functions or libraries easily
>>> with criterion, I can't think of a reasonably easy (and accurate!) way of
>>> benchmarking two versions of the same package. And not necessarily version
>>> as in cabal version -- one of the use cases I have in mind would be running
>>> a benchmark suite whenever a PR gets merged to the main branch of a
>>> library, so the benchmark would need to compare
>>> the performance of the library's-code-before-merging and after.
>>>
>>> This definitely can't be accomplished with something like criterion
>>> because we can't have two different instances of a package in scope for a
>>> module, even with -XPackageImports.
>>>
>>> If we separately build the same program against two instances of the
>>> same library and run the benchmarks separately, this might happen far apart
>>> enough that the machine running this might be under a different load. This
>>> does however seem to be the only actual solution? Run separately at two
>>> different commits, diff the numbers, report.
>>>
>>> I have felt the need for a solution to this for quite some time, but
>>> have never needed it bad enough that it became a priority. Thinking about
>>> it again today, I thought I should drop an email to the list and see if
>>> fellow haskellers have an easy-to-use solution for this? I may very well be
>>> overlooking something.
>>>
>>> --
>>> Alp Mestanogullari
>>>
>>
>>
>>
>> --
>> Alp Mestanogullari
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>
>>
>


-- 
Alp Mestanogullari
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150611/315c5ad3/attachment.html>


More information about the Haskell-Cafe mailing list