Measuring performance of GHC

[Michal Terepeta]

On Mon, Dec 5, 2016 at 12:00 PM Moritz Angermann <moritz at lichtzwerge.de>
wrote:

> Hi,
>
> I’ve started the GHC Performance Regression Collection Proposal[1]
> (Rendered [2])
> a while ago with the idea of having a trivially community curated set of
> small[3]
> real-world examples with performance regressions. I might be at fault here
> for
> not describing this to the best of my abilities. Thus if there is
> interested, and
> this sounds like an useful idea, maybe we should still pursue this
> proposal?
>
> Cheers,
>  moritz
>
> [1]: https://github.com/ghc-proposals/ghc-proposals/pull/26
> [2]:
> https://github.com/angerman/ghc-proposals/blob/prop/perf-regression/proposals/0000-perf-regression.rst
> [3]: for some definition of small
>

Interesting! I must have missed this proposal.  It seems that it didn't meet
with much enthusiasm though (but it also proposes to have a completely
separate
repo on github).

Personally, I'd be happy with something more modest:
- A collection of modules/programs that are more representative of real
Haskell
  programs and stress various aspects of the compiler.
  (this seems to be a weakness of nofib, where >90% of modules compile in
less
  than 0.4s)
- A way to compile all of those and do "before and after" comparisons
easily. To
  measure the time, we should probably try to compile each module at least
a few
  times.
  (it seems that this is not currently possible with `tests/perf/compiler`
and
  nofib only compiles the programs once AFAICS)

Looking at the comments on the proposal from Moritz, most people would
prefer to
extend/improve nofib or `tests/perf/compiler` tests. So I guess the main
question is - what would be better:
- Extending nofib with modules that are compile only (i.e., not runnable)
and
  focus on stressing the compiler?
- Extending `tests/perf/compiler` with ability to run all the tests and do
  easy "before and after" comparisons?

Personally, I'm slightly leaning towards `tests/perf/compiler` since this
would
allow sharing the same module as a test for `validate` and to be used for
comparing the performance of the compiler before and after a change.

What do you think?

Thanks,
Michal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20161205/cb8ee56a/attachment.html>