[GHC] #12758: Bring sanity to our performance testsuite

[GHC]

#12758: Bring sanity to our performance testsuite
-------------------------------------+-------------------------------------
        Reporter:  bgamari           |                Owner:  (none)
            Type:  task              |               Status:  new
        Priority:  high              |            Milestone:  8.8.1
       Component:  Test Suite        |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D3758
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by domenkozar):

 Some insight how I've implemented a similar setting for Snabb (open source
 high performance networking drivers).

 Using Nix, we build a matrix of Snabb (multiple git branches, different
 versions of software dependencies like Qemu, etc) and plot all of those
 graphs.

 There is no smart automation to abort of fail for contributions, but if
 you run these tests before releases - and even as a cronjob every X days,
 you can spot regressions easily and then bisect the commit.

 Nix tags what derivations are benchmarks and those run on specialized
 hardware 1 at the time.

 Example report:
 https://hydra.snabb.co/build/3641949/download/2/report.html

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12758#comment:15>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler